research
∙
09/21/2023
Inferring Capabilities from Task Performance with Bayesian Triangulation
As machine learning models become more general, we need to characterise ...
share
research
∙
06/14/2023
Revealing the structure of language model capabilities
Building a theoretical understanding of the capabilities of large langua...
research
∙
04/22/2023
Dialectical language model evaluation: An initial appraisal of the commonsense spatial reasoning abilities of LLMs
Language models have become very popular recently and many claims have b...
research
∙
09/12/2021
Compute and Energy Consumption Trends in Deep Learning Inference
The progress of some AI paradigms such as deep learning is said to be li...
research
∙
06/29/2021
Conditional Teaching Size
Recent research in machine teaching has explored the instruction of any ...
research
∙
05/12/2021
Automating Data Science: Prospects and Challenges
Given the complexity of typical data science projects and the associated...
research
∙
07/09/2020
Evaluating the Apperception Engine
The Apperception Engine is an unsupervised learning system. Given a sequ...
research
∙
10/05/2019
Making sense of sensory input
This paper attempts to answer a central question in unsupervised learnin...
research
∙
09/12/2019
The Animal-AI Environment: Training and Testing Animal-Like Artificial Cognition
Recent advances in artificial intelligence have been strongly driven by ...
research
∙
05/29/2019
Fairness and Missing Values
The causes underlying unfair decision making are complex, being internal...
research
∙
11/20/2018
Analysing Results from AI Benchmarks: Key Indicators and How to Obtain Them
Item response theory (IRT) can be applied to the analysis of the evaluat...
research
∙
09/26/2018
General-purpose Declarative Inductive Programming with Domain-Specific Background Knowledge for Data Wrangling Automation
Given one or two examples, humans are good at understanding how to solve...
research
∙
07/06/2018
A multidisciplinary task-based perspective for evaluating the impact of AI autonomy and generality on the future of work
This paper presents a multidisciplinary task approach for assessing the ...
research
∙
06/07/2018
Assessing the impact of machine intelligence on human behaviour: an interdisciplinary endeavour
This document contains the outcome of the first Human behaviour and mach...
research
∙
06/02/2018
Accounting for the Neglected Dimensions of AI Progress
We analyze and reframe AI progress. In addition to the prevailing metric...
research
∙
04/19/2018
Finite Biased Teaching with Infinite Concept Classes
We investigate the teaching of infinite concept classes through the effe...
research
∙
09/19/2017
CASP-DM: Context Aware Standard Process for Data Mining
We propose an extension of the Cross Industry Standard Process for Data ...
research
∙
03/26/2015
Universal Psychometrics Tasks: difficulty, composition and decomposition
This note revisits the concepts of task and difficulty. The notion of co...
research
∙
02/19/2015
Forgetting and consolidation for incremental and cumulative knowledge acquisition systems
The application of cognitive mechanisms to support knowledge acquisition...
research
∙
08/29/2014
AI Evaluation: past, present and future
Artificial intelligence develops techniques and systems whose performanc...
research
∙
08/27/2014
Definition and properties to assess multi-agent environments as social intelligence tests
Social intelligence in natural and artificial systems is usually measure...
research
∙
05/09/2013
On the universality of cognitive tests
The analysis of the adaptive behaviour of many different kinds of system...
research
∙
05/07/2013
A short note on estimating intelligence from user profiles in the context of universal psychometrics: prospects and caveats
There has been an increasing interest in inferring some personality trai...
research
∙
02/08/2013
Complexity distribution of agent policies
We analyse the complexity of environments according to the policies that...
research
∙
11/05/2012
Soft (Gaussian CDE) regression models and loss functions
Regression, unlike classification, has lacked a comprehensive and effect...
research
∙
02/03/2012
On the influence of intelligence in (social) intelligence testing environments
This paper analyses the influence of including agents of different degre...
research
∙
12/12/2011
Threshold Choice Methods: the Missing Link
Many performance metrics have been introduced for the evaluation of clas...
research
∙
09/23/2011
Analysis of first prototype universal intelligence tests: evaluating and comparing AI algorithms and humans
Today, available methods that assess AI systems are focused on using emp...
research
∙
07/29/2011
Technical Note: Towards ROC Curves in Cost Space
ROC curves and cost curves are two popular ways of visualising classifie...
research
∙
02/03/2011