A curated, ontology-based, large-scale knowledge graph of artificial intelligence tasks and benchmarks

10/04/2021
by   Kathrin Blagec, et al.
0

Research in artificial intelligence (AI) is addressing a growing number of tasks through a rapidly growing number of models and methodologies. This makes it difficult to keep track of where novel AI methods are successfully – or still unsuccessfully – applied, how progress is measured, how different advances might synergize with each other, and how future research should be prioritized. To help address these issues, we created the Intelligence Task Ontology and Knowledge Graph (ITO), a comprehensive, richly structured and manually curated resource on artificial intelligence tasks, benchmark results and performance metrics. The current version of ITO contain 685,560 edges, 1,100 classes representing AI processes and 1,995 properties representing performance metrics. The goal of ITO is to enable precise and network-based analyses of the global landscape of AI tasks and capabilities. ITO is based on technologies that allow for easy integration and enrichment with external data, automated inference and continuous, collaborative expert curation of underlying ontological models. We make the ITO dataset and a collection of Jupyter notebooks utilising ITO openly available.

READ FULL TEXT
research
01/17/2021

Understanding in Artificial Intelligence

Current Artificial Intelligence (AI) methods, most based on deep learnin...
research
08/22/2019

Report on the First Knowledge Graph Reasoning Challenge 2018 -- Toward the eXplainable AI System

A new challenge for knowledge graph reasoning started in 2018. Deep lear...
research
01/18/2022

Benchmark datasets driving artificial intelligence development fail to capture the needs of medical professionals

Publicly accessible benchmarks that allow for assessing and comparing mo...
research
08/31/2021

Automated Mining of Leaderboards for Empirical AI Research

With the rapid growth of research publications, empowering scientists to...
research
07/26/2021

Measuring Ethics in AI with AI: A Methodology and Dataset Construction

Recently, the use of sound measures and metrics in Artificial Intelligen...
research
08/06/2020

A critical analysis of metrics used for measuring progress in artificial intelligence

Comparing model performances on benchmark datasets is an integral part o...
research
08/24/2023

The intersection of video capsule endoscopy and artificial intelligence: addressing unique challenges using machine learning

Introduction: Technical burdens and time-intensive review processes limi...

Please sign up or login with your details

Forgot password? Click here to reset