Data Engineering for Everyone

02/23/2021
by   Vijay Janapa Reddi, et al.
59

Data engineering is one of the fastest-growing fields within machine learning (ML). As ML becomes more common, the appetite for data grows more ravenous. But ML requires more data than individual teams of data engineers can readily produce, which presents a severe challenge to ML deployment at scale. Much like the software-engineering revolution, where mass adoption of open-source software replaced the closed, in-house development model for infrastructure code, there is a growing need to enable rapid development and open contribution to massive machine learning data sets. This article shows that open-source data sets are the rocket fuel for research and innovation at even some of the largest AI organizations. Our analysis of nearly 2000 research publications from Facebook, Google and Microsoft over the past five years shows the widespread use and adoption of open data sets. Open data sets that are easily accessible to the public are vital to accelerating ML innovation for everyone. But such open resources are scarce in the wild. So, what if we are able to accelerate data-set creation via automatic data set generation tools?

READ FULL TEXT

page 2

page 3

page 4

page 7

research
04/21/2023

Utilizing Online and Open-Source Machine Learning Toolkits to Leverage the Future of Sustainable Engineering

Recently, there has been a national push to use machine learning (ML) an...
research
07/27/2023

AI in Software Engineering: A Survey on Project Management Applications

Artificial Intelligence (AI) refers to the intelligence demonstrated by ...
research
01/05/2022

CFU Playground: Full-Stack Open-Source Framework for Tiny Machine Learning (tinyML) Acceleration on FPGAs

We present CFU Playground, a full-stack open-source framework that enabl...
research
06/08/2020

Summarising Big Data: Common GitHub Dataset for Software Engineering Challenges

In open-source software development environments; textual, numerical and...
research
06/07/2021

Widening Access to Applied Machine Learning with TinyML

Broadening access to both computational and educational resources is cri...
research
03/27/2023

CoCon: A Data Set on Combined Contextualized Research Artifact Use

In the wake of information overload in academia, methodologies and syste...
research
05/13/2019

The Softwarised Network Data Zoo

More and more management and orchestration approaches for (software) net...

Please sign up or login with your details

Forgot password? Click here to reset