DFS: A Dataset File System for Data Discovering Users

05/31/2019
by   Yasith Jayawardana, et al.
0

Many research questions can be answered quickly and efficiently using data already collected for previous research. This practice is called secondary data analysis (SDA), and has gained popularity due to lower costs and improved research efficiency. In this paper we propose DFS, a file system to standardize the metadata representation of datasets, and DDU, a scalable architecture based on DFS for semi-automated metadata generation and data recommendation on the cloud. We discuss how DFS and DDU lays groundwork for automatic dataset aggregation, how it integrates with existing data wrangling and machine learning tools, and explores their implications on datasets stored in digital libraries.

READ FULL TEXT
research
07/02/2019

Methodology for the Automated Metadata-Based Classification of Incriminating Digital Forensic Artefacts

The ever increasing volume of data in digital forensic investigation is ...
research
05/10/2019

Metadata Management for Textual Documents in Data Lakes

Data lakes have emerged as an alternative to data warehouses for the sto...
research
01/09/2022

Camera-Model Identification Using Encoding and Container Characteristics of Video Files

We introduce a new method for camera-model identification. Our approach ...
research
02/17/2019

Let the Cloud Watch Over Your IoT File Systems

Smart devices produce security-sensitive data and keep them in on-device...
research
09/27/2018

FanStore: Enabling Efficient and Scalable I/O for Distributed Deep Learning

Emerging Deep Learning (DL) applications introduce heavy I/O workloads o...
research
05/04/2020

EngMeta – Metadata for Computational Engineering

Computational engineering generates knowledge through the analysis and i...
research
01/05/2021

Efficient Data Management in Neutron Scattering Data Reduction Workflows at ORNL

Oak Ridge National Laboratory (ORNL) experimental neutron science facili...

Please sign up or login with your details

Forgot password? Click here to reset