DeepAI AI Chat
Log In Sign Up

Designing and Implementing Data Warehouse for Agricultural Big Data

by   Vuong M. Ngo, et al.

In recent years, precision agriculture that uses modern information and communication technologies is becoming very popular. Raw and semi-processed agricultural data are usually collected through various sources, such as: Internet of Thing (IoT), sensors, satellites, weather stations, robots, farm equipment, farmers and agribusinesses, etc. Besides, agricultural datasets are very large, complex, unstructured, heterogeneous, non-standardized, and inconsistent. Hence, the agricultural data mining is considered as Big Data application in terms of volume, variety, velocity and veracity. It is a key foundation to establishing a crop intelligence platform, which will enable resource efficient agronomy decision making and recommendations. In this paper, we designed and implemented a continental level agricultural data warehouse by combining Hive, MongoDB and Cassandra. Our data warehouse capabilities: (1) flexible schema; (2) data integration from real agricultural multi datasets; (3) data science and business intelligent support; (4) high performance; (5) high storage; (6) security; (7) governance and monitoring; (8) replication and recovery; (9) consistency, availability and partition tolerant; (10) distributed and cloud deployment. We also evaluate the performance of our data warehouse.


Data Warehouse and Decision Support on Integrated Crop Big Data

In recent years, precision agriculture is becoming very popular. The int...

An Efficient Data Warehouse for Crop Yield Prediction

Nowadays, precision agriculture combined with modern information and com...

Crop Knowledge Discovery Based on Agricultural Big Data Integration

Nowadays, the agricultural data can be generated through various sources...

A big data intelligence marketplace and secure analytics experimentation platform for the aviation industry

The unprecedented volume, diversity and richness of aviation data that c...

Data Mining with Big Data in Intrusion Detection Systems: A Systematic Literature Review

Cloud computing has become a powerful and indispensable technology for c...

Dataspace architecture and manage its components class projection

Big Data technology is described. Big data is a popular term used to des...

Applications of Data Mining (DM) in Science and Engineering: State of the art and perspectives

The continuous increase in the availability of data of any kind, coupled...