DeepAI AI Chat
Log In Sign Up

A Hierarchical Distributed Processing Framework for Big Image Data

by   Le Dong, et al.
University of Electronic Science and Technology of China

This paper introduces an effective processing framework nominated ICP (Image Cloud Processing) to powerfully cope with the data explosion in image processing field. While most previous researches focus on optimizing the image processing algorithms to gain higher efficiency, our work dedicates to providing a general framework for those image processing algorithms, which can be implemented in parallel so as to achieve a boost in time efficiency without compromising the results performance along with the increasing image scale. The proposed ICP framework consists of two mechanisms, i.e. SICP (Static ICP) and DICP (Dynamic ICP). Specifically, SICP is aimed at processing the big image data pre-stored in the distributed system, while DICP is proposed for dynamic input. To accomplish SICP, two novel data representations named P-Image and Big-Image are designed to cooperate with MapReduce to achieve more optimized configuration and higher efficiency. DICP is implemented through a parallel processing procedure working with the traditional processing mechanism of the distributed system. Representative results of comprehensive experiments on the challenging ImageNet dataset are selected to validate the capacity of our proposed ICP framework over the traditional state-of-the-art methods, both in time efficiency and quality of results.


page 8

page 9

page 10

page 12

page 13


Advanced Image Processing for Astronomical Images

Image Processing in Astronomy is a major field of research and involves ...

Scalable Ray Tracing Using the Distributed FrameBuffer

Image- and data-parallel rendering across multiple nodes on high-perform...

RDD-Eclat: Approaches to Parallelize Eclat Algorithm on Spark RDD Framework (Extended Version)

Frequent itemset mining (FIM) is a highly computational and data intensi...

Component-Based Distributed Framework for Coherent and Real-Time Video Dehazing

Traditional dehazing techniques, as a well studied topic in image proces...

Micro-level Modularity of Computaion-intensive Programs in Big Data Platforms: A Case Study with Image Data

With the rapid advancement of Big Data platforms such as Hadoop, Spark, ...

A Distributed Deep Representation Learning Model for Big Image Data Classification

This paper describes an effective and efficient image classification fra...

Guided Optimization for Image Processing Pipelines

Writing high-performance image processing code is challenging and labor-...