To the Cloud and Back: A Distributed Photo Processing Pipeline
نویسنده
چکیده
In this project, we constructed a distributed photo processing pipeline that utilized Dropbox to gather incoming photos, Python-WorkQueue to distribute image processing tasks to a local Condor cluster, and a fault-tolerant daemon that manages these components to automatically process and archive the large collection of data. This system was made to assist a team of archaeologists working in Israel, where collaboration is more difficult, and internet access is very limited. Our paper details the design and implementation of the system and evaluates its effectiveness in the field test.
منابع مشابه
A multi-scale convolutional neural network for automatic cloud and cloud shadow detection from Gaofen-1 images
The reconstruction of the information contaminated by cloud and cloud shadow is an important step in pre-processing of high-resolution satellite images. The cloud and cloud shadow automatic segmentation could be the first step in the process of reconstructing the information contaminated by cloud and cloud shadow. This stage is a remarkable challenge due to the relatively inefficient performanc...
متن کاملAn Efficient Resource Allocation for Processing Healthcare Data in the Cloud Computing Environment
Nowadays, processing large-media healthcare data in the cloud has become an effective way of satisfying the medical userschr('39') QoS (quality of service) demands. Providing healthcare for the community is a complex activity that relies heavily on information processing. Such processing can be very costly for organizations. However, processing healthcare data in cloud has become an effective s...
متن کاملCloud Computing Technology Algorithms Capabilities in Managing and Processing Big Data in Business Organizations: MapReduce, Hadoop, Parallel Programming
The objective of this study is to verify the importance of the capabilities of cloud computing services in managing and analyzing big data in business organizations because the rapid development in the use of information technology in general and network technology in particular, has led to the trend of many organizations to make their applications available for use via electronic platforms hos...
متن کاملCloud manufacturing system
Cloud manufacturing is defined as a relationship between the consumer and a flexible array of production services, managed by an intervening architecture that can match service providers to product and manufacturing processes Cloud manufacturing definitions typically make explicit or imply three groups of actors: consumers, who request and use cloud manufacturing processes; application provider...
متن کاملBuilding a scalable global data processing pipeline for large astronomical photometric datasets
Astronomical photometry is the science of measuring the flux of a celestial object. Since its introduction in the 1970s the CCD has been the principle method of measuring flux to calculate the apparent magnitude of an object. Each CCD image taken must go through a process of cleaning and calibration prior to its use. As the number of research telescopes increases the overall computing resources...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2013