Processing Large Data Sets using a Cluster Computing Framework

Increase in the scientific disciplines has caused large data collections as important community resources. The volume of interesting data is already measured in terabytes and will soon total in peta-bytes. This research proposal presents the issue of processing massive amount of satellite data. A single LEO satellite sends around 2 GB of data in 24 hours of a day. To process this huge amount of data, normal digital computers face constraints like processing time, recourses and cost. A solution is needed that can provide quick way of processing at low cost. Cluster computing is network based distributed environment that can be a solution for fast processing support for huge sized jobs. A middle-ware is typically required in cluster computing. In this proposal a middle-ware is proposed for handling the existing processing problems in distributed environments. In a typical heterogeneous computation, a middleware can be employed to provide incorporation and interoperability in the underlying applications and services.