Available techniques in hadoop small file issue
نویسندگان
چکیده
منابع مشابه
Snapshots in Hadoop Distributed File System
The ability to take snapshots is an essential functionality of any file system, as snapshots enable system administrators to perform data backup and recovery in case of failure. We present a low-overhead snapshot solution for HDFS, a popular distributed file system for large clusters of commodity servers. Our solution obviates the need for complex distributed snapshot algorithms, by taking adva...
متن کاملTechniques for Building Highly Available Distributed File Systems
This paper analyzes recent research in the field of distributed file systems, with a particular emphasis on the problem of high availability. Several of the techniques involved in building such a system are discussed individually: naming, replication, multiple versions, caching, stashing, and logging. These techniques range from extensions of ideas used in centralized file systems, through new ...
متن کاملGoogle File System and Hadoop Distributed File System - An Analogy
Big Data has indeed been the word which IT Industry is talking about lately. With advancement of automation and data being processed in real time, it has now become a necessity for companies to look forward to sustainable solutions to store their huge datasets and compute valuable information out of it. High performance computing heavily relies on distributed environments to process large chunk...
متن کاملThe Hadoop Distributed File System: Balancing Portabilty
Hadoop is a software framework that supports data intensive distributed application. Hadoop creates clusters of machine and coordinates the work among them. It include two major component, HDFS (Hadoop Distributed File System) and MapReduce. HDFS is designed to store large amount of data reliably and provide high availability of data to user application running at client. It creates multiple da...
متن کاملArchitecture for Hadoop Distributed File Systems
The Hadoop Distributed File System (HDFS) is designed to store very large data sets reliably, and to stream those data sets at high bandwidth to user applications. In a large cluster, thousands of servers both host directly attached storage and execute user application tasks. By distributing storage and computation across many servers, the resource can grow with demand while remaining economica...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: International Journal of Electrical and Computer Engineering (IJECE)
سال: 2020
ISSN: 2088-8708,2088-8708
DOI: 10.11591/ijece.v10i2.pp2097-2101