Definition :
Hadoop Distributed File System (HDFS) is the primary storage system used by Hadoop. HDFS is a distributed file system.
That provides high-performance access to data across Hadoop cluster.
Introduction:
HDFC is a distributed file system designed to run on commodity hardware. It existing distributed file System. The most important thing is it differs from other distributed system. HDFS has fault patience and is designed to used on low-cost hardware. HDFS provides high access to application data and is suitable for applications that have large data HDFS built as Infrastructure for the Apache Notch web search engine project. HDFS is now an Apache Hardtop subproject.
Purpose:
Hadoop Distributed file system working along with starting point of applications, either As a part of a Hadoop cluster.
While HDFS is designed to work in many environments, a working knowledge Of HDFS helps with configuration improvements
and diagnostics on a specific cluster.
Assumptions and Goals : -
Hardware Failure:
Hardware failure is the norm rather than the exception. An HDFS instance may consist of thousands Of server machines.
Each storing part of the file system is data.In fact, there are a huge number of Components. So, detection of faults
and quick, automatic recovery from core architectural The goal of HDFS.
Streaming Data Access:
Applications run on HDFS need streaming access to their data sets. HDFS is designed more for batch Processing rather
than interactive use by users. The attention is on high data access Rather than low latency of data access. POSIX
imposes many hard requirements that are not needed for Applications that are targets for HDFS.
Large Data Sets:
Applications that run on HDFS have large data sets. Complicated file in HDFS is changed into gigabytes
To terabytes in size. Thus, HDFS is changed to support large files. It should provide total data bandwidth
And scale to hundreds of nodes in a single cluster. It should support tens of millions of files in a Single instance.
Simple Coherency Model:
HDFS applications need a write-once-read-many access model for files. Once file is created, written, and closed then No
need to changed. This assumption simplifies data coherency issues and enables high throughput data access. A Map-Reduce
application or a web crawler application fits with this model. There is a plan to support Appending writes to files in
the future.