Big Data used to describe a massive volume of both structured and unstructured data that is so large that it's difficult to process using traditional database and software techniques.
On a high level it has the following characteristics:
Volume - The large amount of data generated that must be taken into account and analyzed along with its management to make decisions.
Velocity - The fastness in which data is being produced or changed and the speed with which data must be received, understood, and processed.
Variety - It includes structured, unstructured, semi-structured and complex data which creates difficulties in management and governance of these data.
Hadoop is a technology which basically caters to two of the prime problems which big data brings with it: how to store and work with voluminous data and more important how to understand data and turn it into a competitive advantage.
Hadoop essentially fills a gap in this area by effectively storing and providing computational capabilities over substantial amounts of data. It’s a distributed system made up of a distributed file system and it offers a way to parallelize and execute programs on a cluster of machines.
In short Hadoop provides both distributed storage and computational capabilities.
Architecturally, Hadoop is just the combination of two technologies: the Hadoop Distributed File System (HDFS) that provides storage, and the MapReduce programming model which provides processing.
In this context HDFS exists to split, distribute, and manage chunks of the overall data set, which could be a single file or a directory full of files. These chunks of data are pre-loaded onto the worker nodes, which later process them. By having the data local at process time, HDFS saves all of the headache and inefficiency of shuffling data back and forth across the network.
The HDFS has a master-slave architecture where the HDFS master (name node) is responsible for partitioning the storage across the slave nodes (data node) and keeping track of where data is located.
The fundamental principle which governs HDFS are scalability and availability which are achieved by data replication and fault tolerance.