Hive - High Level Programming Language to Analyse Big Data with Hadoop Eco-System

Hive : it is a SQL-Like High Level Programming Language under the Hadoop Umbrella.

Hive uses Derby as the default storage for its Meta-store.
while derby is an embedaded pure java database it will only available for a single user.

It you want to setup Hive for multi-user, then go for the MySQL for the meta store.

I will keep on posting use full information on using Hive through my blogs.

regards
Ambrish

What is Hadoop

Apache Hadoop is an open-source software framework for storage and large-scale processing of data-sets on clusters of commodity hardware. Hadoop is an Apache top-level project being built and used by a global community of contributors and users.^[2] It is licensed under the Apache License 2.0.

The Apache Hadoop framework is composed of the following modules:

• Hadoop Common – contains libraries and utilities needed by other Hadoop modules.
• Hadoop Distributed File System (HDFS) – a distributed file-system that stores data on commodity machines, providing very high aggregate bandwidth across the cluster.
• Hadoop YARN – a resource-management platform responsible for managing compute resources in clusters and using them for scheduling of users' applications.
• Hadoop MapReduce – a programming model for large scale data processing.

All the modules in Hadoop are designed with a fundamental assumption that hardware failures (of individual machines, or racks of machines) are common and thus should be automatically handled in software by the framework. Apache Hadoop's MapReduce and HDFS components originally derived respectively from Google's MapReduce and Google File System (GFS) papers.

YARN stands for "Yet Another Resource Negotiator" and was added later as part of Hadoop 2.0. YARN takes the resource management capabilities that were in MapReduce and packages them so they can be used by new engines. This also streamlines MapReduce to do what it does best, process data. With YARN, you can now run multiple applications in Hadoop, all sharing a common resource management.

Beyond HDFS, YARN, and MapReduce, the entire Apache Hadoop "platform" is now commonly considered to consist of a number of related projects as well – Apache Pig, Apache Hive,Apache HBase, Apache Spark, and others.^[3]

For the end-users, though MapReduce Java code is common, any programming language can be used with "Hadoop Streaming" to implement the "map" and "reduce" parts of the user's program.^[4] Apache Pig, Apache Hive, Apache Spark among other related projects expose higher level user interfaces like Pig Latin and a SQL variant respectively. The Hadoop framework itself is mostly written in the Java programming language, with some native code in C and command line utilities written as shell-scripts.

Apache Hadoop is a registered trademark of the Apache Software Foundation.

(Source: Wikipedia.org)

Open Letter,

To,
All who is interested,
in hadoop,
and lives in delhi.

Sub: Invitation to a make a group of data scientist.

Dear Friends,

I am Ambrish, Java Tech Lead  from Delhi, recently I have started reading some text on Big Data, No-SQL, Hadoop and related terminology.
I found this is so vast. If could make group / team of people who is also putting their efforts to understand the eco-system of hoodop / big data related stuffs, then we can help each other.


please give me your suggestion and feedback.


regards,
Ambrish Jha