Hadoop installation and Configuration

Tar files : 

/root/hadoop-0.20.2.tar.gz

/root/hbase-0.20.4.tar.gz

Path : 

/opt/hadoop

/opt/hbase

JAR files to place in /server/default/lib :

/opt/hadoop/hadoop-0.20.2-core.jar

/opt/hbase/hbase-0.20.4.jar

Hadoop Configuration Changes : 

1. Added JAVA_HOME path in hadoop/conf/hadoop-env.sh

2. Created Data folder to store data and added the below property in hadoop/conf/core-site.xml

  

    fs.default.name

    hdfs://localhost:9000</value>

  

  hadoop.tmp.dir

  /usr/local/hadoop-datastore/hadoop-${user.name}</value>

  A base for other temporary directories.

 

3. Added below porperty in hadoop/conf/hdfs-site.xml

  

    dfs.replication

    1

  

4. Added below property in hadoop/conf/mapred-site.xml

  

    mapred.job.trackername>

    localhost:9001

  

 Formatting the name node:

You need to do this the first time you set up a Hadoop cluster. Do not format a running Hadoop filesystem, this will cause all your data to be erased.

[root@sindhu11 hadoop]# bin/hadoop namenode -format

10/06/23 09:14:48 INFO namenode.NameNode: STARTUP_MSG:

/************************************************************

STARTUP_MSG: Starting NameNode

STARTUP_MSG:   host = localhost/10.60.11.101

STARTUP_MSG:   args = [-format]

STARTUP_MSG:   version = 0.20.2

STARTUP_MSG:   build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r 911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010

************************************************************/

10/06/23 09:14:48 INFO namenode.FSNamesystem: fsOwner=root,root,bin,daemon,sys,adm,disk,wheel

10/06/23 09:14:48 INFO namenode.FSNamesystem: supergroup=supergroup

10/06/23 09:14:48 INFO namenode.FSNamesystem: isPermissionEnabled=true

10/06/23 09:14:48 INFO common.Storage: Image file of size 94 saved in 0 seconds.

10/06/23 09:14:49 INFO common.Storage: Storage directory /usr/local/hadoop-datastore/hadoop-root/dfs/name has been successfully formatted.

10/06/23 09:14:49 INFO namenode.NameNode: SHUTDOWN_MSG:

/************************************************************

SHUTDOWN_MSG: Shutting down NameNode at http://localhost/10.60.11.101

************************************************************/

Start and Stop Hadoop stand-alone node:

[root@sindhu11 hadoop]# bin/start-all.sh

When you're done, stop the daemons with:

 [root@sindhu11 hadoop]# bin/stop-all.sh

Browse the web interface for the NameNode and the JobTracker; by default they are available at:

• NameNode - http://localhost:50070/
• JobTracker - http://localhost:50030/

HBASE Configurations:

1. Added JAVA_HOME path in hbase/conf/hbase-env.sh

2. Added the below property in hbase/conf/hbase-site.xml (to mention the hadoop server ip and port)

hbase.rootdir

hdfs://localhost:9000/hbase

The directory shared by region servers.

Start and Stop HBase Stand-alone mode:

[root@sindhu11 hadoop]# bin/start-hbase.sh

localhost: starting zookeeper, logging to /opt/hbase/bin/../logs/hbase-root-zookeeper-localhost.out

starting master, logging to /opt/hbase/bin/../logs/hbase-root-master-localhost.out

localhost: starting regionserver, logging to /opt/hbase/bin/../logs/hbase-root-regionserver-localhost.out

To Stop :

[root@sindhu11 hadoop]# bin/stop-hbase.sh

To connect into HBASE Shell : 

[root@sindhu11 hadoop]# bin/hbase shell

HBase Shell; enter 'help' for list of supported commands.

Version: 0.20.4, r941076, Tue May  4 16:23:06 PDT 2010

hbase(main):001:0>

Type 'help' at the shells' prompt to get a list of commands. Test your running install by creating tables, inserting content, viewing content, and then dropping your tables. For example:

hbase> create "mylittletable", "mylittlecolumnfamily"
hbase> # To see the schema for you just created "mylittletable" table and its single "mylittlecolumnfamily", type
hbase> describe "mylittletable"
hbase> # To add a row whose id is "myrow", to the column "mylittlecolumnfamily:x" with a value of 'v', do
hbase> put "mylittletable", "myrow", "mylittlecolumnfamily:x", "v"
hbase> # To get the cell just added, do
hbase> get "mylittletable", "myrow"
hbase> # To scan you new table, do
hbase> scan "mylittletable"