Java Heap Size

JAVA_OPTS="-Xms128m -Xmx1024m -XX:MaxPermSize=256m -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000"


Change in run.conf

IECollection

http://utilu.com/IECollection/

Mozilla Firefox Collection

http://utilu.com/UtiluMFC/

 

Download link

http://java.sun.com/javase/downloads/index.jsp - Jdk
http://www.zend.com/en/products/studio/downloads - Zend Studio
http://www.zend.com/en/products/server/downloads - Zend Server
http://wiki.centos.org/Download  - CentOs ISO Images
http://www.ubuntu.com/desktop/get-ubuntu/download - Ubuntu Desktop Edition
http://fedoraproject.org/get-fedora-all - Fedora ISO Images
https://glassfish.dev.java.net/ - Glassfish Download
http://hudson-ci.org/ - Hudson Download

How to install ImageMagick and RMagick on CEntOS

Install dependencies first.

yum install tcl-devel -y
yum install libpng-devel -y
yum install libjpeg-devel -y
yum install ghostscript-devel -y
yum install bzip2-devel -y
yum install freetype-devel -y
yum install libtiff-devel -y

Download and Install ImageMagick

As root, do the following:

cd /usr/local/src
wget ftp://ftp.imagemagick.org/pub/ImageMagick/ImageMagick.tar.gz
tar xvfz ImageMagick.tar.gz
cd ImageMagick*
./configure
make
make install

HDFS, HBASE, ZooKeeper Cluster Configuration

[hadoop@narmada1 hbase]$ bin/start-hbase.sh

starting master, logging to /u01/hbase/bin/../logs/hbase-hadoop-master-localhost2.out

localhost2: starting regionserver, logging to /u01/hbase/bin/../logs/hbase-hadoop-regionserver-localhost2.out

localhost1 : starting regionserver, logging to /u01/hbase/bin/../logs/hbase-hadoop-regionserver-localhost1.out

[hadoop@narmada1 hbase]$ /opt/jdk1.6.0_20/bin/jps

5179 SecondaryNameNode

5004 DataNode

6198 HRegionServer

5326 QuorumPeerMain

6043 HMaster

6326 Jps

4853 NameNode

Checking the Hbase cluster configuration:

[hadoop@narmada1 hbase]$ bin/hbase shell

HBase Shell; enter 'help' for list of supported commands.

Version: 0.20.4, r941076, Tue May  4 16:23:06 PDT 2010

hbase(main):001:0> create 'Test_table'

0 row(s) in 1.2080 seconds

hbase(main):002:0> describe 'Test_table'

DESCRIPTION                                                             ENABLED

 {NAME => 'Test_table', FAMILIES => []}                                 true

1 row(s) in 0.0380 seconds

hbase(main):003:0> exit

[hadoop@narmada2 bin]$ ./hbase shell

HBase Shell; enter 'help' for list of supported commands.

Version: 0.20.4, r941076, Tue May  4 16:23:06 PDT 2010

hbase(main):001:0> describe 'Test-table'

 {NAME => 'Test_table', FAMILIES => []}                                 true

1 row(s) in 0.0350 seconds

hbase(main):003:0> exit

Netbeans Installation Error

If you get below error while installing netbeans

The wizard cannot continue because of the following error: could not load wizard specified in /wizard.inf (104)

WARNING: could not delete temporary file /tmp/ismp001/1497551

WARNING: could not delete temporary file /tmp/ismp001/7317780

do the below command

./netbeans-5_5_1-linux.bin -is:javahome

Hadoop installation and Configuration

Tar files : 

/root/hadoop-0.20.2.tar.gz

/root/hbase-0.20.4.tar.gz

Path : 

/opt/hadoop

/opt/hbase

JAR files to place in /server/default/lib :

/opt/hadoop/hadoop-0.20.2-core.jar

/opt/hbase/hbase-0.20.4.jar

Hadoop Configuration Changes : 

1. Added JAVA_HOME path in hadoop/conf/hadoop-env.sh

2. Created Data folder to store data and added the below property in hadoop/conf/core-site.xml

  

    fs.default.name

    hdfs://localhost:9000</value>

  

  hadoop.tmp.dir

  /usr/local/hadoop-datastore/hadoop-${user.name}</value>

  A base for other temporary directories.

 

3. Added below porperty in hadoop/conf/hdfs-site.xml

  

    dfs.replication

    1

  

4. Added below property in hadoop/conf/mapred-site.xml

  

    mapred.job.trackername>

    localhost:9001

  

 Formatting the name node:

You need to do this the first time you set up a Hadoop cluster. Do not format a running Hadoop filesystem, this will cause all your data to be erased.

[root@sindhu11 hadoop]# bin/hadoop namenode -format

10/06/23 09:14:48 INFO namenode.NameNode: STARTUP_MSG:

/************************************************************

STARTUP_MSG: Starting NameNode

STARTUP_MSG:   host = localhost/10.60.11.101

STARTUP_MSG:   args = [-format]

STARTUP_MSG:   version = 0.20.2

STARTUP_MSG:   build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r 911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010

************************************************************/

10/06/23 09:14:48 INFO namenode.FSNamesystem: fsOwner=root,root,bin,daemon,sys,adm,disk,wheel

10/06/23 09:14:48 INFO namenode.FSNamesystem: supergroup=supergroup

10/06/23 09:14:48 INFO namenode.FSNamesystem: isPermissionEnabled=true

10/06/23 09:14:48 INFO common.Storage: Image file of size 94 saved in 0 seconds.

10/06/23 09:14:49 INFO common.Storage: Storage directory /usr/local/hadoop-datastore/hadoop-root/dfs/name has been successfully formatted.

10/06/23 09:14:49 INFO namenode.NameNode: SHUTDOWN_MSG:

/************************************************************

SHUTDOWN_MSG: Shutting down NameNode at http://localhost/10.60.11.101

************************************************************/

Start and Stop Hadoop stand-alone node:

[root@sindhu11 hadoop]# bin/start-all.sh

When you're done, stop the daemons with:

 [root@sindhu11 hadoop]# bin/stop-all.sh

Browse the web interface for the NameNode and the JobTracker; by default they are available at:

• NameNode - http://localhost:50070/
• JobTracker - http://localhost:50030/

HBASE Configurations:

1. Added JAVA_HOME path in hbase/conf/hbase-env.sh

2. Added the below property in hbase/conf/hbase-site.xml (to mention the hadoop server ip and port)

hbase.rootdir

hdfs://localhost:9000/hbase

The directory shared by region servers.

Start and Stop HBase Stand-alone mode:

[root@sindhu11 hadoop]# bin/start-hbase.sh

localhost: starting zookeeper, logging to /opt/hbase/bin/../logs/hbase-root-zookeeper-localhost.out

starting master, logging to /opt/hbase/bin/../logs/hbase-root-master-localhost.out

localhost: starting regionserver, logging to /opt/hbase/bin/../logs/hbase-root-regionserver-localhost.out

To Stop :

[root@sindhu11 hadoop]# bin/stop-hbase.sh

To connect into HBASE Shell : 

[root@sindhu11 hadoop]# bin/hbase shell

HBase Shell; enter 'help' for list of supported commands.

Version: 0.20.4, r941076, Tue May  4 16:23:06 PDT 2010

hbase(main):001:0>

Type 'help' at the shells' prompt to get a list of commands. Test your running install by creating tables, inserting content, viewing content, and then dropping your tables. For example:

hbase> create "mylittletable", "mylittlecolumnfamily"
hbase> # To see the schema for you just created "mylittletable" table and its single "mylittlecolumnfamily", type
hbase> describe "mylittletable"
hbase> # To add a row whose id is "myrow", to the column "mylittlecolumnfamily:x" with a value of 'v', do
hbase> put "mylittletable", "myrow", "mylittlecolumnfamily:x", "v"
hbase> # To get the cell just added, do
hbase> get "mylittletable", "myrow"
hbase> # To scan you new table, do
hbase> scan "mylittletable"