[Hadoop-common-user] How to compile HBase code ?

How do you execute the client (command line) do you use the java or the hadoop command?
It seems that there is an error in your classpath when running the client job. The classpath when compiling classes that implement the client is different from the classpath when your client is executed since hadoop and hbase carry their own environment. Maybe tha following link helps:

http://hbase.apache.org/docs/current/api/org/apache/hadoop/hbase/mapreduce/package-summary.html#classpath


regards
Christian

---------------8<--------------------------------

Siemens AG
Corporate Technology
Corporate Research and Technologies
CT T DE IT3
Otto-Hahn-Ring 6
81739 München, Deutschland
Tel.: +49 (89) 636-42722
Fax: +49 (89) 636-41423
mailto:christian.kleegrewe@siemens.com

Siemens Aktiengesellschaft: Vorsitzender des Aufsichtsrats: Gerhard Cromme; Vorstand: Peter Löscher, Vorsitzender; Wolfgang Dehen, Brigitte Ederer, Joe Kaeser, Barbara Kux, Hermann Requardt, Siegfried Russwurm, Peter Y. Solmssen; Sitz der Gesellschaft: Berlin und München, Deutschland; Registergericht: Berlin Charlottenburg, HRB 12300, München, HRB 6684; WEEE-Reg.-Nr. DE 23691322


-----Ursprüngliche Nachricht-----
Von: praveenesh kumar
Gesendet: Dienstag, 24. Mai 2011 11:08
An: common-user@hadoop.apache.org
Betreff: How to compile HBase code ?

Hello guys,

In case any of you are working on HBASE, I just wrote a program by reading
some tutorials..
But no where its mentioned how to run codes on HBASE. In case anyone of you
has done some coding on HBASE , can you please tell me how to run it.

I am able to compile my code by adding hbase-core.jar and hadoop-core.jar in
classpath while compiling it.
But not able to figure out how to run it.

Whenever I am doing java ExampleClient ( which is my Hbase program), I am
getting the following error :

Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/hadoop/hbase/HBaseConfiguration
at ExampleClient.main(ExampleClient.java:20)
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.hbase.HBaseConfiguration
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
... 1 more
Thanks,
Praveenesh

I am simply using HBase API, not doing any Map-reduce work on it.

Following is the code I have written , simply creating the file on HBase:

import java.io.IOException;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.HColumnDescriptor;
import org.apache.hadoop.hbase.HTableDescriptor;
import org.apache.hadoop.hbase.client.Get;
import org.apache.hadoop.hbase.client.HBaseAdmin;
import org.apache.hadoop.hbase.client.HTable;
import org.apache.hadoop.hbase.client.Put;
import org.apache.hadoop.hbase.client.Result;
import org.apache.hadoop.hbase.client.ResultScanner;
import org.apache.hadoop.hbase.client.Scan;
import org.apache.hadoop.hbase.util.Bytes;

public class ExampleClient {
public static void main(String args []) throws IOException
{
HBaseConfiguration config = new HBaseConfiguration();

HBaseAdmin admin = new HBaseAdmin(config);
HTableDescriptor htd = new HTableDescriptor("test");
HColumnDescriptor hcd = new HColumnDescriptor("data");
htd.addFamily(hcd);
admin.createTable(htd);

byte [] tablename = htd.getName();
HTableDescriptor [] tables = admin.listTables();

if(tables.length !=1 && Bytes.equals(tablename, tables[0].getName()))
{
throw new IOException("Failed to create table");
}

HTable table = new HTable(config,tablename);
byte[] row1 = Bytes.toBytes("row1");
Put p1 = new Put(row1);
byte[] databytes = Bytes.toBytes("data");
p1.add(databytes,Bytes.toBytes("1"),Bytes.toBytes("value1"));
table.put(p1);

Get g = new Get(row1);
Result result = table.get(g);
System.out.println("Get : "+ result);
Scan scan = new Scan();
ResultScanner scanner = table.getScanner(scan);
try
{
for(Result scannerResult: scanner)
{
System.out.println("Scan : " + scannerResult);
}
}catch(Exception e ){
e.printStackTrace();
}
finally{
scanner.close();
}
table.close();
}
}

Now I have set the classpath variable in /etc/environment as
MYCLASSPATH="/usr/local/hadoop/hadoop/hadoop-0.20.2-core.jar:/usr/local/hadoop/hbase/hbase/hbase-0.20.6.jar:/usr/local/hadoop/hbase/hbase/lib/zookeeper-3.2.2.jar

now I am compiling my code with javac command

*$javac -classpath $MYCLASSPATH ExampleClient.java*

It is working fine.
While running, I am using java command

*$java -classpath $MYCLASSPATH ExampleClient*, then I am getting the
following error :
Exception in thread "main" java.lang.NoClassDefFoundError: ExampleClient
Caused by: java.lang.ClassNotFoundException: ExampleClient
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
Could not find the main class: ExampleClient. Program will exit.
But I am running the code from the same location. and ExampleClient.class
file exists at that location.

Are you sure that the directory where your ExampleClient.class is locates is part of the MYCLASSPATH?

regards
Christian

---------------8<--------------------------------

Siemens AG
Corporate Technology
Corporate Research and Technologies
CT T DE IT3
Otto-Hahn-Ring 6
81739 München, Deutschland
Tel.: +49 (89) 636-42722
Fax: +49 (89) 636-41423
mailto:christian.kleegrewe@siemens.com

Siemens Aktiengesellschaft: Vorsitzender des Aufsichtsrats: Gerhard Cromme; Vorstand: Peter Löscher, Vorsitzender; Wolfgang Dehen, Brigitte Ederer, Joe Kaeser, Barbara Kux, Hermann Requardt, Siegfried Russwurm, Peter Y. Solmssen; Sitz der Gesellschaft: Berlin und München, Deutschland; Registergericht: Berlin Charlottenburg, HRB 12300, München, HRB 6684; WEEE-Reg.-Nr. DE 23691322


-----Ursprüngliche Nachricht-----
Von: praveenesh kumar
Gesendet: Dienstag, 24. Mai 2011 11:54
An: common-user@hadoop.apache.org
Betreff: Re: How to compile HBase code ?

I am simply using HBase API, not doing any Map-reduce work on it.

Following is the code I have written , simply creating the file on HBase:

import java.io.IOException;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.HColumnDescriptor;
import org.apache.hadoop.hbase.HTableDescriptor;
import org.apache.hadoop.hbase.client.Get;
import org.apache.hadoop.hbase.client.HBaseAdmin;
import org.apache.hadoop.hbase.client.HTable;
import org.apache.hadoop.hbase.client.Put;
import org.apache.hadoop.hbase.client.Result;
import org.apache.hadoop.hbase.client.ResultScanner;
import org.apache.hadoop.hbase.client.Scan;
import org.apache.hadoop.hbase.util.Bytes;

public class ExampleClient {
public static void main(String args []) throws IOException
{
HBaseConfiguration config = new HBaseConfiguration();

HBaseAdmin admin = new HBaseAdmin(config);
HTableDescriptor htd = new HTableDescriptor("test");
HColumnDescriptor hcd = new HColumnDescriptor("data");
htd.addFamily(hcd);
admin.createTable(htd);

byte [] tablename = htd.getName();
HTableDescriptor [] tables = admin.listTables();

if(tables.length !=1 && Bytes.equals(tablename, tables[0].getName()))
{
throw new IOException("Failed to create table");
}

HTable table = new HTable(config,tablename);
byte[] row1 = Bytes.toBytes("row1");
Put p1 = new Put(row1);
byte[] databytes = Bytes.toBytes("data");
p1.add(databytes,Bytes.toBytes("1"),Bytes.toBytes("value1"));
table.put(p1);

Get g = new Get(row1);
Result result = table.get(g);
System.out.println("Get : "+ result);
Scan scan = new Scan();
ResultScanner scanner = table.getScanner(scan);
try
{
for(Result scannerResult: scanner)
{
System.out.println("Scan : " + scannerResult);
}
}catch(Exception e ){
e.printStackTrace();
}
finally{
scanner.close();
}
table.close();
}
}

Now I have set the classpath variable in /etc/environment as
MYCLASSPATH="/usr/local/hadoop/hadoop/hadoop-0.20.2-core.jar:/usr/local/hadoop/hbase/hbase/hbase-0.20.6.jar:/usr/local/hadoop/hbase/hbase/lib/zookeeper-3.2.2.jar

now I am compiling my code with javac command

*$javac -classpath $MYCLASSPATH ExampleClient.java*

It is working fine.
While running, I am using java command

*$java -classpath $MYCLASSPATH ExampleClient*, then I am getting the
following error :
Exception in thread "main" java.lang.NoClassDefFoundError: ExampleClient
Caused by: java.lang.ClassNotFoundException: ExampleClient
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
Could not find the main class: ExampleClient. Program will exit.
But I am running the code from the same location. and ExampleClient.class
file exists at that location.

Praveenesh,

HBase has their own user mailing lists where such queries ought to go.
Am moving the discussion to user@hbase.apache.org and bcc-ing
common-user@ here. Also added you to cc.

Regarding your first error, going forward you can use the useful
`hbase classpath` to generate a HBase-provided classpath list for you
automatically. Something like:

$ MYCLASSPATH=`hbase classpath`

Regarding the second, latest one as below, your ExampleClient.class
isn't on the MYCLASSPATH (nor is the directory it is under, i.e. '.')
so Java can't really find it. This is not a HBase issue.

HTH.

Hey Harsh,

Actually I mailed to HBase mailing list also.. but since I wanted to get
this thing done as soon as possible so I mailed in this group also..
anyways I will take care of this in future , although I got more responses
in this mailing list only :-)

Anyways problem is solved..

What i did is added the folder containing my .class file in the classpath,
along with commons-logging-1.0.4.jar and log4j-1.2.15.jar in my classpath:

so now Myclasspath variable looks like :


*
MYCLASSPATH="/usr/local/hadoop/hadoop/hadoop-0.20.2-core.jar:/usr/local/hadoop/hbase/hbase/hbase-0.20.6.jar:/usr/local/hadoop/hbase/hbase/lib/zookeeper-3.2..2.jar::/usr/local/hadoop/hbase/hbase/lib/commons-logging-1.0.4.jar:/usr/local/hadoop/hbase/hbase/lib/log4j-1.2.15.jar:/usr/local/hadoop/hbase/"
*
* *
and then I used* java -classpath $MYCLASSPATH ExampleClient.java*
now its running..


Thanks.!!!
Praveenesh

Praveenesh,
Good to know your problem resolved. You can also use the `bin/hbase
classpath` utility to generate HBase parts of the classpath
automatically in the future, instead of adding classes manually -
saves you time.

Hey harsh,

I tried that.. its not working.
I am using hbase 0.20.6.
there is no command like bin/hbase classpath :

hadoop@ub6:/usr/local/hadoop/hbase$ hbase
Usage: hbase <command>
where <command> is one of:
shell run the HBase shell
master run an HBase HMaster node
regionserver run an HBase HRegionServer node
rest run an HBase REST server
thrift run an HBase Thrift server
zookeeper run a Zookeeper server
migrate upgrade an hbase.rootdir
or
CLASSNAME run the class named CLASSNAME
Thanks,
Praveenesh

Praveenesh,

Ah yes it would not work on the older 0.20.x releases; The command
exists in the current HBase release.