Grokbase Groups Pig user July 2011
FAQ
Yes I see a few errors in JT logs :-
java.lang.NoClassDefFoundError: com/google/common/collect/Lists
ClassNotFoundException:
org.apache.hadoop.hbase.filter.WritableByteArrayComparable

I think it cannot find some dependent jars? How or where do I add these jars
so that pig can see them

On Fri, Jul 15, 2011 at 3:27 PM, Bill Graham wrote:

What do you see on the map and reduce tasks logs on the JT UI for that job?

This job is failing for some reason, so there should be some hint in the
task logs.

On Fri, Jul 15, 2011 at 2:31 PM, sulabh choudhury wrote:

Bill,

there no useful message in logs (pasted below).
I tried SET pig.usenewlogicalplan 'false' which did not help.
I am using pig-0.8.0-cdh3u0. I have tried both with and without 'hbase://'
prefix

2011-07-15 14:19:58,700 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 100% complete
2011-07-15 14:19:58,702 [main] ERROR
org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!
2011-07-15 14:19:58,703 [main] INFO
org.apache.pig.tools.pigstats.PigStats - Script Statistics:

HadoopVersion PigVersion UserId StartedAt FinishedAt Features
0.20.2-cdh3u0 0.8.0-cdh3u0 cxt 2011-07-15 14:18:11 2011-07-15 14:19:58
GROUP_BY,ORDER_BY

Some jobs have failed! Stop running all dependent jobs

Job Stats (time in seconds):
JobId Maps Reduces MaxMapTime MinMapTIme AvgMapTime MaxReduceTime
MinReduceTime AvgReduceTime Alias Feature Outputs
job_201106212025_0139 1 1 8 8 8 12 12 12 A,ct,grp GROUP_BY,COMBINER
job_201106212025_0140 1 1 3 3 3 12 12 12 sorted SAMPLER

Failed Jobs:
JobId Alias Feature Message Outputs
job_201106212025_0141 result,sorted ORDER_BY Message: Job failed! Error -
NA pig_test,

Input(s):
Successfully read 2583 records (330 bytes) from: "hbase://transaction"

Output(s):
Failed to produce result in "pig_test"

On Fri, Jul 15, 2011 at 1:16 PM, Bill Graham wrote:

What version of Pig are you using and what errors are you seeing?

There was PIG-1870 related to projections that might apply, but I can't
say
so for sure. If that's the case it should work if you disable the new
logical plan with -Dusenewloginalplan=false.

Also, you might try specifying pig_test as 'hbase://pig_test'. I recall
another JIRA about that as well.

On Fri, Jul 15, 2011 at 12:40 PM, sulabh choudhury <sulabhc@gmail.com
wrote:
I have been trying to Store data in HBase suing HbaseStorage class. While I
can store the original read data, it fails when I try to store the
processed
data.
Which means I might be messing up the datatypes somewhere.

My script below is :-

--REGISTER myudfs.jar
--A = load 'hbase://transaction' using
org.apache.pig.backend.hadoop.hbase.HBaseStorage('log:ref2',
'-loadKey') AS
(row:chararray, code:chararray) ;
--grp = group A by myudfs.Parser(code);
--ct = foreach grp generate group,COUNT(A.code) as count;

--sorted = order ct by count desc;
--result = foreach sorted generate $0 as row,(chararray)$1;
--store result into 'pig_test' USING
org.apache.pig.backend.hadoop.hbase.HBaseStorage('log:count');

The dump of "result" works but the store to Hbase fails.
WHen I try to store A it works fine.

Datatypes of A and result are :-
A: {row: chararray,code: chararray}
result: {row: chararray,count: chararray}




--

--
Thanks and Regards,
Sulabh Choudhury

Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 5 of 14 | next ›
Discussion Overview
groupuser @
categoriespig, hadoop
postedJul 15, '11 at 7:41p
activeJul 17, '11 at 4:25a
posts14
users5
websitepig.apache.org

People

Translate

site design / logo © 2021 Grokbase