Grokbase Groups Pig user March 2011
FAQ
Hi,

I am trying to extract data stored in HBase using Pig.
I got Pig to work on Hadoop and then also ran a sample M/R job and got
correct results.
Now when I try to access HBase using the HbaseStorage the compiler just
hangs after
2011-03-14 17:38:23,981 [Thread-4-SendThread] INFO
org.apache.zookeeper.ClientCnxn - Server connection successful
Even after waiting for several minutes it just remains there. There is
no information in the log files too.

Please help!

Search Discussions

  • Bill Graham at Mar 15, 2011 at 2:30 am
    What version of Pig and what version of HBase? If HBase >= 0.89.0
    you'll need PIG-1680:

    https://issues.apache.org/jira/browse/PIG-1680
    On Mon, Mar 14, 2011 at 5:52 PM, SULABH wrote:
    Hi,

    I am trying to extract data stored in HBase using Pig.
    I got Pig to work on Hadoop and then also ran a sample M/R job and got
    correct results.
    Now when I try to access HBase using the HbaseStorage the compiler just
    hangs after
    2011-03-14 17:38:23,981 [Thread-4-SendThread] INFO
    org.apache.zookeeper.ClientCnxn - Server connection successful
    Even after waiting for several minutes it just remains there. There is no
    information in the log files too.

    Please help!
  • Sulabh choudhury at Mar 15, 2011 at 4:06 am
    Thanks Bill for the response.
    Just to be sure for HBase >=0.89 only Pig-1680 would work and no other
    version, even 0.70 ?
    On Mon, Mar 14, 2011 at 7:29 PM, Bill Graham wrote:

    What version of Pig and what version of HBase? If HBase >= 0.89.0
    you'll need PIG-1680:

    https://issues.apache.org/jira/browse/PIG-1680
    On Mon, Mar 14, 2011 at 5:52 PM, SULABH wrote:
    Hi,

    I am trying to extract data stored in HBase using Pig.
    I got Pig to work on Hadoop and then also ran a sample M/R job and got
    correct results.
    Now when I try to access HBase using the HbaseStorage the compiler just
    hangs after
    2011-03-14 17:38:23,981 [Thread-4-SendThread] INFO
    org.apache.zookeeper.ClientCnxn - Server connection successful
    Even after waiting for several minutes it just remains there. There is no
    information in the log files too.

    Please help!
  • Bill Graham at Mar 15, 2011 at 6:25 am
    What I'm saying is that if your version of HBase is 0.89.0 or greater,
    you need to apply the PIG-1680 patch to your pig distro (Pig 0.8.0)
    and rebuild.

    I'm not sure what you mean by 0.70, but if you mean't Pig 0.7.0, you
    can try applying the patch to that version, but I'd expect you'd run
    into a lot of conflicts. You'd be better off to just upgrade Pig to
    0.8.0 and apply the patch, or just build Pig from the trunk.
    On Mon, Mar 14, 2011 at 9:05 PM, sulabh choudhury wrote:
    Thanks Bill for the response.
    Just to be sure for HBase >=0.89 only Pig-1680 would work and no other
    version, even 0.70 ?
    On Mon, Mar 14, 2011 at 7:29 PM, Bill Graham wrote:

    What version of Pig and what version of HBase? If HBase >= 0.89.0
    you'll need PIG-1680:

    https://issues.apache.org/jira/browse/PIG-1680
    On Mon, Mar 14, 2011 at 5:52 PM, SULABH wrote:
    Hi,

    I am trying to extract data stored in HBase using Pig.
    I got Pig to work on Hadoop and then also ran a sample M/R job and got
    correct results.
    Now when I try to access HBase using the HbaseStorage the compiler just
    hangs after
    2011-03-14 17:38:23,981 [Thread-4-SendThread] INFO
    org.apache.zookeeper.ClientCnxn - Server connection successful
    Even after waiting for several minutes it just remains there. There is
    no
    information in the log files too.

    Please help!


  • Dmitriy Ryaboy at Mar 15, 2011 at 8:16 am
    Just hanging doesn't sound like a 20.6 vs 89 behavior; that crashes out
    pretty fast.
    Sulabh, are you sure that your hbase conf is on the Pig path?

    D
    On Mon, Mar 14, 2011 at 11:24 PM, Bill Graham wrote:

    What I'm saying is that if your version of HBase is 0.89.0 or greater,
    you need to apply the PIG-1680 patch to your pig distro (Pig 0.8.0)
    and rebuild.

    I'm not sure what you mean by 0.70, but if you mean't Pig 0.7.0, you
    can try applying the patch to that version, but I'd expect you'd run
    into a lot of conflicts. You'd be better off to just upgrade Pig to
    0.8.0 and apply the patch, or just build Pig from the trunk.
    On Mon, Mar 14, 2011 at 9:05 PM, sulabh choudhury wrote:
    Thanks Bill for the response.
    Just to be sure for HBase >=0.89 only Pig-1680 would work and no other
    version, even 0.70 ?
    On Mon, Mar 14, 2011 at 7:29 PM, Bill Graham wrote:

    What version of Pig and what version of HBase? If HBase >= 0.89.0
    you'll need PIG-1680:

    https://issues.apache.org/jira/browse/PIG-1680
    On Mon, Mar 14, 2011 at 5:52 PM, SULABH wrote:
    Hi,

    I am trying to extract data stored in HBase using Pig.
    I got Pig to work on Hadoop and then also ran a sample M/R job and got
    correct results.
    Now when I try to access HBase using the HbaseStorage the compiler
    just
    hangs after
    2011-03-14 17:38:23,981 [Thread-4-SendThread] INFO
    org.apache.zookeeper.ClientCnxn - Server connection successful
    Even after waiting for several minutes it just remains there. There is
    no
    information in the log files too.

    Please help!


  • Sulabh choudhury at Mar 15, 2011 at 4:56 pm
    I have Pig 0.8.0 installed and Hbase 0.90.1 both on Cloudera (CDH3B4)
    distribution.
    Yes it does not crash it just hangs after the
    "org.apache.zookeeper.ClientCnxn - Server connection successful"

    When I apply the latest patch
    (pig_1680.final.patch<https://issues.apache.org/jira/secure/attachment/12472652/pig_1680.final.patch>)
    all the Hunks failed...

    Also I see that
    Thread-4-SendThread] INFO org.apache.zookeeper.ClientCnxn - Attempting
    connection to server localhost/127.0.0.1:2181
    So that means that it is trying to connect to the correct (2181) zookeeper
    port...





    On Tue, Mar 15, 2011 at 1:15 AM, Dmitriy Ryaboy wrote:

    Just hanging doesn't sound like a 20.6 vs 89 behavior; that crashes out
    pretty fast.
    Sulabh, are you sure that your hbase conf is on the Pig path?

    D

    On Mon, Mar 14, 2011 at 11:24 PM, Bill Graham wrote:

    What I'm saying is that if your version of HBase is 0.89.0 or greater,
    you need to apply the PIG-1680 patch to your pig distro (Pig 0.8.0)
    and rebuild.

    I'm not sure what you mean by 0.70, but if you mean't Pig 0.7.0, you
    can try applying the patch to that version, but I'd expect you'd run
    into a lot of conflicts. You'd be better off to just upgrade Pig to
    0.8.0 and apply the patch, or just build Pig from the trunk.

    On Mon, Mar 14, 2011 at 9:05 PM, sulabh choudhury <sulabhc@gmail.com>
    wrote:
    Thanks Bill for the response.
    Just to be sure for HBase >=0.89 only Pig-1680 would work and no other
    version, even 0.70 ?

    On Mon, Mar 14, 2011 at 7:29 PM, Bill Graham <billgraham@gmail.com>
    wrote:
    What version of Pig and what version of HBase? If HBase >= 0.89.0
    you'll need PIG-1680:

    https://issues.apache.org/jira/browse/PIG-1680
    On Mon, Mar 14, 2011 at 5:52 PM, SULABH wrote:
    Hi,

    I am trying to extract data stored in HBase using Pig.
    I got Pig to work on Hadoop and then also ran a sample M/R job and
    got
    correct results.
    Now when I try to access HBase using the HbaseStorage the compiler
    just
    hangs after
    2011-03-14 17:38:23,981 [Thread-4-SendThread] INFO
    org.apache.zookeeper.ClientCnxn - Server connection successful
    Even after waiting for several minutes it just remains there. There
    is
    no
    information in the log files too.

    Please help!



    --

    --
    Thanks and Regards,
    Sulabh Choudhury
  • Dmitriy Ryaboy at Mar 15, 2011 at 5:19 pm
    The cloudera Pig 0.8 does not work with HBase 0.90 if I recall correctly.
    You can compile a Pig jar from the pig-0.8 branch, and it should work with
    90+

    D
    On Tue, Mar 15, 2011 at 9:55 AM, sulabh choudhury wrote:

    I have Pig 0.8.0 installed and Hbase 0.90.1 both on Cloudera (CDH3B4)
    distribution.
    Yes it does not crash it just hangs after the
    "org.apache.zookeeper.ClientCnxn - Server connection successful"

    When I apply the latest patch (pig_1680.final.patch<https://issues.apache.org/jira/secure/attachment/12472652/pig_1680.final.patch>)
    all the Hunks failed...

    Also I see that
    Thread-4-SendThread] INFO org.apache.zookeeper.ClientCnxn - Attempting
    connection to server localhost/127.0.0.1:2181
    So that means that it is trying to connect to the correct (2181) zookeeper
    port...





    On Tue, Mar 15, 2011 at 1:15 AM, Dmitriy Ryaboy wrote:

    Just hanging doesn't sound like a 20.6 vs 89 behavior; that crashes out
    pretty fast.
    Sulabh, are you sure that your hbase conf is on the Pig path?

    D

    On Mon, Mar 14, 2011 at 11:24 PM, Bill Graham wrote:

    What I'm saying is that if your version of HBase is 0.89.0 or greater,
    you need to apply the PIG-1680 patch to your pig distro (Pig 0.8.0)
    and rebuild.

    I'm not sure what you mean by 0.70, but if you mean't Pig 0.7.0, you
    can try applying the patch to that version, but I'd expect you'd run
    into a lot of conflicts. You'd be better off to just upgrade Pig to
    0.8.0 and apply the patch, or just build Pig from the trunk.

    On Mon, Mar 14, 2011 at 9:05 PM, sulabh choudhury <sulabhc@gmail.com>
    wrote:
    Thanks Bill for the response.
    Just to be sure for HBase >=0.89 only Pig-1680 would work and no other
    version, even 0.70 ?

    On Mon, Mar 14, 2011 at 7:29 PM, Bill Graham <billgraham@gmail.com>
    wrote:
    What version of Pig and what version of HBase? If HBase >= 0.89.0
    you'll need PIG-1680:

    https://issues.apache.org/jira/browse/PIG-1680
    On Mon, Mar 14, 2011 at 5:52 PM, SULABH wrote:
    Hi,

    I am trying to extract data stored in HBase using Pig.
    I got Pig to work on Hadoop and then also ran a sample M/R job and
    got
    correct results.
    Now when I try to access HBase using the HbaseStorage the compiler
    just
    hangs after
    2011-03-14 17:38:23,981 [Thread-4-SendThread] INFO
    org.apache.zookeeper.ClientCnxn - Server connection successful
    Even after waiting for several minutes it just remains there. There
    is
    no
    information in the log files too.

    Please help!



    --

    --
    Thanks and Regards,
    Sulabh Choudhury
  • Sulabh choudhury at Mar 15, 2011 at 5:58 pm
    I am a little confused here....
    Can you elaborate a little more on how to achieve the above ( Pig jar from
    the pig-0.8 branch) and also then how to use the same to make it work with
    Hbase 0.90 ?
    On Tue, Mar 15, 2011 at 10:18 AM, Dmitriy Ryaboy wrote:

    The cloudera Pig 0.8 does not work with HBase 0.90 if I recall correctly.
    You can compile a Pig jar from the pig-0.8 branch, and it should work with
    90+

    D

    On Tue, Mar 15, 2011 at 9:55 AM, sulabh choudhury wrote:

    I have Pig 0.8.0 installed and Hbase 0.90.1 both on Cloudera (CDH3B4)
    distribution.
    Yes it does not crash it just hangs after the
    "org.apache.zookeeper.ClientCnxn - Server connection successful"

    When I apply the latest patch (pig_1680.final.patch<https://issues.apache.org/jira/secure/attachment/12472652/pig_1680.final.patch>)
    all the Hunks failed...

    Also I see that
    Thread-4-SendThread] INFO org.apache.zookeeper.ClientCnxn - Attempting
    connection to server localhost/127.0.0.1:2181
    So that means that it is trying to connect to the correct (2181) zookeeper
    port...





    On Tue, Mar 15, 2011 at 1:15 AM, Dmitriy Ryaboy wrote:

    Just hanging doesn't sound like a 20.6 vs 89 behavior; that crashes out
    pretty fast.
    Sulabh, are you sure that your hbase conf is on the Pig path?

    D

    On Mon, Mar 14, 2011 at 11:24 PM, Bill Graham wrote:

    What I'm saying is that if your version of HBase is 0.89.0 or greater,
    you need to apply the PIG-1680 patch to your pig distro (Pig 0.8.0)
    and rebuild.

    I'm not sure what you mean by 0.70, but if you mean't Pig 0.7.0, you
    can try applying the patch to that version, but I'd expect you'd run
    into a lot of conflicts. You'd be better off to just upgrade Pig to
    0.8.0 and apply the patch, or just build Pig from the trunk.

    On Mon, Mar 14, 2011 at 9:05 PM, sulabh choudhury <sulabhc@gmail.com>
    wrote:
    Thanks Bill for the response.
    Just to be sure for HBase >=0.89 only Pig-1680 would work and no other
    version, even 0.70 ?

    On Mon, Mar 14, 2011 at 7:29 PM, Bill Graham <billgraham@gmail.com>
    wrote:
    What version of Pig and what version of HBase? If HBase >= 0.89.0
    you'll need PIG-1680:

    https://issues.apache.org/jira/browse/PIG-1680
    On Mon, Mar 14, 2011 at 5:52 PM, SULABH wrote:
    Hi,

    I am trying to extract data stored in HBase using Pig.
    I got Pig to work on Hadoop and then also ran a sample M/R job and
    got
    correct results.
    Now when I try to access HBase using the HbaseStorage the compiler
    just
    hangs after
    2011-03-14 17:38:23,981 [Thread-4-SendThread] INFO
    org.apache.zookeeper.ClientCnxn - Server connection successful
    Even after waiting for several minutes it just remains there. There
    is
    no
    information in the log files too.

    Please help!



    --

    --
    Thanks and Regards,
    Sulabh Choudhury

    --

    SC
  • Sulabh choudhury at Mar 15, 2011 at 6:27 pm
    Dmitriy...
    I just found out that after 20 mins I get throws errors

    2011-03-15 10:52:48,662 [Thread-4-SendThread] INFO
    org.apache.zookeeper.ClientCnxn - Server connection successful
    2011-03-15 11:16:39,186 [Thread-4] INFO org.apache.hadoop.mapred.JobClient
    - Cleaning up the staging area
    hdfs://db1/tmp/hadoop-mapred/mapred/staging/cxt/.staging/job_201103080952_0033
    2011-03-15 11:16:39,188 [main] INFO
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
    - job null has failed! Stop running all dependent jobs
    2011-03-15 11:16:39,194 [main] INFO
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
    - 100% complete
    2011-03-15 11:16:39,203 [main] ERROR org.apache.pig.tools.pigstats.PigStats
    - ERROR 2997: Unable to recreate exception from backend error:
    org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Timed out
    trying to locate root region
    2011-03-15 11:16:39,204 [main] ERROR
    org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!

    Are the previous errors related or I am making heads-up and start looking
    into different direction now ?
    On Tue, Mar 15, 2011 at 10:57 AM, sulabh choudhury wrote:

    I am a little confused here....
    Can you elaborate a little more on how to achieve the above ( Pig jar from
    the pig-0.8 branch) and also then how to use the same to make it work with
    Hbase 0.90 ?
    On Tue, Mar 15, 2011 at 10:18 AM, Dmitriy Ryaboy wrote:

    The cloudera Pig 0.8 does not work with HBase 0.90 if I recall correctly.
    You can compile a Pig jar from the pig-0.8 branch, and it should work with
    90+

    D

    On Tue, Mar 15, 2011 at 9:55 AM, sulabh choudhury wrote:

    I have Pig 0.8.0 installed and Hbase 0.90.1 both on Cloudera (CDH3B4)
    distribution.
    Yes it does not crash it just hangs after the
    "org.apache.zookeeper.ClientCnxn - Server connection successful"

    When I apply the latest patch (pig_1680.final.patch<https://issues.apache.org/jira/secure/attachment/12472652/pig_1680.final.patch>)
    all the Hunks failed...

    Also I see that
    Thread-4-SendThread] INFO org.apache.zookeeper.ClientCnxn - Attempting
    connection to server localhost/127.0.0.1:2181
    So that means that it is trying to connect to the correct (2181)
    zookeeper port...





    On Tue, Mar 15, 2011 at 1:15 AM, Dmitriy Ryaboy wrote:

    Just hanging doesn't sound like a 20.6 vs 89 behavior; that crashes out
    pretty fast.
    Sulabh, are you sure that your hbase conf is on the Pig path?

    D

    On Mon, Mar 14, 2011 at 11:24 PM, Bill Graham wrote:

    What I'm saying is that if your version of HBase is 0.89.0 or greater,
    you need to apply the PIG-1680 patch to your pig distro (Pig 0.8.0)
    and rebuild.

    I'm not sure what you mean by 0.70, but if you mean't Pig 0.7.0, you
    can try applying the patch to that version, but I'd expect you'd run
    into a lot of conflicts. You'd be better off to just upgrade Pig to
    0.8.0 and apply the patch, or just build Pig from the trunk.

    On Mon, Mar 14, 2011 at 9:05 PM, sulabh choudhury <sulabhc@gmail.com>
    wrote:
    Thanks Bill for the response.
    Just to be sure for HBase >=0.89 only Pig-1680 would work and no other
    version, even 0.70 ?

    On Mon, Mar 14, 2011 at 7:29 PM, Bill Graham <billgraham@gmail.com>
    wrote:
    What version of Pig and what version of HBase? If HBase >= 0.89.0
    you'll need PIG-1680:

    https://issues.apache.org/jira/browse/PIG-1680
    On Mon, Mar 14, 2011 at 5:52 PM, SULABH wrote:
    Hi,

    I am trying to extract data stored in HBase using Pig.
    I got Pig to work on Hadoop and then also ran a sample M/R job and
    got
    correct results.
    Now when I try to access HBase using the HbaseStorage the compiler
    just
    hangs after
    2011-03-14 17:38:23,981 [Thread-4-SendThread] INFO
    org.apache.zookeeper.ClientCnxn - Server connection successful
    Even after waiting for several minutes it just remains there.
    There is
    no
    information in the log files too.

    Please help!



    --

    --
    Thanks and Regards,
    Sulabh Choudhury

    --

    SC

    --

    SC
  • Dmitriy Ryaboy at Mar 15, 2011 at 10:11 pm
    Anything interesting in Map task logs?

    As for your previous question -- just check out the branch-0.8, run
    "ant jar-withouthadoop"
    and use the the resulting jar.
    On Tue, Mar 15, 2011 at 11:26 AM, sulabh choudhury wrote:

    Dmitriy...
    I just found out that after 20 mins I get throws errors

    2011-03-15 10:52:48,662 [Thread-4-SendThread] INFO
    org.apache.zookeeper.ClientCnxn - Server connection successful
    2011-03-15 11:16:39,186 [Thread-4] INFO org.apache.hadoop.mapred.JobClient
    - Cleaning up the staging area
    hdfs://db1/tmp/hadoop-mapred/mapred/staging/cxt/.staging/job_201103080952_0033
    2011-03-15 11:16:39,188 [main] INFO
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
    - job null has failed! Stop running all dependent jobs
    2011-03-15 11:16:39,194 [main] INFO
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
    - 100% complete
    2011-03-15 11:16:39,203 [main] ERROR org.apache.pig.tools.pigstats.PigStats
    - ERROR 2997: Unable to recreate exception from backend error:
    org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Timed out
    trying to locate root region
    2011-03-15 11:16:39,204 [main] ERROR
    org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!

    Are the previous errors related or I am making heads-up and start looking
    into different direction now ?
    On Tue, Mar 15, 2011 at 10:57 AM, sulabh choudhury wrote:

    I am a little confused here....
    Can you elaborate a little more on how to achieve the above ( Pig jar from
    the pig-0.8 branch) and also then how to use the same to make it work with
    Hbase 0.90 ?
    On Tue, Mar 15, 2011 at 10:18 AM, Dmitriy Ryaboy wrote:

    The cloudera Pig 0.8 does not work with HBase 0.90 if I recall correctly.
    You can compile a Pig jar from the pig-0.8 branch, and it should work
    with 90+

    D

    On Tue, Mar 15, 2011 at 9:55 AM, sulabh choudhury wrote:

    I have Pig 0.8.0 installed and Hbase 0.90.1 both on Cloudera (CDH3B4)
    distribution.
    Yes it does not crash it just hangs after the
    "org.apache.zookeeper.ClientCnxn - Server connection successful"

    When I apply the latest patch (pig_1680.final.patch<https://issues.apache.org/jira/secure/attachment/12472652/pig_1680.final.patch>)
    all the Hunks failed...

    Also I see that
    Thread-4-SendThread] INFO org.apache.zookeeper.ClientCnxn - Attempting
    connection to server localhost/127.0.0.1:2181
    So that means that it is trying to connect to the correct (2181)
    zookeeper port...





    On Tue, Mar 15, 2011 at 1:15 AM, Dmitriy Ryaboy wrote:

    Just hanging doesn't sound like a 20.6 vs 89 behavior; that crashes out
    pretty fast.
    Sulabh, are you sure that your hbase conf is on the Pig path?

    D

    On Mon, Mar 14, 2011 at 11:24 PM, Bill Graham wrote:

    What I'm saying is that if your version of HBase is 0.89.0 or greater,
    you need to apply the PIG-1680 patch to your pig distro (Pig 0.8.0)
    and rebuild.

    I'm not sure what you mean by 0.70, but if you mean't Pig 0.7.0, you
    can try applying the patch to that version, but I'd expect you'd run
    into a lot of conflicts. You'd be better off to just upgrade Pig to
    0.8.0 and apply the patch, or just build Pig from the trunk.

    On Mon, Mar 14, 2011 at 9:05 PM, sulabh choudhury <sulabhc@gmail.com>
    wrote:
    Thanks Bill for the response.
    Just to be sure for HBase >=0.89 only Pig-1680 would work and no other
    version, even 0.70 ?

    On Mon, Mar 14, 2011 at 7:29 PM, Bill Graham <billgraham@gmail.com>
    wrote:
    What version of Pig and what version of HBase? If HBase >= 0.89.0
    you'll need PIG-1680:

    https://issues.apache.org/jira/browse/PIG-1680
    On Mon, Mar 14, 2011 at 5:52 PM, SULABH wrote:
    Hi,

    I am trying to extract data stored in HBase using Pig.
    I got Pig to work on Hadoop and then also ran a sample M/R job
    and got
    correct results.
    Now when I try to access HBase using the HbaseStorage the
    compiler just
    hangs after
    2011-03-14 17:38:23,981 [Thread-4-SendThread] INFO
    org.apache.zookeeper.ClientCnxn - Server connection successful
    Even after waiting for several minutes it just remains there.
    There is
    no
    information in the log files too.

    Please help!



    --

    --
    Thanks and Regards,
    Sulabh Choudhury

    --

    SC

    --

    SC
  • Sulabh choudhury at Mar 15, 2011 at 11:15 pm
    The logs do not provide any useful info.

    I am trying to use the new jar from branch-0.8 now.

    Now it pops other errors
    Cannot find hadoop configurations in classpath (neither hadoop-site.xml nor
    core-site.xml was found in the classpath)
    but since I am using Cloudera dist which does not have hadoop-site.xml and
    core-site.xml instead it has mapred-site.xml, hdfs-site.xml,core-site.xml
    and I have already specified the path to my Hadoop conf folder.


    On Tue, Mar 15, 2011 at 3:11 PM, Dmitriy Ryaboy wrote:

    Anything interesting in Map task logs?

    As for your previous question -- just check out the branch-0.8, run "ant jar-withouthadoop"
    and use the the resulting jar.

    On Tue, Mar 15, 2011 at 11:26 AM, sulabh choudhury wrote:

    Dmitriy...
    I just found out that after 20 mins I get throws errors

    2011-03-15 10:52:48,662 [Thread-4-SendThread] INFO
    org.apache.zookeeper.ClientCnxn - Server connection successful
    2011-03-15 11:16:39,186 [Thread-4] INFO
    org.apache.hadoop.mapred.JobClient - Cleaning up the staging area
    hdfs://db1/tmp/hadoop-mapred/mapred/staging/cxt/.staging/job_201103080952_0033
    2011-03-15 11:16:39,188 [main] INFO
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
    - job null has failed! Stop running all dependent jobs
    2011-03-15 11:16:39,194 [main] INFO
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
    - 100% complete
    2011-03-15 11:16:39,203 [main] ERROR
    org.apache.pig.tools.pigstats.PigStats - ERROR 2997: Unable to recreate
    exception from backend error:
    org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Timed out
    trying to locate root region
    2011-03-15 11:16:39,204 [main] ERROR
    org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!

    Are the previous errors related or I am making heads-up and start looking
    into different direction now ?
    On Tue, Mar 15, 2011 at 10:57 AM, sulabh choudhury wrote:

    I am a little confused here....
    Can you elaborate a little more on how to achieve the above ( Pig jar
    from the pig-0.8 branch) and also then how to use the same to make it work
    with Hbase 0.90 ?
    On Tue, Mar 15, 2011 at 10:18 AM, Dmitriy Ryaboy wrote:

    The cloudera Pig 0.8 does not work with HBase 0.90 if I recall
    correctly.
    You can compile a Pig jar from the pig-0.8 branch, and it should work
    with 90+

    D

    On Tue, Mar 15, 2011 at 9:55 AM, sulabh choudhury wrote:

    I have Pig 0.8.0 installed and Hbase 0.90.1 both on Cloudera (CDH3B4)
    distribution.
    Yes it does not crash it just hangs after the
    "org.apache.zookeeper.ClientCnxn - Server connection successful"

    When I apply the latest patch (pig_1680.final.patch<https://issues.apache.org/jira/secure/attachment/12472652/pig_1680.final.patch>)
    all the Hunks failed...

    Also I see that
    Thread-4-SendThread] INFO org.apache.zookeeper.ClientCnxn - Attempting
    connection to server localhost/127.0.0.1:2181
    So that means that it is trying to connect to the correct (2181)
    zookeeper port...





    On Tue, Mar 15, 2011 at 1:15 AM, Dmitriy Ryaboy wrote:

    Just hanging doesn't sound like a 20.6 vs 89 behavior; that crashes
    out pretty fast.
    Sulabh, are you sure that your hbase conf is on the Pig path?

    D

    On Mon, Mar 14, 2011 at 11:24 PM, Bill Graham wrote:

    What I'm saying is that if your version of HBase is 0.89.0 or
    greater,
    you need to apply the PIG-1680 patch to your pig distro (Pig 0.8.0)
    and rebuild.

    I'm not sure what you mean by 0.70, but if you mean't Pig 0.7.0, you
    can try applying the patch to that version, but I'd expect you'd run
    into a lot of conflicts. You'd be better off to just upgrade Pig to
    0.8.0 and apply the patch, or just build Pig from the trunk.

    On Mon, Mar 14, 2011 at 9:05 PM, sulabh choudhury <sulabhc@gmail.com>
    wrote:
    Thanks Bill for the response.
    Just to be sure for HBase >=0.89 only Pig-1680 would work and no other
    version, even 0.70 ?

    On Mon, Mar 14, 2011 at 7:29 PM, Bill Graham <billgraham@gmail.com>
    wrote:
    What version of Pig and what version of HBase? If HBase >= 0.89.0
    you'll need PIG-1680:

    https://issues.apache.org/jira/browse/PIG-1680

    On Mon, Mar 14, 2011 at 5:52 PM, SULABH <sulabhc@gmail.com>
    wrote:
    Hi,

    I am trying to extract data stored in HBase using Pig.
    I got Pig to work on Hadoop and then also ran a sample M/R job
    and got
    correct results.
    Now when I try to access HBase using the HbaseStorage the
    compiler just
    hangs after
    2011-03-14 17:38:23,981 [Thread-4-SendThread] INFO
    org.apache.zookeeper.ClientCnxn - Server connection successful
    Even after waiting for several minutes it just remains there.
    There is
    no
    information in the log files too.

    Please help!



    --

    --
    Thanks and Regards,
    Sulabh Choudhury

    --

    SC

    --

    SC

    --
  • Dmitriy Ryaboy at Mar 15, 2011 at 11:39 pm
    I don't think the hadoop conf directory is on the pig classpath.
    You need to make sure that the following things are on the classpath (the
    easiest way to do that is to set PIG_CLASSPATH env var):
    the hadoop conf directory
    all the jars in the hadoop lib directory
    guava jar
    hbase jar
    zookeeper jar

    -D

    On Tue, Mar 15, 2011 at 4:13 PM, sulabh choudhury wrote:

    The logs do not provide any useful info.

    I am trying to use the new jar from branch-0.8 now.

    Now it pops other errors
    Cannot find hadoop configurations in classpath (neither hadoop-site.xml nor
    core-site.xml was found in the classpath)
    but since I am using Cloudera dist which does not have hadoop-site.xml and
    core-site.xml instead it has mapred-site.xml, hdfs-site.xml,core-site.xml
    and I have already specified the path to my Hadoop conf folder.


    On Tue, Mar 15, 2011 at 3:11 PM, Dmitriy Ryaboy wrote:

    Anything interesting in Map task logs?

    As for your previous question -- just check out the branch-0.8, run "ant jar-withouthadoop"
    and use the the resulting jar.

    On Tue, Mar 15, 2011 at 11:26 AM, sulabh choudhury wrote:

    Dmitriy...
    I just found out that after 20 mins I get throws errors

    2011-03-15 10:52:48,662 [Thread-4-SendThread] INFO
    org.apache.zookeeper.ClientCnxn - Server connection successful
    2011-03-15 11:16:39,186 [Thread-4] INFO
    org.apache.hadoop.mapred.JobClient - Cleaning up the staging area
    hdfs://db1/tmp/hadoop-mapred/mapred/staging/cxt/.staging/job_201103080952_0033
    2011-03-15 11:16:39,188 [main] INFO
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
    - job null has failed! Stop running all dependent jobs
    2011-03-15 11:16:39,194 [main] INFO
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
    - 100% complete
    2011-03-15 11:16:39,203 [main] ERROR
    org.apache.pig.tools.pigstats.PigStats - ERROR 2997: Unable to recreate
    exception from backend error:
    org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Timed out
    trying to locate root region
    2011-03-15 11:16:39,204 [main] ERROR
    org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!

    Are the previous errors related or I am making heads-up and start looking
    into different direction now ?
    On Tue, Mar 15, 2011 at 10:57 AM, sulabh choudhury wrote:

    I am a little confused here....
    Can you elaborate a little more on how to achieve the above ( Pig jar
    from the pig-0.8 branch) and also then how to use the same to make it work
    with Hbase 0.90 ?
    On Tue, Mar 15, 2011 at 10:18 AM, Dmitriy Ryaboy wrote:

    The cloudera Pig 0.8 does not work with HBase 0.90 if I recall
    correctly.
    You can compile a Pig jar from the pig-0.8 branch, and it should work
    with 90+

    D

    On Tue, Mar 15, 2011 at 9:55 AM, sulabh choudhury wrote:

    I have Pig 0.8.0 installed and Hbase 0.90.1 both on Cloudera (CDH3B4)
    distribution.
    Yes it does not crash it just hangs after the
    "org.apache.zookeeper.ClientCnxn - Server connection successful"

    When I apply the latest patch (pig_1680.final.patch<https://issues.apache.org/jira/secure/attachment/12472652/pig_1680.final.patch>)
    all the Hunks failed...

    Also I see that
    Thread-4-SendThread] INFO org.apache.zookeeper.ClientCnxn -
    Attempting connection to server localhost/127.0.0.1:2181
    So that means that it is trying to connect to the correct (2181)
    zookeeper port...





    On Tue, Mar 15, 2011 at 1:15 AM, Dmitriy Ryaboy wrote:

    Just hanging doesn't sound like a 20.6 vs 89 behavior; that crashes
    out pretty fast.
    Sulabh, are you sure that your hbase conf is on the Pig path?

    D

    On Mon, Mar 14, 2011 at 11:24 PM, Bill Graham wrote:

    What I'm saying is that if your version of HBase is 0.89.0 or
    greater,
    you need to apply the PIG-1680 patch to your pig distro (Pig 0.8.0)
    and rebuild.

    I'm not sure what you mean by 0.70, but if you mean't Pig 0.7.0, you
    can try applying the patch to that version, but I'd expect you'd run
    into a lot of conflicts. You'd be better off to just upgrade Pig to
    0.8.0 and apply the patch, or just build Pig from the trunk.

    On Mon, Mar 14, 2011 at 9:05 PM, sulabh choudhury <
    sulabhc@gmail.com> wrote:
    Thanks Bill for the response.
    Just to be sure for HBase >=0.89 only Pig-1680 would work and no other
    version, even 0.70 ?

    On Mon, Mar 14, 2011 at 7:29 PM, Bill Graham <
    billgraham@gmail.com> wrote:
    What version of Pig and what version of HBase? If HBase >= 0.89.0
    you'll need PIG-1680:

    https://issues.apache.org/jira/browse/PIG-1680

    On Mon, Mar 14, 2011 at 5:52 PM, SULABH <sulabhc@gmail.com>
    wrote:
    Hi,

    I am trying to extract data stored in HBase using Pig.
    I got Pig to work on Hadoop and then also ran a sample M/R job
    and got
    correct results.
    Now when I try to access HBase using the HbaseStorage the
    compiler just
    hangs after
    2011-03-14 17:38:23,981 [Thread-4-SendThread] INFO
    org.apache.zookeeper.ClientCnxn - Server connection successful
    Even after waiting for several minutes it just remains there.
    There is
    no
    information in the log files too.

    Please help!



    --

    --
    Thanks and Regards,
    Sulabh Choudhury

    --

    SC

    --

    SC

    --

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriespig, hadoop
postedMar 15, '11 at 12:53a
activeMar 15, '11 at 11:39p
posts12
users3
websitepig.apache.org

People

Translate

site design / logo © 2022 Grokbase