Grokbase Groups Pig user October 2010
FAQ
Hi,

Couple of Questions:

1. What's the best way to get "Counters" out of a Pig Job execution ? (e.g. Counters object exposed by "org.apache.mapreduce.hadoop.Job")

2. Is there anything special needs to be done to have PigStats taken (i.e. any property etc ) ? I get empty PigStats when I run my job in local mode.

Thanks,
-Rakesh

Search Discussions

  • Dmitriy Ryaboy at Oct 8, 2010 at 3:10 am
    What version of pig are you using?
    On Thu, Oct 7, 2010 at 4:48 PM, rakesh kothari wrote:


    Hi,

    Couple of Questions:

    1. What's the best way to get "Counters" out of a Pig Job execution ? (e.g.
    Counters object exposed by "org.apache.mapreduce.hadoop.Job")

    2. Is there anything special needs to be done to have PigStats taken (i.e.
    any property etc ) ? I get empty PigStats when I run my job in local mode.

    Thanks,
    -Rakesh
  • Ashutosh Chauhan at Oct 8, 2010 at 4:10 pm
    Assuming you are either on trunk / 0.8 branch. Way to get counters is
    through PigStats. But, in local mode Hadoop doesnt provide any
    counters to Pig, so they arent available. You will get all the
    counters in map-reduce mode via PigStats.

    Hope it helps,
    Ashutosh
    On Thu, Oct 7, 2010 at 20:10, Dmitriy Ryaboy wrote:
    What version of pig are you using?
    On Thu, Oct 7, 2010 at 4:48 PM, rakesh kothari wrote:


    Hi,

    Couple of Questions:

    1. What's the best way to get "Counters" out of a Pig Job execution ? (e.g.
    Counters object exposed by "org.apache.mapreduce.hadoop.Job")

    2. Is there anything special needs to be done to have PigStats taken (i.e.
    any property etc ) ? I get empty PigStats when I run my job in local mode.

    Thanks,
    -Rakesh
  • Rakesh kothari at Oct 8, 2010 at 8:26 pm
    I am using PIG 0.7 and I am able to get counters from PigStats when running the job in mapreduce mode.

    Thanks,
    -Rakesh
    From: hashutosh@apache.org
    Date: Thu, 7 Oct 2010 20:34:08 -0700
    Subject: Re: Counters from a PIG Job
    To: user@pig.apache.org
    CC: pig-user@hadoop.apache.org

    Assuming you are either on trunk / 0.8 branch. Way to get counters is
    through PigStats. But, in local mode Hadoop doesnt provide any
    counters to Pig, so they arent available. You will get all the
    counters in map-reduce mode via PigStats.

    Hope it helps,
    Ashutosh
    On Thu, Oct 7, 2010 at 20:10, Dmitriy Ryaboy wrote:
    What version of pig are you using?
    On Thu, Oct 7, 2010 at 4:48 PM, rakesh kothari wrote:


    Hi,

    Couple of Questions:

    1. What's the best way to get "Counters" out of a Pig Job execution ? (e.g.
    Counters object exposed by "org.apache.mapreduce.hadoop.Job")

    2. Is there anything special needs to be done to have PigStats taken (i.e.
    any property etc ) ? I get empty PigStats when I run my job in local mode.

    Thanks,
    -Rakesh

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriespig, hadoop
postedOct 7, '10 at 11:48p
activeOct 8, '10 at 8:26p
posts4
users3
websitepig.apache.org

People

Translate

site design / logo © 2021 Grokbase