Grokbase Groups Pig user June 2011
FAQ
Hi,

I wanted to profile the jobs being created by my pig-script. As per the
hadoop documentation, I can use hprof and set suitable parameters.

So I added the following lines to my pig-script:
set mapred.task.profile true
set mapred.task.profile.params
-agentlib:hprof=heap=sites,file=output_hprof.txt

I do see that the first three mappers are significantly slower after the
addition of these two lines, implying that the profiler is working. I,
however, do not see any output in the directory I ran pig from. Does anyone
know the location of profiler output? Has anyone used hprof with pig to
profile the mappers before?

Thanks,
Shubham.

Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriespig, hadoop
postedJun 27, '11 at 5:44p
activeJun 27, '11 at 5:44p
posts1
users1
websitepig.apache.org

1 user in discussion

Shubham Chopra: 1 post

People

Translate

site design / logo © 2021 Grokbase