Grokbase Groups Pig user March 2009
FAQ
Yiping,

Thank-you, that worked! ( ... the register command needs the local path and not the hdfs path. )


Any thoughts on what to do when confronted with an "Unable to open iterator" error ?

Also, the illustrate command rarely works for me, any ideas there?
Here is a typical error I get from the illustrate command:

��0e����&e������`Xa����������/~����>1�D��K���9�9��H���@zUbï¿N�"1F���qm���n/�O4���3�M t(g���pud9=C��)����8==��j�aÙ¢ ��4��!�C�O�Ü�l׳Ð���%, G}{]��0����97��=������H ��V�1�c F����t;�e~�B×Ӭ�]L�14~�s�I6r�렿0od�G�c��D�\�"w��k����&q�g��f�h����M�X�E�Q�O��d�I�I��8 ��oL2~�R�)4�&�\.���-DY{Í���x×22�ï¿ï¿½ï¿½m=�F̼F�r������'D�ï¿ï¿½ï¿½(Tuple.java:176)
at org.apache.pig.pen.DisplayExamples.MakeArray(DisplayExamples.java:127)
at org.apache.pig.pen.DisplayExamples.PrintTabular(DisplayExamples.java:60)
at org.apache.pig.pen.DisplayExamples.PrintTabular(DisplayExamples.java:47)
at org.apache.pig.PigServer.showExamples(PigServer.java:570)
at org.apache.pig.PigServer.showExamples(PigServer.java:565)
at org.apache.pig.tools.grunt.GruntParser.processIllustrate(GruntParser.java:150)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:172)
at org.apache.pig.tools.grunt.GruntParser.parseContOnError(GruntParser.java:91)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:54)
at org.apache.pig.Main.main(Main.java:270)


Thanks for your help,
Avram




-----Original Message-----
From: Yiping Han
Sent: Tuesday, March 17, 2009 2:33 PM
To: pig-user@hadoop.apache.org
Subject: Re: How are you using Pig?

Avram,


You mentioned you use copyFromLocal to copy the jar file onto HDFS. What
grunt shows you is your HDFS home directory. You should have the file on
local file system (could be the same directory where your .pig script stays,
for example).


--Yiping

On 3/17/09 2:28 PM, "Avram Aelony" wrote:


Yiping,

That is where the tutorial.jar is.

grunt> ls tutorial.jar
tutorial.jar<r 1> 7400

-A



-----Original Message-----
From: Yiping Han
Sent: Tuesday, March 17, 2009 2:25 PM
To: pig-user@hadoop.apache.org
Subject: Re: How are you using Pig?

Avram,

Your tutorial.jar should be put on local filesystem on the node where you
launch your pig job. Pig will pack it and deliver it to every tasktracker.

--Yiping

On 3/17/09 2:15 PM, "Avram Aelony" wrote:

Hello Pig list,

I wish I could count myself among "pig users" but at this point my experience
is that small test files work well, but pig will generally complain when
presented with real data. When pig complains, it is difficult to tell why.

The most dreaded error that I do not understand is the "Unable to open
iterator" error where pig will run for a while then fail in the reducer
steps.
Can you recommend steps on how to determine the cause for this error and how
to fix?


Another problem I run into is that sometimes I can register UDF jar files,
and
at other times Pig refuses to read the jar file. I don't believe the jar
file
to be at fault. To illustrate this point, below I try to register the
tutorial.jar file that is distributed with pig.

grunt> register tutorial.jar
2009-03-17 13:57:53,430 [main] ERROR org.apache.pig.tools.grunt.GruntParser -
java.io.IOException: Can't read jar file: tutorial.jar
at org.apache.pig.PigServer.registerJar(PigServer.java:242)
at
org.apache.pig.tools.grunt.GruntParser.processRegister(GruntParser.java:158)
at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.j>>
a
va:246)
at
org.apache.pig.tools.grunt.GruntParser.parseContOnError(GruntParser.java:91)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:54)
at org.apache.pig.Main.main(Main.java:270)

2009-03-17 13:57:53,430 [main] ERROR org.apache.pig.tools.grunt.GruntParser -
java.io.IOException: Can't read jar file: tutorial.jar

Note that the tutorial.jar file is readily visible via ls and the file was
put
into hdfs via copyFromLocal. I have tried permissions of 777 and 775 on the
jar file and it does not resolve the issue.

My feeling is that until pig becomes more user-friendly, fewer people will
use
it.

Thanks for your help.

Avram
----------------------
Yiping Han
2MC 8127
2811 Mission College Blvd.,
Santa Clara, CA 95054
(408)349-4403
yhan@yahoo-inc.com
----------------------
Yiping Han
2MC 8127
2811 Mission College Blvd.,
Santa Clara, CA 95054
(408)349-4403
yhan@yahoo-inc.com

Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

People

Translate

site design / logo © 2021 Grokbase