FAQ
Most likely you, when build PIG, were using the older version 1.0.3 of log4j logger. See https://jira.jboss.org/browse/JBAS-1781. The root cause might be due to https://issues.apache.org/jira/browse/PIG-1582. So the first thing you probably need to do is upgrade your local copy of PIG trunk.

Yan

-----Original Message-----
From: Bill Graham
Sent: Tuesday, September 07, 2010 10:00 AM
To: pig-user@hadoop.apache.org
Subject: PigServer.executeBatch throwing NotSerializableException

Hi,

I've just deployed some new Pig jobs live (Pig version 0.7.0) and I'm
getting the error shown below. Has anyone seen this before? What's strange
is that I have a tier of 4 load-balanced machines that serve as my job
runners, all with identical code and hardware, but 2 of the four will fail
consistently with this error. The other 2 run the same job without problem.
I'm racking my brain to understand why...

java.io.NotSerializableException:
org.apache.commons.logging.impl.Log4JLogger
at
java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1156)
at
java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1509)
at
java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1474)
at
java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1392)
at
java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1150)
at
java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:326)
at java.util.HashMap.writeObject(HashMap.java:1000)
at sun.reflect.GeneratedMethodAccessor624.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:945)
at
java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1461)
at
java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1392)
at
java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1150)
at
java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1509)
at
java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1474)
at
java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1392)
at
java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1150)
at
java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1509)
at
java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1474)
at
java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1392)
at
java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1150)
at
java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:326)
at
org.apache.pig.impl.util.ObjectSerializer.serialize(ObjectSerializer.java:40)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:511)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:246)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:131)
at
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:308)
at
org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:844)
at org.apache.pig.PigServer.execute(PigServer.java:837)
at org.apache.pig.PigServer.access$100(PigServer.java:107)
at org.apache.pig.PigServer$Graph.execute(PigServer.java:1089)
at org.apache.pig.PigServer.executeBatch(PigServer.java:290)

thanks,
Bill

Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 2 of 3 | next ›
Discussion Overview
groupuser @
categoriespig, hadoop
postedSep 7, '10 at 5:01p
activeSep 7, '10 at 6:40p
posts3
users2
websitepig.apache.org

2 users in discussion

Bill Graham: 2 posts Yan Zhou: 1 post

People

Translate

site design / logo © 2022 Grokbase