I will spawn a sub process and handle third party errors!
-- Ashwin!
Date: Tue, 9 Aug 2011 18:11:40 +0000
Subject: Re: Skipping Bad Records in M/R Job
From: owen@hortonworks.com
To: common-user@hadoop.apache.org
On Tue, Aug 9, 2011 at 5:28 PM, Maheshwaran Janarthanan <
ashwinwaran@hotmail.com> wrote:
experimental and I don't think it has been well maintained.
If your 3rd part library crashes the jvm, I'd suggest using a subprocess to
call it and handle the errors yourself.
-- Owen
Subject: Re: Skipping Bad Records in M/R Job
From: owen@hortonworks.com
To: common-user@hadoop.apache.org
On Tue, Aug 9, 2011 at 5:28 PM, Maheshwaran Janarthanan <
ashwinwaran@hotmail.com> wrote:
Hi,
I have written a Map reduce job which uses third party libraries to process
unseen data which makes job fail because of errors in records.
I realized 'Skipping Bad Records' feature in Hadoop Map/Reduce. Can Anyone
send me the code snippet which enables this feature by setting properties on
JobConf
I wouldn't recommend using the bad record skipping, since it was alwaysI have written a Map reduce job which uses third party libraries to process
unseen data which makes job fail because of errors in records.
I realized 'Skipping Bad Records' feature in Hadoop Map/Reduce. Can Anyone
send me the code snippet which enables this feature by setting properties on
JobConf
experimental and I don't think it has been well maintained.
If your 3rd part library crashes the jvm, I'd suggest using a subprocess to
call it and handle the errors yourself.
-- Owen
