FAQ
Thank you Owen and Aaron,

I will spawn a sub process and handle third party errors!

-- Ashwin!
Date: Tue, 9 Aug 2011 18:11:40 +0000
Subject: Re: Skipping Bad Records in M/R Job
From: owen@hortonworks.com
To: common-user@hadoop.apache.org

On Tue, Aug 9, 2011 at 5:28 PM, Maheshwaran Janarthanan <
ashwinwaran@hotmail.com> wrote:
Hi,

I have written a Map reduce job which uses third party libraries to process
unseen data which makes job fail because of errors in records.

I realized 'Skipping Bad Records' feature in Hadoop Map/Reduce. Can Anyone
send me the code snippet which enables this feature by setting properties on
JobConf
I wouldn't recommend using the bad record skipping, since it was always
experimental and I don't think it has been well maintained.

If your 3rd part library crashes the jvm, I'd suggest using a subprocess to
call it and handle the errors yourself.

-- Owen

Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 7 of 8 | next ›
Discussion Overview
groupcommon-user @
categorieshadoop
postedAug 6, '11 at 7:42p
activeAug 13, '11 at 7:53p
posts8
users5
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase