Grokbase Groups Pig user October 2010
FAQ
anyone ever read a pig output file with bags/tuples into a java map reduce program?

Search Discussions

  • Jeff Zhang at Oct 8, 2010 at 1:30 am
    Hi Corbin,

    As far as I know it is possible for you to read the output using java
    code. I suggest you look at PigServer.openIterator() which will read
    the output of pig script. But it will run the pig script. And I
    believe you already run the pig script in other places and want to use
    the output in another mapreduce job, right ? Then I suggest you look
    at org.apache.pig.impl.io.ReadToEndLoader which actually do the
    reading task. But I believe you have to do some hacking by yourself.

    Another work around I can think of is that You can store the output in
    the plain text format, and then read the output using Hadoop
    FileSystem API, and then using org.apache.pig.data.DataReaderWriter to
    read the output line by line.



    On Fri, Oct 8, 2010 at 3:03 AM, Corbin Hoenes wrote:
    anyone ever read a pig output file with bags/tuples into a java map reduce program?


    --
    Best Regards

    Jeff Zhang

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriespig, hadoop
postedOct 7, '10 at 7:04p
activeOct 8, '10 at 1:30a
posts2
users2
websitepig.apache.org

2 users in discussion

Jeff Zhang: 1 post Corbin Hoenes: 1 post

People

Translate

site design / logo © 2021 Grokbase