Grokbase Groups Pig user June 2009
FAQ
Thank you Kevin, this is one option. But my question to the Pig guru is, is
there API for file IO between HDFS and local system, or will be there in the
future?
George

2009/6/16 Kevin Weil <kevinweil@gmail.com>
If you're already writing Pig from within Java, your best bet is to
probably
go through the standard HDFS interfaces. In particular, see

http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/fs/FileUtil.htmlfor
a utility class that exposes copy method from HDFS to the local file
system.

Kevin
On Thu, Jun 11, 2009 at 11:46 AM, George Pang wrote:

Hi pig users,
I tried to copyToLocal my stored result from pig queries to my local
workspace. My lines of code in Java are:

........
pigServer.store("B","output");
*pigServer.registerQuery("copyToLocal output WorkingDir output ");*

And I know Pig Latin statements will execute only at "store" and "dump", so
I think the last line of code won't get executed.
so I tried to add another line:
pigServer.registerQuery( "quit;" );
*
*
This will work in Grunt, but not here. So what's the best practice to copy
the file to a local working directory for reading or further processing?
Should I use the interface in Hadoop?
Thank you.

George

Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 3 of 4 | next ›
Discussion Overview
groupuser @
categoriespig, hadoop
postedJun 11, '09 at 6:46p
activeJun 16, '09 at 5:27p
posts4
users3
websitepig.apache.org

People

Translate

site design / logo © 2021 Grokbase