Grokbase Groups Pig user March 2011
FAQ
Hello PigUsers,

I¹m running a series of Pig scripts from a java program, and I find that
it¹s tough to swap out configurations

For instance,
- when I run a local build I¹d like to run pig in local mode,
- when I run an automated build I¹d like to connect to a QA Hadoop cluster,
- when I deploy to prod I¹d like to connect to a larger Production cluster.

I was under the impression that the PigServer(ExecType, Properties)
consturctor would let me set "fs.default.name" and "mapred.job.tracker" -
which would accomplish exactly what I want - I could externalize those
properties and swap them out depending on the environment. However, it looks
like HExecutionEngine is going to try to pick the XML files up regardless of
what I put into the constructor argument for Properties.
(ref: line 167 of HExecutionEngine)
http://search-hadoop.com/c/Pig:/src/org/apache/pig/backend/hadoop/executione
ngine/HExecutionEngine.java%7C%7ChexecutionEngine

Does anyone know..
1) If there's a way to programmatically set "fs.default.name"/
"mapred.job.tracker" WITHOUT requiring XML files to be in the classpath?
2) What does the properties parameter in PigServer(ExecType, Properties) do?
I could find it anywhere in the javadocs or the source code - I would be
happy to update the javadocs if someone can help me figure out what this is
doing.


Thank you,
- Andrew

Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriespig, hadoop
postedMar 31, '11 at 1:21a
activeMar 31, '11 at 1:21a
posts1
users1
websitepig.apache.org

1 user in discussion

Andrew Look: 1 post

People

Translate

site design / logo © 2022 Grokbase