Grokbase Groups Pig user January 2011
FAQ
I can connect to hdfs while running in script mode but am having trouble doing so in embedded mode(local mode is fine)
I've tried all possible export settings, could someone please point out what I'm doing wrong in trying to connect to HDFS
Do I have to do anything else in the java file other than pigServer = new PigServer("mapreduce")

which hadoop
/usr/local/software/hadoop-0.20.2/bin/hadoop

pig -x mapreduce try1.pig is fine ... Connecting to hadoop file system at: hdfs://master-hadoop:54310
java -cp pig.jar:. test.java is fine too (pigServer = new PigServer("local"))

echo $HADOOPDIR gives /usr/local/software/hadoop-0.20.2/conf
however
java -cp pig.jar:.:$HADOOPDIR test is not OK (pigServer = new PigServer("mapreduce") Connecting to hadoop file system at: file:/// instead of hdfs://)
Also it shows: ERROR mapReduceLayer.MapReduceLauncher: java.io.IOException: sample2.txt does not exist

I have sample2.txt in local dir as well as in hdfs
ls -l sample2.txt
-rw-r--r-- 1 hadoop hadoop 225 2011-01-13 17:47 sample2.txt
hadoop dfs -ls sample2.txt
Found 1 items
-rw-r--r-- 3 hadoop supergroup 225 2011-01-20 18:34 /user/hadoop/sample2.txt
-----
export JAVA_HOME=/usr/local/jdk1.6.0_18
HADOOP_HOME=/usr/local/software/hadoop-0.20.2
PIG_HOME=/home/hadoop/code/pig-0.7.0/pig-0.7.0
export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$PIG_HOME/bin

export HADOOPDIR=$HADOOP_HOME/conf
export PIGDIR=$PIG_HOME
export PIG_HADOOP_VERSION=20
export PIG_CLASSPATH=$PIG_HOME/pig.jar:.:$HADOOPDIR

thanks a lot!

Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriespig, hadoop
postedJan 21, '11 at 2:09a
activeJan 21, '11 at 2:09a
posts1
users1
websitepig.apache.org

1 user in discussion

Anindita Mahapatra: 1 post

People

Translate

site design / logo © 2021 Grokbase