FAQ
That depends on your ssh implementation. Most will look at the file
~/.ssh/id_dsa.pub or id_rsa.pub in the same directory. You can generate
your keys on the linux machine ('man ssh-keygen' for more info), which
will generate files named id_Xsa and id_Xsa.pub (Depending on the key
type). The private key file goes in ~/.ssh/ on the Linux machine. The
public key file (should also be copied there), and goes in the specified
directory on your Windows machine depending on what ssh client you're
using. (If you're using cygwin's, ~/.ssh/ should be the correct one.)

If you have a sane cygwin install, 'man ssh' for more.

- Aaron

dcave555 wrote:
Thx again for your reply

Ok maybe you right

I have my ubuntu hadoop node on ip 168.13.51.13

So i write in hadoop-site.xml (windows application)

<configuration>
<property>
<name>hadoop.tmp.dir</name>
<value>//168.13.51.13/usr/local/hadoop-datastore/hadoop-hadoop</value>
<description>A base for other temporary directories.</description>
</property>

<property>
<name>fs.default.name</name>
<value>hdfs://168.13.51.13:54310</value>
</property>

<property>
<name>mapred.job.tracker</name>
<value>168.13.51.13:54311</value>
</property>

<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>


The same xml i have on ubuntu machine.


after running my application i have exception:

Login failded.

i suspect that i have problem with ssh public key

so i think i need to copy ssh public key to windows machine

but how can i tell to my application to use this key?

Thx again


Ted Dunning-3 wrote:
Your program may be trying to run locally on your windows box instead of
talking to the ubuntu micro cluster.

On 10/13/07 11:43 AM, "dcave555" wrote:

Thx for your reply

But i think i don't understand something.
i have installed hadoop single node on ubuntu.
my test is running on windows.

i changed property in hadoop-site.xml in my application to

<property>
<name>hadoop.tmp.dir</name>
<value>//172.16.50.13/usr/local/hadoop-datastore/hadoop-hadoop</value>
<description>A base for other temporary directories.</description>
</property>

amd i am getting Login failed.

If i want to save file from windows to hadoop on ubuntu i need to import
ssh
public key?
how can i tell my test application on windows to use this pulic key?

Or i need to do something else?

Thx





Ted Dunning-3 wrote:
Hadoop is having trouble spawning a process to measure how much disk
space
you have available.

It really is much easier to run hadoop on Linux. In case you can't do
that,
make sure that you really do have cygwin installed in a standard way on
each
hadoop node.

On 10/13/07 8:05 AM, "dcave555" wrote:

Hello all

I am new to hadoop .

I am trying to write file to single cluster and getting this exception
when
i am trying to close output stream

java.io.IOException: CreateProcess: df -k
C:\usr\local\hadoop-datastore\hadoop-hadoop\dfs\tmp error=2
at java.lang.ProcessImpl.create(Native Method)
at java.lang.ProcessImpl.<init>(Unknown Source)
at java.lang.ProcessImpl.start(Unknown Source)
at java.lang.ProcessBuilder.start(Unknown Source)
at java.lang.Runtime.exec(Unknown Source)
at java.lang.Runtime.exec(Unknown Source)
at org.apache.hadoop.fs.DF.doDF(DF.java:60)
at org.apache.hadoop.fs.DF.<init>(DF.java:53)
at
org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.confChanged(Local
Di
rAllocator.java:198)
at
org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWr
it
e(LocalDirAllocator.java:235)
at
org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.createTmpFileForW
ri
te(LocalDirAllocator.java:276)
at
org.apache.hadoop.fs.LocalDirAllocator.createTmpFileForWrite(LocalDirAllocat
or
.java:155)
at
org.apache.hadoop.dfs.DFSClient$DFSOutputStream.newBackupFile(DFSClient.java
:1
475)
at
org.apache.hadoop.dfs.DFSClient$DFSOutputStream.openBackupStream(DFSClient.j
av
a:1442)
at
org.apache.hadoop.dfs.DFSClient$DFSOutputStream.writeChunk(DFSClient.java:160
0>
)
at
org.apache.hadoop.fs.FSOutputSummer.writeChecksumChunk(FSOutputSummer.java:14
0>
)
at
org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:122)
at
org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:1739)
at
org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStre
am
.java:49)
at
org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:64)
at Test1.main(Test1.java:23)



My test is:

Configuration configuration = new
Configuration();
FileSystem fileSystem =
FileSystem.get(configuration);
Path path = new Path("/testfile");
//writing:
FSDataOutputStream dataOutputStream = fileSystem.create(path);
dataOutputStream.writeUTF("hello world");
dataOutputStream.close();
//reading
FSDataInputStream dataInputStream = fileSystem.open(path);
System.out.println(dataInputStream.readUTF());
dataInputStream.close();
fileSystem.close();

i added hadoop-site.xml to classpath :


<configuration>

<property>
<name>hadoop.tmp.dir</name>
<value>/usr/local/hadoop-datastore/hadoop-hadoop</value>
<description>A base for other temporary directories.</description>
</property>

<property>
<name>fs.default.name</name>
<value>hdfs://172.16.50.13:54310</value>
<description>The name of the default file system. A URI whose
scheme and authority determine the FileSystem implementation. The
uri's scheme determines the config property (fs.SCHEME.impl) naming
the FileSystem implementation class. The uri's authority is used to
determine the host, port, etc. for a filesystem.</description>
</property>

<property>
<name>mapred.job.tracker</name>
<value>172.16.50.13:54311</value>
<description>The host and port that the MapReduce job tracker runs
at. If "local", then jobs are run in-process as a single map
and reduce task.
</description>
</property>

<property>
<name>dfs.replication</name>
<value>1</value>
<description>Default block replication.
The actual number of replications can be specified when the file is
created.
The default is used if replication is not specified in create time.
</description>
</property>
</configuration>



Please help me
thx

Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 6 of 7 | next ›
Discussion Overview
groupcommon-user @
categorieshadoop
postedOct 13, '07 at 3:06p
activeOct 13, '07 at 10:21p
posts7
users3
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase