FAQ
df command doesn't exist under windows
--------------------------------------

Key: HADOOP-1792
URL: https://issues.apache.org/jira/browse/HADOOP-1792
Project: Hadoop
Issue Type: Bug
Reporter: Benjamin Francisoud
Fix For: 0.14.0


My code use to work with previous version of hadoop, I upgraded to 0.14 and now:
java.io.IOException: CreateProcess: df -k "C:\Documents and Settings\Benjamin\Local Settings\Temp\test14906test\mapredLocal" error=2
at java.lang.ProcessImpl.create(Native Method)
at java.lang.ProcessImpl.<init>(Unknown Source)
at java.lang.ProcessImpl.start(Unknown Source)
at java.lang.ProcessBuilder.start(Unknown Source)
at java.lang.Runtime.exec(Unknown Source)
at java.lang.Runtime.exec(Unknown Source)
at org.apache.hadoop.fs.DF.doDF(DF.java:60)
at org.apache.hadoop.fs.DF.(LocalDirAllocator.java:198)
at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:235)
at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:124)
at org.apache.hadoop.mapred.MapOutputFile.getSpillFileForWrite(MapOutputFile.java:88)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpillToDisk(MapTask.java:373)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:593)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:190)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:137)
at org.apache.hadoop.mapred.LocalJobRunner.submitJob(LocalJobRunner.java:283)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:397)
...

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Benjamin Francisoud (JIRA) at Aug 28, 2007 at 10:39 am
    [ https://issues.apache.org/jira/browse/HADOOP-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Benjamin Francisoud updated HADOOP-1792:
    ----------------------------------------

    Component/s: fs
    Fix Version/s: (was: 0.14.0)
    Affects Version/s: 0.14.0
    df command doesn't exist under windows
    --------------------------------------

    Key: HADOOP-1792
    URL: https://issues.apache.org/jira/browse/HADOOP-1792
    Project: Hadoop
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.14.0
    Reporter: Benjamin Francisoud

    My code use to work with previous version of hadoop, I upgraded to 0.14 and now:
    java.io.IOException: CreateProcess: df -k "C:\Documents and Settings\Benjamin\Local Settings\Temp\test14906test\mapredLocal" error=2
    at java.lang.ProcessImpl.create(Native Method)
    at java.lang.ProcessImpl.<init>(Unknown Source)
    at java.lang.ProcessImpl.start(Unknown Source)
    at java.lang.ProcessBuilder.start(Unknown Source)
    at java.lang.Runtime.exec(Unknown Source)
    at java.lang.Runtime.exec(Unknown Source)
    at org.apache.hadoop.fs.DF.doDF(DF.java:60)
    at org.apache.hadoop.fs.DF.<init>(DF.java:53)
    at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.confChanged(LocalDirAllocator.java:198)
    at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:235)
    at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:124)
    at org.apache.hadoop.mapred.MapOutputFile.getSpillFileForWrite(MapOutputFile.java:88)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpillToDisk(MapTask.java:373)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:593)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:190)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:137)
    at org.apache.hadoop.mapred.LocalJobRunner.submitJob(LocalJobRunner.java:283)
    at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:397)
    ...
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Doug Cutting (JIRA) at Aug 28, 2007 at 5:37 pm
    [ https://issues.apache.org/jira/browse/HADOOP-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12523290 ]

    Doug Cutting commented on HADOOP-1792:
    --------------------------------------

    Is anyone else seeing this? There were changes to DF.java in 0.14, but none that should cause this. Are you still running under a Cygwin shell? We depend on Cygwin binaries being on $PATH, including at least 'df' and 'bash'.
    df command doesn't exist under windows
    --------------------------------------

    Key: HADOOP-1792
    URL: https://issues.apache.org/jira/browse/HADOOP-1792
    Project: Hadoop
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.14.0
    Reporter: Benjamin Francisoud

    My code use to work with previous version of hadoop, I upgraded to 0.14 and now:
    java.io.IOException: CreateProcess: df -k "C:\Documents and Settings\Benjamin\Local Settings\Temp\test14906test\mapredLocal" error=2
    at java.lang.ProcessImpl.create(Native Method)
    at java.lang.ProcessImpl.<init>(Unknown Source)
    at java.lang.ProcessImpl.start(Unknown Source)
    at java.lang.ProcessBuilder.start(Unknown Source)
    at java.lang.Runtime.exec(Unknown Source)
    at java.lang.Runtime.exec(Unknown Source)
    at org.apache.hadoop.fs.DF.doDF(DF.java:60)
    at org.apache.hadoop.fs.DF.<init>(DF.java:53)
    at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.confChanged(LocalDirAllocator.java:198)
    at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:235)
    at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:124)
    at org.apache.hadoop.mapred.MapOutputFile.getSpillFileForWrite(MapOutputFile.java:88)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpillToDisk(MapTask.java:373)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:593)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:190)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:137)
    at org.apache.hadoop.mapred.LocalJobRunner.submitJob(LocalJobRunner.java:283)
    at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:397)
    ...
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Benjamin Francisoud (JIRA) at Aug 29, 2007 at 7:33 am
    [ https://issues.apache.org/jira/browse/HADOOP-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12523487 ]

    Benjamin Francisoud commented on HADOOP-1792:
    ---------------------------------------------

    It use to work without cygwin in the path var.

    I added it and it work yes.

    But I don't understand why it use to work before, that's weird ?!
    df command doesn't exist under windows
    --------------------------------------

    Key: HADOOP-1792
    URL: https://issues.apache.org/jira/browse/HADOOP-1792
    Project: Hadoop
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.14.0
    Reporter: Benjamin Francisoud

    My code use to work with previous version of hadoop, I upgraded to 0.14 and now:
    java.io.IOException: CreateProcess: df -k "C:\Documents and Settings\Benjamin\Local Settings\Temp\test14906test\mapredLocal" error=2
    at java.lang.ProcessImpl.create(Native Method)
    at java.lang.ProcessImpl.<init>(Unknown Source)
    at java.lang.ProcessImpl.start(Unknown Source)
    at java.lang.ProcessBuilder.start(Unknown Source)
    at java.lang.Runtime.exec(Unknown Source)
    at java.lang.Runtime.exec(Unknown Source)
    at org.apache.hadoop.fs.DF.doDF(DF.java:60)
    at org.apache.hadoop.fs.DF.<init>(DF.java:53)
    at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.confChanged(LocalDirAllocator.java:198)
    at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:235)
    at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:124)
    at org.apache.hadoop.mapred.MapOutputFile.getSpillFileForWrite(MapOutputFile.java:88)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpillToDisk(MapTask.java:373)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:593)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:190)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:137)
    at org.apache.hadoop.mapred.LocalJobRunner.submitJob(LocalJobRunner.java:283)
    at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:397)
    ...
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Olivier Dagenais (JIRA) at Nov 30, 2007 at 8:50 pm
    [ https://issues.apache.org/jira/browse/HADOOP-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12547282 ]

    Olivier Dagenais commented on HADOOP-1792:
    ------------------------------------------

    I got this problem too: It appears DF is being used to compute available disk space before creating files. While Java does not appears to have support for this (see [bug 4057701|http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4057701]), it does appear that another Apache project (sort of) does:

    [org.apache.commons.FileSystemUtils.freeSpaceKb|http://commons.apache.org/io/api-release/org/apache/commons/io/FileSystemUtils.html#freeSpaceKb(java.lang.String)]

    ...at least it wouldn't require that Hadoop users on Windows install Cygwin.
    df command doesn't exist under windows
    --------------------------------------

    Key: HADOOP-1792
    URL: https://issues.apache.org/jira/browse/HADOOP-1792
    Project: Hadoop
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.14.0
    Reporter: Benjamin Francisoud

    My code use to work with previous version of hadoop, I upgraded to 0.14 and now:
    java.io.IOException: CreateProcess: df -k "C:\Documents and Settings\Benjamin\Local Settings\Temp\test14906test\mapredLocal" error=2
    at java.lang.ProcessImpl.create(Native Method)
    at java.lang.ProcessImpl.<init>(Unknown Source)
    at java.lang.ProcessImpl.start(Unknown Source)
    at java.lang.ProcessBuilder.start(Unknown Source)
    at java.lang.Runtime.exec(Unknown Source)
    at java.lang.Runtime.exec(Unknown Source)
    at org.apache.hadoop.fs.DF.doDF(DF.java:60)
    at org.apache.hadoop.fs.DF.<init>(DF.java:53)
    at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.confChanged(LocalDirAllocator.java:198)
    at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:235)
    at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:124)
    at org.apache.hadoop.mapred.MapOutputFile.getSpillFileForWrite(MapOutputFile.java:88)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpillToDisk(MapTask.java:373)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:593)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:190)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:137)
    at org.apache.hadoop.mapred.LocalJobRunner.submitJob(LocalJobRunner.java:283)
    at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:397)
    ...
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedAug 28, '07 at 10:37a
activeNov 30, '07 at 8:50p
posts5
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Olivier Dagenais (JIRA): 5 posts

People

Translate

site design / logo © 2022 Grokbase