Grokbase Groups Pig user June 2012
FAQ
Hi all,

I'm trying to run the mahout canopy clustering algorithm through a
Python-embedded Pig script. The embedded Pig part of the script works (using
compileFromFile, bind, runSingle), but I can't figure out how to run mahout
from the same script. Originally I tried running mahout via subprocess.call,
but when trying to import subprocess, I get:

ImportError: No module named subprocess

Similar errors occur when I try to import sys or os modules.

Next I tried just instantiating the CanopyClustering class, but got a
similar error when using the following import statement:

from org.apache.mahout.clustering.canopy import CanopyDriver

#=> ImportError: No module named mahout

The ImportErrors don't occur when I run Python interactively. Is this a
Jython problem? Am I not setting some path properly?

Other possibly useful info:
- I'm including the mahout jars in the pig.additional.jars property.
- I'm running the script using Pig, i.e., `pig myscript.py`

Thanks,
Chun

Search Discussions

  • Daniel Dai at Jun 18, 2012 at 2:07 am
    I see subprocess problem before. This is because we bundle jython.jar
    instead of jython-standalone.jar, see PIG-2665.

    On Fri, Jun 15, 2012 at 6:30 PM, Chun Yang
    wrote:
    Hi all,

    I'm trying to run the mahout canopy clustering algorithm through a
    Python-embedded Pig script. The embedded Pig part of the script works
    (using
    compileFromFile, bind, runSingle), but I can't figure out how to run mahout
    from the same script. Originally I tried running mahout via
    subprocess.call,
    but when trying to import subprocess, I get:

    ImportError: No module named subprocess

    Similar errors occur when I try to import sys or os modules.

    Next I tried just instantiating the CanopyClustering class, but got a
    similar error when using the following import statement:

    from org.apache.mahout.clustering.canopy import CanopyDriver

    #=> ImportError: No module named mahout

    The ImportErrors don't occur when I run Python interactively. Is this a
    Jython problem? Am I not setting some path properly?

    Other possibly useful info:
    - I'm including the mahout jars in the pig.additional.jars property.
    - I'm running the script using Pig, i.e., `pig myscript.py`

    Thanks,
    Chun
  • Chun Yang at Jun 18, 2012 at 4:56 pm
    Thanks Daniel!

    That was exactly what I was looking for.

    Cheers,
    Chun

    On 6/17/12 7:07 PM, "Daniel Dai" wrote:

    I see subprocess problem before. This is because we bundle jython.jar
    instead of jython-standalone.jar, see PIG-2665.

    On Fri, Jun 15, 2012 at 6:30 PM, Chun Yang
    wrote:
    Hi all,

    I'm trying to run the mahout canopy clustering algorithm through a
    Python-embedded Pig script. The embedded Pig part of the script works
    (using
    compileFromFile, bind, runSingle), but I can't figure out how to run mahout
    from the same script. Originally I tried running mahout via
    subprocess.call,
    but when trying to import subprocess, I get:

    ImportError: No module named subprocess

    Similar errors occur when I try to import sys or os modules.

    Next I tried just instantiating the CanopyClustering class, but got a
    similar error when using the following import statement:

    from org.apache.mahout.clustering.canopy import CanopyDriver

    #=> ImportError: No module named mahout

    The ImportErrors don't occur when I run Python interactively. Is this a
    Jython problem? Am I not setting some path properly?

    Other possibly useful info:
    - I'm including the mahout jars in the pig.additional.jars property.
    - I'm running the script using Pig, i.e., `pig myscript.py`

    Thanks,
    Chun

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriespig, hadoop
postedJun 16, '12 at 1:30a
activeJun 18, '12 at 4:56p
posts3
users2
websitepig.apache.org

2 users in discussion

Chun Yang: 2 posts Daniel Dai: 1 post

People

Translate

site design / logo © 2022 Grokbase