Grokbase Groups Hive user May 2010
FAQ
Hi,

I have two python scripts that is loaded into Hive, one of the python script
reference the other file as class file. However, when I run the transform
statement with the first python script, it says it can't reference the other
file in the import header.

So, is there a way to reference the other python script? Or do I have embed
all the files into one file? BTW, when I add file, which directory on the
Slaves does the files copy to?

Thanks,
-ray

Python: foo.py

#!/usr/bin/env python

from bar import bar1


Hive:

add file foo.py
add file bar.py

select
transform(x, y)
using 'python foo.py'
as x, y, z
from
footable;

*stderr logs*

Traceback (most recent call last):
File "foo.py", line 6, in ?
from bar import bar1
ImportError: No module named bar

Search Discussions

  • Dilip Joseph at May 13, 2010 at 8:06 pm
    I had the same problem. Adding the following lines before the import
    solved the problem:

    import sys
    import os
    sys.path.append(os.getcwd())

    Dilip
    On Thu, May 13, 2010 at 10:58 AM, Ray Duong wrote:
    Hi,
    I have two python scripts that is loaded into Hive, one of the python script
    reference the other file as class file.  However, when I run the transform
    statement with the first python script, it says it can't reference the other
    file in the import header.
    So, is there a way to reference the other python script?  Or do I have embed
    all the files into one file?  BTW, when I add file, which directory on the
    Slaves does the files copy to?
    Thanks,
    -ray
    Python: foo.py
    #!/usr/bin/env python
    from bar import bar1

    Hive:
    add file foo.py
    add file bar.py
    select
    transform(x, y)
    using 'python foo.py'
    as x, y, z
    from
    footable;
    stderr logs

    Traceback (most recent call last):
    File "foo.py", line 6, in ?
    from bar import bar1
    ImportError: No module named bar


    --
    _________________________________________
    Dilip Antony Joseph
    http://www.marydilip.info
  • Ray Duong at May 13, 2010 at 8:32 pm
    Thanks, that worked.

    -ray
    On Thu, May 13, 2010 at 1:06 PM, Dilip Joseph wrote:

    I had the same problem. Adding the following lines before the import
    solved the problem:

    import sys
    import os
    sys.path.append(os.getcwd())

    Dilip
    On Thu, May 13, 2010 at 10:58 AM, Ray Duong wrote:
    Hi,
    I have two python scripts that is loaded into Hive, one of the python script
    reference the other file as class file. However, when I run the transform
    statement with the first python script, it says it can't reference the other
    file in the import header.
    So, is there a way to reference the other python script? Or do I have embed
    all the files into one file? BTW, when I add file, which directory on the
    Slaves does the files copy to?
    Thanks,
    -ray
    Python: foo.py
    #!/usr/bin/env python
    from bar import bar1

    Hive:
    add file foo.py
    add file bar.py
    select
    transform(x, y)
    using 'python foo.py'
    as x, y, z
    from
    footable;
    stderr logs

    Traceback (most recent call last):
    File "foo.py", line 6, in ?
    from bar import bar1
    ImportError: No module named bar


    --
    _________________________________________
    Dilip Antony Joseph
    http://www.marydilip.info

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categorieshive, hadoop
postedMay 13, '10 at 5:59p
activeMay 13, '10 at 8:32p
posts3
users2
websitehive.apache.org

2 users in discussion

Ray Duong: 2 posts Dilip Joseph: 1 post

People

Translate

site design / logo © 2021 Grokbase