Grokbase Groups Pig user June 2011
FAQ
Hello,

I just tried the example from the pig udf manual step by step. But I got the
error information. Can anyone tell me how to solve it?

grunt> REGISTER /home/huyong/test/myudfs.jar;
grunt> A = LOAD '/home/huyong/test/student.txt' as (name:chararray);
grunt> B = FOREACH A GENERATE myudfs.UPPER(name);
2011-06-18 11:15:38,892 [main] ERROR org.apache.pig.tools.grunt.Grunt -
ERROR 1070: Could not resolve myudfs.UPPER using imports: [,
org.apache.pig.builtin., org.apache.pig.impl.builtin.]
Details at logfile: /home/huyong/test/pig_1308388238352.log

I have already registered the udf, why pig tries to search from the builtin
path.

Thanks for your help!

Yong Hu

Search Discussions

  • Jonathan Coveney at Jun 18, 2011 at 1:46 pm
    Can you paste the content of the UDF?

    2011/6/18 勇胡 <yongyong313@gmail.com>
    Hello,

    I just tried the example from the pig udf manual step by step. But I got
    the
    error information. Can anyone tell me how to solve it?

    grunt> REGISTER /home/huyong/test/myudfs.jar;
    grunt> A = LOAD '/home/huyong/test/student.txt' as (name:chararray);
    grunt> B = FOREACH A GENERATE myudfs.UPPER(name);
    2011-06-18 11:15:38,892 [main] ERROR org.apache.pig.tools.grunt.Grunt -
    ERROR 1070: Could not resolve myudfs.UPPER using imports: [,
    org.apache.pig.builtin., org.apache.pig.impl.builtin.]
    Details at logfile: /home/huyong/test/pig_1308388238352.log

    I have already registered the udf, why pig tries to search from the builtin
    path.

    Thanks for your help!

    Yong Hu
  • 勇胡 at Jun 18, 2011 at 1:51 pm
    Hi,

    package myudfs;
    import java.io.IOException;
    import org.apache.pig.EvalFunc;
    import org.apache.pig.data.Tuple;
    import org.apache.pig.impl.util.*;

    public class UPPER extends EvalFunc<String>
    {
    public String exec(Tuple input) throws IOException {
    if (input == null || input.size() == 0)
    return null;
    try{
    String str = (String)input.get(0);
    return str.toUpperCase();
    }catch(Exception e){
    throw new IOException(e);
    }
    }
    }

    This is as same as the example from the Pig website. By the way, I also
    added the PIG_CLASS. But it still didn't work.

    Yong

    2011/6/18 Jonathan Coveney <jcoveney@gmail.com>
    Can you paste the content of the UDF?

    2011/6/18 勇胡 <yongyong313@gmail.com>
    Hello,

    I just tried the example from the pig udf manual step by step. But I got
    the
    error information. Can anyone tell me how to solve it?

    grunt> REGISTER /home/huyong/test/myudfs.jar;
    grunt> A = LOAD '/home/huyong/test/student.txt' as (name:chararray);
    grunt> B = FOREACH A GENERATE myudfs.UPPER(name);
    2011-06-18 11:15:38,892 [main] ERROR org.apache.pig.tools.grunt.Grunt -
    ERROR 1070: Could not resolve myudfs.UPPER using imports: [,
    org.apache.pig.builtin., org.apache.pig.impl.builtin.]
    Details at logfile: /home/huyong/test/pig_1308388238352.log

    I have already registered the udf, why pig tries to search from the builtin
    path.

    Thanks for your help!

    Yong Hu
  • Dmitriy Ryaboy at Jun 18, 2011 at 2:30 pm
    This usually hapens when you aren't registering what you think you are
    registering.
    try `jar tf /home/huyong/test/myudfs.jar | grep UPPER` and see if you
    get anything.

    D

    2011/6/18 勇胡 <yongyong313@gmail.com>:
    Hi,

    package myudfs;
    import java.io.IOException;
    import org.apache.pig.EvalFunc;
    import org.apache.pig.data.Tuple;
    import org.apache.pig.impl.util.*;

    public class UPPER extends EvalFunc<String>
    {
    public String exec(Tuple input) throws IOException {
    if (input == null || input.size() == 0)
    return null;
    try{
    String str = (String)input.get(0);
    return str.toUpperCase();
    }catch(Exception e){
    throw new IOException(e);
    }
    }
    }

    This is as same as the example from the Pig website. By the way, I also
    added the PIG_CLASS. But it still didn't work.

    Yong

    2011/6/18 Jonathan Coveney <jcoveney@gmail.com>
    Can you paste the content of the UDF?

    2011/6/18 勇胡 <yongyong313@gmail.com>
    Hello,

    I just tried the example from the pig udf manual step by step. But I got
    the
    error information. Can anyone tell me how to solve it?

    grunt> REGISTER /home/huyong/test/myudfs.jar;
    grunt> A = LOAD '/home/huyong/test/student.txt' as (name:chararray);
    grunt> B = FOREACH A GENERATE myudfs.UPPER(name);
    2011-06-18 11:15:38,892 [main] ERROR org.apache.pig.tools.grunt.Grunt -
    ERROR 1070: Could not resolve myudfs.UPPER using imports: [,
    org.apache.pig.builtin., org.apache.pig.impl.builtin.]
    Details at logfile: /home/huyong/test/pig_1308388238352.log

    I have already registered the udf, why pig tries to search from the builtin
    path.

    Thanks for your help!

    Yong Hu
  • 勇胡 at Jun 18, 2011 at 7:33 pm
    I tried your command and then it shows me as following:
    /home/huyong/test/UPPER.class
    /home/huyong/test/UPPER.java

    Yong
    在 2011年6月18日 下午4:29,Dmitriy Ryaboy <dvryaboy@gmail.com>写道:
    This usually hapens when you aren't registering what you think you are
    registering.
    try `jar tf /home/huyong/test/myudfs.jar | grep UPPER` and see if you
    get anything.

    D

    2011/6/18 勇胡 <yongyong313@gmail.com>:
    Hi,

    package myudfs;
    import java.io.IOException;
    import org.apache.pig.EvalFunc;
    import org.apache.pig.data.Tuple;
    import org.apache.pig.impl.util.*;

    public class UPPER extends EvalFunc<String>
    {
    public String exec(Tuple input) throws IOException {
    if (input == null || input.size() == 0)
    return null;
    try{
    String str = (String)input.get(0);
    return str.toUpperCase();
    }catch(Exception e){
    throw new IOException(e);
    }
    }
    }

    This is as same as the example from the Pig website. By the way, I also
    added the PIG_CLASS. But it still didn't work.

    Yong

    2011/6/18 Jonathan Coveney <jcoveney@gmail.com>
    Can you paste the content of the UDF?

    2011/6/18 勇胡 <yongyong313@gmail.com>
    Hello,

    I just tried the example from the pig udf manual step by step. But I
    got
    the
    error information. Can anyone tell me how to solve it?

    grunt> REGISTER /home/huyong/test/myudfs.jar;
    grunt> A = LOAD '/home/huyong/test/student.txt' as (name:chararray);
    grunt> B = FOREACH A GENERATE myudfs.UPPER(name);
    2011-06-18 11:15:38,892 [main] ERROR org.apache.pig.tools.grunt.Grunt
    -
    ERROR 1070: Could not resolve myudfs.UPPER using imports: [,
    org.apache.pig.builtin., org.apache.pig.impl.builtin.]
    Details at logfile: /home/huyong/test/pig_1308388238352.log

    I have already registered the udf, why pig tries to search from the builtin
    path.

    Thanks for your help!

    Yong Hu
  • Dexin Wang at Jun 19, 2011 at 2:12 am
    You need to have your class file in this path

    /home/huyong/test/myudfs/UPPER.class

    since it's in myudfs directory.

    On Jun 18, 2011, at 12:33 PM, 勇胡 wrote:

    I tried your command and then it shows me as following:
    /home/huyong/test/UPPER.class
    /home/huyong/test/UPPER.java

    Yong
    在 2011年6月18日 下午4:29,Dmitriy Ryaboy <dvryaboy@gmail.com>写道:
    This usually hapens when you aren't registering what you think you are
    registering.
    try `jar tf /home/huyong/test/myudfs.jar | grep UPPER` and see if you
    get anything.

    D

    2011/6/18 勇胡 <yongyong313@gmail.com>:
    Hi,

    package myudfs;
    import java.io.IOException;
    import org.apache.pig.EvalFunc;
    import org.apache.pig.data.Tuple;
    import org.apache.pig.impl.util.*;

    public class UPPER extends EvalFunc<String>
    {
    public String exec(Tuple input) throws IOException {
    if (input == null || input.size() == 0)
    return null;
    try{
    String str = (String)input.get(0);
    return str.toUpperCase();
    }catch(Exception e){
    throw new IOException(e);
    }
    }
    }

    This is as same as the example from the Pig website. By the way, I also
    added the PIG_CLASS. But it still didn't work.

    Yong

    2011/6/18 Jonathan Coveney <jcoveney@gmail.com>
    Can you paste the content of the UDF?

    2011/6/18 勇胡 <yongyong313@gmail.com>
    Hello,

    I just tried the example from the pig udf manual step by step. But I
    got
    the
    error information. Can anyone tell me how to solve it?

    grunt> REGISTER /home/huyong/test/myudfs.jar;
    grunt> A = LOAD '/home/huyong/test/student.txt' as (name:chararray);
    grunt> B = FOREACH A GENERATE myudfs.UPPER(name);
    2011-06-18 11:15:38,892 [main] ERROR org.apache.pig.tools.grunt.Grunt
    -
    ERROR 1070: Could not resolve myudfs.UPPER using imports: [,
    org.apache.pig.builtin., org.apache.pig.impl.builtin.]
    Details at logfile: /home/huyong/test/pig_1308388238352.log

    I have already registered the udf, why pig tries to search from the builtin
    path.

    Thanks for your help!

    Yong Hu
  • 勇胡 at Jun 19, 2011 at 8:35 am
    In this path, it has already contained UPPER.class and UPPER.java. And the
    myudfs.jar is also in the path. But it did not work. By the way, my Pig
    version is 0.8.1. Is there something, for example, classpath I have to set
    up in the Pig configuration?

    Yong

    在 2011年6月19日 上午4:10,Dexin Wang <wangdexin@gmail.com>写道:
    You need to have your class file in this path

    /home/huyong/test/myudfs/UPPER.class

    since it's in myudfs directory.

    On Jun 18, 2011, at 12:33 PM, 勇胡 wrote:

    I tried your command and then it shows me as following:
    /home/huyong/test/UPPER.class
    /home/huyong/test/UPPER.java

    Yong
    在 2011年6月18日 下午4:29,Dmitriy Ryaboy <dvryaboy@gmail.com>写道:
    This usually hapens when you aren't registering what you think you are
    registering.
    try `jar tf /home/huyong/test/myudfs.jar | grep UPPER` and see if you
    get anything.

    D

    2011/6/18 勇胡 <yongyong313@gmail.com>:
    Hi,

    package myudfs;
    import java.io.IOException;
    import org.apache.pig.EvalFunc;
    import org.apache.pig.data.Tuple;
    import org.apache.pig.impl.util.*;

    public class UPPER extends EvalFunc<String>
    {
    public String exec(Tuple input) throws IOException {
    if (input == null || input.size() == 0)
    return null;
    try{
    String str = (String)input.get(0);
    return str.toUpperCase();
    }catch(Exception e){
    throw new IOException(e);
    }
    }
    }

    This is as same as the example from the Pig website. By the way, I also
    added the PIG_CLASS. But it still didn't work.

    Yong

    2011/6/18 Jonathan Coveney <jcoveney@gmail.com>
    Can you paste the content of the UDF?

    2011/6/18 勇胡 <yongyong313@gmail.com>
    Hello,

    I just tried the example from the pig udf manual step by step. But I
    got
    the
    error information. Can anyone tell me how to solve it?

    grunt> REGISTER /home/huyong/test/myudfs.jar;
    grunt> A = LOAD '/home/huyong/test/student.txt' as (name:chararray);
    grunt> B = FOREACH A GENERATE myudfs.UPPER(name);
    2011-06-18 11:15:38,892 [main] ERROR org.apache.pig.tools.grunt.Grunt
    -
    ERROR 1070: Could not resolve myudfs.UPPER using imports: [,
    org.apache.pig.builtin., org.apache.pig.impl.builtin.]
    Details at logfile: /home/huyong/test/pig_1308388238352.log

    I have already registered the udf, why pig tries to search from the builtin
    path.

    Thanks for your help!

    Yong Hu
  • Dexin Wang at Jun 19, 2011 at 5:25 pm
    Looks like you need to learn a bit about how java package works. The path
    for the class file needs to be

    /home/huyong/test/*myudfs*/UPPER.class

    Or remove the first line in your UDF code "package myudfs;" and redo
    everything you did, it will work.

    2011/6/19 勇胡 <yongyong313@gmail.com>
    In this path, it has already contained UPPER.class and UPPER.java. And the
    myudfs.jar is also in the path. But it did not work. By the way, my Pig
    version is 0.8.1. Is there something, for example, classpath I have to set
    up in the Pig configuration?

    Yong

    在 2011年6月19日 上午4:10,Dexin Wang <wangdexin@gmail.com>写道:
    You need to have your class file in this path

    /home/huyong/test/myudfs/UPPER.class

    since it's in myudfs directory.

    On Jun 18, 2011, at 12:33 PM, 勇胡 wrote:

    I tried your command and then it shows me as following:
    /home/huyong/test/UPPER.class
    /home/huyong/test/UPPER.java

    Yong
    在 2011年6月18日 下午4:29,Dmitriy Ryaboy <dvryaboy@gmail.com>写道:
    This usually hapens when you aren't registering what you think you are
    registering.
    try `jar tf /home/huyong/test/myudfs.jar | grep UPPER` and see if you
    get anything.

    D

    2011/6/18 勇胡 <yongyong313@gmail.com>:
    Hi,

    package myudfs;
    import java.io.IOException;
    import org.apache.pig.EvalFunc;
    import org.apache.pig.data.Tuple;
    import org.apache.pig.impl.util.*;

    public class UPPER extends EvalFunc<String>
    {
    public String exec(Tuple input) throws IOException {
    if (input == null || input.size() == 0)
    return null;
    try{
    String str = (String)input.get(0);
    return str.toUpperCase();
    }catch(Exception e){
    throw new IOException(e);
    }
    }
    }

    This is as same as the example from the Pig website. By the way, I
    also
    added the PIG_CLASS. But it still didn't work.

    Yong

    2011/6/18 Jonathan Coveney <jcoveney@gmail.com>
    Can you paste the content of the UDF?

    2011/6/18 勇胡 <yongyong313@gmail.com>
    Hello,

    I just tried the example from the pig udf manual step by step. But
    I
    got
    the
    error information. Can anyone tell me how to solve it?

    grunt> REGISTER /home/huyong/test/myudfs.jar;
    grunt> A = LOAD '/home/huyong/test/student.txt' as
    (name:chararray);
    grunt> B = FOREACH A GENERATE myudfs.UPPER(name);
    2011-06-18 11:15:38,892 [main] ERROR
    org.apache.pig.tools.grunt.Grunt
    -
    ERROR 1070: Could not resolve myudfs.UPPER using imports: [,
    org.apache.pig.builtin., org.apache.pig.impl.builtin.]
    Details at logfile: /home/huyong/test/pig_1308388238352.log

    I have already registered the udf, why pig tries to search from the builtin
    path.

    Thanks for your help!

    Yong Hu
  • Dmitriy Ryaboy at Jun 19, 2011 at 9:41 pm
    It's even worse than just the missing package name.

    When you create a jar, its root is supposed to be the root of your package tree.

    So a class "com.foo.Bar" has to be in "com/foo/Bar.class" when you run
    jar tf on the jarfile. If it's in "home/me/src/com/foo/Bar.class" or
    just in "Bar.class" or in "something/else/Bar.class" java won't find
    it.

    D

    2011/6/19 Dexin Wang <wangdexin@gmail.com>:
    Looks like you need to learn a bit about how java package works. The path
    for the class file needs to be

    /home/huyong/test/*myudfs*/UPPER.class

    Or remove the first line in your UDF code "package myudfs;" and redo
    everything you did, it will work.

    2011/6/19 勇胡 <yongyong313@gmail.com>
    In this path, it has already contained UPPER.class and UPPER.java. And the
    myudfs.jar is also in the path. But it did not work. By the way, my Pig
    version is 0.8.1. Is there something, for example, classpath I have to set
    up in the Pig configuration?

    Yong

    在 2011年6月19日 上午4:10,Dexin Wang <wangdexin@gmail.com>写道:
    You need to have your class file in this path

    /home/huyong/test/myudfs/UPPER.class

    since it's in myudfs directory.

    On Jun 18, 2011, at 12:33 PM, 勇胡 wrote:

    I tried your command and then it shows me as following:
    /home/huyong/test/UPPER.class
    /home/huyong/test/UPPER.java

    Yong
    在 2011年6月18日 下午4:29,Dmitriy Ryaboy <dvryaboy@gmail.com>写道:
    This usually hapens when you aren't registering what you think you are
    registering.
    try `jar tf /home/huyong/test/myudfs.jar | grep UPPER` and see if you
    get anything.

    D

    2011/6/18 勇胡 <yongyong313@gmail.com>:
    Hi,

    package myudfs;
    import java.io.IOException;
    import org.apache.pig.EvalFunc;
    import org.apache.pig.data.Tuple;
    import org.apache.pig.impl.util.*;

    public class UPPER extends EvalFunc<String>
    {
    public String exec(Tuple input) throws IOException {
    if (input == null || input.size() == 0)
    return null;
    try{
    String str = (String)input.get(0);
    return str.toUpperCase();
    }catch(Exception e){
    throw new IOException(e);
    }
    }
    }

    This is as same as the example from the Pig website. By the way, I
    also
    added the PIG_CLASS. But it still didn't work.

    Yong

    2011/6/18 Jonathan Coveney <jcoveney@gmail.com>
    Can you paste the content of the UDF?

    2011/6/18 勇胡 <yongyong313@gmail.com>
    Hello,

    I just tried the example from the pig udf manual step by step. But
    I
    got
    the
    error information. Can anyone tell me how to solve it?

    grunt> REGISTER /home/huyong/test/myudfs.jar;
    grunt> A = LOAD '/home/huyong/test/student.txt' as
    (name:chararray);
    grunt> B = FOREACH A GENERATE myudfs.UPPER(name);
    2011-06-18 11:15:38,892 [main] ERROR
    org.apache.pig.tools.grunt.Grunt
    -
    ERROR 1070: Could not resolve myudfs.UPPER using imports: [,
    org.apache.pig.builtin., org.apache.pig.impl.builtin.]
    Details at logfile: /home/huyong/test/pig_1308388238352.log

    I have already registered the udf, why pig tries to search from the builtin
    path.

    Thanks for your help!

    Yong Hu
  • 勇胡 at Jun 20, 2011 at 7:16 am
    Thanks for your help. I didn't even notice that. Now it works.

    yong

    2011/6/19 Dmitriy Ryaboy <dvryaboy@gmail.com>
    It's even worse than just the missing package name.

    When you create a jar, its root is supposed to be the root of your package
    tree.

    So a class "com.foo.Bar" has to be in "com/foo/Bar.class" when you run
    jar tf on the jarfile. If it's in "home/me/src/com/foo/Bar.class" or
    just in "Bar.class" or in "something/else/Bar.class" java won't find
    it.

    D

    2011/6/19 Dexin Wang <wangdexin@gmail.com>:
    Looks like you need to learn a bit about how java package works. The path
    for the class file needs to be

    /home/huyong/test/*myudfs*/UPPER.class

    Or remove the first line in your UDF code "package myudfs;" and redo
    everything you did, it will work.

    2011/6/19 勇胡 <yongyong313@gmail.com>
    In this path, it has already contained UPPER.class and UPPER.java. And
    the
    myudfs.jar is also in the path. But it did not work. By the way, my Pig
    version is 0.8.1. Is there something, for example, classpath I have to
    set
    up in the Pig configuration?

    Yong

    在 2011年6月19日 上午4:10,Dexin Wang <wangdexin@gmail.com>写道:
    You need to have your class file in this path

    /home/huyong/test/myudfs/UPPER.class

    since it's in myudfs directory.

    On Jun 18, 2011, at 12:33 PM, 勇胡 wrote:

    I tried your command and then it shows me as following:
    /home/huyong/test/UPPER.class
    /home/huyong/test/UPPER.java

    Yong
    在 2011年6月18日 下午4:29,Dmitriy Ryaboy <dvryaboy@gmail.com>写道:
    This usually hapens when you aren't registering what you think you
    are
    registering.
    try `jar tf /home/huyong/test/myudfs.jar | grep UPPER` and see if
    you
    get anything.

    D

    2011/6/18 勇胡 <yongyong313@gmail.com>:
    Hi,

    package myudfs;
    import java.io.IOException;
    import org.apache.pig.EvalFunc;
    import org.apache.pig.data.Tuple;
    import org.apache.pig.impl.util.*;

    public class UPPER extends EvalFunc<String>
    {
    public String exec(Tuple input) throws IOException {
    if (input == null || input.size() == 0)
    return null;
    try{
    String str = (String)input.get(0);
    return str.toUpperCase();
    }catch(Exception e){
    throw new IOException(e);
    }
    }
    }

    This is as same as the example from the Pig website. By the way, I
    also
    added the PIG_CLASS. But it still didn't work.

    Yong

    2011/6/18 Jonathan Coveney <jcoveney@gmail.com>
    Can you paste the content of the UDF?

    2011/6/18 勇胡 <yongyong313@gmail.com>
    Hello,

    I just tried the example from the pig udf manual step by step.
    But
    I
    got
    the
    error information. Can anyone tell me how to solve it?

    grunt> REGISTER /home/huyong/test/myudfs.jar;
    grunt> A = LOAD '/home/huyong/test/student.txt' as
    (name:chararray);
    grunt> B = FOREACH A GENERATE myudfs.UPPER(name);
    2011-06-18 11:15:38,892 [main] ERROR
    org.apache.pig.tools.grunt.Grunt
    -
    ERROR 1070: Could not resolve myudfs.UPPER using imports: [,
    org.apache.pig.builtin., org.apache.pig.impl.builtin.]
    Details at logfile: /home/huyong/test/pig_1308388238352.log

    I have already registered the udf, why pig tries to search from
    the
    builtin
    path.

    Thanks for your help!

    Yong Hu

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriespig, hadoop
postedJun 18, '11 at 9:23a
activeJun 20, '11 at 7:16a
posts10
users4
websitepig.apache.org

People

Translate

site design / logo © 2021 Grokbase