Grokbase Groups Pig user July 2010
FAQ
Hi all,

I have a strange issue with data types. We have a custom loader which
loads data from logs similar to apache logs.
It sets the type of a field(var) to be int. When I examine its type, it
shows it to be int(using describe). However, when I do a numeric
comparison (for example: var==200) it throws an error.

java.lang.ClassCastException: java.lang.Integer cannot be cast to
java.lang.String
at java.lang.String.compareTo(String.java:92)

And when a string comparison is done (using var == '200'), it again
throws an error.

ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1039: Incompatible types
in EqualTo Operator left hand side:int right hand side:chararray

When I explicitly type cast or use a UDF to convert it from string to
integer or long and then do the numeric comparison, it works fine.

I searched if there are any previously such issues have been reported
before, but I couldnt find anything related.
It would be great if any one could throw more light on this if I am
missing something here.

Thanks,
-Rohini

Search Discussions

  • Mridul Muralidharan at Jul 29, 2010 at 10:54 pm
    Are you returning the appropriate schema and using the correct schema in
    the pig script ?
    More info might help though !

    Regards,
    Mridul
    On Thursday 29 July 2010 09:16 PM, Uppuluri, Rohini wrote:
    Hi all,

    I have a strange issue with data types. We have a custom loader which
    loads data from logs similar to apache logs.
    It sets the type of a field(var) to be int. When I examine its type, it
    shows it to be int(using describe). However, when I do a numeric
    comparison (for example: var==200) it throws an error.

    java.lang.ClassCastException: java.lang.Integer cannot be cast to
    java.lang.String
    at java.lang.String.compareTo(String.java:92)

    And when a string comparison is done (using var == '200'), it again
    throws an error.

    ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1039: Incompatible types
    in EqualTo Operator left hand side:int right hand side:chararray

    When I explicitly type cast or use a UDF to convert it from string to
    integer or long and then do the numeric comparison, it works fine.

    I searched if there are any previously such issues have been reported
    before, but I couldnt find anything related.
    It would be great if any one could throw more light on this if I am
    missing something here.

    Thanks,
    -Rohini
  • Mridul Muralidharan at Jul 30, 2010 at 7:39 am
    Can you try explicitly using the schema in the load ?
    raw = LOAD '$inputPath' using my_custom_loader() AS (..., myint:int, ...);


    I think pig will flag a schema mismatch if the declared schema does not
    match what is returned by loader.


    Regards,
    Mridul

    On Friday 30 July 2010 11:21 AM, Uppuluri, Rohini wrote:
    Hi,

    The schema is being set as follows:
    s.add(new Schema.FieldSchema("var", DataType.INTEGER));

    In the pig script, using describe, I am getting the schema back as
    raw: { ...... var:int, ......}

    An excerpt from the pig script is as follows:
    ----
    raw = LOAD '$inputPath' using my_custom_loader();
    describe raw;
    ------


    Regards,
    -Rohini


    -----Original Message-----
    From: Mridul Muralidharan
    Sent: Friday, July 30, 2010 4:24 AM
    To: pig-user@hadoop.apache.org
    Cc: Uppuluri, Rohini
    Subject: Re: Pig Data types issue


    Are you returning the appropriate schema and using the correct schema in
    the pig script ?
    More info might help though !

    Regards,
    Mridul
    On Thursday 29 July 2010 09:16 PM, Uppuluri, Rohini wrote:
    Hi all,

    I have a strange issue with data types. We have a custom loader which
    loads data from logs similar to apache logs.
    It sets the type of a field(var) to be int. When I examine its type,
    it shows it to be int(using describe). However, when I do a numeric
    comparison (for example: var==200) it throws an error.

    java.lang.ClassCastException: java.lang.Integer cannot be cast to
    java.lang.String
    at java.lang.String.compareTo(String.java:92)

    And when a string comparison is done (using var == '200'), it again
    throws an error.

    ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1039: Incompatible
    types in EqualTo Operator left hand side:int right hand side:chararray

    When I explicitly type cast or use a UDF to convert it from string to
    integer or long and then do the numeric comparison, it works fine.

    I searched if there are any previously such issues have been reported
    before, but I couldnt find anything related.
    It would be great if any one could throw more light on this if I am
    missing something here.

    Thanks,
    -Rohini

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriespig, hadoop
postedJul 29, '10 at 7:18p
activeJul 30, '10 at 7:39a
posts3
users2
websitepig.apache.org

People

Translate

site design / logo © 2021 Grokbase