FAQ
Guys, I'm seeing this one

2998

Unexpected internal error.


Can we be more specific or dump a stack trace when this happens?

Search Discussions

  • Jeff Zhang at Sep 30, 2010 at 3:13 am
    No other stack trace ? And in what situation does this happen ?


    On Thu, Sep 30, 2010 at 11:09 AM, hc busy wrote:
    Guys, I'm seeing this one

    2998

    Unexpected internal error.


    Can we be more specific or dump a stack trace when this happens?


    --
    Best Regards

    Jeff Zhang
  • Hc busy at Sep 30, 2010 at 3:19 am
    "null" was the error. this 60k PigLatin script that I'm running hasn't
    changed that much, but suddenly started erroring out. I've rebuilt pig
    release 7 from scratch, checked java version, err... checked PiggyBank and
    our own libraries, not there changed.

    You know, some comercial software that has professional support will
    actually send emails when an "unhandled error" occurs. The email is received
    by the developer/support and diagnosed. And in that email, it would contain
    all the gory details that the product doesn't want to display to the user.


    I wonder if you guys are up to doing something like that for pig?


    On Wed, Sep 29, 2010 at 8:13 PM, Jeff Zhang wrote:

    No other stack trace ? And in what situation does this happen ?


    On Thu, Sep 30, 2010 at 11:09 AM, hc busy wrote:
    Guys, I'm seeing this one

    2998

    Unexpected internal error.


    Can we be more specific or dump a stack trace when this happens?


    --
    Best Regards

    Jeff Zhang
  • Hc busy at Sep 30, 2010 at 3:21 am
    But having complained about error, I want to say this seems like a
    really courteous error message


    ERROR 1007: Found duplicates in schema. : 2 columns, column_name: 2 columns.
    *Please* alias the columns with unique names.


    Where it explains nicely how to fix the error.


    On Wed, Sep 29, 2010 at 8:18 PM, hc busy wrote:

    "null" was the error. this 60k PigLatin script that I'm running hasn't
    changed that much, but suddenly started erroring out. I've rebuilt pig
    release 7 from scratch, checked java version, err... checked PiggyBank and
    our own libraries, not there changed.

    You know, some comercial software that has professional support will
    actually send emails when an "unhandled error" occurs. The email is received
    by the developer/support and diagnosed. And in that email, it would contain
    all the gory details that the product doesn't want to display to the user.


    I wonder if you guys are up to doing something like that for pig?


    On Wed, Sep 29, 2010 at 8:13 PM, Jeff Zhang wrote:

    No other stack trace ? And in what situation does this happen ?


    On Thu, Sep 30, 2010 at 11:09 AM, hc busy wrote:
    Guys, I'm seeing this one

    2998

    Unexpected internal error.


    Can we be more specific or dump a stack trace when this happens?


    --
    Best Regards

    Jeff Zhang
  • Santhosh Srinivasan at Sep 30, 2010 at 5:33 am
    Are you sure that the 2998 and 1007 both popped up?

    -----Original Message-----
    From: hc busy
    Sent: Wednesday, September 29, 2010 8:21 PM
    To: pig-user@hadoop.apache.org
    Subject: Re: funny error

    But having complained about error, I want to say this seems like a really courteous error message


    ERROR 1007: Found duplicates in schema. : 2 columns, column_name: 2 columns.
    *Please* alias the columns with unique names.


    Where it explains nicely how to fix the error.


    On Wed, Sep 29, 2010 at 8:18 PM, hc busy wrote:

    "null" was the error. this 60k PigLatin script that I'm running hasn't
    changed that much, but suddenly started erroring out. I've rebuilt pig
    release 7 from scratch, checked java version, err... checked PiggyBank
    and our own libraries, not there changed.

    You know, some comercial software that has professional support will
    actually send emails when an "unhandled error" occurs. The email is
    received by the developer/support and diagnosed. And in that email, it
    would contain all the gory details that the product doesn't want to display to the user.


    I wonder if you guys are up to doing something like that for pig?


    On Wed, Sep 29, 2010 at 8:13 PM, Jeff Zhang wrote:

    No other stack trace ? And in what situation does this happen ?


    On Thu, Sep 30, 2010 at 11:09 AM, hc busy wrote:
    Guys, I'm seeing this one

    2998

    Unexpected internal error.


    Can we be more specific or dump a stack trace when this happens?


    --
    Best Regards

    Jeff Zhang
  • Dmitriy Ryaboy at Sep 30, 2010 at 9:02 pm
    Yeah, sure, as soon as we all quit our respective jobs and start a support
    company :)

    (on a more serious note, improving the error messages is a big item for Pig
    0.9)

    (also, good god, 60k line Pig script. Look into workflow management tools
    that allow you to avid creating such monoliths).
    On Wed, Sep 29, 2010 at 8:18 PM, hc busy wrote:

    "null" was the error. this 60k PigLatin script that I'm running hasn't
    changed that much, but suddenly started erroring out. I've rebuilt pig
    release 7 from scratch, checked java version, err... checked PiggyBank and
    our own libraries, not there changed.

    You know, some comercial software that has professional support will
    actually send emails when an "unhandled error" occurs. The email is
    received
    by the developer/support and diagnosed. And in that email, it would contain
    all the gory details that the product doesn't want to display to the user.


    I wonder if you guys are up to doing something like that for pig?


    On Wed, Sep 29, 2010 at 8:13 PM, Jeff Zhang wrote:

    No other stack trace ? And in what situation does this happen ?


    On Thu, Sep 30, 2010 at 11:09 AM, hc busy wrote:
    Guys, I'm seeing this one

    2998

    Unexpected internal error.


    Can we be more specific or dump a stack trace when this happens?


    --
    Best Regards

    Jeff Zhang
  • Hc busy at Sep 30, 2010 at 9:32 pm
    Well, that's a great point! The only reason why I'm creating a large 60k
    pigscript to run (among over half a dozen people adding/modifying scripts)
    is that when pig compiles script, it optimizes to give the best performance.

    I mean, otherwise we'd actually have to manually create optimized pieces of
    scripts that have clearly defined interface in the form of an HDFS file. But
    even then, I don't believe it will run as fast as a pig script concatenation
    of all of the scripts.

    (btw, can somebody confirm this fact for me?)


    That's why when I actually run the script on a full set of data, it is a
    huge piece of script post concatenation, marco expansion, var substitution,
    etc., etc., etc... rendering it even more like assembly language. I mean the
    alias names are like


    temp_count_group_by_group_b_sales_by_metro_month_1

    And then I freak out when I see something like the 2998 error. My eyes glaze
    over, and I'm like, okay, it's time to email pig-user group... And to answer
    the other question, I eventually got to the duplicate alias after rebuilding
    everything yet again and redeploying. My current hypothesis is that maybe an
    scp failed and I didn't see the error message and one of our own libraries
    was corrupt ?

    It might have been an UDF that we're using that crashed it...

    if only I had the stacktrace...



    On Thu, Sep 30, 2010 at 2:02 PM, Dmitriy Ryaboy wrote:

    Yeah, sure, as soon as we all quit our respective jobs and start a support
    company :)

    (on a more serious note, improving the error messages is a big item for Pig
    0.9)

    (also, good god, 60k line Pig script. Look into workflow management tools
    that allow you to avid creating such monoliths).
    On Wed, Sep 29, 2010 at 8:18 PM, hc busy wrote:

    "null" was the error. this 60k PigLatin script that I'm running hasn't
    changed that much, but suddenly started erroring out. I've rebuilt pig
    release 7 from scratch, checked java version, err... checked PiggyBank and
    our own libraries, not there changed.

    You know, some comercial software that has professional support will
    actually send emails when an "unhandled error" occurs. The email is
    received
    by the developer/support and diagnosed. And in that email, it would contain
    all the gory details that the product doesn't want to display to the user.

    I wonder if you guys are up to doing something like that for pig?


    On Wed, Sep 29, 2010 at 8:13 PM, Jeff Zhang wrote:

    No other stack trace ? And in what situation does this happen ?


    On Thu, Sep 30, 2010 at 11:09 AM, hc busy wrote:
    Guys, I'm seeing this one

    2998

    Unexpected internal error.


    Can we be more specific or dump a stack trace when this happens?


    --
    Best Regards

    Jeff Zhang
  • Thejas M Nair at Sep 30, 2010 at 11:13 pm
    If it crashed while executing MR job , you might have more information in the logs of the failed hadoop tasks.
    Its possible that for some reason (bug?) pig might have failed to get the complete error from hadoop task.
    -Thejas



    On 9/30/10 2:32 PM, "hc busy" wrote:

    Well, that's a great point! The only reason why I'm creating a large 60k
    pigscript to run (among over half a dozen people adding/modifying scripts)
    is that when pig compiles script, it optimizes to give the best performance.

    I mean, otherwise we'd actually have to manually create optimized pieces of
    scripts that have clearly defined interface in the form of an HDFS file. But
    even then, I don't believe it will run as fast as a pig script concatenation
    of all of the scripts.

    (btw, can somebody confirm this fact for me?)


    That's why when I actually run the script on a full set of data, it is a
    huge piece of script post concatenation, marco expansion, var substitution,
    etc., etc., etc... rendering it even more like assembly language. I mean the
    alias names are like


    temp_count_group_by_group_b_sales_by_metro_month_1

    And then I freak out when I see something like the 2998 error. My eyes glaze
    over, and I'm like, okay, it's time to email pig-user group... And to answer
    the other question, I eventually got to the duplicate alias after rebuilding
    everything yet again and redeploying. My current hypothesis is that maybe an
    scp failed and I didn't see the error message and one of our own libraries
    was corrupt ?

    It might have been an UDF that we're using that crashed it...

    if only I had the stacktrace...



    On Thu, Sep 30, 2010 at 2:02 PM, Dmitriy Ryaboy wrote:

    Yeah, sure, as soon as we all quit our respective jobs and start a support
    company :)

    (on a more serious note, improving the error messages is a big item for Pig
    0.9)

    (also, good god, 60k line Pig script. Look into workflow management tools
    that allow you to avid creating such monoliths).
    On Wed, Sep 29, 2010 at 8:18 PM, hc busy wrote:

    "null" was the error. this 60k PigLatin script that I'm running hasn't
    changed that much, but suddenly started erroring out. I've rebuilt pig
    release 7 from scratch, checked java version, err... checked PiggyBank and
    our own libraries, not there changed.

    You know, some comercial software that has professional support will
    actually send emails when an "unhandled error" occurs. The email is
    received
    by the developer/support and diagnosed. And in that email, it would contain
    all the gory details that the product doesn't want to display to the user.

    I wonder if you guys are up to doing something like that for pig?


    On Wed, Sep 29, 2010 at 8:13 PM, Jeff Zhang wrote:

    No other stack trace ? And in what situation does this happen ?


    On Thu, Sep 30, 2010 at 11:09 AM, hc busy wrote:
    Guys, I'm seeing this one

    2998

    Unexpected internal error.


    Can we be more specific or dump a stack trace when this happens?


    --
    Best Regards

    Jeff Zhang

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriespig, hadoop
postedSep 30, '10 at 3:10a
activeSep 30, '10 at 11:13p
posts8
users5
websitepig.apache.org

People

Translate

site design / logo © 2021 Grokbase