FAQ
Problem when schema contains more columns that actual data
----------------------------------------------------------

Key: PIG-446
URL: https://issues.apache.org/jira/browse/PIG-446
Project: Pig
Issue Type: Bug
Reporter: Olga Natkovich
Assignee: Alan Gates


Script:

-- extra1 and extra2 are not present in the data
A = load 'data' as (name, age: int, gpa, extra1, extra2);
B = limit A 10;
dump B;

Error:

Out of bounds access: Request for field number 3 exceeds tuple size of 3
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:223)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:121)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:391)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)



--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Olga Natkovich (JIRA) at Sep 23, 2008 at 3:43 am
    [ https://issues.apache.org/jira/browse/PIG-446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Olga Natkovich updated PIG-446:
    -------------------------------

    Fix Version/s: types_branch
    Affects Version/s: types_branch
    Problem when schema contains more columns that actual data
    ----------------------------------------------------------

    Key: PIG-446
    URL: https://issues.apache.org/jira/browse/PIG-446
    Project: Pig
    Issue Type: Bug
    Affects Versions: types_branch
    Reporter: Olga Natkovich
    Assignee: Alan Gates
    Fix For: types_branch


    Script:
    -- extra1 and extra2 are not present in the data
    A = load 'data' as (name, age: int, gpa, extra1, extra2);
    B = limit A 10;
    dump B;
    Error:
    Out of bounds access: Request for field number 3 exceeds tuple size of 3
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:223)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:121)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:391)
    at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Alan Gates (JIRA) at Sep 23, 2008 at 5:19 am
    [ https://issues.apache.org/jira/browse/PIG-446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Alan Gates updated PIG-446:
    ---------------------------

    Status: Patch Available (was: Open)
    Problem when schema contains more columns that actual data
    ----------------------------------------------------------

    Key: PIG-446
    URL: https://issues.apache.org/jira/browse/PIG-446
    Project: Pig
    Issue Type: Bug
    Affects Versions: types_branch
    Reporter: Olga Natkovich
    Assignee: Alan Gates
    Fix For: types_branch

    Attachments: PIG-446.patch


    Script:
    -- extra1 and extra2 are not present in the data
    A = load 'data' as (name, age: int, gpa, extra1, extra2);
    B = limit A 10;
    dump B;
    Error:
    Out of bounds access: Request for field number 3 exceeds tuple size of 3
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:223)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:121)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:391)
    at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Alan Gates (JIRA) at Sep 23, 2008 at 5:19 am
    [ https://issues.apache.org/jira/browse/PIG-446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Alan Gates updated PIG-446:
    ---------------------------

    Attachment: PIG-446.patch

    Changed POProject to catch exceptions from Tuple.get(int) and insert a null instead of pass the exceptions along.
    Problem when schema contains more columns that actual data
    ----------------------------------------------------------

    Key: PIG-446
    URL: https://issues.apache.org/jira/browse/PIG-446
    Project: Pig
    Issue Type: Bug
    Affects Versions: types_branch
    Reporter: Olga Natkovich
    Assignee: Alan Gates
    Fix For: types_branch

    Attachments: PIG-446.patch


    Script:
    -- extra1 and extra2 are not present in the data
    A = load 'data' as (name, age: int, gpa, extra1, extra2);
    B = limit A 10;
    dump B;
    Error:
    Out of bounds access: Request for field number 3 exceeds tuple size of 3
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:223)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:121)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:391)
    at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Olga Natkovich (JIRA) at Sep 23, 2008 at 4:12 pm
    [ https://issues.apache.org/jira/browse/PIG-446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12633781#action_12633781 ]

    Olga Natkovich commented on PIG-446:
    ------------------------------------

    +1. My only concer is that if there is enough data like this we will be logging a lot. Buit we already doing this with UDFs so at least this is consistent. Once we address logging we could aggregate warnings.
    Problem when schema contains more columns that actual data
    ----------------------------------------------------------

    Key: PIG-446
    URL: https://issues.apache.org/jira/browse/PIG-446
    Project: Pig
    Issue Type: Bug
    Affects Versions: types_branch
    Reporter: Olga Natkovich
    Assignee: Alan Gates
    Fix For: types_branch

    Attachments: PIG-446.patch


    Script:
    -- extra1 and extra2 are not present in the data
    A = load 'data' as (name, age: int, gpa, extra1, extra2);
    B = limit A 10;
    dump B;
    Error:
    Out of bounds access: Request for field number 3 exceeds tuple size of 3
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:223)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:121)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:391)
    at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Alan Gates (JIRA) at Sep 23, 2008 at 7:30 pm
    [ https://issues.apache.org/jira/browse/PIG-446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Alan Gates updated PIG-446:
    ---------------------------

    Resolution: Fixed
    Status: Resolved (was: Patch Available)

    Patch checked in.
    Problem when schema contains more columns that actual data
    ----------------------------------------------------------

    Key: PIG-446
    URL: https://issues.apache.org/jira/browse/PIG-446
    Project: Pig
    Issue Type: Bug
    Affects Versions: types_branch
    Reporter: Olga Natkovich
    Assignee: Alan Gates
    Fix For: types_branch

    Attachments: PIG-446.patch


    Script:
    -- extra1 and extra2 are not present in the data
    A = load 'data' as (name, age: int, gpa, extra1, extra2);
    B = limit A 10;
    dump B;
    Error:
    Out of bounds access: Request for field number 3 exceeds tuple size of 3
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:223)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:121)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:391)
    at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupdev @
categoriespig, hadoop
postedSep 22, '08 at 11:41p
activeSep 23, '08 at 7:30p
posts6
users1
websitepig.apache.org

1 user in discussion

Alan Gates (JIRA): 6 posts

People

Translate

site design / logo © 2022 Grokbase