Grokbase Groups Pig dev August 2008
FAQ
Porting of the slicer into types branch caused all scripts to run just one map.
-------------------------------------------------------------------------------

Key: PIG-403
URL: https://issues.apache.org/jira/browse/PIG-403
Project: Pig
Issue Type: Bug
Affects Versions: types_branch
Reporter: Alan Gates
Assignee: Alan Gates
Priority: Critical
Fix For: types_branch


In PigSlicer.java, splittable is set to false by default. This causes the system to scan an entire file in a single map. It should be set to true by default, so that files are only scanned in a single map if the user explicitly requests it.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Alan Gates (JIRA) at Aug 28, 2008 at 8:38 pm
    [ https://issues.apache.org/jira/browse/PIG-403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Alan Gates updated PIG-403:
    ---------------------------

    Status: Patch Available (was: Open)
    Porting of the slicer into types branch caused all scripts to run just one map.
    -------------------------------------------------------------------------------

    Key: PIG-403
    URL: https://issues.apache.org/jira/browse/PIG-403
    Project: Pig
    Issue Type: Bug
    Affects Versions: types_branch
    Reporter: Alan Gates
    Assignee: Alan Gates
    Priority: Critical
    Fix For: types_branch

    Attachments: PIG-403.patch


    In PigSlicer.java, splittable is set to false by default. This causes the system to scan an entire file in a single map. It should be set to true by default, so that files are only scanned in a single map if the user explicitly requests it.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Alan Gates (JIRA) at Aug 28, 2008 at 8:38 pm
    [ https://issues.apache.org/jira/browse/PIG-403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Alan Gates updated PIG-403:
    ---------------------------

    Attachment: PIG-403.patch

    Changed PigSlicer.splittable default to true from false.
    Porting of the slicer into types branch caused all scripts to run just one map.
    -------------------------------------------------------------------------------

    Key: PIG-403
    URL: https://issues.apache.org/jira/browse/PIG-403
    Project: Pig
    Issue Type: Bug
    Affects Versions: types_branch
    Reporter: Alan Gates
    Assignee: Alan Gates
    Priority: Critical
    Fix For: types_branch

    Attachments: PIG-403.patch


    In PigSlicer.java, splittable is set to false by default. This causes the system to scan an entire file in a single map. It should be set to true by default, so that files are only scanned in a single map if the user explicitly requests it.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Pradeep Kamath (JIRA) at Aug 28, 2008 at 8:56 pm
    [ https://issues.apache.org/jira/browse/PIG-403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12626735#action_12626735 ]

    Pradeep Kamath commented on PIG-403:
    ------------------------------------

    Review comments:
    I think this one line change should be good - "splittable" field is used in the slice() method in PigSlicer and that looks at filename (to see if it ends with .gz) or if splittable is set to false. The only way we allow splittable to set is through the setSplittable() which is not being called currently. I will be changing PigInputFormat to call setSplittable(false) when the script has "SPLIT by file" as part of the patch for streaming
    Porting of the slicer into types branch caused all scripts to run just one map.
    -------------------------------------------------------------------------------

    Key: PIG-403
    URL: https://issues.apache.org/jira/browse/PIG-403
    Project: Pig
    Issue Type: Bug
    Affects Versions: types_branch
    Reporter: Alan Gates
    Assignee: Alan Gates
    Priority: Critical
    Fix For: types_branch

    Attachments: PIG-403.patch


    In PigSlicer.java, splittable is set to false by default. This causes the system to scan an entire file in a single map. It should be set to true by default, so that files are only scanned in a single map if the user explicitly requests it.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Alan Gates (JIRA) at Aug 28, 2008 at 9:08 pm
    [ https://issues.apache.org/jira/browse/PIG-403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Alan Gates updated PIG-403:
    ---------------------------

    Resolution: Fixed
    Status: Resolved (was: Patch Available)

    Patch checked in.
    Porting of the slicer into types branch caused all scripts to run just one map.
    -------------------------------------------------------------------------------

    Key: PIG-403
    URL: https://issues.apache.org/jira/browse/PIG-403
    Project: Pig
    Issue Type: Bug
    Affects Versions: types_branch
    Reporter: Alan Gates
    Assignee: Alan Gates
    Priority: Critical
    Fix For: types_branch

    Attachments: PIG-403.patch


    In PigSlicer.java, splittable is set to false by default. This causes the system to scan an entire file in a single map. It should be set to true by default, so that files are only scanned in a single map if the user explicitly requests it.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupdev @
categoriespig, hadoop
postedAug 28, '08 at 8:00p
activeAug 28, '08 at 9:08p
posts5
users1
websitepig.apache.org

1 user in discussion

Alan Gates (JIRA): 5 posts

People

Translate

site design / logo © 2022 Grokbase