Grokbase Groups Pig dev October 2010
FAQ
Performance: Use System.arraycopy() instead of manually copying the bytes while reading the data
------------------------------------------------------------------------------------------------

Key: PIG-1696
URL: https://issues.apache.org/jira/browse/PIG-1696
Project: Pig
Issue Type: Improvement
Reporter: Ashutosh Chauhan
Priority: Minor


System.arraycopy() is said to be faster as compared to iterating over bytes and copying them over as it is implemented natively in JVM. Since every single byte read through PigStorage() will go through this code-path, this will have a performance impact.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Ashutosh Chauhan (JIRA) at Oct 23, 2010 at 8:29 am
    [ https://issues.apache.org/jira/browse/PIG-1696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Ashutosh Chauhan updated PIG-1696:
    ----------------------------------

    Attachment: dba_syscp.patch

    Attached patch implements the change.
    Performance: Use System.arraycopy() instead of manually copying the bytes while reading the data
    ------------------------------------------------------------------------------------------------

    Key: PIG-1696
    URL: https://issues.apache.org/jira/browse/PIG-1696
    Project: Pig
    Issue Type: Improvement
    Reporter: Ashutosh Chauhan
    Priority: Minor
    Attachments: dba_syscp.patch


    System.arraycopy() is said to be faster as compared to iterating over bytes and copying them over as it is implemented natively in JVM. Since every single byte read through PigStorage() will go through this code-path, this will have a performance impact.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Ashutosh Chauhan (JIRA) at Oct 28, 2010 at 3:37 am
    [ https://issues.apache.org/jira/browse/PIG-1696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Ashutosh Chauhan updated PIG-1696:
    ----------------------------------

    Fix Version/s: 0.9.0
    Assignee: Ashutosh Chauhan
    Status: Patch Available (was: Open)

    This requires no new tests as it changes only the internal implementation. Existing testcases covers the functionality provided. They are in TestDataByteArray.java Additionally, PigStorage heavily uses this, so all the tests using PigStorage provides additional coverage. I ran all the unit tests. All of them passes. This is ready for review.
    Performance: Use System.arraycopy() instead of manually copying the bytes while reading the data
    ------------------------------------------------------------------------------------------------

    Key: PIG-1696
    URL: https://issues.apache.org/jira/browse/PIG-1696
    Project: Pig
    Issue Type: Improvement
    Reporter: Ashutosh Chauhan
    Assignee: Ashutosh Chauhan
    Priority: Minor
    Fix For: 0.9.0

    Attachments: dba_syscp.patch


    System.arraycopy() is said to be faster as compared to iterating over bytes and copying them over as it is implemented natively in JVM. Since every single byte read through PigStorage() will go through this code-path, this will have a performance impact.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Daniel Dai (JIRA) at Oct 28, 2010 at 10:20 pm
    [ https://issues.apache.org/jira/browse/PIG-1696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12925969#action_12925969 ]

    Daniel Dai commented on PIG-1696:
    ---------------------------------

    +1
    Performance: Use System.arraycopy() instead of manually copying the bytes while reading the data
    ------------------------------------------------------------------------------------------------

    Key: PIG-1696
    URL: https://issues.apache.org/jira/browse/PIG-1696
    Project: Pig
    Issue Type: Improvement
    Reporter: Ashutosh Chauhan
    Assignee: Ashutosh Chauhan
    Priority: Minor
    Fix For: 0.9.0

    Attachments: dba_syscp.patch


    System.arraycopy() is said to be faster as compared to iterating over bytes and copying them over as it is implemented natively in JVM. Since every single byte read through PigStorage() will go through this code-path, this will have a performance impact.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Ashutosh Chauhan (JIRA) at Oct 29, 2010 at 10:34 am
    [ https://issues.apache.org/jira/browse/PIG-1696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Ashutosh Chauhan updated PIG-1696:
    ----------------------------------

    Resolution: Fixed
    Status: Resolved (was: Patch Available)

    Patch committed to trunk.
    Performance: Use System.arraycopy() instead of manually copying the bytes while reading the data
    ------------------------------------------------------------------------------------------------

    Key: PIG-1696
    URL: https://issues.apache.org/jira/browse/PIG-1696
    Project: Pig
    Issue Type: Improvement
    Reporter: Ashutosh Chauhan
    Assignee: Ashutosh Chauhan
    Priority: Minor
    Fix For: 0.9.0

    Attachments: dba_syscp.patch


    System.arraycopy() is said to be faster as compared to iterating over bytes and copying them over as it is implemented natively in JVM. Since every single byte read through PigStorage() will go through this code-path, this will have a performance impact.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupdev @
categoriespig, hadoop
postedOct 23, '10 at 8:27a
activeOct 29, '10 at 10:34a
posts5
users1
websitepig.apache.org

1 user in discussion

Ashutosh Chauhan (JIRA): 5 posts

People

Translate

site design / logo © 2022 Grokbase