FAQ
Guys,

Does Pig read the _log directories from an output script ?

What I want is to read an pig output dir (or multiples) from pig scripts.
But I just want the part-XXXX files not the .part-crc or _logs files.

Thanks

--
*Charles Ferreira Gonçalves *
http://homepages.dcc.ufmg.br/~charles/
UFMG - ICEx - Dcc
Cel.: 55 31 87741485
Tel.: 55 31 34741485
Lab.: 55 31 34095840

Search Discussions

  • Ramesh, Amit at Feb 17, 2011 at 11:23 pm
    Directory names starting with underscores are ignored, but I am not certain
    about .* files/directories.

    Amit

    On 2/17/11 3:12 PM, "Charles Gonçalves" wrote:

    Guys,

    Does Pig read the _log directories from an output script ?

    What I want is to read an pig output dir (or multiples) from pig scripts.
    But I just want the part-XXXX files not the .part-crc or _logs files.

    Thanks
  • Charles Gonçalves at Feb 17, 2011 at 11:28 pm
    Files starting with .* are also ignored (hdfs does)
    https://issues.apache.org/jira/browse/PIG-950

    <https://issues.apache.org/jira/browse/PIG-950>But thanks, I didn't found
    any documentation about files starting with _ files!
    On Thu, Feb 17, 2011 at 9:23 PM, Ramesh, Amit wrote:


    Directory names starting with underscores are ignored, but I am not certain
    about .* files/directories.

    Amit

    On 2/17/11 3:12 PM, "Charles Gonçalves" wrote:

    Guys,

    Does Pig read the _log directories from an output script ?

    What I want is to read an pig output dir (or multiples) from pig scripts.
    But I just want the part-XXXX files not the .part-crc or _logs files.

    Thanks

    --
    *Charles Ferreira Gonçalves *
    http://homepages.dcc.ufmg.br/~charles/
    UFMG - ICEx - Dcc
    Cel.: 55 31 87741485
    Tel.: 55 31 34741485
    Lab.: 55 31 34095840
  • Richard Ding at Feb 17, 2011 at 11:28 pm
    Files starting with . are also ignored.

    -Richard


    On 2/17/11 3:23 PM, "Ramesh, Amit" wrote:



    Directory names starting with underscores are ignored, but I am not certain
    about .* files/directories.

    Amit

    On 2/17/11 3:12 PM, "Charles Gonçalves" wrote:

    Guys,

    Does Pig read the _log directories from an output script ?

    What I want is to read an pig output dir (or multiples) from pig scripts.
    But I just want the part-XXXX files not the .part-crc or _logs files.

    Thanks

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriespig, hadoop
postedFeb 17, '11 at 11:12p
activeFeb 17, '11 at 11:28p
posts4
users3
websitepig.apache.org

People

Translate

site design / logo © 2021 Grokbase