Grokbase Groups Pig user May 2011
FAQ
Hi, I have a sequence of jobs which are run daily and usually the logs
and results are erased every time they have to be re-run. Now we want
to keep those logs and results, but if the results already exist, the
pig job fails. I thought that maybe setting the results' name + date
would solve it for me. Can I do that from pig? Do you guys have any
other suggestions?
Thanks in advance.


Renato M.

Search Discussions

  • Dexin Wang at May 21, 2011 at 2:45 am
    Yeah I do that all the time.

    STORE result INTO 'out-$date';

    Or you could run the pig script then after it's done move the result aside.


    On May 20, 2011, at 6:51 PM, Renato Marroquín Mogrovejowrote:
    Hi, I have a sequence of jobs which are run daily and usually the logs
    and results are erased every time they have to be re-run. Now we want
    to keep those logs and results, but if the results already exist, the
    pig job fails. I thought that maybe setting the results' name + date
    would solve it for me. Can I do that from pig? Do you guys have any
    other suggestions?
    Thanks in advance.


    Renato M.
  • Renato Marroquín Mogrovejo at May 21, 2011 at 8:16 pm
    Thanks Dexin! I tried that but that did not work for me. I am using
    Pig 0.7, what version of Pig do you use?
    Yeah I think I could move it aside, but the problem is that I need to
    keep track of the results, and if I move them aside then I would have
    to rename the results of each job sequentially because my jobs can
    repeat many times, but their results are different.
    Thanks again.

    Renato M.

    2011/5/20 Dexin Wang <wangdexin@gmail.com>:
    Yeah I do that all the time.

    STORE result INTO 'out-$date';

    Or you could run the pig script then after it's done move the result aside.


    On May 20, 2011, at 6:51 PM, Renato Marroquín Mogrovejowrote:
    Hi, I have a sequence of jobs which are run daily and usually the logs
    and results are erased every time they have to be re-run. Now we want
    to keep those logs and results, but if the results already exist, the
    pig job fails. I thought that maybe setting the results' name + date
    would solve it for me. Can I do that from pig? Do you guys have any
    other suggestions?
    Thanks in advance.


    Renato M.
  • Dexin Wang at May 23, 2011 at 4:19 pm
    I don't think version is a problem. variables is probably supported from the
    start of the Pig.

    Using

    STORE result INTO 'out-$date';

    I mentioned about, when you run the pig script, you just add "-param
    date=20110522'" to your command line. What is the problem you see?

    2011/5/21 Renato Marroquín Mogrovejo <renatoj.marroquin@gmail.com>
    Thanks Dexin! I tried that but that did not work for me. I am using
    Pig 0.7, what version of Pig do you use?
    Yeah I think I could move it aside, but the problem is that I need to
    keep track of the results, and if I move them aside then I would have
    to rename the results of each job sequentially because my jobs can
    repeat many times, but their results are different.
    Thanks again.

    Renato M.

    2011/5/20 Dexin Wang <wangdexin@gmail.com>:
    Yeah I do that all the time.

    STORE result INTO 'out-$date';

    Or you could run the pig script then after it's done move the result aside.

    On May 20, 2011, at 6:51 PM, Renato Marroquín Mogrovejo<
    renatoj.marroquin@gmail.com> wrote:
    Hi, I have a sequence of jobs which are run daily and usually the logs
    and results are erased every time they have to be re-run. Now we want
    to keep those logs and results, but if the results already exist, the
    pig job fails. I thought that maybe setting the results' name + date
    would solve it for me. Can I do that from pig? Do you guys have any
    other suggestions?
    Thanks in advance.


    Renato M.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriespig, hadoop
postedMay 21, '11 at 1:51a
activeMay 23, '11 at 4:19p
posts4
users2
websitepig.apache.org

People

Translate

site design / logo © 2022 Grokbase