Search Discussions
-
Hi, I am trying to load a large gzip file and process using pig. Everytime I run the following script, I get outofmemory errors. The hadoop-site.xml is attached. The pig and the hadoop jobtracker ...
Irfan Mohammed
Sep 8, 2009 at 2:58 pm
Sep 10, 2009 at 3:56 am -
I'm having trouble thinking about to use pig to do a pairwise document similarity. If i have a huge list of word counts: (using dummy names to explain easier) doc_id, word, count doc1, testword1, ...
Tommy Chheng
Sep 9, 2009 at 8:22 pm
Sep 10, 2009 at 7:37 am -
Hi all, I'm becoming a bit more comfortable writing scripts, but still not always sure what the best way to structure/frame my statements in order to optimize performance. When it comes to Split and ...
Zaki rahaman
Sep 3, 2009 at 8:04 pm
Sep 9, 2009 at 2:35 am -
Hello, In the process of to trying to add the support for HBase 0.20.0 in PIG (trunk) I was trying the tutorial from PIG documentation: http://hadoop.apache.org/pig/docs/r0.3.0/udf.html#Custom+Slicer ...
Vincent BARAT
Sep 14, 2009 at 1:39 pm
Sep 17, 2009 at 8:08 pm -
Hi, I have a pig script reading/writing to S3. $ export PIG_OPTS="-Dfs.default.name=s3n://bucket_1/"; $ pig r0 = LOAD 'input2/transaction_ar20090909_14*' using PigStorage('\u0002'); r1 = FILTER r0 by ...
Irfan Mohammed
Sep 9, 2009 at 10:03 pm
Sep 15, 2009 at 12:21 am -
I'm writing a document similarity script. I created an inverted index and trying to create a pairwise comparison list with docs of shared words. I got my data reduced my data to : ({(233),(534)}) ...
Tommy Chheng
Sep 4, 2009 at 1:48 am
Sep 4, 2009 at 6:49 pm -
I want to read from s3 and write to local hdfs. But when I am setting -Dfs... it is setting for all load/store pig latin statements. Is there a way I can instruct pig to read from a s3 filesystem ( ...
Prasenjit mukherjee
Sep 24, 2009 at 5:53 pm
Sep 29, 2009 at 4:22 pm -
Okay... I can see why some of this would be "Removing duplicate links to the documentation per discussion on the user list." ... but why leave that vendor link there? That's already on the Pig home ...
Greg Stein
Sep 11, 2009 at 9:09 pm
Sep 11, 2009 at 9:59 pm -
Hi everyone! Recently, I updated my HDFS from 0.18.0 to 0.20.1 and installed PigPen (0.04) for Eclipse Galileo under Windows. With the older version of Hadoop (0.18) the plugin worked correctly and I ...
Alberto Luengo Cabanillas
Sep 30, 2009 at 11:07 pm
Oct 1, 2009 at 8:54 am -
Hi, I would like to use Pig Latin to process the contents of Nutch files. I understand that the Nutch crawler stores the contents of pages crawled in MapFile's, that is, a binary database. MapReduce ...
Guillermo Garrido
Sep 8, 2009 at 4:51 pm
Sep 15, 2009 at 4:25 pm -
It is probably more appropriate for hadoop forum. I am looking for a web-based debugging interface for hadoop on Amazon/EC2. Is there a way I can see ( in Amazon/EC2 ) what is the current status of ...
Prasenjit mukherjee
Sep 15, 2009 at 12:49 am
Sep 15, 2009 at 2:42 am -
Hello, I'm new to pig, I use it on MacOS, and I wonder if there is a way to avoid the double log traces in the grunt console, and if there is a way to make the ^D key work (the DEL key). I think this ...
Vincent BARAT
Sep 11, 2009 at 10:29 am
Sep 11, 2009 at 4:50 pm -
I found the following in pig-docs : A = LOAD 'data'; B = STREAM A THROUGH `stream.pl -n 5`; What does stream.pl do ? Can I get a simpler example of stream ? -Thanks, Prasenjit
Prasenjit mukherjee
Sep 7, 2009 at 6:59 pm
Sep 7, 2009 at 8:51 pm -
Hi, everyone. I am trying to use pig, but now I am confused about what the difference is between Pig and Hive, which is more powerful or more convenient for users. In addition, what is their ...
刘祥龙
Sep 8, 2009 at 2:21 pm
Sep 19, 2009 at 8:55 pm -
Hello, For those who need this, I have attached to this email a small PIG patch to support HBase 0.20.0. It can be applied on the trunk as of today. It is a minimal patch that only modifies the ...
Vincent BARAT
Sep 18, 2009 at 1:34 pm
Sep 19, 2009 at 4:58 pm -
Hi, I have a file which uses non-default delimiters. Here is an example: type::click;;date::Sun Apr 26 23:57:20 CDT 2009;; It uses '::' to delimit between the field name, and the value, and ';;' to ...
Craig Hamilton
Sep 18, 2009 at 11:35 pm
Sep 19, 2009 at 1:43 am -
I am trying to track my pig job's progress till now via inspecting hadoop/pig logs. I see the following set of logs in my hadoop logdir : 1. seemingly Hadoop's logs : datanode, jobtracker, namenode, ...
Prasenjit mukherjee
Sep 16, 2009 at 1:02 am
Sep 16, 2009 at 3:26 pm -
I would like to filter a set of rows ( of groups ) based on its arity of a specified field . Can I do this in my pig script : r1 = LOAD ...... AS f1,f2,f3; gr1 = GROUP r1 BY (f1,f2); gr1 = FILTER gr1 ...
Prasenjit mukherjee
Sep 14, 2009 at 10:29 pm
Sep 15, 2009 at 12:36 pm -
Hello, I'm trying to use pig (trunk) with hbase 0.20.0 and hadoop 0.20.0. No success so far (I'm still unable to make the TestHBaseStorage unit test run correctly, and I'm lost in the code ...
Vincent BARAT
Sep 11, 2009 at 9:19 am
Sep 11, 2009 at 4:47 pm -
I have a set of logfiles that I'm parsing and analyzing using Pig in various ways. As of right now, for each different dimension (time, geography, etc.) I am writing a new script each time to ...
Zaki rahaman
Sep 3, 2009 at 8:09 pm
Sep 8, 2009 at 6:15 pm -
Hi all, Is there a provision to turn off speculative execution in Pig similar to hadoop. If yes, can someone kindly let me know how to do that. I tried to search over the web but couldn't find any ...
Palleti, Pallavi
Sep 2, 2009 at 9:26 am
Sep 2, 2009 at 1:44 pm -
I am randomly getting the following stack trace, and having otherwise simple jobs fail. We're running the version of pig-0.4 that comes with Cloudera's new 0.20.1-based distro, so I know it's a ...
Kevin Weil
Sep 25, 2009 at 7:37 pm
Sep 25, 2009 at 8:11 pm -
Hi, As you know, a lot of work this year went into performance optimization of Pig. One of the main sources of performance problems is high memory usage. In an effort to address this problem we ...
Olga Natkovich
Sep 11, 2009 at 6:56 pm
Sep 21, 2009 at 8:01 pm -
Any idea why I get this following error? When I run pig script in local mode it works but when i run it in hadoop/mapreduce mode it throws the following error: ERROR org.apache.pig.tools.grunt.Grunt ...
Seshadri bashyam
Sep 10, 2009 at 4:22 pm
Sep 11, 2009 at 4:30 pm -
So for a given pig script 'A = LOAD 'test' USING PigStorage(',') AS (...); B = GROUP A BY (FOO,BAR); Is their a way to dump each different foo,bar group into a different directory?
Alex Newman
Sep 2, 2009 at 5:32 pm
Sep 2, 2009 at 8:17 pm -
i'm trying to run my pig script on ec2 large instance using the Cloudera 0.18 distribution.The pig script itself works in local mode on a reduced data set. The map phase went by fast but the script ...
Tommy Chheng
Sep 25, 2009 at 5:15 am
Sep 25, 2009 at 5:15 am -
Hi, all With PIG-891 checked in (credit to Jeff Zhang), Pig implements complete set of hadoop dfs commands. The syntax for dfs commands are exactly the same with hadoop (actually, Pig delegates all ...
Daniel Dai
Sep 21, 2009 at 6:13 pm
Sep 21, 2009 at 6:13 pm -
Hadoop Fans, we're getting down to the wire for Hadoop World. We couldn't be happier with how the schedule has come together. For a full list of speakers and registration details, see ...
Christophe Bisciglia
Sep 15, 2009 at 9:41 pm
Sep 15, 2009 at 9:41 pm -
Hi all. I´ve integrated the PigPen plugin in Eclipse 3.5 (Galileo) succesfully so as the Hadoop plugin (0.18.1). Also I did execute the example script 'id.pig' (the one who splits the contents from ...
Alberto Luengo Cabanillas
Sep 15, 2009 at 4:18 pm
Sep 15, 2009 at 4:18 pm -
Hi, There seems to be some confusion of why user documentation links were removed from the wiki page. The main reason is that they migrated to the main site (hadoop.apache.org/pig). They have ...
Olga Natkovich
Sep 10, 2009 at 8:03 pm
Sep 10, 2009 at 8:03 pm -
Message forwarded from Irfan : ---------- Forwarded message ---------- From: Irfan Mohammed <irfan.ma@gmail.com Date: Wed, Sep 9, 2009 at 11:26 AM Subject: [Fwd: Re: OutOfMemory Errors when loading a ...
Prasenjit mukherjee
Sep 9, 2009 at 3:48 pm
Sep 9, 2009 at 3:48 pm -
So right off the bat, I fixed the regex patterns in my split, but what I kept getting an error from the multiquery optimize. Specifically, the following: ERROR 2146: Internal Error. Inconsistency in ...
Zaki rahaman
Sep 4, 2009 at 6:35 pm
Sep 4, 2009 at 6:35 pm -
Hi, I want to make sure whether pig is integrated with Hbase. I mean, does pig support some query and other operations like create\write\append to HBase tables. I found ...
刘祥龙
Sep 3, 2009 at 2:04 pm
Sep 3, 2009 at 2:04 pm -
Hi, I want to make sure whether pig is integrated with Hbase. I mean, does pig support some query and other operations like create\write\append to HBase tables. I found ...
刘祥龙
Sep 3, 2009 at 8:47 am
Sep 3, 2009 at 8:47 am
Group Overview
group | user |
categories | pig, hadoop |
discussions | 34 |
posts | 134 |
users | 37 |
website | pig.apache.org |
37 users for September 2009
Archives
- May 2013 (92)
- April 2013 (226)
- March 2013 (362)
- February 2013 (192)
- January 2013 (166)
- December 2012 (115)
- November 2012 (223)
- October 2012 (249)
- September 2012 (275)
- August 2012 (249)
- July 2012 (219)
- June 2012 (371)
- May 2012 (281)
- April 2012 (377)
- March 2012 (341)
- February 2012 (323)
- January 2012 (364)
- December 2011 (266)
- November 2011 (234)
- October 2011 (207)
- September 2011 (321)
- August 2011 (271)
- July 2011 (253)
- June 2011 (249)
- May 2011 (239)
- April 2011 (341)
- March 2011 (321)
- February 2011 (276)
- January 2011 (320)
- December 2010 (244)
- November 2010 (136)
- October 2010 (251)
- September 2010 (161)
- August 2010 (201)
- July 2010 (198)
- June 2010 (171)
- May 2010 (205)
- April 2010 (192)
- March 2010 (237)
- February 2010 (192)
- January 2010 (182)
- December 2009 (106)
- November 2009 (169)
- October 2009 (105)
- September 2009 (134)
- August 2009 (108)
- July 2009 (140)
- June 2009 (151)
- May 2009 (150)
- April 2009 (133)
- March 2009 (124)
- February 2009 (119)
- January 2009 (66)
- December 2008 (45)
- November 2008 (80)
- October 2008 (102)
- September 2008 (112)
- August 2008 (32)
- July 2008 (46)
- June 2008 (78)
- May 2008 (79)
- April 2008 (26)
- March 2008 (42)
- February 2008 (30)
- January 2008 (15)
- December 2007 (31)
- November 2007 (13)
- October 2007 (9)