Grokbase Groups Pig dev January 2009
Hi All,
I have a custom loader that returns a set of fields after
reading a log line. One of the fields returned is of type DataType.Map.
My question is how can I set the data types for this map's (key, value)
pair. In my script I try to generate a record from k,v of this map and
get the error Unknown type Unknown
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$
at org.apache.hadoop.mapred.TaskTracker$Child.main(

here is my script
raw = LOAD 'myfile' USING myUdf.MyCustomLoader() ;
filtered = FILTER raw BY (ARG_MAP#'key' is not null);
entry = FOREACH filtered GENERATE A, B, myUdf.MySplit(ARG_MAP#'key',
'|') as FIELDS; // This returns a map with String (key, value) pairs
// The MySplit UDF line splits the value in the map which is "|"
separated and puts the splits it into another Map and returns it. Each
split is keyed by 'field0', 'field1'...'fieldn' where n is the number of

result = FOREACH entry GENERATE A, B, FIELDS#'field0' as CLIENT_ID,
FIELDS#'field1' as CHANNEL_ID, FIELDS#'field2' as OTHER_ID;
// Here another tuple is generated

store results into 'location' using PigStorage();

Any help here is appreciated.


Search Discussions

Discussion Posts

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 1 of 2 | next ›
Discussion Overview
groupdev @
categoriespig, hadoop
postedJan 15, '09 at 2:32p
activeJan 15, '09 at 6:34p

2 users in discussion

ANKUR GOEL: 1 post Santhosh Srinivasan: 1 post



site design / logo © 2022 Grokbase