FAQ
Hi,

This isn't cascalog related since I use plain Cascading, but more
Pail-related, but since many people here are good with pail, i guess you'll
have some advice...

I created my test Pail where I used my custom dummy PailStructure
- FirstDigitIntegerPailStructure (partitions written Integers by first
digit). Everythings work OK, since I can write few Integer "records" to it,
and read it afterwards. Pail's directory structure is properly created.

But I'm trying to use it in my cascading flow that just copies contents
from one PailTap to another one, and it fails with:



Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to
java.lang.Integer
at
vmarcinko.mxitlab.FirstDigitIntegerPailStructure.getTarget(FirstDigitIntegerPailStructure.java:9)
~[mxitlabhort/:na]
at com.backtype.cascading.tap.PailTap$PailScheme.sink(PailTap.java:172)
~[dfs-datastores-cascading-1.3.6.jar:na]
at
cascading.tuple.TupleEntrySchemeCollector.collect(TupleEntrySchemeCollector.java:153)
~[cascading-core-2.5.4.jar:na]

My flow is defined as:
PailTap source = constructSourcePailTap(sourcePail);
PailTap sink = constructSinkPailTap(destinationPailPath,
destinationPailStructure);

Pipe copyPipe = new Pipe("copy");

// connect the taps, pipes, etc., into a flow
FlowDef flowDef = FlowDef.flowDef()
addSource(copyPipe, source)
addTailSink(copyPipe, sink);

Now, I looked at PailTap source(...) and sink(...) methods for possible
clue.....

Method source() emits read Integer "records" from Pail using these lines:

String relPath = ((Text) k).toString();
Object value = deserialize((BytesWritable) v);
sourceCall.getIncomingEntry().setTuple(new Tuple(relPath, value));

When I debug it, I can see that "relPath" is partition directory path
(String), and value is my Integer "record". All is well I guess.

But when it tries to process that emitted tuple in sink method when wanting
to write it to target PailTap, then it fetches that tuple via:
Object obj = tuple.getObject(0);
which is actually partition directory path (String), not Integer "record",
but treats it as if it were a value (Integer), and few line below tries to
find sink's Pail new partition directory by calling:
getStructure().getTarget(obj)
and that's why class cast exception is raised since it is not a Integer,
but String.

What am I doing wrong?

Regards,
Vjeran

--
You received this message because you are subscribed to the Google Groups "cascalog-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cascalog-user+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

Discussion Posts

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 1 of 2 | next ›
Discussion Overview
groupcascalog-user @
categoriesclojure, hadoop
postedApr 27, '14 at 2:46p
activeApr 27, '14 at 3:50p
posts2
users1
websiteclojure.org
irc#clojure

1 user in discussion

Vjeran Marcinko: 2 posts

People

Translate

site design / logo © 2021 Grokbase