I am trying to use a template tap in ":update" mode, so that the
results of multiple queries will accumulate in the output files. This
appears to usually not work (although in < 10% of my test runs it
actually does work). Below is code to reproduce the issue.
I looked through the Cascalog source and convinced myself that it
should be passing the sinkmode on to Cascading, even for template
taps. Can anyone confirm this is a bug, or perhaps provide advice on
how to use a template tap in update mode?
Thanks.
-David
====
(use 'cascalog.api)
(let [age [["alice" 28]
["bob" 33]]]
(?<- (hfs-seqfile "foo"
:sinkmode :update
:sink-template "%s/"
:templatefields ["?n"]
:outfields ["?a"])
[?n ?a]
(age :> ?n ?a)))
(??- (hfs-seqfile "foo/alice"))
;;=> (([28]))
(let [age [["alice" 280]
["bob" 330]]]
(?<- (hfs-seqfile "foo"
:sinkmode :update
:sink-template "%s/"
:templatefields ["?n"]
:outfields ["?a"])
[?n ?a]
(age :> ?n ?a)))
(??- (hfs-seqfile "foo/alice"))
;;=> (([280]))
;; This is surprising! I expected to see both [28] and [280] in
the results.