Grokbase › Groups › Clojure › cascalog-user › September 2012
FAQ
Hello all,

New to cascalog, cascading and clojure.

I'm trying to read from and save to a (Maple) HBase tap. Reading works
fine but the following code:

(defn trial-age-b ;; fails could not select fields
   []
   (let [src-table (hbase-tap "src-table" "person" "cf" "age" "sex")
         tgt-table (hbase-tap "tgt-table" "person" "cf" "age" "sex" )
         ]
     (?<- tgt-table [?person ?age ?sex] (src-table ?b-person ?b-age ?b-sex)
(to-string ?b-person :> ?person) (to-string ?b-age :> ?age) (to-string
?b-sex :> ?sex))
     ))

causes the error:

FieldsResolverException could not select fields: [{1}:'person'], from:
[{3}:'?person', '?age', '?sex'] cascading.tuple.Fields.indexOf
(Fields.java:823)

If I add '?' to the field names on the tgt-table tap, the HBase table is
written but the column qualifiers retain the ? in them.

This *appears* to be a similar problem to this post<https://groups.google.com/forum/#!msg/cascalog-user/3ZEu6yx0BBs/K33U2h6gGOQJ> but
I'm using cascalog 1.10

Any guidance would be appreciated.

Thanks in advance.

FYI: project.clj is:

(defproject aud-cas "0.1.0-SNAPSHOT"
   :description "Using cascalog, cascading and clojure for auditd log ETL"
   :url "http://example.com/FIXME"
   :license {:name "Eclipse Public License"
             :url "http://www.eclipse.org/legal/epl-v10.html"}
   :dependencies [[org.clojure/clojure "1.4.0"]
                  [cascalog "1.10.0"]
                  ;;[cascalog "1.9.0"]
                  [org.clojure/data.json "0.1.2"]
                  [clj-time "0.4.3"]
                  ;;[com.twitter/maple "0.2.0"]
                  [com.twitter/maple "0.2.3"]
                  [org.apache.hadoop/hadoop-core "0.20.2-dev"]
                  [org.apache.hbase/hbase "0.94.1"]
                  [cascading/cascading-hadoop "2.0.6-wip-358"]
                  [cascading/cascading-core "2.0.6-wip-358"]
                  ]
   :dev-dependencies [[org.apache.hadoop/hadoop-core "0.20.2-dev"]]
   :profiles { :dev {:dependencies [[org.apache.hadoop/hadoop-core
"0.20.2-dev"]]}}
   :repositories {"conjars" "http://conjars.org/repo"})

Search Discussions

  • Paul Lam at Oct 5, 2012 at 10:47 am ⇧
    Don't know much about hbase, but other taps make use of :outfields to
    overcome this by declaring the query fields to use for the tap

    On Thursday, 27 September 2012 16:24:29 UTC+1, Ian Rumford wrote:

    Hello all,

    New to cascalog, cascading and clojure.

    I'm trying to read from and save to a (Maple) HBase tap. Reading works
    fine but the following code:

    (defn trial-age-b ;; fails could not select fields
    []
    (let [src-table (hbase-tap "src-table" "person" "cf" "age" "sex")
    tgt-table (hbase-tap "tgt-table" "person" "cf" "age" "sex" )
    ]
    (?<- tgt-table [?person ?age ?sex] (src-table ?b-person ?b-age ?b-sex)
    (to-string ?b-person :> ?person) (to-string ?b-age :> ?age) (to-string
    ?b-sex :> ?sex))
    ))

    causes the error:

    FieldsResolverException could not select fields: [{1}:'person'], from:
    [{3}:'?person', '?age', '?sex'] cascading.tuple.Fields.indexOf
    (Fields.java:823)

    If I add '?' to the field names on the tgt-table tap, the HBase table is
    written but the column qualifiers retain the ? in them.

    This *appears* to be a similar problem to this post<https://groups.google.com/forum/#!msg/cascalog-user/3ZEu6yx0BBs/K33U2h6gGOQJ> but
    I'm using cascalog 1.10

    Any guidance would be appreciated.

    Thanks in advance.

    FYI: project.clj is:

    (defproject aud-cas "0.1.0-SNAPSHOT"
    :description "Using cascalog, cascading and clojure for auditd log ETL"
    :url "http://example.com/FIXME"
    :license {:name "Eclipse Public License"
    :url "http://www.eclipse.org/legal/epl-v10.html"}
    :dependencies [[org.clojure/clojure "1.4.0"]
    [cascalog "1.10.0"]
    ;;[cascalog "1.9.0"]
    [org.clojure/data.json "0.1.2"]
    [clj-time "0.4.3"]
    ;;[com.twitter/maple "0.2.0"]
    [com.twitter/maple "0.2.3"]
    [org.apache.hadoop/hadoop-core "0.20.2-dev"]
    [org.apache.hbase/hbase "0.94.1"]
    [cascading/cascading-hadoop "2.0.6-wip-358"]
    [cascading/cascading-core "2.0.6-wip-358"]
    ]
    :dev-dependencies [[org.apache.hadoop/hadoop-core "0.20.2-dev"]]
    :profiles { :dev {:dependencies [[org.apache.hadoop/hadoop-core
    "0.20.2-dev"]]}}
    :repositories {"conjars" "http://conjars.org/repo"})

  • Ian Rumford at Oct 8, 2012 at 11:21 am ⇧
    Paul,

    Thanks, I will try that.
    On Friday, October 5, 2012 11:47:19 AM UTC+1, Paul Lam wrote:

    Don't know much about hbase, but other taps make use of :outfields to
    overcome this by declaring the query fields to use for the tap

    On Thursday, 27 September 2012 16:24:29 UTC+1, Ian Rumford wrote:

    Hello all,

    New to cascalog, cascading and clojure.

    I'm trying to read from and save to a (Maple) HBase tap. Reading works
    fine but the following code:

    (defn trial-age-b ;; fails could not select fields
    []
    (let [src-table (hbase-tap "src-table" "person" "cf" "age" "sex")
    tgt-table (hbase-tap "tgt-table" "person" "cf" "age" "sex" )
    ]
    (?<- tgt-table [?person ?age ?sex] (src-table ?b-person ?b-age
    ?b-sex) (to-string ?b-person :> ?person) (to-string ?b-age :> ?age)
    (to-string ?b-sex :> ?sex))
    ))

    causes the error:

    FieldsResolverException could not select fields: [{1}:'person'], from:
    [{3}:'?person', '?age', '?sex'] cascading.tuple.Fields.indexOf
    (Fields.java:823)

    If I add '?' to the field names on the tgt-table tap, the HBase table is
    written but the column qualifiers retain the ? in them.

    This *appears* to be a similar problem to this post<https://groups.google.com/forum/#!msg/cascalog-user/3ZEu6yx0BBs/K33U2h6gGOQJ> but
    I'm using cascalog 1.10

    Any guidance would be appreciated.

    Thanks in advance.

    FYI: project.clj is:

    (defproject aud-cas "0.1.0-SNAPSHOT"
    :description "Using cascalog, cascading and clojure for auditd log ETL"
    :url "http://example.com/FIXME"
    :license {:name "Eclipse Public License"
    :url "http://www.eclipse.org/legal/epl-v10.html"}
    :dependencies [[org.clojure/clojure "1.4.0"]
    [cascalog "1.10.0"]
    ;;[cascalog "1.9.0"]
    [org.clojure/data.json "0.1.2"]
    [clj-time "0.4.3"]
    ;;[com.twitter/maple "0.2.0"]
    [com.twitter/maple "0.2.3"]
    [org.apache.hadoop/hadoop-core "0.20.2-dev"]
    [org.apache.hbase/hbase "0.94.1"]
    [cascading/cascading-hadoop "2.0.6-wip-358"]
    [cascading/cascading-core "2.0.6-wip-358"]
    ]
    :dev-dependencies [[org.apache.hadoop/hadoop-core "0.20.2-dev"]]
    :profiles { :dev {:dependencies [[org.apache.hadoop/hadoop-core
    "0.20.2-dev"]]}}
    :repositories {"conjars" "http://conjars.org/repo"})

  • Toy at Jul 15, 2013 at 4:04 pm ⇧
    Was this ever tried and does the fix work? Is this still an issue?
    On Monday, October 8, 2012 7:21:56 AM UTC-4, Ian Rumford wrote:

    Paul,

    Thanks, I will try that.
    On Friday, October 5, 2012 11:47:19 AM UTC+1, Paul Lam wrote:

    Don't know much about hbase, but other taps make use of :outfields to
    overcome this by declaring the query fields to use for the tap

    On Thursday, 27 September 2012 16:24:29 UTC+1, Ian Rumford wrote:

    Hello all,

    New to cascalog, cascading and clojure.

    I'm trying to read from and save to a (Maple) HBase tap. Reading works
    fine but the following code:

    (defn trial-age-b ;; fails could not select fields
    []
    (let [src-table (hbase-tap "src-table" "person" "cf" "age" "sex")
    tgt-table (hbase-tap "tgt-table" "person" "cf" "age" "sex" )
    ]
    (?<- tgt-table [?person ?age ?sex] (src-table ?b-person ?b-age
    ?b-sex) (to-string ?b-person :> ?person) (to-string ?b-age :> ?age)
    (to-string ?b-sex :> ?sex))
    ))

    causes the error:

    FieldsResolverException could not select fields: [{1}:'person'], from:
    [{3}:'?person', '?age', '?sex'] cascading.tuple.Fields.indexOf
    (Fields.java:823)

    If I add '?' to the field names on the tgt-table tap, the HBase table is
    written but the column qualifiers retain the ? in them.

    This *appears* to be a similar problem to this post<https://groups.google.com/forum/#!msg/cascalog-user/3ZEu6yx0BBs/K33U2h6gGOQJ> but
    I'm using cascalog 1.10

    Any guidance would be appreciated.

    Thanks in advance.

    FYI: project.clj is:

    (defproject aud-cas "0.1.0-SNAPSHOT"
    :description "Using cascalog, cascading and clojure for auditd log ETL"
    :url "http://example.com/FIXME"
    :license {:name "Eclipse Public License"
    :url "http://www.eclipse.org/legal/epl-v10.html"}
    :dependencies [[org.clojure/clojure "1.4.0"]
    [cascalog "1.10.0"]
    ;;[cascalog "1.9.0"]
    [org.clojure/data.json "0.1.2"]
    [clj-time "0.4.3"]
    ;;[com.twitter/maple "0.2.0"]
    [com.twitter/maple "0.2.3"]
    [org.apache.hadoop/hadoop-core "0.20.2-dev"]
    [org.apache.hbase/hbase "0.94.1"]
    [cascading/cascading-hadoop "2.0.6-wip-358"]
    [cascading/cascading-core "2.0.6-wip-358"]
    ]
    :dev-dependencies [[org.apache.hadoop/hadoop-core "0.20.2-dev"]]
    :profiles { :dev {:dependencies [[org.apache.hadoop/hadoop-core
    "0.20.2-dev"]]}}
    :repositories {"conjars" "http://conjars.org/repo"})

    --
    You received this message because you are subscribed to the Google Groups "cascalog-user" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
    For more options, visit https://groups.google.com/groups/opt_out.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcascalog-user @
categoriesclojure, hadoop
postedSep 27, '12 at 5:22p
activeJul 15, '13 at 4:04p
posts4
users3
websiteclojure.org
irc#clojure

3 users in discussion

Ian Rumford: 2 posts Paul Lam: 1 post Toy: 1 post

Content

People

Support

Translate

site design / logo © 2023 Grokbase