FAQ
Hi -- I am seeing errors when jobs go into the reduce phase and it can't
find a resource -- stack trace looks like this:

Caused by: java\.lang\.IllegalArgumentException: URI is not hierarchical
at java\.io\.File\.(Native Method)
at sun\.reflect\.NativeConstructorAccessorImpl\.newInstance(NativeConstructorAccessorImpl\.java:39)
at sun\.reflect\.DelegatingConstructorAccessorImpl\.newInstance(DelegatingConstructorAccessorImpl\.java:27)
at java\.lang\.reflect\.Constructor\.newInstance(Constructor\.java:513)
at clojure\.lang\.Reflector\.invokeConstructor(Reflector\.java:183)
at schema_reader$read_json\.invoke(schema_reader\.clj:12)
at schema_reader$load_schema\.invoke(schema_reader\.clj:15)
at schema_reader$get_fields\.invoke(schema_reader\.clj:20)
at sql_dump_tap$get_sql_dump_tap\.invoke(sql_dump_tap\.clj:35)
at sql_dump_tap$get_taps$fn__1968\.invoke(sql_dump_tap\.clj:52)
at clojure\.core$map$fn__3811\.invoke(core\.clj:2430)
at clojure\.lang\.LazySeq\.sval(LazySeq\.java:42)
at clojure\.lang\.LazySeq\.seq(LazySeq\.java:60)
at clojure\.lang\.RT\.seq(RT\.java:466)
at clojure\.core$seq\.invoke(core\.clj:133)
at clojure\.core$into_array\.invoke(core\.clj:3040)
at sql_dump_tap$multitap\.invoke(sql_dump_tap\.clj:60)
at sql_dump_tap$get_multi_tap\.invoke(sql_dump_tap\.clj:62)
at sql_dump_tap$get_query_distinct\.invoke(sql_dump_tap\.clj:70)
at views\.silverpop\.silverpopeventfacts$mk_user_id_lookup_query\.invoke(silverpopeventfacts\.clj:44)
at views\.silverpop\.silverpopeventfacts$mk_silverpop_event_facts_q\.invoke(silverpopeventfacts\.clj:50)
at views\.silverpop\.silverpopeventfacts__init\.load(Unknown Source)
at views\.silverpop\.silverpopeventfacts__init\.<clinit>(Unknown Source)
\.\.\. 60 more


so the issue is that when the reduce task starts, it has to init my class (in this case silverpopeventfacts). it fails when trying to load schema.json which is a resource i have put in my /resources folder.


it seems odd that this is failing only on the reduce tasks -- the maps tasks have no issue loading this resource on init.


part of this is sloppiness on my part -- i bind a (<- ) to a def so it forces a query to be built (and it tries to init a tap using schema.json). i simply removed the bindings and the job works -- however i am curious why only fails on reduce tasks. any ideas? thanks!

Search Discussions

  • Andy Xue at Jan 11, 2013 at 4:43 pm
    more details:

    i call (clojure.java.io/resource "schema.json") to get my resource

    again, this functionality works in all other cases except on the class init
    during the reduce task
    On Friday, January 11, 2013 11:41:21 AM UTC-5, Andy Xue wrote:

    Hi -- I am seeing errors when jobs go into the reduce phase and it can't
    find a resource -- stack trace looks like this:

    Caused by: java\.lang\.IllegalArgumentException: URI is not hierarchical
    at java\.io\.File\.<init>(File\.java:363)
    at sun\.reflect\.NativeConstructorAccessorImpl\.newInstance0(Native Method)
    at sun\.reflect\.NativeConstructorAccessorImpl\.newInstance(NativeConstructorAccessorImpl\.java:39)
    at sun\.reflect\.DelegatingConstructorAccessorImpl\.newInstance(DelegatingConstructorAccessorImpl\.java:27)
    at java\.lang\.reflect\.Constructor\.newInstance(Constructor\.java:513)
    at clojure\.lang\.Reflector\.invokeConstructor(Reflector\.java:183)
    at schema_reader$read_json\.invoke(schema_reader\.clj:12)
    at schema_reader$load_schema\.invoke(schema_reader\.clj:15)
    at schema_reader$get_fields\.invoke(schema_reader\.clj:20)
    at sql_dump_tap$get_sql_dump_tap\.invoke(sql_dump_tap\.clj:35)
    at sql_dump_tap$get_taps$fn__1968\.invoke(sql_dump_tap\.clj:52)
    at clojure\.core$map$fn__3811\.invoke(core\.clj:2430)
    at clojure\.lang\.LazySeq\.sval(LazySeq\.java:42)
    at clojure\.lang\.LazySeq\.seq(LazySeq\.java:60)
    at clojure\.lang\.RT\.seq(RT\.java:466)
    at clojure\.core$seq\.invoke(core\.clj:133)
    at clojure\.core$into_array\.invoke(core\.clj:3040)
    at sql_dump_tap$multitap\.invoke(sql_dump_tap\.clj:60)
    at sql_dump_tap$get_multi_tap\.invoke(sql_dump_tap\.clj:62)
    at sql_dump_tap$get_query_distinct\.invoke(sql_dump_tap\.clj:70)
    at views\.silverpop\.silverpopeventfacts$mk_user_id_lookup_query\.invoke(silverpopeventfacts\.clj:44)
    at views\.silverpop\.silverpopeventfacts$mk_silverpop_event_facts_q\.invoke(silverpopeventfacts\.clj:50)
    at views\.silverpop\.silverpopeventfacts__init\.load(Unknown Source)
    at views\.silverpop\.silverpopeventfacts__init\.<clinit>(Unknown Source)
    \.\.\. 60 more


    so the issue is that when the reduce task starts, it has to init my class (in this case silverpopeventfacts). it fails when trying to load schema.json which is a resource i have put in my /resources folder.


    it seems odd that this is failing only on the reduce tasks -- the maps tasks have no issue loading this resource on init.


    part of this is sloppiness on my part -- i bind a (<- ) to a def so it forces a query to be built (and it tries to init a tap using schema.json). i simply removed the bindings and the job works -- however i am curious why only fails on reduce tasks. any ideas? thanks!

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcascalog-user @
categoriesclojure, hadoop
postedJan 11, '13 at 4:43p
activeJan 11, '13 at 4:47p
posts2
users1
websiteclojure.org
irc#clojure

1 user in discussion

Andy Xue: 2 posts

People

Translate

site design / logo © 2021 Grokbase