FAQ
As I think i mentioned in a previous email I'm trying to build a
DBStore, that writes the output to a database instead of the filesystem.

The only problem is I want to be able to specify a parameter file that
contains all the cruft instead of putting it in the script file.

Sadly the StoreFunc interface doesn't get passed the FS so it can't be done.

Is there any reason not to pass this through to the load/store functions?

regards
Ian

Search Discussions

  • Mridul Muralidharan at Nov 10, 2008 at 12:42 pm
    You could probably use a define to pass the db details through constructor ?

    Regards,
    Mridul

    Ian Holsman wrote:
    As I think i mentioned in a previous email I'm trying to build a
    DBStore, that writes the output to a database instead of the filesystem.

    The only problem is I want to be able to specify a parameter file that
    contains all the cruft instead of putting it in the script file.

    Sadly the StoreFunc interface doesn't get passed the FS so it can't be
    done.

    Is there any reason not to pass this through to the load/store functions?

    regards
    Ian
  • Ian Holsman at Nov 10, 2008 at 8:45 pm

    Mridul Muralidharan wrote:
    You could probably use a define to pass the db details through
    constructor ?
    yes.. that is how I'm getting there now. but i'd like to keep the
    password out of the scripts ;-)
    Regards,
    Mridul

    Ian Holsman wrote:
    As I think i mentioned in a previous email I'm trying to build a
    DBStore, that writes the output to a database instead of the filesystem.

    The only problem is I want to be able to specify a parameter file
    that contains all the cruft instead of putting it in the script file.

    Sadly the StoreFunc interface doesn't get passed the FS so it can't
    be done.

    Is there any reason not to pass this through to the load/store
    functions?

    regards
    Ian
  • Mridul Muralidharan at Nov 10, 2008 at 9:06 pm
    Other than using -param (which is also not safe - a "ps -ef | grep"
    could be used to find it) ... I cant think of anything offhand.
    Unless you have users to either include it in the jar file (and use
    resource/property related api to fetch it) or have it copy to dfs & read
    it off the dfs and to use it.

    Thinking out loud, one possibility is to directly use
    PigServer.registerQuery related api (use some template engine ?),
    generate the appropriate load/store funcspec with the db info
    (table/passwords/etc read off the config file) and execute the
    appropriate stores.

    Regards,
    Mridul

    Ian Holsman wrote:
    Mridul Muralidharan wrote:
    You could probably use a define to pass the db details through
    constructor ?
    yes.. that is how I'm getting there now. but i'd like to keep the
    password out of the scripts ;-)
    Regards,
    Mridul

    Ian Holsman wrote:
    As I think i mentioned in a previous email I'm trying to build a
    DBStore, that writes the output to a database instead of the filesystem.

    The only problem is I want to be able to specify a parameter file
    that contains all the cruft instead of putting it in the script file.

    Sadly the StoreFunc interface doesn't get passed the FS so it can't
    be done.

    Is there any reason not to pass this through to the load/store
    functions?

    regards
    Ian

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriespig, hadoop
postedNov 10, '08 at 8:11a
activeNov 10, '08 at 9:06p
posts4
users2
websitepig.apache.org

People

Translate

site design / logo © 2021 Grokbase