FAQ
Is it possible to load data into a hive table that is fed in using a unix pipe or streamed via a script? I've only seen streaming options when using SELECT TRANSFORM.

Thanks in advance,

Adriaan

Search Discussions

  • Vikas Srivastava at Sep 12, 2011 at 1:24 pm
    ya you can simple load the data by a simple script without using any api or
    anything!!

    On Mon, Sep 12, 2011 at 6:29 PM, Adriaan Tijsseling
    wrote:
    Is it possible to load data into a hive table that is fed in using a unix
    pipe or streamed via a script? I've only seen streaming options when using
    SELECT TRANSFORM.

    Thanks in advance,

    Adriaan

    --
    With Regards
    Vikas Srivastava

    DWH & Analytics Team
    Mob:+91 9560885900
    One97 | Let's get talking !
  • Adriaan Tijsseling at Sep 12, 2011 at 1:35 pm
    Do you have the syntax for the proper hive QL command?

    Thanks!
    On 2011/09/12, at 15:23, Vikas Srivastava wrote:

    ya you can simple load the data by a simple script without using any api or
    anything!!

    On Mon, Sep 12, 2011 at 6:29 PM, Adriaan Tijsseling
    wrote:
    Is it possible to load data into a hive table that is fed in using a unix
    pipe or streamed via a script? I've only seen streaming options when using
    SELECT TRANSFORM.

    Thanks in advance,

    Adriaan

    --
    With Regards
    Vikas Srivastava

    DWH & Analytics Team
    Mob:+91 9560885900
    One97 | Let's get talking !
  • Vikas Srivastava at Sep 12, 2011 at 1:37 pm
    hive -e "load data local in path '$path' into table $table
    partition(date='$date')"

    On Mon, Sep 12, 2011 at 7:04 PM, Adriaan Tijsseling
    wrote:
    Do you have the syntax for the proper hive QL command?

    Thanks!
    On 2011/09/12, at 15:23, Vikas Srivastava wrote:

    ya you can simple load the data by a simple script without using any api or
    anything!!

    On Mon, Sep 12, 2011 at 6:29 PM, Adriaan Tijsseling
    wrote:
    Is it possible to load data into a hive table that is fed in using a
    unix
    pipe or streamed via a script? I've only seen streaming options when
    using
    SELECT TRANSFORM.

    Thanks in advance,

    Adriaan

    --
    With Regards
    Vikas Srivastava

    DWH & Analytics Team
    Mob:+91 9560885900
    One97 | Let's get talking !

    --
    With Regards
    Vikas Srivastava

    DWH & Analytics Team
    Mob:+91 9560885900
    One97 | Let's get talking !
  • Adriaan Tijsseling at Sep 12, 2011 at 1:59 pm
    Sorry, I wasn't clear. What I meant was if the data in path $path can itself be a script that feeds the data in streaming fashion? Something like "load data using script 'loader.py' into table foo".
    On 2011/09/12, at 15:36, Vikas Srivastava wrote:

    hive -e "load data local in path '$path' into table $table
    partition(date='$date')"

    On Mon, Sep 12, 2011 at 7:04 PM, Adriaan Tijsseling
    wrote:
    Do you have the syntax for the proper hive QL command?

    Thanks!
    On 2011/09/12, at 15:23, Vikas Srivastava wrote:

    ya you can simple load the data by a simple script without using any api or
    anything!!

    On Mon, Sep 12, 2011 at 6:29 PM, Adriaan Tijsseling
    wrote:
    Is it possible to load data into a hive table that is fed in using a
    unix
    pipe or streamed via a script? I've only seen streaming options when
    using
    SELECT TRANSFORM.

    Thanks in advance,

    Adriaan

    --
    With Regards
    Vikas Srivastava

    DWH & Analytics Team
    Mob:+91 9560885900
    One97 | Let's get talking !

    --
    With Regards
    Vikas Srivastava

    DWH & Analytics Team
    Mob:+91 9560885900
    One97 | Let's get talking !
  • Vikas Srivastava at Sep 12, 2011 at 2:10 pm
    No,
    but what you can do is to write a general script to read files from the
    folder and store it into list then retrive the file from list and load them
    into desire path.

    On Mon, Sep 12, 2011 at 7:24 PM, Adriaan Tijsseling
    wrote:
    Sorry, I wasn't clear. What I meant was if the data in path $path can
    itself be a script that feeds the data in streaming fashion? Something like
    "load data using script 'loader.py' into table foo".
    On 2011/09/12, at 15:36, Vikas Srivastava wrote:

    hive -e "load data local in path '$path' into table $table
    partition(date='$date')"

    On Mon, Sep 12, 2011 at 7:04 PM, Adriaan Tijsseling
    wrote:
    Do you have the syntax for the proper hive QL command?

    Thanks!
    On 2011/09/12, at 15:23, Vikas Srivastava wrote:

    ya you can simple load the data by a simple script without using any
    api
    or
    anything!!

    On Mon, Sep 12, 2011 at 6:29 PM, Adriaan Tijsseling
    wrote:
    Is it possible to load data into a hive table that is fed in using a
    unix
    pipe or streamed via a script? I've only seen streaming options when
    using
    SELECT TRANSFORM.

    Thanks in advance,

    Adriaan

    --
    With Regards
    Vikas Srivastava

    DWH & Analytics Team
    Mob:+91 9560885900
    One97 | Let's get talking !

    --
    With Regards
    Vikas Srivastava

    DWH & Analytics Team
    Mob:+91 9560885900
    One97 | Let's get talking !

    --
    With Regards
    Vikas Srivastava

    DWH & Analytics Team
    Mob:+91 9560885900
    One97 | Let's get talking !
  • Adriaan Tijsseling at Sep 12, 2011 at 2:26 pm
    Thanks, that seems to be the proper alternative.

    Adriaan
    On 2011/09/12, at 16:05, Vikas Srivastava wrote:

    No,
    but what you can do is to write a general script to read files from the
    folder and store it into list then retrive the file from list and load them
    into desire path.

    On Mon, Sep 12, 2011 at 7:24 PM, Adriaan Tijsseling
    wrote:
    Sorry, I wasn't clear. What I meant was if the data in path $path can
    itself be a script that feeds the data in streaming fashion? Something like
    "load data using script 'loader.py' into table foo".
    On 2011/09/12, at 15:36, Vikas Srivastava wrote:

    hive -e "load data local in path '$path' into table $table
    partition(date='$date')"

    On Mon, Sep 12, 2011 at 7:04 PM, Adriaan Tijsseling
    wrote:
    Do you have the syntax for the proper hive QL command?

    Thanks!
    On 2011/09/12, at 15:23, Vikas Srivastava wrote:

    ya you can simple load the data by a simple script without using any
    api
    or
    anything!!

    On Mon, Sep 12, 2011 at 6:29 PM, Adriaan Tijsseling
    wrote:
    Is it possible to load data into a hive table that is fed in using a
    unix
    pipe or streamed via a script? I've only seen streaming options when
    using
    SELECT TRANSFORM.

    Thanks in advance,

    Adriaan

    --
    With Regards
    Vikas Srivastava

    DWH & Analytics Team
    Mob:+91 9560885900
    One97 | Let's get talking !

    --
    With Regards
    Vikas Srivastava

    DWH & Analytics Team
    Mob:+91 9560885900
    One97 | Let's get talking !

    --
    With Regards
    Vikas Srivastava

    DWH & Analytics Team
    Mob:+91 9560885900
    One97 | Let's get talking !

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categorieshive, hadoop
postedSep 12, '11 at 1:00p
activeSep 12, '11 at 2:26p
posts7
users2
websitehive.apache.org

People

Translate

site design / logo © 2021 Grokbase