Grokbase Groups Hive user June 2011
FAQ
Hi,

Just throwing this out to get some good ideas. Is anyone aware of any sink
for flume that would write / load data directly to the hive tables? If not,
one solution that I could think of is dump the data to hdfs or s3 and have a
periodic map reduce job load it to hive.

--
- Prash

Search Discussions

  • Sean Bigdatafun at Jun 5, 2011 at 1:06 am
    Is not this just the HDFS sink?


    On Sat, Jun 4, 2011 at 1:22 PM, Prashanth R wrote:

    Hi,

    Just throwing this out to get some good ideas. Is anyone aware of any sink
    for flume that would write / load data directly to the hive tables? If not,
    one solution that I could think of is dump the data to hdfs or s3 and have a
    periodic map reduce job load it to hive.

    --
    - Prash


    --
    --Sean
  • Bharath vissapragada at Jun 5, 2011 at 5:42 am
    Hey,

    Hive tables are nothing but some meta-data overlay on top of folders in HDFS
    containing table data. So I guess hdfs-sink of flume suffices.

    Please correct me if I am wrong.

    Thanks
    On Sun, Jun 5, 2011 at 1:52 AM, Prashanth R wrote:

    Hi,

    Just throwing this out to get some good ideas. Is anyone aware of any sink
    for flume that would write / load data directly to the hive tables? If not,
    one solution that I could think of is dump the data to hdfs or s3 and have a
    periodic map reduce job load it to hive.

    --
    - Prash


    --
    Regards,
    Bharath .V
    w:http://researchweb.iiit.ac.in/~bharath.v
  • Ankit Jain at Jun 6, 2011 at 4:22 am
    Hi,

    Load data into hbase using hbase-sink. Ones data store into hbase, create
    hive external table corresponding to that hbase table.
    On Sun, Jun 5, 2011 at 1:41 AM, bharath vissapragada wrote:

    Hey,

    Hive tables are nothing but some meta-data overlay on top of folders in
    HDFS containing table data. So I guess hdfs-sink of flume suffices.

    Please correct me if I am wrong.

    Thanks

    On Sun, Jun 5, 2011 at 1:52 AM, Prashanth R wrote:

    Hi,

    Just throwing this out to get some good ideas. Is anyone aware of any sink
    for flume that would write / load data directly to the hive tables? If not,
    one solution that I could think of is dump the data to hdfs or s3 and have a
    periodic map reduce job load it to hive.

    --
    - Prash


    --
    Regards,
    Bharath .V
    w:http://researchweb.iiit.ac.in/~bharath.v

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categorieshive, hadoop
postedJun 4, '11 at 8:23p
activeJun 6, '11 at 4:22a
posts4
users4
websitehive.apache.org

People

Translate

site design / logo © 2021 Grokbase