Grokbase › Groups › Hive › user › December 2010
FAQ
just wondering if i have a pipe delimited file, how i can just import this data into hive:

basically i am using the microlens database, which is pipe separated. for example:

user id | age | gender | occupation | zip code

translates to

123 | 24 | M | worker | 12345


i'd like to just import this straight into Hive. my initial thoughts:
1. use unix substitute command and change all "|" to "Ctrl-A".
2. import into mysql, then use sqoop

seems it should be easier than this. can someone help?

thanks for the help.

sean

Search Discussions

  • Bryan Talbot at Dec 15, 2010 at 4:36 am ⇧
    I'll help by recommending that you get started by looking at the "Getting
    Started Guide".

    http://wiki.apache.org/hadoop/Hive/GettingStarted


    -Bryan

    On Tue, Dec 14, 2010 at 8:23 PM, Sean Curtis wrote:

    just wondering if i have a pipe delimited file, how i can just import this
    data into hive:

    basically i am using the microlens database, which is pipe separated. for
    example:

    user id | age | gender | occupation | zip code

    translates to

    123 | 24 | M | worker | 12345


    i'd like to just import this straight into Hive. my initial thoughts:
    1. use unix substitute command and change all "|" to "Ctrl-A".
    2. import into mysql, then use sqoop

    seems it should be easier than this. can someone help?

    thanks for the help.

    sean
  • Sean Curtis at Dec 15, 2010 at 4:59 am ⇧
    thanks Bryan

    should have been more specific. checked that guide and it didnt seem obvious with LOAD INFILE how to take a file that was already pipe-delimited and straight import that.

    is there a section of the doc i may have missed or some tip that can help there?

    sean
    On Dec 14, 2010, at 11:36 PM, Bryan Talbot wrote:

    I'll help by recommending that you get started by looking at the "Getting Started Guide".

    http://wiki.apache.org/hadoop/Hive/GettingStarted


    -Bryan


    On Tue, Dec 14, 2010 at 8:23 PM, Sean Curtis wrote:
    just wondering if i have a pipe delimited file, how i can just import this data into hive:

    basically i am using the microlens database, which is pipe separated. for example:

    user id | age | gender | occupation | zip code

    translates to

    123 | 24 | M | worker | 12345


    i'd like to just import this straight into Hive. my initial thoughts:
    1. use unix substitute command and change all "|" to "Ctrl-A".
    2. import into mysql, then use sqoop

    seems it should be easier than this. can someone help?

    thanks for the help.

    sean
  • Kirk True at Dec 15, 2010 at 5:01 am ⇧
    Hi Sean,

    The delimiter used is specified in the create table statement, not the
    LOAD DATA statement.

    Kirk
    On 12/14/2010 08:59 PM, Sean Curtis wrote:
    thanks Bryan

    should have been more specific. checked that guide and it didnt seem
    obvious with LOAD INFILE how to take a file that was already
    pipe-delimited and straight import that.

    is there a section of the doc i may have missed or some tip that can
    help there?

    sean
    On Dec 14, 2010, at 11:36 PM, Bryan Talbot wrote:

    I'll help by recommending that you get started by looking at the
    "Getting Started Guide".

    http://wiki.apache.org/hadoop/Hive/GettingStarted


    -Bryan


    On Tue, Dec 14, 2010 at 8:23 PM, Sean Curtis <sean.curtis@gmail.com
    wrote:

    just wondering if i have a pipe delimited file, how i can just
    import this data into hive:

    basically i am using the microlens database, which is pipe
    separated. for example:

    user id | age | gender | occupation | zip code

    translates to

    123 | 24 | M | worker | 12345


    i'd like to just import this straight into Hive. my initial thoughts:
    1. use unix substitute command and change all "|" to "Ctrl-A".
    2. import into mysql, then use sqoop

    seems it should be easier than this. can someone help?

    thanks for the help.

    sean
  • Sonal Goyal at Dec 15, 2010 at 5:19 am ⇧
    Sean,

    You can refer to
    http://wiki.apache.org/hadoop/Hive/LanguageManual/DDL#Create.2BAC8-Drop_Table

    You can define the ROW FORMAT DELIMITED as part of the table definition and
    then load your data into the table.

    Thanks and Regards,
    Sonal
    <https://github.com/sonalgoyal/hiho>Connect Hadoop with databases,
    Salesforce, FTP servers and others <https://github.com/sonalgoyal/hiho>
    Nube Technologies <http://www.nubetech.co>

    <http://in.linkedin.com/in/sonalgoyal>




    On Wed, Dec 15, 2010 at 10:29 AM, Sean Curtis wrote:

    thanks Bryan

    should have been more specific. checked that guide and it didnt seem
    obvious with LOAD INFILE how to take a file that was already pipe-delimited
    and straight import that.

    is there a section of the doc i may have missed or some tip that can help
    there?

    sean

    On Dec 14, 2010, at 11:36 PM, Bryan Talbot wrote:

    I'll help by recommending that you get started by looking at the "Getting
    Started Guide".

    http://wiki.apache.org/hadoop/Hive/GettingStarted


    -Bryan

    On Tue, Dec 14, 2010 at 8:23 PM, Sean Curtis wrote:

    just wondering if i have a pipe delimited file, how i can just import this
    data into hive:

    basically i am using the microlens database, which is pipe separated. for
    example:

    user id | age | gender | occupation | zip code

    translates to

    123 | 24 | M | worker | 12345


    i'd like to just import this straight into Hive. my initial thoughts:
    1. use unix substitute command and change all "|" to "Ctrl-A".
    2. import into mysql, then use sqoop

    seems it should be easier than this. can someone help?

    thanks for the help.

    sean

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categorieshive, hadoop
postedDec 15, '10 at 4:23a
activeDec 15, '10 at 5:19a
posts5
users4
websitehive.apache.org

4 users in discussion

Sean Curtis: 2 posts Sonal Goyal: 1 post Kirk True: 1 post Bryan Talbot: 1 post

Content

People

Support

Translate

site design / logo © 2022 Grokbase