Grokbase Groups Hive user April 2011
FAQ

[Hive-user] can I change the default field delimiter of the hive?

Erix Yao
Apr 23, 2011 at 5:08 am
As I know, all the data exported from hive use ASCII \001 as the default
field delimiter, and I want to change it, How can I achieve this?

Thanks


--
haitao.yao@Beijing
reply

Search Discussions

4 responses

  • Soren Flexner at Apr 23, 2011 at 6:09 pm
    The data in hive sits in (tabular) files on HDFS. The "tables" in hive are
    metadata overlays to those files. If the files you're interested in
    pointing Hive to are tab delimited, you need to tell Hive that fact when you
    "create" the table (metadata). This happens in CREATE TABLE.

    See the second-to-last line:

    CREATE TABLE u_data (
    userid INT,
    movieid INT,
    rating INT,
    unixtime STRING)
    ROW FORMAT DELIMITED
    FIELDS TERMINATED BY '\t'
    STORED AS TEXTFILE;


    http://wiki.apache.org/hadoop/Hive/GettingStarted


    If you want to have Hive output a different delimiter than the one that's
    native to the data you have stored on HDFS, you could try altering it just
    after the fact. For instance,

    hive -e 'select * from myTable' | sed 's/oldDelimiter/newDelimiter/g' >
    outputFile.newDelimiter

    -s
    On Fri, Apr 22, 2011 at 10:07 PM, Erix Yao wrote:

    As I know, all the data exported from hive use ASCII \001 as the default
    field delimiter, and I want to change it, How can I achieve this?

    Thanks


    --
    haitao.yao@Beijing



  • Bharath vissapragada at Apr 23, 2011 at 6:21 pm
    Hi ,

    Please see

    http://wiki.apache.org/hadoop/Hive/LanguageManual/DDL
    <http://wiki.apache.org/hadoop/Hive/LanguageManual/DDL>
    http://wiki.apache.org/hadoop/Hive/LanguageManual/DML

    <http://wiki.apache.org/hadoop/Hive/LanguageManual/DML>These links are
    pretty good for starters !

    Thanks
    On Sat, Apr 23, 2011 at 10:37 AM, Erix Yao wrote:

    As I know, all the data exported from hive use ASCII \001 as the default
    field delimiter, and I want to change it, How can I achieve this?

    Thanks


    --
    haitao.yao@Beijing




    --
    Regards,
    Bharath .V
    w:http://research.iiit.ac.in/~bharath.v
  • Erix Yao at Apr 24, 2011 at 6:34 am
    I mean if I want to export data into local directory using HQL like '

    INSERT OVERWRITE LOCAL DIRECTORY '/tmp/local_out' SELECT a.* FROM pokes a;

    '

    How can I change the exported file delimiter, which is ASCII '\001'?



    2011/4/24 bharath vissapragada <bharathvissapragada1990@gmail.com>
    Hi ,

    Please see

    http://wiki.apache.org/hadoop/Hive/LanguageManual/DDL
    <http://wiki.apache.org/hadoop/Hive/LanguageManual/DDL>
    http://wiki.apache.org/hadoop/Hive/LanguageManual/DML

    <http://wiki.apache.org/hadoop/Hive/LanguageManual/DML>These links are
    pretty good for starters !

    Thanks

    On Sat, Apr 23, 2011 at 10:37 AM, Erix Yao wrote:

    As I know, all the data exported from hive use ASCII \001 as the default
    field delimiter, and I want to change it, How can I achieve this?

    Thanks


    --
    haitao.yao@Beijing




    --
    Regards,
    Bharath .V
    w:http://research.iiit.ac.in/~bharath.v


    --
    haitao.yao@Beijing
  • Edward Capriolo at Apr 24, 2011 at 2:28 pm

    On Sun, Apr 24, 2011 at 2:34 AM, Erix Yao wrote:
    I mean if I want to export data into local directory using HQL like '

    INSERT OVERWRITE LOCAL DIRECTORY '/tmp/local_out' SELECT a.* FROM pokes a;

    '

    How can I change the exported file delimiter, which is ASCII '\001'?



    2011/4/24 bharath vissapragada <bharathvissapragada1990@gmail.com>
    Hi ,
    Please see
    http://wiki.apache.org/hadoop/Hive/LanguageManual/DDL
    http://wiki.apache.org/hadoop/Hive/LanguageManual/DML
    These links are pretty good for starters !
    Thanks
    On Sat, Apr 23, 2011 at 10:37 AM, Erix Yao wrote:

    As I know, all the data exported from hive use ASCII \001 as the default
    field delimiter, and I want to change it, How can I achieve this?
    Thanks

    --
    haitao.yao@Beijing




    --
    Regards,
    Bharath .V
    w:http://research.iiit.ac.in/~bharath.v


    --
    haitao.yao@Beijing



    No create a table with a format you like then select into it

    create table xzy as select x, z,y from table row format delimited
    fields terminated by '\t'

    Then your files are in /user/hive/warehouse/xzy

Related Discussions

Discussion Navigation
viewthread | post