FAQ
Hello,

I was trying the following in the Cloudera training vm 0.3.2 (hadoop 0.20):

I'm parsing a syslog file with RegexSerDe. I created the table thus:

CREATE TABLE syslog (month STRING, day STRING, time STRING, host STRING, logline STRING)
ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
WITH SERDEPROPERTIES (
"input.regex" = "^([^ ]+) ([^ ]+) ([^ ]+) ([^ ]+) (.+)$",
"output.format.string" = "%1$s %2$s %3$s %4$s %5$s"
)
STORED AS TEXTFILE;

and was able to successfully load the data in the right columns with:

LOAD DATA LOCAL INPATH '/home/training/sf45-cc2/mail' INTO TABLE syslog;

When I select *all* the columns (select * from syslog), I get the results just fine. But if I select just ONE column, it fails!
The input data file is quite small & the VM was freshly installed so I don't think I had a space issue....
Below are the exceptions I got - any insights appreciated.

Thanks
Ken

I tried this several times, rebooted a couple times. One time it just said:

2009-11-18 16:29:26,149 ERROR exec.ExecDriver (SessionState.java:printError(279)) - Ended Job = job_200911181622_0001 with errors
2009-11-18 16:29:26,168 ERROR ql.Driver (SessionState.java:printError(279)) - FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.ExecDriver

On another occasion, I got:

hive> select month from syslog limit 2;
FAILED: Unknown exception : org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot create directory /tmp/hive-training/1241259183. Name node is in safe mode.
The reported blocks 22 has reached the threshold 0.9990 of total blocks 22. Safe mode will be turned off automatically in 24 seconds.
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInternal(FSNamesystem.java:1757)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:1731)
at org.apache.hadoop.hdfs.server.namenode.NameNode.mkdirs(NameNode.java:554)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)

Search Discussions

  • Jerome Boulon at Nov 20, 2009 at 12:45 am
    When you¹re doing a select *, you¹re not using a Map/Reduce, just reading
    data directly

    Select some column will try to create a new directory on HDFS and will fail
    because of safe mode.

    Wait for your HDFS to no longer be on safe mode: Name node is in safe mode.

    /Jerome.

    On 11/19/09 5:27 PM, "Ken.Barclay@wellsfargo.com"
    wrote:
    Hello,

    I was trying the following in the Cloudera training vm 0.3.2 (hadoop 0.20):

    I¹m parsing a syslog file with RegexSerDe. I created the table thus:

    CREATE TABLE syslog (month STRING, day STRING, time STRING, host STRING,
    logline STRING)
    ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
    WITH SERDEPROPERTIES (
    "input.regex" = "^([^ ]+) ([^ ]+) ([^ ]+) ([^ ]+) (.+)$",
    "output.format.string" = "%1$s %2$s %3$s %4$s %5$s"
    )
    STORED AS TEXTFILE;

    and was able to successfully load the data in the right columns with:

    LOAD DATA LOCAL INPATH '/home/training/sf45-cc2/mail' INTO TABLE syslog;

    When I select *all* the columns (select * from syslog), I get the results just
    fine. But if I select just ONE column, it fails!
    The input data file is quite small & the VM was freshly installed so I don¹t
    think I had a space issueŠ.
    Below are the exceptions I got ­ any insights appreciated.

    Thanks
    Ken

    I tried this several times, rebooted a couple times. One time it just said:

    2009-11-18 16:29:26,149 ERROR exec.ExecDriver
    (SessionState.java:printError(279)) - Ended Job = job_200911181622_0001 with
    errors
    2009-11-18 16:29:26,168 ERROR ql.Driver (SessionState.java:printError(279)) -
    FAILED: Execution Error, return code 2 from
    org.apache.hadoop.hive.ql.exec.ExecDriver

    On another occasion, I got:

    hive> select month from syslog limit 2;
    FAILED: Unknown exception : org.apache.hadoop.ipc.RemoteException:
    org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot create
    directory /tmp/hive-training/1241259183. Name node is in safe mode.
    The reported blocks 22 has reached the threshold 0.9990 of total blocks 22.
    Safe mode will be turned off automatically in 24 seconds.
    at
    org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInternal(FSNamesyste
    m.java:1757)
    at
    org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:1
    731)
    at
    org.apache.hadoop.hdfs.server.namenode.NameNode.mkdirs(NameNode.java:554)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.j
    ava:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categorieshive, hadoop
postedNov 20, '09 at 12:27a
activeNov 20, '09 at 12:45a
posts2
users2
websitehive.apache.org

2 users in discussion

Ken Barclay: 1 post Jerome Boulon: 1 post

People

Translate

site design / logo © 2021 Grokbase