FAQ
Hello, I've used your cloudera manager to install YARN, zookeeper, flume
and of course hadoop on clean ubuntu 12.04.
But I am facing a problem with persmissions although I've change the
mountpoint for namenode "/" from newly created "/hdfs"

I still get same error in logs :
/var/log/hadoop-hdfs# tail -f hadoop-cmf-hdfs1-NAMENODE-userver.log.out

2012-11-27 11:42:25,528 ERROR
org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException
as:flume (auth:SIMPLE)
cause:org.apache.hadoop.security.AccessControlException: Permission denied:
user=flume, access=WRITE, inode="/":hdfs:supergroup:drwxr-xr-x
2012-11-27 11:42:25,529 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 1 on 8020, call
org.apache.hadoop.hdfs.protocol.ClientProtocol.create from
192.168.56.101:57926: error:
org.apache.hadoop.security.AccessControlException: Permission denied:
user=flume, access=WRITE, inode="/":hdfs:supergroup:drwxr-xr-x
org.apache.hadoop.security.AccessControlException: Permission denied:
user=flume, access=WRITE, inode="/":hdfs:supergroup:drwxr-xr-x
at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:205)
at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:186)
at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:135)

and flume
/var/log/flume-ng# tail -f flume-cmf-flume1-AGENT-userver.log

2012-11-27 11:47:23,840 WARN org.apache.flume.sink.hdfs.HDFSEventSink: HDFS
IO error
org.apache.hadoop.security.AccessControlException: Permission denied:
user=flume, access=WRITE, inode="/":hdfs:supergroup:drwxr-xr-x
at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:205)
at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:186)


please help

Search Discussions

  • Harsh J at Nov 27, 2012 at 11:00 am
    Hi Johnny,

    Your flume app is attempting to write to HDFS on a path thats not been
    created/granted for it.

    If you know the path, create it manually and grant flume write access for it.

    For example, if you want a /hdfs path on HDFS, for user flume to write to:

    sudo -u hdfs hadoop fs -mkdir /hdfs
    sudo -u hdfs hadoop fs -chown flume:flume /hdfs

    And retry your flume app, and it should work.

    On Tue, Nov 27, 2012 at 4:18 PM, Johnny Kowalski
    wrote:
    Hello, I've used your cloudera manager to install YARN, zookeeper, flume and
    of course hadoop on clean ubuntu 12.04.
    But I am facing a problem with persmissions although I've change the
    mountpoint for namenode "/" from newly created "/hdfs"

    I still get same error in logs :
    /var/log/hadoop-hdfs# tail -f hadoop-cmf-hdfs1-NAMENODE-userver.log.out

    2012-11-27 11:42:25,528 ERROR
    org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException
    as:flume (auth:SIMPLE)
    cause:org.apache.hadoop.security.AccessControlException: Permission denied:
    user=flume, access=WRITE, inode="/":hdfs:supergroup:drwxr-xr-x
    2012-11-27 11:42:25,529 INFO org.apache.hadoop.ipc.Server: IPC Server
    handler 1 on 8020, call
    org.apache.hadoop.hdfs.protocol.ClientProtocol.create from
    192.168.56.101:57926: error:
    org.apache.hadoop.security.AccessControlException: Permission denied:
    user=flume, access=WRITE, inode="/":hdfs:supergroup:drwxr-xr-x
    org.apache.hadoop.security.AccessControlException: Permission denied:
    user=flume, access=WRITE, inode="/":hdfs:supergroup:drwxr-xr-x
    at
    org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:205)
    at
    org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:186)
    at
    org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:135)

    and flume
    /var/log/flume-ng# tail -f flume-cmf-flume1-AGENT-userver.log

    2012-11-27 11:47:23,840 WARN org.apache.flume.sink.hdfs.HDFSEventSink: HDFS
    IO error
    org.apache.hadoop.security.AccessControlException: Permission denied:
    user=flume, access=WRITE, inode="/":hdfs:supergroup:drwxr-xr-x
    at
    org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:205)
    at
    org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:186)


    please help




    --
    Harsh J
  • Johnny Kowalski at Nov 27, 2012 at 12:40 pm
    Amazing! It helped for flume! Thank you so much.

    I had also an issue to start JobHistory

    /var/log/hadoop-mapreduce# less hadoop-cmf-yarn1-JOBHISTORY-userver.log.out
    2012-11-27 10:17:41,897 FATAL
    org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer: Error starting
    JobHistoryServer
    org.apache.hadoop.yarn.YarnException: Error creating done directory:
    [hdfs://userver:8020/user/history/done]
    at
    org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.init(HistoryFileManager.java:398)
    at
    org.apache.hadoop.mapreduce.v2.hs.JobHistory.init(JobHistory.java:87)
    at
    org.apache.hadoop.yarn.service.CompositeService.init(CompositeService.java:58)
    at
    org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.init(JobHistoryServer.java:78)
    at
    org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.main(JobHistoryServer.java:132)
    Caused by: org.apache.hadoop.security.AccessControlException: Permission
    denied: user=mapred, access=WRITE, inode="/":hdfs:supergroup:drwxr-xr-x
    at
    org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:205)

    But this "sudo hadoop chown" commands helped a lot with JobHisotry too:
    sudo -u hdfs hadoop fs -chown mapred:mapred /user/history

    At the begging when I was querying "hadoop fs -ls /" it showed me all the
    structure from my local root (as "/" was the mountpoint I assume)

    I've changed mounting point and restarted HDFS but still it didn't change
    anything, just after I've rebooted the whole vm it change to those paths
    I've provided in mounting points!

    Best regards Harsh!

    W dniu wtorek, 27 listopada 2012 11:55:55 UTC+1 użytkownik Harsh J napisał:
    Hi Johnny,

    Your flume app is attempting to write to HDFS on a path thats not been
    created/granted for it.

    If you know the path, create it manually and grant flume write access for
    it.

    For example, if you want a /hdfs path on HDFS, for user flume to write to:

    sudo -u hdfs hadoop fs -mkdir /hdfs
    sudo -u hdfs hadoop fs -chown flume:flume /hdfs

    And retry your flume app, and it should work.

    On Tue, Nov 27, 2012 at 4:18 PM, Johnny Kowalski
    <johnny....@gmail.com <javascript:>> wrote:
    Hello, I've used your cloudera manager to install YARN, zookeeper, flume and
    of course hadoop on clean ubuntu 12.04.
    But I am facing a problem with persmissions although I've change the
    mountpoint for namenode "/" from newly created "/hdfs"

    I still get same error in logs :
    /var/log/hadoop-hdfs# tail -f hadoop-cmf-hdfs1-NAMENODE-userver.log.out

    2012-11-27 11:42:25,528 ERROR
    org.apache.hadoop.security.UserGroupInformation:
    PriviledgedActionException
    as:flume (auth:SIMPLE)
    cause:org.apache.hadoop.security.AccessControlException: Permission denied:
    user=flume, access=WRITE, inode="/":hdfs:supergroup:drwxr-xr-x
    2012-11-27 11:42:25,529 INFO org.apache.hadoop.ipc.Server: IPC Server
    handler 1 on 8020, call
    org.apache.hadoop.hdfs.protocol.ClientProtocol.create from
    192.168.56.101:57926: error:
    org.apache.hadoop.security.AccessControlException: Permission denied:
    user=flume, access=WRITE, inode="/":hdfs:supergroup:drwxr-xr-x
    org.apache.hadoop.security.AccessControlException: Permission denied:
    user=flume, access=WRITE, inode="/":hdfs:supergroup:drwxr-xr-x
    at
    org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:205)
    at
    org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:186)
    at
    org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:135)
    and flume
    /var/log/flume-ng# tail -f flume-cmf-flume1-AGENT-userver.log

    2012-11-27 11:47:23,840 WARN org.apache.flume.sink.hdfs.HDFSEventSink: HDFS
    IO error
    org.apache.hadoop.security.AccessControlException: Permission denied:
    user=flume, access=WRITE, inode="/":hdfs:supergroup:drwxr-xr-x
    at
    org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:205)
    at
    org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:186)

    please help




    --
    Harsh J

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupscm-users @
categorieshadoop
postedNov 27, '12 at 10:48a
activeNov 27, '12 at 12:40p
posts3
users2
websitecloudera.com
irc#hadoop

2 users in discussion

Johnny Kowalski: 2 posts Harsh J: 1 post

People

Translate

site design / logo © 2022 Grokbase