FAQ
Hi,

I have just install hue 2.0 using rpm packages from CDH4.0.1 and I finding
myself been unable to submit jobs using hue web interface.
There is 1 master and two slaves cluster setup (no kerberos), hue is
installed in the master node, in core-site.xml and mapred-site.xml (all
nodes) I have place this config and restarted:

           <!-- Hue WebHDFS proxy user setting -->
         <property>
                 <name>hadoop.proxyuser.hue.hosts</name>
                 <value>*</value>
         </property>
         <property>
                 <name>hadoop.proxyuser.hue.groups</name>
                 <value>hadoop</value>
         </property>

From the cluster master and other machines in the same network I can issue
the following HTTP REST request has user hue impersonating desarrollo (both
are members of hadoop group)
curl -i
"http://srvhdp1clm.pragsis.local:50070/webhdfs/v1/user/?user.name=hue&doas=desarrollo&op=LISTSTATUS"
HTTP/1.1 200 OK
Content-Type: application/json
Expires: Thu, 01-Jan-1970 00:00:00 GMT
Set-Cookie:
hadoop.auth="u=hue&p=hue&t=simple&e=1344535113047&s=Hm34ibMxgH3m6Fvu0Rn39RzPutU=";Path=/
Content-Length: 1136
Server: Jetty(6.1.26.cloudera.1)

{"FileStatuses":{"FileStatus":[
{"accessTime":0,"blockSize":0,"group":"hadoop","length":0,"modificationTime":1344425879124,"owner":"desarrollo","pathSuffix":"desarrollo","permission":"755","replication":0,"type":"DIRECTORY"},
{"accessTime":0,"blockSize":0,"group":"hadoop","length":0,"modificationTime":1344439829276,"owner":"hdfs","pathSuffix":"hdfs","permission":"755","replication":0,"type":"DIRECTORY"},
{"accessTime":0,"blockSize":0,"group":"hadoop","length":0,"modificationTime":1343995271079,"owner":"hdfs","pathSuffix":"hive","permission":"755","replication":0,"type":"DIRECTORY"},
{"accessTime":0,"blockSize":0,"group":"hue","length":0,"modificationTime":1344427321022,"owner":"hue","pathSuffix":"hue","permission":"755","replication":0,"type":"DIRECTORY"},
{"accessTime":0,"blockSize":0,"group":"hadoop","length":0,"modificationTime":1344433455887,"owner":"oozie","pathSuffix":"oozie","permission":"755","replication":0,"type":"DIRECTORY"},
{"accessTime":0,"blockSize":0,"group":"hadoop","length":0,"modificationTime":1344420773372,"owner":"root","pathSuffix":"root","permission":"755","replication":0,"type":"DIRECTORY"}
]}}

But when I try to submit a job using user desarrollo I only get these line
at

[09/Aug/2012 10:34:59 +0000] middleware INFO Processing exception:
SecurityException: Failed to obtain user group information:
org.apache.hadoop.security.authorize.AuthorizationException: User: hue is
not allowed to impersonate desarrollo (error 401): Traceback (most recent
call last):
   File
"/usr/share/hue/build/env/lib/python2.6/site-packages/Django-1.2.3-py2.6.egg/django/core/handlers/base.py",
line 100, in get_response
     response = callback(request, *callback_args, **callback_kwargs)
   File "/usr/share/hue/apps/jobsub/src/jobsub/views.py", line 254, in
submit_design
     jobid = submission.run()
   File "/usr/share/hue/apps/jobsub/src/jobsub/submit.py", line 83, in run
     self._do_as(self._username, self._copy_files, wf_dir, wf_xml)
   File "/usr/share/hue/apps/jobsub/src/jobsub/submit.py", line 59, in _do_as
     fn(*args, **kwargs)
   File "/usr/share/hue/apps/jobsub/src/jobsub/submit.py", line 117, in
_copy_files
     self._fs.create(xml_path, overwrite=True, permission=0644, data=wf_xml)
   File "/usr/share/hue/desktop/libs/hadoop/src/hadoop/fs/webhdfs.py", line
376, in create
     self._invoke_with_redirect('PUT', path, params, data)
   File "/usr/share/hue/desktop/libs/hadoop/src/hadoop/fs/webhdfs.py", line
473, in _invoke_with_redirect
     return resource.Resource(client).invoke(method, data=data,
headers=headers)
   File "/usr/share/hue/desktop/core/src/desktop/lib/rest/resource.py", line
58, in invoke
     headers=headers)
   File "/usr/share/hue/desktop/core/src/desktop/lib/rest/http_client.py",
line 172, in execute
     raise self._exc_class(ex)
WebHdfsException: SecurityException: Failed to obtain user group
information: org.apache.hadoop.security.authorize.AuthorizationException:
User: hue is not allowed to impersonate desarrollo (error 401)

Can anyone point me in the right direction?

Kind Regards

Search Discussions

  • Romain Rigaux at Aug 9, 2012 at 3:44 pm
    Hi Gerardo,

    "Failed to obtain user group information"

    Maybe it is related to the "hadoop" group you put in:
             <property>
                     <name>hadoop.proxyuser.hue.groups</name>
                     <value>hadoop</value>
             </property>

    Have you started by trying with * instead of Hadoop?

    https://ccp.cloudera.com/display/CDH4DOC/Hue+Installation#HueInstallation-HadoopConfigurationsforHue

    Romain
    On Thu, Aug 9, 2012 at 1:52 AM, Gerardo Vázquez wrote:

    Hi,

    I have just install hue 2.0 using rpm packages from CDH4.0.1 and I finding
    myself been unable to submit jobs using hue web interface.
    There is 1 master and two slaves cluster setup (no kerberos), hue is
    installed in the master node, in core-site.xml and mapred-site.xml (all
    nodes) I have place this config and restarted:

    <!-- Hue WebHDFS proxy user setting -->
    <property>
    <name>hadoop.proxyuser.hue.hosts</name>
    <value>*</value>
    </property>
    <property>
    <name>hadoop.proxyuser.hue.groups</name>
    <value>hadoop</value>
    </property>

    From the cluster master and other machines in the same network I can issue
    the following HTTP REST request has user hue impersonating desarrollo (both
    are members of hadoop group)
    curl -i "
    http://srvhdp1clm.pragsis.local:50070/webhdfs/v1/user/?user.name=hue&doas=desarrollo&op=LISTSTATUS
    "
    HTTP/1.1 200 OK
    Content-Type: application/json
    Expires: Thu, 01-Jan-1970 00:00:00 GMT
    Set-Cookie:
    hadoop.auth="u=hue&p=hue&t=simple&e=1344535113047&s=Hm34ibMxgH3m6Fvu0Rn39RzPutU=";Path=/
    Content-Length: 1136
    Server: Jetty(6.1.26.cloudera.1)

    {"FileStatuses":{"FileStatus":[

    {"accessTime":0,"blockSize":0,"group":"hadoop","length":0,"modificationTime":1344425879124,"owner":"desarrollo","pathSuffix":"desarrollo","permission":"755","replication":0,"type":"DIRECTORY"},

    {"accessTime":0,"blockSize":0,"group":"hadoop","length":0,"modificationTime":1344439829276,"owner":"hdfs","pathSuffix":"hdfs","permission":"755","replication":0,"type":"DIRECTORY"},

    {"accessTime":0,"blockSize":0,"group":"hadoop","length":0,"modificationTime":1343995271079,"owner":"hdfs","pathSuffix":"hive","permission":"755","replication":0,"type":"DIRECTORY"},

    {"accessTime":0,"blockSize":0,"group":"hue","length":0,"modificationTime":1344427321022,"owner":"hue","pathSuffix":"hue","permission":"755","replication":0,"type":"DIRECTORY"},

    {"accessTime":0,"blockSize":0,"group":"hadoop","length":0,"modificationTime":1344433455887,"owner":"oozie","pathSuffix":"oozie","permission":"755","replication":0,"type":"DIRECTORY"},

    {"accessTime":0,"blockSize":0,"group":"hadoop","length":0,"modificationTime":1344420773372,"owner":"root","pathSuffix":"root","permission":"755","replication":0,"type":"DIRECTORY"}
    ]}}

    But when I try to submit a job using user desarrollo I only get these line
    at

    [09/Aug/2012 10:34:59 +0000] middleware INFO Processing exception:
    SecurityException: Failed to obtain user group information:
    org.apache.hadoop.security.authorize.AuthorizationException: User: hue is
    not allowed to impersonate desarrollo (error 401): Traceback (most recent
    call last):
    File
    "/usr/share/hue/build/env/lib/python2.6/site-packages/Django-1.2.3-py2.6.egg/django/core/handlers/base.py",
    line 100, in get_response
    response = callback(request, *callback_args, **callback_kwargs)
    File "/usr/share/hue/apps/jobsub/src/jobsub/views.py", line 254, in
    submit_design
    jobid = submission.run()
    File "/usr/share/hue/apps/jobsub/src/jobsub/submit.py", line 83, in run
    self._do_as(self._username, self._copy_files, wf_dir, wf_xml)
    File "/usr/share/hue/apps/jobsub/src/jobsub/submit.py", line 59, in
    _do_as
    fn(*args, **kwargs)
    File "/usr/share/hue/apps/jobsub/src/jobsub/submit.py", line 117, in
    _copy_files
    self._fs.create(xml_path, overwrite=True, permission=0644, data=wf_xml)
    File "/usr/share/hue/desktop/libs/hadoop/src/hadoop/fs/webhdfs.py", line
    376, in create
    self._invoke_with_redirect('PUT', path, params, data)
    File "/usr/share/hue/desktop/libs/hadoop/src/hadoop/fs/webhdfs.py", line
    473, in _invoke_with_redirect
    return resource.Resource(client).invoke(method, data=data,
    headers=headers)
    File "/usr/share/hue/desktop/core/src/desktop/lib/rest/resource.py",
    line 58, in invoke
    headers=headers)
    File "/usr/share/hue/desktop/core/src/desktop/lib/rest/http_client.py",
    line 172, in execute
    raise self._exc_class(ex)
    WebHdfsException: SecurityException: Failed to obtain user group
    information: org.apache.hadoop.security.authorize.AuthorizationException:
    User: hue is not allowed to impersonate desarrollo (error 401)

    Can anyone point me in the right direction?

    Kind Regards
  • Gerardo Vázquez at Aug 10, 2012 at 6:48 am
    Hi Romain,

        Thanks for your reply. Yes, the first try was with '*' for groups and
    hosts with the same results. One of my collegues has noticed that user
    desarrollo is a local user of the master node (namenode) and is not present
    in the (datanodes) slave nodes, after creating it in the slave nodes /(same
    name/pwd, uid and gid, and also member of hadoop group) the error message
    has change to:

    IOException: Failed to add a datanode. User may turn off this feature
    by setting dfs.client.block.write.replace-datanode-on-failure.policy
    in configuration, where the current policy is DEFAULT. (Nodes:
    current=[192.168.10.92:50010, 192.168.10.91:50010],
    original=[192.168.10.92:50010, 192.168.10.91:50010]) (error 403)

    I will post the answer has soon has I can :)

    Kind Regards


    2012/8/9 Romain Rigaux <romain@cloudera.com>
    Hi Gerardo,


    "Failed to obtain user group information"

    Maybe it is related to the "hadoop" group you put in:

    <property>
    <name>hadoop.proxyuser.hue.groups</name>
    <value>hadoop</value>
    </property>

    Have you started by trying with * instead of Hadoop?


    https://ccp.cloudera.com/display/CDH4DOC/Hue+Installation#HueInstallation-HadoopConfigurationsforHue

    Romain

    On Thu, Aug 9, 2012 at 1:52 AM, Gerardo Vázquez wrote:

    Hi,

    I have just install hue 2.0 using rpm packages from CDH4.0.1 and I
    finding myself been unable to submit jobs using hue web interface.
    There is 1 master and two slaves cluster setup (no kerberos), hue is
    installed in the master node, in core-site.xml and mapred-site.xml (all
    nodes) I have place this config and restarted:

    <!-- Hue WebHDFS proxy user setting -->
    <property>
    <name>hadoop.proxyuser.hue.hosts</name>
    <value>*</value>
    </property>
    <property>
    <name>hadoop.proxyuser.hue.groups</name>
    <value>hadoop</value>
    </property>

    From the cluster master and other machines in the same network I can
    issue the following HTTP REST request has user hue impersonating desarrollo
    (both are members of hadoop group)
    curl -i "
    http://srvhdp1clm.pragsis.local:50070/webhdfs/v1/user/?user.name=hue&doas=desarrollo&op=LISTSTATUS
    "
    HTTP/1.1 200 OK
    Content-Type: application/json
    Expires: Thu, 01-Jan-1970 00:00:00 GMT
    Set-Cookie:
    hadoop.auth="u=hue&p=hue&t=simple&e=1344535113047&s=Hm34ibMxgH3m6Fvu0Rn39RzPutU=";Path=/
    Content-Length: 1136
    Server: Jetty(6.1.26.cloudera.1)

    {"FileStatuses":{"FileStatus":[

    {"accessTime":0,"blockSize":0,"group":"hadoop","length":0,"modificationTime":1344425879124,"owner":"desarrollo","pathSuffix":"desarrollo","permission":"755","replication":0,"type":"DIRECTORY"},

    {"accessTime":0,"blockSize":0,"group":"hadoop","length":0,"modificationTime":1344439829276,"owner":"hdfs","pathSuffix":"hdfs","permission":"755","replication":0,"type":"DIRECTORY"},

    {"accessTime":0,"blockSize":0,"group":"hadoop","length":0,"modificationTime":1343995271079,"owner":"hdfs","pathSuffix":"hive","permission":"755","replication":0,"type":"DIRECTORY"},

    {"accessTime":0,"blockSize":0,"group":"hue","length":0,"modificationTime":1344427321022,"owner":"hue","pathSuffix":"hue","permission":"755","replication":0,"type":"DIRECTORY"},

    {"accessTime":0,"blockSize":0,"group":"hadoop","length":0,"modificationTime":1344433455887,"owner":"oozie","pathSuffix":"oozie","permission":"755","replication":0,"type":"DIRECTORY"},

    {"accessTime":0,"blockSize":0,"group":"hadoop","length":0,"modificationTime":1344420773372,"owner":"root","pathSuffix":"root","permission":"755","replication":0,"type":"DIRECTORY"}
    ]}}

    But when I try to submit a job using user desarrollo I only get these
    line at

    [09/Aug/2012 10:34:59 +0000] middleware INFO Processing exception:
    SecurityException: Failed to obtain user group information:
    org.apache.hadoop.security.authorize.AuthorizationException: User: hue is
    not allowed to impersonate desarrollo (error 401): Traceback (most recent
    call last):
    File
    "/usr/share/hue/build/env/lib/python2.6/site-packages/Django-1.2.3-py2.6.egg/django/core/handlers/base.py",
    line 100, in get_response
    response = callback(request, *callback_args, **callback_kwargs)
    File "/usr/share/hue/apps/jobsub/src/jobsub/views.py", line 254, in
    submit_design
    jobid = submission.run()
    File "/usr/share/hue/apps/jobsub/src/jobsub/submit.py", line 83, in run
    self._do_as(self._username, self._copy_files, wf_dir, wf_xml)
    File "/usr/share/hue/apps/jobsub/src/jobsub/submit.py", line 59, in
    _do_as
    fn(*args, **kwargs)
    File "/usr/share/hue/apps/jobsub/src/jobsub/submit.py", line 117, in
    _copy_files
    self._fs.create(xml_path, overwrite=True, permission=0644,
    data=wf_xml)
    File "/usr/share/hue/desktop/libs/hadoop/src/hadoop/fs/webhdfs.py",
    line 376, in create
    self._invoke_with_redirect('PUT', path, params, data)
    File "/usr/share/hue/desktop/libs/hadoop/src/hadoop/fs/webhdfs.py",
    line 473, in _invoke_with_redirect
    return resource.Resource(client).invoke(method, data=data,
    headers=headers)
    File "/usr/share/hue/desktop/core/src/desktop/lib/rest/resource.py",
    line 58, in invoke
    headers=headers)
    File "/usr/share/hue/desktop/core/src/desktop/lib/rest/http_client.py",
    line 172, in execute
    raise self._exc_class(ex)
    WebHdfsException: SecurityException: Failed to obtain user group
    information: org.apache.hadoop.security.authorize.AuthorizationException:
    User: hue is not allowed to impersonate desarrollo (error 401)

    Can anyone point me in the right direction?

    Kind Regards
  • Gerardo Vázquez at Aug 10, 2012 at 10:54 am
    Hi,
          Another update, the previous error was due replication set to 3 with
    only two available datanodes, changing replication to 2 removes the
    IOException and we are back to and auth error.

    HTTP Error 401: No Autorizado
    /usr/share/hue/desktop/core/src/desktop/lib/rest/http_client.py in
    execute, line 172

    Regards


    2012/8/10 Gerardo Vázquez <gerardovr@gmail.com>
    Hi Romain,

    Thanks for your reply. Yes, the first try was with '*' for groups and
    hosts with the same results. One of my collegues has noticed that user
    desarrollo is a local user of the master node (namenode) and is not present
    in the (datanodes) slave nodes, after creating it in the slave nodes /(same
    name/pwd, uid and gid, and also member of hadoop group) the error message
    has change to:

    IOException: Failed to add a datanode. User may turn off this feature by setting dfs.client.block.write.replace-datanode-on-failure.policy in configuration, where the current policy is DEFAULT. (Nodes: current=[192.168.10.92:50010, 192.168.10.91:50010], original=[192.168.10.92:50010, 192.168.10.91:50010]) (error 403)

    I will post the answer has soon has I can :)

    Kind Regards



    2012/8/9 Romain Rigaux <romain@cloudera.com>
    Hi Gerardo,


    "Failed to obtain user group information"

    Maybe it is related to the "hadoop" group you put in:

    <property>
    <name>hadoop.proxyuser.hue.groups</name>
    <value>hadoop</value>
    </property>

    Have you started by trying with * instead of Hadoop?


    https://ccp.cloudera.com/display/CDH4DOC/Hue+Installation#HueInstallation-HadoopConfigurationsforHue

    Romain

    On Thu, Aug 9, 2012 at 1:52 AM, Gerardo Vázquez wrote:

    Hi,

    I have just install hue 2.0 using rpm packages from CDH4.0.1 and I
    finding myself been unable to submit jobs using hue web interface.
    There is 1 master and two slaves cluster setup (no kerberos), hue is
    installed in the master node, in core-site.xml and mapred-site.xml (all
    nodes) I have place this config and restarted:

    <!-- Hue WebHDFS proxy user setting -->
    <property>
    <name>hadoop.proxyuser.hue.hosts</name>
    <value>*</value>
    </property>
    <property>
    <name>hadoop.proxyuser.hue.groups</name>
    <value>hadoop</value>
    </property>

    From the cluster master and other machines in the same network I can
    issue the following HTTP REST request has user hue impersonating desarrollo
    (both are members of hadoop group)
    curl -i "
    http://srvhdp1clm.pragsis.local:50070/webhdfs/v1/user/?user.name=hue&doas=desarrollo&op=LISTSTATUS
    "
    HTTP/1.1 200 OK
    Content-Type: application/json
    Expires: Thu, 01-Jan-1970 00:00:00 GMT
    Set-Cookie:
    hadoop.auth="u=hue&p=hue&t=simple&e=1344535113047&s=Hm34ibMxgH3m6Fvu0Rn39RzPutU=";Path=/
    Content-Length: 1136
    Server: Jetty(6.1.26.cloudera.1)

    {"FileStatuses":{"FileStatus":[

    {"accessTime":0,"blockSize":0,"group":"hadoop","length":0,"modificationTime":1344425879124,"owner":"desarrollo","pathSuffix":"desarrollo","permission":"755","replication":0,"type":"DIRECTORY"},

    {"accessTime":0,"blockSize":0,"group":"hadoop","length":0,"modificationTime":1344439829276,"owner":"hdfs","pathSuffix":"hdfs","permission":"755","replication":0,"type":"DIRECTORY"},

    {"accessTime":0,"blockSize":0,"group":"hadoop","length":0,"modificationTime":1343995271079,"owner":"hdfs","pathSuffix":"hive","permission":"755","replication":0,"type":"DIRECTORY"},

    {"accessTime":0,"blockSize":0,"group":"hue","length":0,"modificationTime":1344427321022,"owner":"hue","pathSuffix":"hue","permission":"755","replication":0,"type":"DIRECTORY"},

    {"accessTime":0,"blockSize":0,"group":"hadoop","length":0,"modificationTime":1344433455887,"owner":"oozie","pathSuffix":"oozie","permission":"755","replication":0,"type":"DIRECTORY"},

    {"accessTime":0,"blockSize":0,"group":"hadoop","length":0,"modificationTime":1344420773372,"owner":"root","pathSuffix":"root","permission":"755","replication":0,"type":"DIRECTORY"}
    ]}}

    But when I try to submit a job using user desarrollo I only get these
    line at

    [09/Aug/2012 10:34:59 +0000] middleware INFO Processing exception:
    SecurityException: Failed to obtain user group information:
    org.apache.hadoop.security.authorize.AuthorizationException: User: hue is
    not allowed to impersonate desarrollo (error 401): Traceback (most recent
    call last):
    File
    "/usr/share/hue/build/env/lib/python2.6/site-packages/Django-1.2.3-py2.6.egg/django/core/handlers/base.py",
    line 100, in get_response
    response = callback(request, *callback_args, **callback_kwargs)
    File "/usr/share/hue/apps/jobsub/src/jobsub/views.py", line 254, in
    submit_design
    jobid = submission.run()
    File "/usr/share/hue/apps/jobsub/src/jobsub/submit.py", line 83, in run
    self._do_as(self._username, self._copy_files, wf_dir, wf_xml)
    File "/usr/share/hue/apps/jobsub/src/jobsub/submit.py", line 59, in
    _do_as
    fn(*args, **kwargs)
    File "/usr/share/hue/apps/jobsub/src/jobsub/submit.py", line 117, in
    _copy_files
    self._fs.create(xml_path, overwrite=True, permission=0644,
    data=wf_xml)
    File "/usr/share/hue/desktop/libs/hadoop/src/hadoop/fs/webhdfs.py",
    line 376, in create
    self._invoke_with_redirect('PUT', path, params, data)
    File "/usr/share/hue/desktop/libs/hadoop/src/hadoop/fs/webhdfs.py",
    line 473, in _invoke_with_redirect
    return resource.Resource(client).invoke(method, data=data,
    headers=headers)
    File "/usr/share/hue/desktop/core/src/desktop/lib/rest/resource.py",
    line 58, in invoke
    headers=headers)
    File
    "/usr/share/hue/desktop/core/src/desktop/lib/rest/http_client.py", line
    172, in execute
    raise self._exc_class(ex)
    WebHdfsException: SecurityException: Failed to obtain user group
    information: org.apache.hadoop.security.authorize.AuthorizationException:
    User: hue is not allowed to impersonate desarrollo (error 401)

    Can anyone point me in the right direction?

    Kind Regards
  • Gerardo Vázquez at Aug 10, 2012 at 11:24 am
    [Solved]

    Hi,
    The problem was missing configuration for user oozie to impersonate, after
    adding these lines I'm able to submit jobs

               <!-- Oozie WebHDFS proxy user setting -->
             <property>
                     <name>hadoop.proxyuser.oozie.hosts</name>
                      <value>*</value>
             </property>
             <property>
                     <name>hadoop.proxyuser.ozzie.groups</name>
                     <value>*</value>
             </property>

    >

    Kind regards

    2012/8/10 Gerardo Vázquez <gerardovr@gmail.com>
    Hi,
    Another update, the previous error was due replication set to 3 with
    only two available datanodes, changing replication to 2 removes the
    IOException and we are back to and auth error.

    HTTP Error 401: No Autorizado

    /usr/share/hue/desktop/core/src/desktop/lib/rest/http_client.py in execute, line 172

    Regards



    2012/8/10 Gerardo Vázquez <gerardovr@gmail.com>
    Hi Romain,

    Thanks for your reply. Yes, the first try was with '*' for groups and
    hosts with the same results. One of my collegues has noticed that user
    desarrollo is a local user of the master node (namenode) and is not present
    in the (datanodes) slave nodes, after creating it in the slave nodes /(same
    name/pwd, uid and gid, and also member of hadoop group) the error message
    has change to:

    IOException: Failed to add a datanode. User may turn off this feature by setting dfs.client.block.write.replace-datanode-on-failure.policy in configuration, where the current policy is DEFAULT. (Nodes: current=[192.168.10.92:50010, 192.168.10.91:50010], original=[192.168.10.92:50010, 192.168.10.91:50010]) (error 403)


    I will post the answer has soon has I can :)

    Kind Regards



    2012/8/9 Romain Rigaux <romain@cloudera.com>
    Hi Gerardo,


    "Failed to obtain user group information"

    Maybe it is related to the "hadoop" group you put in:

    <property>
    <name>hadoop.proxyuser.hue.groups</name>
    <value>hadoop</value>
    </property>

    Have you started by trying with * instead of Hadoop?


    https://ccp.cloudera.com/display/CDH4DOC/Hue+Installation#HueInstallation-HadoopConfigurationsforHue

    Romain

    On Thu, Aug 9, 2012 at 1:52 AM, Gerardo Vázquez wrote:

    Hi,

    I have just install hue 2.0 using rpm packages from CDH4.0.1 and I
    finding myself been unable to submit jobs using hue web interface.
    There is 1 master and two slaves cluster setup (no kerberos), hue is
    installed in the master node, in core-site.xml and mapred-site.xml (all
    nodes) I have place this config and restarted:

    <!-- Hue WebHDFS proxy user setting -->
    <property>
    <name>hadoop.proxyuser.hue.hosts</name>
    <value>*</value>
    </property>
    <property>
    <name>hadoop.proxyuser.hue.groups</name>
    <value>hadoop</value>
    </property>

    From the cluster master and other machines in the same network I can
    issue the following HTTP REST request has user hue impersonating desarrollo
    (both are members of hadoop group)
    curl -i "
    http://srvhdp1clm.pragsis.local:50070/webhdfs/v1/user/?user.name=hue&doas=desarrollo&op=LISTSTATUS
    "
    HTTP/1.1 200 OK
    Content-Type: application/json
    Expires: Thu, 01-Jan-1970 00:00:00 GMT
    Set-Cookie:
    hadoop.auth="u=hue&p=hue&t=simple&e=1344535113047&s=Hm34ibMxgH3m6Fvu0Rn39RzPutU=";Path=/
    Content-Length: 1136
    Server: Jetty(6.1.26.cloudera.1)

    {"FileStatuses":{"FileStatus":[

    {"accessTime":0,"blockSize":0,"group":"hadoop","length":0,"modificationTime":1344425879124,"owner":"desarrollo","pathSuffix":"desarrollo","permission":"755","replication":0,"type":"DIRECTORY"},

    {"accessTime":0,"blockSize":0,"group":"hadoop","length":0,"modificationTime":1344439829276,"owner":"hdfs","pathSuffix":"hdfs","permission":"755","replication":0,"type":"DIRECTORY"},

    {"accessTime":0,"blockSize":0,"group":"hadoop","length":0,"modificationTime":1343995271079,"owner":"hdfs","pathSuffix":"hive","permission":"755","replication":0,"type":"DIRECTORY"},

    {"accessTime":0,"blockSize":0,"group":"hue","length":0,"modificationTime":1344427321022,"owner":"hue","pathSuffix":"hue","permission":"755","replication":0,"type":"DIRECTORY"},

    {"accessTime":0,"blockSize":0,"group":"hadoop","length":0,"modificationTime":1344433455887,"owner":"oozie","pathSuffix":"oozie","permission":"755","replication":0,"type":"DIRECTORY"},

    {"accessTime":0,"blockSize":0,"group":"hadoop","length":0,"modificationTime":1344420773372,"owner":"root","pathSuffix":"root","permission":"755","replication":0,"type":"DIRECTORY"}
    ]}}

    But when I try to submit a job using user desarrollo I only get these
    line at

    [09/Aug/2012 10:34:59 +0000] middleware INFO Processing
    exception: SecurityException: Failed to obtain user group information:
    org.apache.hadoop.security.authorize.AuthorizationException: User: hue is
    not allowed to impersonate desarrollo (error 401): Traceback (most recent
    call last):
    File
    "/usr/share/hue/build/env/lib/python2.6/site-packages/Django-1.2.3-py2.6.egg/django/core/handlers/base.py",
    line 100, in get_response
    response = callback(request, *callback_args, **callback_kwargs)
    File "/usr/share/hue/apps/jobsub/src/jobsub/views.py", line 254, in
    submit_design
    jobid = submission.run()
    File "/usr/share/hue/apps/jobsub/src/jobsub/submit.py", line 83, in
    run
    self._do_as(self._username, self._copy_files, wf_dir, wf_xml)
    File "/usr/share/hue/apps/jobsub/src/jobsub/submit.py", line 59, in
    _do_as
    fn(*args, **kwargs)
    File "/usr/share/hue/apps/jobsub/src/jobsub/submit.py", line 117, in
    _copy_files
    self._fs.create(xml_path, overwrite=True, permission=0644,
    data=wf_xml)
    File "/usr/share/hue/desktop/libs/hadoop/src/hadoop/fs/webhdfs.py",
    line 376, in create
    self._invoke_with_redirect('PUT', path, params, data)
    File "/usr/share/hue/desktop/libs/hadoop/src/hadoop/fs/webhdfs.py",
    line 473, in _invoke_with_redirect
    return resource.Resource(client).invoke(method, data=data,
    headers=headers)
    File "/usr/share/hue/desktop/core/src/desktop/lib/rest/resource.py",
    line 58, in invoke
    headers=headers)
    File
    "/usr/share/hue/desktop/core/src/desktop/lib/rest/http_client.py", line
    172, in execute
    raise self._exc_class(ex)
    WebHdfsException: SecurityException: Failed to obtain user group
    information: org.apache.hadoop.security.authorize.AuthorizationException:
    User: hue is not allowed to impersonate desarrollo (error 401)

    Can anyone point me in the right direction?

    Kind Regards
  • Romain Rigaux at Aug 10, 2012 at 4:56 pm
    Thank you for the feedback and glad that it is working now!

    Romain

    On Fri, Aug 10, 2012 at 4:24 AM, Gerardo Vázquez [Solved]
    Hi,
    The problem was missing configuration for user oozie to impersonate, after
    adding these lines I'm able to submit jobs

    <!-- Oozie WebHDFS proxy user setting -->
    <property>
    <name>hadoop.proxyuser.oozie.hosts</name>

    <value>*</value>
    </property>
    <property>
    <name>hadoop.proxyuser.ozzie.groups</name>
    <value>*</value>
    </property>
    Kind regards


    2012/8/10 Gerardo Vázquez <gerardovr@gmail.com>
    Hi,
    Another update, the previous error was due replication set to 3 with
    only two available datanodes, changing replication to 2 removes the
    IOException and we are back to and auth error.

    HTTP Error 401: No Autorizado

    /usr/share/hue/desktop/core/src/desktop/lib/rest/http_client.py in execute, line 172

    Regards



    2012/8/10 Gerardo Vázquez <gerardovr@gmail.com>
    Hi Romain,

    Thanks for your reply. Yes, the first try was with '*' for groups and
    hosts with the same results. One of my collegues has noticed that user
    desarrollo is a local user of the master node (namenode) and is not present
    in the (datanodes) slave nodes, after creating it in the slave nodes /(same
    name/pwd, uid and gid, and also member of hadoop group) the error message
    has change to:

    IOException: Failed to add a datanode. User may turn off this feature by setting dfs.client.block.write.replace-datanode-on-failure.policy in configuration, where the current policy is DEFAULT. (Nodes: current=[192.168.10.92:50010, 192.168.10.91:50010], original=[192.168.10.92:50010, 192.168.10.91:50010]) (error 403)




    I will post the answer has soon has I can :)

    Kind Regards



    2012/8/9 Romain Rigaux <romain@cloudera.com>
    Hi Gerardo,


    "Failed to obtain user group information"

    Maybe it is related to the "hadoop" group you put in:

    <property>
    <name>hadoop.proxyuser.hue.groups</name>
    <value>hadoop</value>
    </property>

    Have you started by trying with * instead of Hadoop?


    https://ccp.cloudera.com/display/CDH4DOC/Hue+Installation#HueInstallation-HadoopConfigurationsforHue

    Romain

    On Thu, Aug 9, 2012 at 1:52 AM, Gerardo Vázquez wrote:

    Hi,

    I have just install hue 2.0 using rpm packages from CDH4.0.1 and I
    finding myself been unable to submit jobs using hue web interface.
    There is 1 master and two slaves cluster setup (no kerberos), hue is
    installed in the master node, in core-site.xml and mapred-site.xml (all
    nodes) I have place this config and restarted:

    <!-- Hue WebHDFS proxy user setting -->
    <property>
    <name>hadoop.proxyuser.hue.hosts</name>
    <value>*</value>
    </property>
    <property>
    <name>hadoop.proxyuser.hue.groups</name>
    <value>hadoop</value>
    </property>

    From the cluster master and other machines in the same network I can
    issue the following HTTP REST request has user hue impersonating desarrollo
    (both are members of hadoop group)
    curl -i "
    http://srvhdp1clm.pragsis.local:50070/webhdfs/v1/user/?user.name=hue&doas=desarrollo&op=LISTSTATUS
    "
    HTTP/1.1 200 OK
    Content-Type: application/json
    Expires: Thu, 01-Jan-1970 00:00:00 GMT
    Set-Cookie:
    hadoop.auth="u=hue&p=hue&t=simple&e=1344535113047&s=Hm34ibMxgH3m6Fvu0Rn39RzPutU=";Path=/
    Content-Length: 1136
    Server: Jetty(6.1.26.cloudera.1)

    {"FileStatuses":{"FileStatus":[

    {"accessTime":0,"blockSize":0,"group":"hadoop","length":0,"modificationTime":1344425879124,"owner":"desarrollo","pathSuffix":"desarrollo","permission":"755","replication":0,"type":"DIRECTORY"},

    {"accessTime":0,"blockSize":0,"group":"hadoop","length":0,"modificationTime":1344439829276,"owner":"hdfs","pathSuffix":"hdfs","permission":"755","replication":0,"type":"DIRECTORY"},

    {"accessTime":0,"blockSize":0,"group":"hadoop","length":0,"modificationTime":1343995271079,"owner":"hdfs","pathSuffix":"hive","permission":"755","replication":0,"type":"DIRECTORY"},

    {"accessTime":0,"blockSize":0,"group":"hue","length":0,"modificationTime":1344427321022,"owner":"hue","pathSuffix":"hue","permission":"755","replication":0,"type":"DIRECTORY"},

    {"accessTime":0,"blockSize":0,"group":"hadoop","length":0,"modificationTime":1344433455887,"owner":"oozie","pathSuffix":"oozie","permission":"755","replication":0,"type":"DIRECTORY"},

    {"accessTime":0,"blockSize":0,"group":"hadoop","length":0,"modificationTime":1344420773372,"owner":"root","pathSuffix":"root","permission":"755","replication":0,"type":"DIRECTORY"}
    ]}}

    But when I try to submit a job using user desarrollo I only get these
    line at

    [09/Aug/2012 10:34:59 +0000] middleware INFO Processing
    exception: SecurityException: Failed to obtain user group information:
    org.apache.hadoop.security.authorize.AuthorizationException: User: hue is
    not allowed to impersonate desarrollo (error 401): Traceback (most recent
    call last):
    File
    "/usr/share/hue/build/env/lib/python2.6/site-packages/Django-1.2.3-py2.6.egg/django/core/handlers/base.py",
    line 100, in get_response
    response = callback(request, *callback_args, **callback_kwargs)
    File "/usr/share/hue/apps/jobsub/src/jobsub/views.py", line 254, in
    submit_design
    jobid = submission.run()
    File "/usr/share/hue/apps/jobsub/src/jobsub/submit.py", line 83, in
    run
    self._do_as(self._username, self._copy_files, wf_dir, wf_xml)
    File "/usr/share/hue/apps/jobsub/src/jobsub/submit.py", line 59, in
    _do_as
    fn(*args, **kwargs)
    File "/usr/share/hue/apps/jobsub/src/jobsub/submit.py", line 117, in
    _copy_files
    self._fs.create(xml_path, overwrite=True, permission=0644,
    data=wf_xml)
    File "/usr/share/hue/desktop/libs/hadoop/src/hadoop/fs/webhdfs.py",
    line 376, in create
    self._invoke_with_redirect('PUT', path, params, data)
    File "/usr/share/hue/desktop/libs/hadoop/src/hadoop/fs/webhdfs.py",
    line 473, in _invoke_with_redirect
    return resource.Resource(client).invoke(method, data=data,
    headers=headers)
    File "/usr/share/hue/desktop/core/src/desktop/lib/rest/resource.py",
    line 58, in invoke
    headers=headers)
    File
    "/usr/share/hue/desktop/core/src/desktop/lib/rest/http_client.py", line
    172, in execute
    raise self._exc_class(ex)
    WebHdfsException: SecurityException: Failed to obtain user group
    information: org.apache.hadoop.security.authorize.AuthorizationException:
    User: hue is not allowed to impersonate desarrollo (error 401)

    Can anyone point me in the right direction?

    Kind Regards

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouphue-user @
categorieshadoop
postedAug 9, '12 at 8:52a
activeAug 10, '12 at 4:56p
posts6
users2
websitecloudera.com
irc#hadoop

2 users in discussion

Gerardo Vázquez: 4 posts Romain Rigaux: 2 posts

People

Translate

site design / logo © 2022 Grokbase