Grokbase › Groups › Hadoop › impala-user › January 2013
FAQ
I have impala and other services running on a node. I have built impalad on
the same node - following the instructions.

Now when I'm trying to start impala service along with other services
through cloudera manager but it is not starting and logs are showing this
error -

/**
+ exec impalad
--flagfile=/var/run/cloudera-scm-agent/process/157-impala-IMPALAD/impala-conf/impalad_flags
/usr/lib64/cmf/service/impala/impala.sh: line 23: exec: impalad: not found

*/

When I search in my system I see that I have impalad at my installed
location which is /usr/lib/imapla/sbin/impalad

# find / -name impalad
/var/log/impalad
/usr/lib/impala/sbin/impalad
/usr/lib64/cmf/agent/src/cmf/monitor/impalad
/home/src/impala/impala/be/build/debug/service/impalad

The highlighted location is the one where I have build impalad.

Where is imapla.sh looking for impalad? I didn't modify anything - Does the
build process delete impalad? Where should I copy impalad so that this
script file finds it again and start the service?

Thanks.

--

Search Discussions

  • Deepsoft at Jan 25, 2013 at 12:30 am ⇧
    Vikas,

    Thanks for replying. After linking back impalad and statestored from
    /usr/lib/impala/sbin, I was able to start statestored but got some error
    while starting impalad

    /**

    + date
    + date
    Thu Jan 24 16:18:36 PST 2013
    + export IMPALA_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf
    + IMPALA_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf
    + export HADOOP_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hadoop-conf
    + HADOOP_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hadoop-conf
    + export HIVE_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hive-conf
    + HIVE_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hive-conf
    + export HBASE_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hbase-conf
    + HBASE_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hbase-conf
    + FLAG_FILE=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf/impalad_flags
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/cloudera-scm-agent/process/197-impala-IMPALAD#g' /var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad --flagfile=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf/impalad_flags
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    E0124 16:18:36.749694 19556 impalad-main.cc:81] Failed to get/create JVM
    + date
    + date
    Thu Jan 24 16:18:37 PST 2013
    + export IMPALA_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf
    + IMPALA_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf
    + export HADOOP_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hadoop-conf
    + HADOOP_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hadoop-conf
    + export HIVE_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hive-conf
    + HIVE_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hive-conf
    + export HBASE_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hbase-conf
    + HBASE_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hbase-conf
    + FLAG_FILE=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf/impalad_flags
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/cloudera-scm-agent/process/197-impala-IMPALAD#g' /var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad --flagfile=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf/impalad_flags
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    E0124 16:18:37.914975 19591 impalad-main.cc:81] Failed to get/create JVM
    + date
    + date
    Thu Jan 24 16:18:39 PST 2013
    + export IMPALA_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf
    + IMPALA_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf
    + export HADOOP_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hadoop-conf
    + HADOOP_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hadoop-conf
    + export HIVE_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hive-conf
    + HIVE_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hive-conf
    + export HBASE_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hbase-conf
    + HBASE_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hbase-conf
    + FLAG_FILE=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf/impalad_flags
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/cloudera-scm-agent/process/197-impala-IMPALAD#g' /var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad --flagfile=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf/impalad_flags
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    E0124 16:18:39.950352 19595 impalad-main.cc:81] Failed to get/create JVM
    + date
    + date
    Thu Jan 24 16:18:42 PST 2013
    + export IMPALA_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf
    + IMPALA_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf
    + export HADOOP_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hadoop-conf
    + HADOOP_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hadoop-conf
    + export HIVE_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hive-conf
    + HIVE_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hive-conf
    + export HBASE_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hbase-conf
    + HBASE_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hbase-conf
    + FLAG_FILE=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf/impalad_flags
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/cloudera-scm-agent/process/197-impala-IMPALAD#g' /var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad --flagfile=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf/impalad_flags
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    E0124 16:18:42.983335 19599 impalad-main.cc:81] Failed to get/create JVM

    **/

    My install was working fine before I started building impalad - so I'm not sure whether build script somehow deletes existing
    impalad/statestored if it finds in /usr/bin folder?

    Now it is complaining about some missing environment variables - I wasn't setting these before as well. Did I lose a script file which is setting environment
    variable?

    Yes, I have not updated my CM yet. When I tried to udpate using the instructions here https://ccp.cloudera.com/display/FREE41DOC/Upgrade+from+Cloudera+Manager+4+to+the+Latest+Version+of+Cloudera+Manager+4

    I got the following message -

    [root@khadi01-rhvm01 local]# sudo yum update --disablerepo='*' --enablerepo=cloudera-manager
    Loaded plugins: product-id, refresh-packagekit, rhnplugin, subscription-manager
    Updating certificate-based repositories.
    Unable to read consumer identity
    Repository 'dvd' is missing name in configuration, using id
    cloudera-manager | 951 B 00:00
    cloudera-manager/primary | 3.6 kB 00:00
    cloudera-manager 7/7
    Setting up Update Process
    No Packages marked for Update

    Now when I check "host inspector" result, I see that there exists mixed CDH versions. It seems my upgrade didn't go well. Correct?

    Thanks.
    -Dipesh


    Thanks.
    -Dipesh



    On Thursday, January 24, 2013 3:03:20 PM UTC-8, vikas wrote:

    Hi Dipesh,

    The binary is looked in the default PATH and impalad isn't present in it.
    You might want to symlink the build you created to /usr/bin.

    Also, it seems you aren't using the latest CM build, it has been updated
    to use full impala path when calling 'exec' instead of just 'impalad'.

    Vikas


    On Thu, Jan 24, 2013 at 2:41 PM, Deepsoft <dipeshs...@gmail.com<javascript:>
    wrote:
    I have impala and other services running on a node. I have built impalad
    on the same node - following the instructions.

    Now when I'm trying to start impala service along with other services
    through cloudera manager but it is not starting and logs are showing this
    error -

    /**
    + exec impalad
    --flagfile=/var/run/cloudera-scm-agent/process/157-impala-IMPALAD/impala-conf/impalad_flags
    /usr/lib64/cmf/service/impala/impala.sh: line 23: exec: impalad: not found

    */

    When I search in my system I see that I have impalad at my installed
    location which is /usr/lib/imapla/sbin/impalad

    # find / -name impalad
    /var/log/impalad
    /usr/lib/impala/sbin/impalad
    /usr/lib64/cmf/agent/src/cmf/monitor/impalad
    /home/src/impala/impala/be/build/debug/service/impalad

    The highlighted location is the one where I have build impalad.

    Where is imapla.sh looking for impalad? I didn't modify anything - Does
    the build process delete impalad? Where should I copy impalad so that this
    script file finds it again and start the service?

    Thanks.

    --

  • Vikas Singh at Jan 25, 2013 at 12:47 am ⇧
    Hi Dipesh,

    I am actually not sure what happens if you build impala on a machine where
    you already have impala installed from package. If you install impala, the
    file '/usr/bin/impalad' is a bash file that takes care of setting all
    environment before it exec the actual 'impalad' binary. For some reason in
    your setup that file is missing and launching impalad directly fails
    because env insn't setup correctly.

    And upgrading CM shouldn't change CDH component versions, so mixed CDH
    versions were probably the case before too. What versions does it report?

    Vikas

    On Thu, Jan 24, 2013 at 4:30 PM, Deepsoft wrote:

    Vikas,

    Thanks for replying. After linking back impalad and statestored from
    /usr/lib/impala/sbin, I was able to start statestored but got some error
    while starting impalad

    /**

    + date
    + date
    Thu Jan 24 16:18:36 PST 2013
    + export IMPALA_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf
    + IMPALA_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf
    + export HADOOP_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hadoop-conf
    + HADOOP_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hadoop-conf
    + export HIVE_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hive-conf
    + HIVE_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hive-conf
    + export HBASE_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hbase-conf
    + HBASE_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hbase-conf
    + FLAG_FILE=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf/impalad_flags
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/cloudera-scm-agent/process/197-impala-IMPALAD#g' /var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad --flagfile=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf/impalad_flags
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    E0124 16:18:36.749694 19556 impalad-main.cc:81] Failed to get/create JVM
    + date
    + date
    Thu Jan 24 16:18:37 PST 2013
    + export IMPALA_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf
    + IMPALA_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf
    + export HADOOP_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hadoop-conf
    + HADOOP_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hadoop-conf
    + export HIVE_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hive-conf
    + HIVE_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hive-conf
    + export HBASE_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hbase-conf
    + HBASE_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hbase-conf
    + FLAG_FILE=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf/impalad_flags
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/cloudera-scm-agent/process/197-impala-IMPALAD#g' /var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad --flagfile=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf/impalad_flags
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    E0124 16:18:37.914975 19591 impalad-main.cc:81] Failed to get/create JVM
    + date
    + date
    Thu Jan 24 16:18:39 PST 2013
    + export IMPALA_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf
    + IMPALA_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf
    + export HADOOP_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hadoop-conf
    + HADOOP_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hadoop-conf
    + export HIVE_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hive-conf
    + HIVE_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hive-conf
    + export HBASE_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hbase-conf
    + HBASE_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hbase-conf
    + FLAG_FILE=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf/impalad_flags
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/cloudera-scm-agent/process/197-impala-IMPALAD#g' /var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad --flagfile=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf/impalad_flags
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    E0124 16:18:39.950352 19595 impalad-main.cc:81] Failed to get/create JVM
    + date
    + date
    Thu Jan 24 16:18:42 PST 2013
    + export IMPALA_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf
    + IMPALA_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf
    + export HADOOP_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hadoop-conf
    + HADOOP_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hadoop-conf
    + export HIVE_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hive-conf
    + HIVE_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hive-conf
    + export HBASE_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hbase-conf
    + HBASE_CONF_DIR=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/hbase-conf
    + FLAG_FILE=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf/impalad_flags
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/cloudera-scm-agent/process/197-impala-IMPALAD#g' /var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad --flagfile=/var/run/cloudera-scm-agent/process/197-impala-IMPALAD/impala-conf/impalad_flags
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    E0124 16:18:42.983335 19599 impalad-main.cc:81] Failed to get/create JVM

    **/

    My install was working fine before I started building impalad - so I'm not sure whether build script somehow deletes existing
    impalad/statestored if it finds in /usr/bin folder?

    Now it is complaining about some missing environment variables - I wasn't setting these before as well. Did I lose a script file which is setting environment
    variable?

    Yes, I have not updated my CM yet. When I tried to udpate using the instructions here https://ccp.cloudera.com/display/FREE41DOC/Upgrade+from+Cloudera+Manager+4+to+the+Latest+Version+of+Cloudera+Manager+4

    I got the following message -

    [root@khadi01-rhvm01 local]# sudo yum update --disablerepo='*' --enablerepo=cloudera-manager
    Loaded plugins: product-id, refresh-packagekit, rhnplugin, subscription-manager
    Updating certificate-based repositories.
    Unable to read consumer identity
    Repository 'dvd' is missing name in configuration, using id
    cloudera-manager | 951 B 00:00
    cloudera-manager/primary | 3.6 kB 00:00
    cloudera-manager 7/7
    Setting up Update Process
    No Packages marked for Update

    Now when I check "host inspector" result, I see that there exists mixed CDH versions. It seems my upgrade didn't go well. Correct?

    Thanks.
    -Dipesh


    Thanks.
    -Dipesh



    On Thursday, January 24, 2013 3:03:20 PM UTC-8, vikas wrote:

    Hi Dipesh,

    The binary is looked in the default PATH and impalad isn't present in it.
    You might want to symlink the build you created to /usr/bin.

    Also, it seems you aren't using the latest CM build, it has been updated
    to use full impala path when calling 'exec' instead of just 'impalad'.

    Vikas

    On Thu, Jan 24, 2013 at 2:41 PM, Deepsoft wrote:

    I have impala and other services running on a node. I have built impalad
    on the same node - following the instructions.

    Now when I'm trying to start impala service along with other services
    through cloudera manager but it is not starting and logs are showing this
    error -

    /**
    + exec impalad --flagfile=/var/run/cloudera-**
    scm-agent/process/157-impala-**IMPALAD/impala-conf/impalad_**flags
    /usr/lib64/cmf/service/impala/**impala.sh: line 23: exec: impalad: not
    found

    */

    When I search in my system I see that I have impalad at my installed
    location which is /usr/lib/imapla/sbin/impalad

    # find / -name impalad
    /var/log/impalad
    /usr/lib/impala/sbin/impalad
    /usr/lib64/cmf/agent/src/cmf/**monitor/impalad
    /home/src/impala/impala/be/**build/debug/service/impalad

    The highlighted location is the one where I have build impalad.

    Where is imapla.sh looking for impalad? I didn't modify anything - Does
    the build process delete impalad? Where should I copy impalad so that this
    script file finds it again and start the service?

    Thanks.

    --

  • Vikas Singh at Jan 25, 2013 at 6:25 pm ⇧
    yes, /etc/default/impala is the script that sets the classpath, so you are
    probably missing the jars that come packaged with Impala. I think we need
    to know what you are trying to achieve here. That will help us provide a
    solution to the issue rather than responding to point questions. From the
    mail thread, here what I understand:

    1. You had working Impala installation with CM.
    2. On one of the nodes, you downloaded Impala source code and build it.
    3. As a result of that (or some command that got executed in that process),
    bunch of Impala installation files are missing
    4. Now Impala service will not start.

    Now are you trying to get the cluster back on track? Why not just uninstall
    Impala package and install it again? That will get all files back from yum
    in pristine state and it should work.

    - Vikas


    On Fri, Jan 25, 2013 at 10:11 AM, Deepsoft wrote:

    Vikas,

    Thanks. I will try upgrade using the below mentioned base url.

    I had same impala installation (CDH 4.1.2 and Imapla 0.3) on another node,
    so I took these bash files (impalad and statestored) and copied to my
    problematic node but then i go the following errors -

    /**

    + date
    + date
    Thu Jan 24 17:12:40 PST 2013
    + export IMPALA_CONF_DIR=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/impala-conf
    + IMPALA_CONF_DIR=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/impala-conf
    + export HADOOP_CONF_DIR=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/hadoop-conf
    + HADOOP_CONF_DIR=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/hadoop-conf
    + export HIVE_CONF_DIR=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/hive-conf
    + HIVE_CONF_DIR=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/hive-conf
    + export HBASE_CONF_DIR=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/hbase-conf
    + HBASE_CONF_DIR=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/hbase-conf
    + FLAG_FILE=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/impala-conf/impalad_flags
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/cloudera-scm-agent/process/202-impala-IMPALAD#g' /var/run/cloudera-scm-agent/process/202-impala-IMPALAD/impala-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad --flagfile=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/impala-conf/impalad_flags
    /usr/bin/impalad: line 2: /etc/default/impala: No such file or directory
    /usr/bin/impalad: line 16: /impalad: No such file or directory
    /usr/bin/impalad: line 16: exec: /impalad: cannot execute: No such file or directory

    **/

    After copying /etc/default/impala from my second node, i was able to start statestored service but impalad service is still not starting and now
    throwing following error



    /**

    + date
    + date
    Fri Jan 25 10:02:48 PST 2013
    + export
    IMPALA_CONF_DIR=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/impala-conf
    +
    IMPALA_CONF_DIR=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/impala-conf
    + export
    HADOOP_CONF_DIR=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/hadoop-conf
    +
    HADOOP_CONF_DIR=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/hadoop-conf
    + export
    HIVE_CONF_DIR=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/hive-conf
    +
    HIVE_CONF_DIR=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/hive-conf
    + export
    HBASE_CONF_DIR=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/hbase-conf
    +
    HBASE_CONF_DIR=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/hbase-conf
    +
    FLAG_FILE=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/impala-conf/impalad_flags
    + perl -pi -e
    's#{{CMF_CONF_DIR}}#/var/run/cloudera-scm-agent/process/204-impala-IMPALAD#g'
    /var/run/cloudera-scm-agent/process/204-impala-IMPALAD/impala-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad
    --flagfile=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/impala-conf/impalad_flags
    loadFileSystems error:
    (unable to get stack trace for java.lang.NoClassDefFoundError exception:
    ExceptionUtils::getStackTrace error.)
    hdfsBuilderConnect(forceNewInstance=0, nn=default, port=0,
    kerbTicketCachePath=(NULL), userName=(NULL)) error:
    (unable to get stack trace for java.lang.NoClassDefFoundError exception:
    ExceptionUtils::getStackTrace error.)
    Exception in thread "main" java.lang.NoClassDefFoundError:
    com/cloudera/impala/common/JniUtil
    Caused by: java.lang.ClassNotFoundException:
    com.cloudera.impala.common.JniUtil
    at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
    E0125 10:02:50.076205 29022 impalad-main.cc:81] Failed to find JniUtil
    class.

    */

    Seems it is a classpath issue now?

    Thanks.
    -Dipesh

    On Thursday, January 24, 2013 5:24:15 PM UTC-8, vikas wrote:

    Hi Dipesh,

    I think you need to change the baseurl to "http://archive.cloudera.com/**
    cm4/redhat/6/x86_64/cm/4/<http://archive.cloudera.com/cm4/redhat/6/x86_64/cm/4/>",
    which is the url that will fetch the latest CM whenever you run "yum
    upgrade" command. For Imapla upgrade you may have to do something similar
    (depending on the content of your impala repo file)

    The file /usr/bin/impalad (which is a bash file) is the one that sets
    environment variables (and is missing in your setup).

    - Vikas

    On Thu, Jan 24, 2013 at 5:02 PM, Deepsoft wrote:

    Vikas,

    It is reporting version 4.1.2. I wanted to upgrade it to 4.1.3 so that I
    can upgrade to impala 0.4. Here is the content of my cloudera-manager.repo
    file

    [cloudera-manager]
    name = Cloudera Manager, Version 4.1.2
    baseurl = http://archive.cloudera.com/**cm4/redhat/6/x86_64/cm/4.1.2/<http://archive.cloudera.com/cm4/redhat/6/x86_64/cm/4.1.2/>
    gpgkey = http://archive.cloudera.com/**redhat/cdh/RPM-GPG-KEY-**cloudera<http://archive.cloudera.com/redhat/cdh/RPM-GPG-KEY-cloudera>
    gpgcheck = 1

    Can you please confirm which file sets all the required enviornment
    variables? I have created a sym link to impalad in /usr/bin from the
    installed folder /usr/lib/impala/sbin/impalad.

    Thanks.
    -Dipesh


    On Thursday, January 24, 2013 4:47:22 PM UTC-8, vikas wrote:

    Hi Dipesh,

    I am actually not sure what happens if you build impala on a machine
    where you already have impala installed from package. If you install
    impala, the file '/usr/bin/impalad' is a bash file that takes care of
    setting all environment before it exec the actual 'impalad' binary. For
    some reason in your setup that file is missing and launching impalad
    directly fails because env insn't setup correctly.

    And upgrading CM shouldn't change CDH component versions, so mixed CDH
    versions were probably the case before too. What versions does it report?

    Vikas

    On Thu, Jan 24, 2013 at 4:30 PM, Deepsoft wrote:

    Vikas,

    Thanks for replying. After linking back impalad and statestored from
    /usr/lib/impala/sbin, I was able to start statestored but got some error
    while starting impalad

    /**

    + date
    + date
    Thu Jan 24 16:18:36 PST 2013
    + export IMPALA_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/impala-conf
    + IMPALA_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/impala-conf
    + export HADOOP_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/hadoop-conf
    + HADOOP_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/hadoop-conf
    + export HIVE_CONF_DIR=/var/run/**clouder**a-scm-agent/process/**197-**impala-IMPALAD/hive-conf
    + HIVE_CONF_DIR=/var/run/**clouder**a-scm-agent/process/**197-**impala-IMPALAD/hive-conf
    + export HBASE_CONF_DIR=/var/run/**cloude**ra-scm-agent/process/**197-**impala-IMPALAD/hbase-conf
    + HBASE_CONF_DIR=/var/run/**cloude**ra-scm-agent/process/**197-**impala-IMPALAD/hbase-conf
    + FLAG_FILE=/var/run/cloudera-**sc**m-agent/process/197-impala-**IMP**ALAD/impala-conf/impalad_**flags
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/**c**loudera-scm-agent/process/**197-**impala-IMPALAD#g' /var/run/cloudera-scm-agent/**pr**ocess/197-impala-IMPALAD/**impal**a-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad --flagfile=/var/run/cloudera-**s**cm-agent/process/197-impala-**IM**PALAD/impala-conf/impalad_**flag**s
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    E0124 16:18:36.749694 19556 impalad-main.cc:81] Failed to get/create JVM
    + date
    + date
    Thu Jan 24 16:18:37 PST 2013
    + export IMPALA_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/impala-conf
    + IMPALA_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/impala-conf
    + export HADOOP_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/hadoop-conf
    + HADOOP_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/hadoop-conf
    + export HIVE_CONF_DIR=/var/run/**clouder**a-scm-agent/process/**197-**impala-IMPALAD/hive-conf
    + HIVE_CONF_DIR=/var/run/**clouder**a-scm-agent/process/**197-**impala-IMPALAD/hive-conf
    + export HBASE_CONF_DIR=/var/run/**cloude**ra-scm-agent/process/**197-**impala-IMPALAD/hbase-conf
    + HBASE_CONF_DIR=/var/run/**cloude**ra-scm-agent/process/**197-**impala-IMPALAD/hbase-conf
    + FLAG_FILE=/var/run/cloudera-**sc**m-agent/process/197-impala-**IMP**ALAD/impala-conf/impalad_**flags
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/**c**loudera-scm-agent/process/**197-**impala-IMPALAD#g' /var/run/cloudera-scm-agent/**pr**ocess/197-impala-IMPALAD/**impal**a-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad --flagfile=/var/run/cloudera-**s**cm-agent/process/197-impala-**IM**PALAD/impala-conf/impalad_**flag**s
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    E0124 16:18:37.914975 19591 impalad-main.cc:81] Failed to get/create JVM
    + date
    + date
    Thu Jan 24 16:18:39 PST 2013
    + export IMPALA_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/impala-conf
    + IMPALA_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/impala-conf
    + export HADOOP_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/hadoop-conf
    + HADOOP_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/hadoop-conf
    + export HIVE_CONF_DIR=/var/run/**clouder**a-scm-agent/process/**197-**impala-IMPALAD/hive-conf
    + HIVE_CONF_DIR=/var/run/**clouder**a-scm-agent/process/**197-**impala-IMPALAD/hive-conf
    + export HBASE_CONF_DIR=/var/run/**cloude**ra-scm-agent/process/**197-**impala-IMPALAD/hbase-conf
    + HBASE_CONF_DIR=/var/run/**cloude**ra-scm-agent/process/**197-**impala-IMPALAD/hbase-conf
    + FLAG_FILE=/var/run/cloudera-**sc**m-agent/process/197-impala-**IMP**ALAD/impala-conf/impalad_**flags
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/**c**loudera-scm-agent/process/**197-**impala-IMPALAD#g' /var/run/cloudera-scm-agent/**pr**ocess/197-impala-IMPALAD/**impal**a-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad --flagfile=/var/run/cloudera-**s**cm-agent/process/197-impala-**IM**PALAD/impala-conf/impalad_**flag**s
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    E0124 16:18:39.950352 19595 impalad-main.cc:81] Failed to get/create JVM
    + date
    + date
    Thu Jan 24 16:18:42 PST 2013
    + export IMPALA_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/impala-conf
    + IMPALA_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/impala-conf
    + export HADOOP_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/hadoop-conf
    + HADOOP_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/hadoop-conf
    + export HIVE_CONF_DIR=/var/run/**clouder**a-scm-agent/process/**197-**impala-IMPALAD/hive-conf
    + HIVE_CONF_DIR=/var/run/**clouder**a-scm-agent/process/**197-**impala-IMPALAD/hive-conf
    + export HBASE_CONF_DIR=/var/run/**cloude**ra-scm-agent/process/**197-**impala-IMPALAD/hbase-conf
    + HBASE_CONF_DIR=/var/run/**cloude**ra-scm-agent/process/**197-**impala-IMPALAD/hbase-conf
    + FLAG_FILE=/var/run/cloudera-**sc**m-agent/process/197-impala-**IMP**ALAD/impala-conf/impalad_**flags
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/**c**loudera-scm-agent/process/**197-**impala-IMPALAD#g' /var/run/cloudera-scm-agent/**pr**ocess/197-impala-IMPALAD/**impal**a-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad --flagfile=/var/run/cloudera-**s**cm-agent/process/197-impala-**IM**PALAD/impala-conf/impalad_**flag**s
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    E0124 16:18:42.983335 19599 impalad-main.cc:81] Failed to get/create JVM

    **/

    My install was working fine before I started building impalad - so I'm not sure whether build script somehow deletes existing


    impalad/statestored if it finds in /usr/bin folder?

    Now it is complaining about some missing environment variables - I wasn't setting these before as well. Did I lose a script file which is setting environment


    variable?

    Yes, I have not updated my CM yet. When I tried to udpate using the instructions here https://ccp.cloudera.com/**displ**ay/FREE41DOC/Upgrade+**from+**Cloudera+Manager+4+to+**the+**Latest+Version+of+**Cloudera+**Manager+4 <https://ccp.cloudera.com/display/FREE41DOC/Upgrade+from+Cloudera+Manager+4+to+the+Latest+Version+of+Cloudera+Manager+4>


    I got the following message -

    [root@khadi01-rhvm01 local]# sudo yum update --disablerepo='*' --enablerepo=cloudera-manager
    Loaded plugins: product-id, refresh-packagekit, rhnplugin, subscription-manager


    Updating certificate-based repositories.
    Unable to read consumer identity
    Repository 'dvd' is missing name in configuration, using id
    cloudera-manager | 951 B 00:00


    cloudera-manager/primary | 3.6 kB 00:00
    cloudera-manager 7/7
    Setting up Update Process
    No Packages marked for Update


    Now when I check "host inspector" result, I see that there exists mixed CDH versions. It seems my upgrade didn't go well. Correct?

    Thanks.
    -Dipesh


    Thanks.
    -Dipesh



    On Thursday, January 24, 2013 3:03:20 PM UTC-8, vikas wrote:

    Hi Dipesh,

    The binary is looked in the default PATH and impalad isn't present in
    it. You might want to symlink the build you created to /usr/bin.

    Also, it seems you aren't using the latest CM build, it has been
    updated to use full impala path when calling 'exec' instead of just
    'impalad'.

    Vikas

    On Thu, Jan 24, 2013 at 2:41 PM, Deepsoft wrote:

    I have impala and other services running on a node. I have built
    impalad on the same node - following the instructions.

    Now when I'm trying to start impala service along with other
    services through cloudera manager but it is not starting and logs are
    showing this error -

    /**
    + exec impalad --flagfile=/var/run/cloudera-**s****
    cm-agent/process/157-impala-**IM****PALAD/impala-conf/impalad_**flag
    ****s
    /usr/lib64/cmf/service/impala/******impala.sh: line 23: exec:
    impalad: not found

    */

    When I search in my system I see that I have impalad at my
    installed location which is /usr/lib/imapla/sbin/impalad

    # find / -name impalad
    /var/log/impalad
    /usr/lib/impala/sbin/impalad
    /usr/lib64/cmf/agent/src/cmf/**m****onitor/impalad
    /home/src/impala/impala/be/**bui****ld/debug/service/impalad

    The highlighted location is the one where I have build impalad.

    Where is imapla.sh looking for impalad? I didn't modify anything -
    Does the build process delete impalad? Where should I copy impalad so that
    this script file finds it again and start the service?

    Thanks.

    --

    --
    --
  • Deepsoft at Jan 25, 2013 at 6:46 pm ⇧
    Vikas,

    Thanks. Hmnn...I had a working version of Impala so I'm not sure whether
    some jars got missing or got deleted. Here is the sequence of events -

    1. I have installed CDH 4.1.2 and impala 0.3 on Node1 and Nod2.
    2. Configured and ran impala on Node1.
    3. Built impala on Node1.
    4. Now when i tried to start impala on Node1, i found that following files
    were missing
    /usr/bin/imapad
    /usr/bin/statestored
    /etc/default/impala
    5. Copied these files from Node2.

    After this I'm getting CNFE (ClassNotFoundException) when I'm trying to
    start impala service.

    I didn't do cluster set up - wanted to make sure that it works on single
    node correctly then I was planning to add more nodes (well initially it
    happened that I had installed CDH and impala on two nodes). I'm trying to
    figure what went wrong after builiding and why the working state went
    berserk.

    Now I see that my /usr/lib/impala has only two folders -

    conf
    sbin

    When i compare it with my Node2, i see that number of directories have been
    nuked (most probably when I have built impala on this node??) and it is
    missing

    bin
    cloudera
    llvm-ir
    lib
    sbin-debug
    sbin-retail
    www

    Thanks.
    -Dipesh



    On Friday, January 25, 2013 10:25:01 AM UTC-8, vikas wrote:

    yes, /etc/default/impala is the script that sets the classpath, so you are
    probably missing the jars that come packaged with Impala. I think we need
    to know what you are trying to achieve here. That will help us provide a
    solution to the issue rather than responding to point questions. From the
    mail thread, here what I understand:

    1. You had working Impala installation with CM.
    2. On one of the nodes, you downloaded Impala source code and build it.
    3. As a result of that (or some command that got executed in that
    process), bunch of Impala installation files are missing
    4. Now Impala service will not start.

    Now are you trying to get the cluster back on track? Why not just
    uninstall Impala package and install it again? That will get all files back
    from yum in pristine state and it should work.

    - Vikas



    On Fri, Jan 25, 2013 at 10:11 AM, Deepsoft <dipeshs...@gmail.com<javascript:>
    wrote:
    Vikas,

    Thanks. I will try upgrade using the below mentioned base url.

    I had same impala installation (CDH 4.1.2 and Imapla 0.3) on another
    node, so I took these bash files (impalad and statestored) and copied to my
    problematic node but then i go the following errors -

    /**

    + date
    + date
    Thu Jan 24 17:12:40 PST 2013
    + export IMPALA_CONF_DIR=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/impala-conf
    + IMPALA_CONF_DIR=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/impala-conf
    + export HADOOP_CONF_DIR=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/hadoop-conf
    + HADOOP_CONF_DIR=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/hadoop-conf
    + export HIVE_CONF_DIR=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/hive-conf
    + HIVE_CONF_DIR=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/hive-conf
    + export HBASE_CONF_DIR=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/hbase-conf
    + HBASE_CONF_DIR=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/hbase-conf
    + FLAG_FILE=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/impala-conf/impalad_flags
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/cloudera-scm-agent/process/202-impala-IMPALAD#g' /var/run/cloudera-scm-agent/process/202-impala-IMPALAD/impala-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad --flagfile=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/impala-conf/impalad_flags
    /usr/bin/impalad: line 2: /etc/default/impala: No such file or directory
    /usr/bin/impalad: line 16: /impalad: No such file or directory
    /usr/bin/impalad: line 16: exec: /impalad: cannot execute: No such file or directory

    **/

    After copying /etc/default/impala from my second node, i was able to start statestored service but impalad service is still not starting and now
    throwing following error



    /**

    + date
    + date
    Fri Jan 25 10:02:48 PST 2013
    + export
    IMPALA_CONF_DIR=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/impala-conf
    +
    IMPALA_CONF_DIR=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/impala-conf
    + export
    HADOOP_CONF_DIR=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/hadoop-conf
    +
    HADOOP_CONF_DIR=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/hadoop-conf
    + export
    HIVE_CONF_DIR=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/hive-conf
    +
    HIVE_CONF_DIR=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/hive-conf
    + export
    HBASE_CONF_DIR=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/hbase-conf
    +
    HBASE_CONF_DIR=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/hbase-conf
    +
    FLAG_FILE=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/impala-conf/impalad_flags
    + perl -pi -e
    's#{{CMF_CONF_DIR}}#/var/run/cloudera-scm-agent/process/204-impala-IMPALAD#g'
    /var/run/cloudera-scm-agent/process/204-impala-IMPALAD/impala-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad
    --flagfile=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/impala-conf/impalad_flags
    loadFileSystems error:
    (unable to get stack trace for java.lang.NoClassDefFoundError exception:
    ExceptionUtils::getStackTrace error.)
    hdfsBuilderConnect(forceNewInstance=0, nn=default, port=0,
    kerbTicketCachePath=(NULL), userName=(NULL)) error:
    (unable to get stack trace for java.lang.NoClassDefFoundError exception:
    ExceptionUtils::getStackTrace error.)
    Exception in thread "main" java.lang.NoClassDefFoundError:
    com/cloudera/impala/common/JniUtil
    Caused by: java.lang.ClassNotFoundException:
    com.cloudera.impala.common.JniUtil
    at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
    E0125 10:02:50.076205 29022 impalad-main.cc:81] Failed to find JniUtil
    class.

    */

    Seems it is a classpath issue now?

    Thanks.
    -Dipesh

    On Thursday, January 24, 2013 5:24:15 PM UTC-8, vikas wrote:

    Hi Dipesh,

    I think you need to change the baseurl to "http://archive.cloudera.com/*
    *cm4/redhat/6/x86_64/cm/4/<http://archive.cloudera.com/cm4/redhat/6/x86_64/cm/4/>",
    which is the url that will fetch the latest CM whenever you run "yum
    upgrade" command. For Imapla upgrade you may have to do something similar
    (depending on the content of your impala repo file)

    The file /usr/bin/impalad (which is a bash file) is the one that sets
    environment variables (and is missing in your setup).

    - Vikas

    On Thu, Jan 24, 2013 at 5:02 PM, Deepsoft wrote:

    Vikas,

    It is reporting version 4.1.2. I wanted to upgrade it to 4.1.3 so that
    I can upgrade to impala 0.4. Here is the content of my
    cloudera-manager.repo file

    [cloudera-manager]
    name = Cloudera Manager, Version 4.1.2
    baseurl = http://archive.cloudera.com/**cm4/redhat/6/x86_64/cm/4.1.2/<http://archive.cloudera.com/cm4/redhat/6/x86_64/cm/4.1.2/>
    gpgkey = http://archive.cloudera.com/**redhat/cdh/RPM-GPG-KEY-**
    cloudera <http://archive.cloudera.com/redhat/cdh/RPM-GPG-KEY-cloudera>
    gpgcheck = 1

    Can you please confirm which file sets all the required enviornment
    variables? I have created a sym link to impalad in /usr/bin from the
    installed folder /usr/lib/impala/sbin/impalad.

    Thanks.
    -Dipesh


    On Thursday, January 24, 2013 4:47:22 PM UTC-8, vikas wrote:

    Hi Dipesh,

    I am actually not sure what happens if you build impala on a machine
    where you already have impala installed from package. If you install
    impala, the file '/usr/bin/impalad' is a bash file that takes care of
    setting all environment before it exec the actual 'impalad' binary. For
    some reason in your setup that file is missing and launching impalad
    directly fails because env insn't setup correctly.

    And upgrading CM shouldn't change CDH component versions, so mixed CDH
    versions were probably the case before too. What versions does it report?

    Vikas

    On Thu, Jan 24, 2013 at 4:30 PM, Deepsoft wrote:

    Vikas,

    Thanks for replying. After linking back impalad and statestored from
    /usr/lib/impala/sbin, I was able to start statestored but got some error
    while starting impalad

    /**

    + date
    + date
    Thu Jan 24 16:18:36 PST 2013
    + export IMPALA_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/impala-conf
    + IMPALA_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/impala-conf
    + export HADOOP_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/hadoop-conf
    + HADOOP_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/hadoop-conf
    + export HIVE_CONF_DIR=/var/run/**clouder**a-scm-agent/process/**197-**impala-IMPALAD/hive-conf
    + HIVE_CONF_DIR=/var/run/**clouder**a-scm-agent/process/**197-**impala-IMPALAD/hive-conf
    + export HBASE_CONF_DIR=/var/run/**cloude**ra-scm-agent/process/**197-**impala-IMPALAD/hbase-conf
    + HBASE_CONF_DIR=/var/run/**cloude**ra-scm-agent/process/**197-**impala-IMPALAD/hbase-conf
    + FLAG_FILE=/var/run/cloudera-**sc**m-agent/process/197-impala-**IMP**ALAD/impala-conf/impalad_**flags
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/**c**loudera-scm-agent/process/**197-**impala-IMPALAD#g' /var/run/cloudera-scm-agent/**pr**ocess/197-impala-IMPALAD/**impal**a-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad --flagfile=/var/run/cloudera-**s**cm-agent/process/197-impala-**IM**PALAD/impala-conf/impalad_**flag**s
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    E0124 16:18:36.749694 19556 impalad-main.cc:81] Failed to get/create JVM
    + date
    + date
    Thu Jan 24 16:18:37 PST 2013
    + export IMPALA_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/impala-conf
    + IMPALA_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/impala-conf
    + export HADOOP_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/hadoop-conf
    + HADOOP_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/hadoop-conf
    + export HIVE_CONF_DIR=/var/run/**clouder**a-scm-agent/process/**197-**impala-IMPALAD/hive-conf
    + HIVE_CONF_DIR=/var/run/**clouder**a-scm-agent/process/**197-**impala-IMPALAD/hive-conf
    + export HBASE_CONF_DIR=/var/run/**cloude**ra-scm-agent/process/**197-**impala-IMPALAD/hbase-conf
    + HBASE_CONF_DIR=/var/run/**cloude**ra-scm-agent/process/**197-**impala-IMPALAD/hbase-conf
    + FLAG_FILE=/var/run/cloudera-**sc**m-agent/process/197-impala-**IMP**ALAD/impala-conf/impalad_**flags
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/**c**loudera-scm-agent/process/**197-**impala-IMPALAD#g' /var/run/cloudera-scm-agent/**pr**ocess/197-impala-IMPALAD/**impal**a-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad --flagfile=/var/run/cloudera-**s**cm-agent/process/197-impala-**IM**PALAD/impala-conf/impalad_**flag**s
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    E0124 16:18:37.914975 19591 impalad-main.cc:81] Failed to get/create JVM
    + date
    + date
    Thu Jan 24 16:18:39 PST 2013
    + export IMPALA_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/impala-conf
    + IMPALA_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/impala-conf
    + export HADOOP_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/hadoop-conf
    + HADOOP_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/hadoop-conf
    + export HIVE_CONF_DIR=/var/run/**clouder**a-scm-agent/process/**197-**impala-IMPALAD/hive-conf
    + HIVE_CONF_DIR=/var/run/**clouder**a-scm-agent/process/**197-**impala-IMPALAD/hive-conf
    + export HBASE_CONF_DIR=/var/run/**cloude**ra-scm-agent/process/**197-**impala-IMPALAD/hbase-conf
    + HBASE_CONF_DIR=/var/run/**cloude**ra-scm-agent/process/**197-**impala-IMPALAD/hbase-conf
    + FLAG_FILE=/var/run/cloudera-**sc**m-agent/process/197-impala-**IMP**ALAD/impala-conf/impalad_**flags
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/**c**loudera-scm-agent/process/**197-**impala-IMPALAD#g' /var/run/cloudera-scm-agent/**pr**ocess/197-impala-IMPALAD/**impal**a-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad --flagfile=/var/run/cloudera-**s**cm-agent/process/197-impala-**IM**PALAD/impala-conf/impalad_**flag**s
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    E0124 16:18:39.950352 19595 impalad-main.cc:81] Failed to get/create JVM
    + date
    + date
    Thu Jan 24 16:18:42 PST 2013
    + export IMPALA_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/impala-conf
    + IMPALA_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/impala-conf
    + export HADOOP_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/hadoop-conf
    + HADOOP_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/hadoop-conf
    + export HIVE_CONF_DIR=/var/run/**clouder**a-scm-agent/process/**197-**impala-IMPALAD/hive-conf
    + HIVE_CONF_DIR=/var/run/**clouder**a-scm-agent/process/**197-**impala-IMPALAD/hive-conf
    + export HBASE_CONF_DIR=/var/run/**cloude**ra-scm-agent/process/**197-**impala-IMPALAD/hbase-conf
    + HBASE_CONF_DIR=/var/run/**cloude**ra-scm-agent/process/**197-**impala-IMPALAD/hbase-conf
    + FLAG_FILE=/var/run/cloudera-**sc**m-agent/process/197-impala-**IMP**ALAD/impala-conf/impalad_**flags
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/**c**loudera-scm-agent/process/**197-**impala-IMPALAD#g' /var/run/cloudera-scm-agent/**pr**ocess/197-impala-IMPALAD/**impal**a-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad --flagfile=/var/run/cloudera-**s**cm-agent/process/197-impala-**IM**PALAD/impala-conf/impalad_**flag**s
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    E0124 16:18:42.983335 19599 impalad-main.cc:81] Failed to get/create JVM

    **/

    My install was working fine before I started building impalad - so I'm not sure whether build script somehow deletes existing



    impalad/statestored if it finds in /usr/bin folder?

    Now it is complaining about some missing environment variables - I wasn't setting these before as well. Did I lose a script file which is setting environment



    variable?

    Yes, I have not updated my CM yet. When I tried to udpate using the instructions here https://ccp.cloudera.com/**displ**ay/FREE41DOC/Upgrade+**from+**Cloudera+Manager+4+to+**the+**Latest+Version+of+**Cloudera+**Manager+4 <https://ccp.cloudera.com/display/FREE41DOC/Upgrade+from+Cloudera+Manager+4+to+the+Latest+Version+of+Cloudera+Manager+4>



    I got the following message -

    [root@khadi01-rhvm01 local]# sudo yum update --disablerepo='*' --enablerepo=cloudera-manager
    Loaded plugins: product-id, refresh-packagekit, rhnplugin, subscription-manager



    Updating certificate-based repositories.
    Unable to read consumer identity
    Repository 'dvd' is missing name in configuration, using id
    cloudera-manager | 951 B 00:00



    cloudera-manager/primary | 3.6 kB 00:00
    cloudera-manager 7/7
    Setting up Update Process
    No Packages marked for Update



    Now when I check "host inspector" result, I see that there exists mixed CDH versions. It seems my upgrade didn't go well. Correct?

    Thanks.
    -Dipesh


    Thanks.
    -Dipesh



    On Thursday, January 24, 2013 3:03:20 PM UTC-8, vikas wrote:

    Hi Dipesh,

    The binary is looked in the default PATH and impalad isn't present
    in it. You might want to symlink the build you created to /usr/bin.

    Also, it seems you aren't using the latest CM build, it has been
    updated to use full impala path when calling 'exec' instead of just
    'impalad'.

    Vikas

    On Thu, Jan 24, 2013 at 2:41 PM, Deepsoft wrote:

    I have impala and other services running on a node. I have built
    impalad on the same node - following the instructions.

    Now when I'm trying to start impala service along with other
    services through cloudera manager but it is not starting and logs are
    showing this error -

    /**
    + exec impalad --flagfile=/var/run/cloudera-**s****
    cm-agent/process/157-impala-**IM****PALAD/impala-conf/impalad_**
    flag****s
    /usr/lib64/cmf/service/impala/******impala.sh: line 23: exec:
    impalad: not found

    */

    When I search in my system I see that I have impalad at my
    installed location which is /usr/lib/imapla/sbin/impalad

    # find / -name impalad
    /var/log/impalad
    /usr/lib/impala/sbin/impalad
    /usr/lib64/cmf/agent/src/cmf/**m****onitor/impalad
    /home/src/impala/impala/be/**bui****ld/debug/service/impalad

    The highlighted location is the one where I have build impalad.

    Where is imapla.sh looking for impalad? I didn't modify anything -
    Does the build process delete impalad? Where should I copy impalad so that
    this script file finds it again and start the service?

    Thanks.

    --

    --
    --
  • Deepsoft at Jan 25, 2013 at 7:20 pm ⇧
    Vikas,

    After copying jar directory from Node2 to Node1, I'm able to start impalad
    service. Thanks for providing useful information about install locations.

    I will upgrade to the latest version.

    I'm not sure whether it is a bug that build scripts are deleting installled
    files/folders - please confirm and inform me I will file a jira for it.

    Thanks.
    -Dipesh

    On Friday, January 25, 2013 10:46:08 AM UTC-8, Deepsoft wrote:

    Vikas,

    Thanks. Hmnn...I had a working version of Impala so I'm not sure whether
    some jars got missing or got deleted. Here is the sequence of events -

    1. I have installed CDH 4.1.2 and impala 0.3 on Node1 and Nod2.
    2. Configured and ran impala on Node1.
    3. Built impala on Node1.
    4. Now when i tried to start impala on Node1, i found that following files
    were missing
    /usr/bin/imapad
    /usr/bin/statestored
    /etc/default/impala
    5. Copied these files from Node2.

    After this I'm getting CNFE (ClassNotFoundException) when I'm trying to
    start impala service.

    I didn't do cluster set up - wanted to make sure that it works on single
    node correctly then I was planning to add more nodes (well initially it
    happened that I had installed CDH and impala on two nodes). I'm trying to
    figure what went wrong after builiding and why the working state went
    berserk.

    Now I see that my /usr/lib/impala has only two folders -

    conf
    sbin

    When i compare it with my Node2, i see that number of directories have
    been nuked (most probably when I have built impala on this node??) and it
    is missing

    bin
    cloudera
    llvm-ir
    lib
    sbin-debug
    sbin-retail
    www

    Thanks.
    -Dipesh



    On Friday, January 25, 2013 10:25:01 AM UTC-8, vikas wrote:

    yes, /etc/default/impala is the script that sets the classpath, so you
    are probably missing the jars that come packaged with Impala. I think we
    need to know what you are trying to achieve here. That will help us provide
    a solution to the issue rather than responding to point questions. From the
    mail thread, here what I understand:

    1. You had working Impala installation with CM.
    2. On one of the nodes, you downloaded Impala source code and build it.
    3. As a result of that (or some command that got executed in that
    process), bunch of Impala installation files are missing
    4. Now Impala service will not start.

    Now are you trying to get the cluster back on track? Why not just
    uninstall Impala package and install it again? That will get all files back
    from yum in pristine state and it should work.

    - Vikas


    On Fri, Jan 25, 2013 at 10:11 AM, Deepsoft wrote:

    Vikas,

    Thanks. I will try upgrade using the below mentioned base url.

    I had same impala installation (CDH 4.1.2 and Imapla 0.3) on another
    node, so I took these bash files (impalad and statestored) and copied to my
    problematic node but then i go the following errors -

    /**

    + date
    + date
    Thu Jan 24 17:12:40 PST 2013
    + export IMPALA_CONF_DIR=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/impala-conf
    + IMPALA_CONF_DIR=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/impala-conf
    + export HADOOP_CONF_DIR=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/hadoop-conf
    + HADOOP_CONF_DIR=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/hadoop-conf
    + export HIVE_CONF_DIR=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/hive-conf
    + HIVE_CONF_DIR=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/hive-conf
    + export HBASE_CONF_DIR=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/hbase-conf
    + HBASE_CONF_DIR=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/hbase-conf
    + FLAG_FILE=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/impala-conf/impalad_flags
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/cloudera-scm-agent/process/202-impala-IMPALAD#g' /var/run/cloudera-scm-agent/process/202-impala-IMPALAD/impala-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad --flagfile=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/impala-conf/impalad_flags
    /usr/bin/impalad: line 2: /etc/default/impala: No such file or directory
    /usr/bin/impalad: line 16: /impalad: No such file or directory
    /usr/bin/impalad: line 16: exec: /impalad: cannot execute: No such file or directory

    **/

    After copying /etc/default/impala from my second node, i was able to start statestored service but impalad service is still not starting and now
    throwing following error



    /**

    + date
    + date
    Fri Jan 25 10:02:48 PST 2013
    + export
    IMPALA_CONF_DIR=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/impala-conf
    +
    IMPALA_CONF_DIR=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/impala-conf
    + export
    HADOOP_CONF_DIR=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/hadoop-conf
    +
    HADOOP_CONF_DIR=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/hadoop-conf
    + export
    HIVE_CONF_DIR=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/hive-conf
    +
    HIVE_CONF_DIR=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/hive-conf
    + export
    HBASE_CONF_DIR=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/hbase-conf
    +
    HBASE_CONF_DIR=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/hbase-conf
    +
    FLAG_FILE=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/impala-conf/impalad_flags
    + perl -pi -e
    's#{{CMF_CONF_DIR}}#/var/run/cloudera-scm-agent/process/204-impala-IMPALAD#g'
    /var/run/cloudera-scm-agent/process/204-impala-IMPALAD/impala-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad
    --flagfile=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/impala-conf/impalad_flags
    loadFileSystems error:
    (unable to get stack trace for java.lang.NoClassDefFoundError exception:
    ExceptionUtils::getStackTrace error.)
    hdfsBuilderConnect(forceNewInstance=0, nn=default, port=0,
    kerbTicketCachePath=(NULL), userName=(NULL)) error:
    (unable to get stack trace for java.lang.NoClassDefFoundError exception:
    ExceptionUtils::getStackTrace error.)
    Exception in thread "main" java.lang.NoClassDefFoundError:
    com/cloudera/impala/common/JniUtil
    Caused by: java.lang.ClassNotFoundException:
    com.cloudera.impala.common.JniUtil
    at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
    E0125 10:02:50.076205 29022 impalad-main.cc:81] Failed to find JniUtil
    class.

    */

    Seems it is a classpath issue now?

    Thanks.
    -Dipesh

    On Thursday, January 24, 2013 5:24:15 PM UTC-8, vikas wrote:

    Hi Dipesh,

    I think you need to change the baseurl to "http://archive.cloudera.com/
    **cm4/redhat/6/x86_64/cm/4/<http://archive.cloudera.com/cm4/redhat/6/x86_64/cm/4/>",
    which is the url that will fetch the latest CM whenever you run "yum
    upgrade" command. For Imapla upgrade you may have to do something similar
    (depending on the content of your impala repo file)

    The file /usr/bin/impalad (which is a bash file) is the one that sets
    environment variables (and is missing in your setup).

    - Vikas

    On Thu, Jan 24, 2013 at 5:02 PM, Deepsoft wrote:

    Vikas,

    It is reporting version 4.1.2. I wanted to upgrade it to 4.1.3 so that
    I can upgrade to impala 0.4. Here is the content of my
    cloudera-manager.repo file

    [cloudera-manager]
    name = Cloudera Manager, Version 4.1.2
    baseurl = http://archive.cloudera.com/**cm4/redhat/6/x86_64/cm/4.1.2/<http://archive.cloudera.com/cm4/redhat/6/x86_64/cm/4.1.2/>
    gpgkey = http://archive.cloudera.com/**redhat/cdh/RPM-GPG-KEY-**
    cloudera <http://archive.cloudera.com/redhat/cdh/RPM-GPG-KEY-cloudera>
    gpgcheck = 1

    Can you please confirm which file sets all the required enviornment
    variables? I have created a sym link to impalad in /usr/bin from the
    installed folder /usr/lib/impala/sbin/impalad.

    Thanks.
    -Dipesh


    On Thursday, January 24, 2013 4:47:22 PM UTC-8, vikas wrote:

    Hi Dipesh,

    I am actually not sure what happens if you build impala on a machine
    where you already have impala installed from package. If you install
    impala, the file '/usr/bin/impalad' is a bash file that takes care of
    setting all environment before it exec the actual 'impalad' binary. For
    some reason in your setup that file is missing and launching impalad
    directly fails because env insn't setup correctly.

    And upgrading CM shouldn't change CDH component versions, so mixed
    CDH versions were probably the case before too. What versions does it
    report?

    Vikas

    On Thu, Jan 24, 2013 at 4:30 PM, Deepsoft wrote:

    Vikas,

    Thanks for replying. After linking back impalad and statestored from
    /usr/lib/impala/sbin, I was able to start statestored but got some error
    while starting impalad

    /**

    + date
    + date
    Thu Jan 24 16:18:36 PST 2013
    + export IMPALA_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/impala-conf
    + IMPALA_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/impala-conf
    + export HADOOP_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/hadoop-conf
    + HADOOP_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/hadoop-conf
    + export HIVE_CONF_DIR=/var/run/**clouder**a-scm-agent/process/**197-**impala-IMPALAD/hive-conf
    + HIVE_CONF_DIR=/var/run/**clouder**a-scm-agent/process/**197-**impala-IMPALAD/hive-conf
    + export HBASE_CONF_DIR=/var/run/**cloude**ra-scm-agent/process/**197-**impala-IMPALAD/hbase-conf
    + HBASE_CONF_DIR=/var/run/**cloude**ra-scm-agent/process/**197-**impala-IMPALAD/hbase-conf
    + FLAG_FILE=/var/run/cloudera-**sc**m-agent/process/197-impala-**IMP**ALAD/impala-conf/impalad_**flags
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/**c**loudera-scm-agent/process/**197-**impala-IMPALAD#g' /var/run/cloudera-scm-agent/**pr**ocess/197-impala-IMPALAD/**impal**a-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad --flagfile=/var/run/cloudera-**s**cm-agent/process/197-impala-**IM**PALAD/impala-conf/impalad_**flag**s
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    E0124 16:18:36.749694 19556 impalad-main.cc:81] Failed to get/create JVM
    + date
    + date
    Thu Jan 24 16:18:37 PST 2013
    + export IMPALA_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/impala-conf
    + IMPALA_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/impala-conf
    + export HADOOP_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/hadoop-conf
    + HADOOP_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/hadoop-conf
    + export HIVE_CONF_DIR=/var/run/**clouder**a-scm-agent/process/**197-**impala-IMPALAD/hive-conf
    + HIVE_CONF_DIR=/var/run/**clouder**a-scm-agent/process/**197-**impala-IMPALAD/hive-conf
    + export HBASE_CONF_DIR=/var/run/**cloude**ra-scm-agent/process/**197-**impala-IMPALAD/hbase-conf
    + HBASE_CONF_DIR=/var/run/**cloude**ra-scm-agent/process/**197-**impala-IMPALAD/hbase-conf
    + FLAG_FILE=/var/run/cloudera-**sc**m-agent/process/197-impala-**IMP**ALAD/impala-conf/impalad_**flags
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/**c**loudera-scm-agent/process/**197-**impala-IMPALAD#g' /var/run/cloudera-scm-agent/**pr**ocess/197-impala-IMPALAD/**impal**a-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad --flagfile=/var/run/cloudera-**s**cm-agent/process/197-impala-**IM**PALAD/impala-conf/impalad_**flag**s
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    E0124 16:18:37.914975 19591 impalad-main.cc:81] Failed to get/create JVM
    + date
    + date
    Thu Jan 24 16:18:39 PST 2013
    + export IMPALA_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/impala-conf
    + IMPALA_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/impala-conf
    + export HADOOP_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/hadoop-conf
    + HADOOP_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/hadoop-conf
    + export HIVE_CONF_DIR=/var/run/**clouder**a-scm-agent/process/**197-**impala-IMPALAD/hive-conf
    + HIVE_CONF_DIR=/var/run/**clouder**a-scm-agent/process/**197-**impala-IMPALAD/hive-conf
    + export HBASE_CONF_DIR=/var/run/**cloude**ra-scm-agent/process/**197-**impala-IMPALAD/hbase-conf
    + HBASE_CONF_DIR=/var/run/**cloude**ra-scm-agent/process/**197-**impala-IMPALAD/hbase-conf
    + FLAG_FILE=/var/run/cloudera-**sc**m-agent/process/197-impala-**IMP**ALAD/impala-conf/impalad_**flags
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/**c**loudera-scm-agent/process/**197-**impala-IMPALAD#g' /var/run/cloudera-scm-agent/**pr**ocess/197-impala-IMPALAD/**impal**a-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad --flagfile=/var/run/cloudera-**s**cm-agent/process/197-impala-**IM**PALAD/impala-conf/impalad_**flag**s
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    E0124 16:18:39.950352 19595 impalad-main.cc:81] Failed to get/create JVM
    + date
    + date
    Thu Jan 24 16:18:42 PST 2013
    + export IMPALA_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/impala-conf
    + IMPALA_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/impala-conf
    + export HADOOP_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/hadoop-conf
    + HADOOP_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/hadoop-conf
    + export HIVE_CONF_DIR=/var/run/**clouder**a-scm-agent/process/**197-**impala-IMPALAD/hive-conf
    + HIVE_CONF_DIR=/var/run/**clouder**a-scm-agent/process/**197-**impala-IMPALAD/hive-conf
    + export HBASE_CONF_DIR=/var/run/**cloude**ra-scm-agent/process/**197-**impala-IMPALAD/hbase-conf
    + HBASE_CONF_DIR=/var/run/**cloude**ra-scm-agent/process/**197-**impala-IMPALAD/hbase-conf
    + FLAG_FILE=/var/run/cloudera-**sc**m-agent/process/197-impala-**IMP**ALAD/impala-conf/impalad_**flags
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/**c**loudera-scm-agent/process/**197-**impala-IMPALAD#g' /var/run/cloudera-scm-agent/**pr**ocess/197-impala-IMPALAD/**impal**a-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad --flagfile=/var/run/cloudera-**s**cm-agent/process/197-impala-**IM**PALAD/impala-conf/impalad_**flag**s
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    E0124 16:18:42.983335 19599 impalad-main.cc:81] Failed to get/create JVM

    **/

    My install was working fine before I started building impalad - so I'm not sure whether build script somehow deletes existing



    impalad/statestored if it finds in /usr/bin folder?

    Now it is complaining about some missing environment variables - I wasn't setting these before as well. Did I lose a script file which is setting environment



    variable?

    Yes, I have not updated my CM yet. When I tried to udpate using the instructions here https://ccp.cloudera.com/**displ**ay/FREE41DOC/Upgrade+**from+**Cloudera+Manager+4+to+**the+**Latest+Version+of+**Cloudera+**Manager+4 <https://ccp.cloudera.com/display/FREE41DOC/Upgrade+from+Cloudera+Manager+4+to+the+Latest+Version+of+Cloudera+Manager+4>



    I got the following message -

    [root@khadi01-rhvm01 local]# sudo yum update --disablerepo='*' --enablerepo=cloudera-manager
    Loaded plugins: product-id, refresh-packagekit, rhnplugin, subscription-manager



    Updating certificate-based repositories.
    Unable to read consumer identity
    Repository 'dvd' is missing name in configuration, using id
    cloudera-manager | 951 B 00:00



    cloudera-manager/primary | 3.6 kB 00:00
    cloudera-manager 7/7
    Setting up Update Process
    No Packages marked for Update



    Now when I check "host inspector" result, I see that there exists mixed CDH versions. It seems my upgrade didn't go well. Correct?

    Thanks.
    -Dipesh


    Thanks.
    -Dipesh



    On Thursday, January 24, 2013 3:03:20 PM UTC-8, vikas wrote:

    Hi Dipesh,

    The binary is looked in the default PATH and impalad isn't present
    in it. You might want to symlink the build you created to /usr/bin.

    Also, it seems you aren't using the latest CM build, it has been
    updated to use full impala path when calling 'exec' instead of just
    'impalad'.

    Vikas

    On Thu, Jan 24, 2013 at 2:41 PM, Deepsoft wrote:

    I have impala and other services running on a node. I have built
    impalad on the same node - following the instructions.

    Now when I'm trying to start impala service along with other
    services through cloudera manager but it is not starting and logs are
    showing this error -

    /**
    + exec impalad --flagfile=/var/run/cloudera-**s****
    cm-agent/process/157-impala-**IM****PALAD/impala-conf/impalad_**
    flag****s
    /usr/lib64/cmf/service/impala/******impala.sh: line 23: exec:
    impalad: not found

    */

    When I search in my system I see that I have impalad at my
    installed location which is /usr/lib/imapla/sbin/impalad

    # find / -name impalad
    /var/log/impalad
    /usr/lib/impala/sbin/impalad
    /usr/lib64/cmf/agent/src/cmf/**m****onitor/impalad
    /home/src/impala/impala/be/**bui****ld/debug/service/impalad

    The highlighted location is the one where I have build impalad.

    Where is imapla.sh looking for impalad? I didn't modify anything -
    Does the build process delete impalad? Where should I copy impalad so that
    this script file finds it again and start the service?

    Thanks.

    --

    --
    --
  • Deepsoft at Jan 25, 2013 at 7:22 pm ⇧
    I meant jars in /usr/lib/imapla/lib directory
    On Friday, January 25, 2013 11:20:10 AM UTC-8, Deepsoft wrote:

    Vikas,

    After copying jar directory from Node2 to Node1, I'm able to start impalad
    service. Thanks for providing useful information about install locations.

    I will upgrade to the latest version.

    I'm not sure whether it is a bug that build scripts are deleting
    installled files/folders - please confirm and inform me I will file a jira
    for it.

    Thanks.
    -Dipesh

    On Friday, January 25, 2013 10:46:08 AM UTC-8, Deepsoft wrote:

    Vikas,

    Thanks. Hmnn...I had a working version of Impala so I'm not sure whether
    some jars got missing or got deleted. Here is the sequence of events -

    1. I have installed CDH 4.1.2 and impala 0.3 on Node1 and Nod2.
    2. Configured and ran impala on Node1.
    3. Built impala on Node1.
    4. Now when i tried to start impala on Node1, i found that following
    files were missing
    /usr/bin/imapad
    /usr/bin/statestored
    /etc/default/impala
    5. Copied these files from Node2.

    After this I'm getting CNFE (ClassNotFoundException) when I'm trying to
    start impala service.

    I didn't do cluster set up - wanted to make sure that it works on single
    node correctly then I was planning to add more nodes (well initially it
    happened that I had installed CDH and impala on two nodes). I'm trying to
    figure what went wrong after builiding and why the working state went
    berserk.

    Now I see that my /usr/lib/impala has only two folders -

    conf
    sbin

    When i compare it with my Node2, i see that number of directories have
    been nuked (most probably when I have built impala on this node??) and it
    is missing

    bin
    cloudera
    llvm-ir
    lib
    sbin-debug
    sbin-retail
    www

    Thanks.
    -Dipesh



    On Friday, January 25, 2013 10:25:01 AM UTC-8, vikas wrote:

    yes, /etc/default/impala is the script that sets the classpath, so you
    are probably missing the jars that come packaged with Impala. I think we
    need to know what you are trying to achieve here. That will help us provide
    a solution to the issue rather than responding to point questions. From the
    mail thread, here what I understand:

    1. You had working Impala installation with CM.
    2. On one of the nodes, you downloaded Impala source code and build it.
    3. As a result of that (or some command that got executed in that
    process), bunch of Impala installation files are missing
    4. Now Impala service will not start.

    Now are you trying to get the cluster back on track? Why not just
    uninstall Impala package and install it again? That will get all files back
    from yum in pristine state and it should work.

    - Vikas


    On Fri, Jan 25, 2013 at 10:11 AM, Deepsoft wrote:

    Vikas,

    Thanks. I will try upgrade using the below mentioned base url.

    I had same impala installation (CDH 4.1.2 and Imapla 0.3) on another
    node, so I took these bash files (impalad and statestored) and copied to my
    problematic node but then i go the following errors -

    /**

    + date
    + date
    Thu Jan 24 17:12:40 PST 2013
    + export IMPALA_CONF_DIR=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/impala-conf
    + IMPALA_CONF_DIR=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/impala-conf
    + export HADOOP_CONF_DIR=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/hadoop-conf
    + HADOOP_CONF_DIR=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/hadoop-conf
    + export HIVE_CONF_DIR=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/hive-conf
    + HIVE_CONF_DIR=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/hive-conf
    + export HBASE_CONF_DIR=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/hbase-conf
    + HBASE_CONF_DIR=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/hbase-conf
    + FLAG_FILE=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/impala-conf/impalad_flags
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/cloudera-scm-agent/process/202-impala-IMPALAD#g' /var/run/cloudera-scm-agent/process/202-impala-IMPALAD/impala-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad --flagfile=/var/run/cloudera-scm-agent/process/202-impala-IMPALAD/impala-conf/impalad_flags
    /usr/bin/impalad: line 2: /etc/default/impala: No such file or directory
    /usr/bin/impalad: line 16: /impalad: No such file or directory
    /usr/bin/impalad: line 16: exec: /impalad: cannot execute: No such file or directory

    **/

    After copying /etc/default/impala from my second node, i was able to start statestored service but impalad service is still not starting and now
    throwing following error



    /**

    + date
    + date
    Fri Jan 25 10:02:48 PST 2013
    + export
    IMPALA_CONF_DIR=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/impala-conf
    +
    IMPALA_CONF_DIR=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/impala-conf
    + export
    HADOOP_CONF_DIR=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/hadoop-conf
    +
    HADOOP_CONF_DIR=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/hadoop-conf
    + export
    HIVE_CONF_DIR=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/hive-conf
    +
    HIVE_CONF_DIR=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/hive-conf
    + export
    HBASE_CONF_DIR=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/hbase-conf
    +
    HBASE_CONF_DIR=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/hbase-conf
    +
    FLAG_FILE=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/impala-conf/impalad_flags
    + perl -pi -e
    's#{{CMF_CONF_DIR}}#/var/run/cloudera-scm-agent/process/204-impala-IMPALAD#g'
    /var/run/cloudera-scm-agent/process/204-impala-IMPALAD/impala-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad
    --flagfile=/var/run/cloudera-scm-agent/process/204-impala-IMPALAD/impala-conf/impalad_flags
    loadFileSystems error:
    (unable to get stack trace for java.lang.NoClassDefFoundError
    exception: ExceptionUtils::getStackTrace error.)
    hdfsBuilderConnect(forceNewInstance=0, nn=default, port=0,
    kerbTicketCachePath=(NULL), userName=(NULL)) error:
    (unable to get stack trace for java.lang.NoClassDefFoundError
    exception: ExceptionUtils::getStackTrace error.)
    Exception in thread "main" java.lang.NoClassDefFoundError:
    com/cloudera/impala/common/JniUtil
    Caused by: java.lang.ClassNotFoundException:
    com.cloudera.impala.common.JniUtil
    at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
    E0125 10:02:50.076205 29022 impalad-main.cc:81] Failed to find JniUtil
    class.

    */

    Seems it is a classpath issue now?

    Thanks.
    -Dipesh

    On Thursday, January 24, 2013 5:24:15 PM UTC-8, vikas wrote:

    Hi Dipesh,

    I think you need to change the baseurl to "
    http://archive.cloudera.com/**cm4/redhat/6/x86_64/cm/4/<http://archive.cloudera.com/cm4/redhat/6/x86_64/cm/4/>",
    which is the url that will fetch the latest CM whenever you run "yum
    upgrade" command. For Imapla upgrade you may have to do something similar
    (depending on the content of your impala repo file)

    The file /usr/bin/impalad (which is a bash file) is the one that sets
    environment variables (and is missing in your setup).

    - Vikas

    On Thu, Jan 24, 2013 at 5:02 PM, Deepsoft wrote:

    Vikas,

    It is reporting version 4.1.2. I wanted to upgrade it to 4.1.3 so
    that I can upgrade to impala 0.4. Here is the content of my
    cloudera-manager.repo file

    [cloudera-manager]
    name = Cloudera Manager, Version 4.1.2
    baseurl = http://archive.cloudera.com/**cm4/redhat/6/x86_64/cm/4.1.2/<http://archive.cloudera.com/cm4/redhat/6/x86_64/cm/4.1.2/>
    gpgkey = http://archive.cloudera.com/**redhat/cdh/RPM-GPG-KEY-**
    cloudera<http://archive.cloudera.com/redhat/cdh/RPM-GPG-KEY-cloudera>
    gpgcheck = 1

    Can you please confirm which file sets all the required enviornment
    variables? I have created a sym link to impalad in /usr/bin from the
    installed folder /usr/lib/impala/sbin/impalad.

    Thanks.
    -Dipesh


    On Thursday, January 24, 2013 4:47:22 PM UTC-8, vikas wrote:

    Hi Dipesh,

    I am actually not sure what happens if you build impala on a machine
    where you already have impala installed from package. If you install
    impala, the file '/usr/bin/impalad' is a bash file that takes care of
    setting all environment before it exec the actual 'impalad' binary. For
    some reason in your setup that file is missing and launching impalad
    directly fails because env insn't setup correctly.

    And upgrading CM shouldn't change CDH component versions, so mixed
    CDH versions were probably the case before too. What versions does it
    report?

    Vikas

    On Thu, Jan 24, 2013 at 4:30 PM, Deepsoft wrote:

    Vikas,

    Thanks for replying. After linking back impalad and statestored
    from /usr/lib/impala/sbin, I was able to start statestored but got some
    error while starting impalad

    /**

    + date
    + date
    Thu Jan 24 16:18:36 PST 2013
    + export IMPALA_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/impala-conf
    + IMPALA_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/impala-conf
    + export HADOOP_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/hadoop-conf
    + HADOOP_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/hadoop-conf
    + export HIVE_CONF_DIR=/var/run/**clouder**a-scm-agent/process/**197-**impala-IMPALAD/hive-conf
    + HIVE_CONF_DIR=/var/run/**clouder**a-scm-agent/process/**197-**impala-IMPALAD/hive-conf
    + export HBASE_CONF_DIR=/var/run/**cloude**ra-scm-agent/process/**197-**impala-IMPALAD/hbase-conf
    + HBASE_CONF_DIR=/var/run/**cloude**ra-scm-agent/process/**197-**impala-IMPALAD/hbase-conf
    + FLAG_FILE=/var/run/cloudera-**sc**m-agent/process/197-impala-**IMP**ALAD/impala-conf/impalad_**flags
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/**c**loudera-scm-agent/process/**197-**impala-IMPALAD#g' /var/run/cloudera-scm-agent/**pr**ocess/197-impala-IMPALAD/**impal**a-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad --flagfile=/var/run/cloudera-**s**cm-agent/process/197-impala-**IM**PALAD/impala-conf/impalad_**flag**s
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    E0124 16:18:36.749694 19556 impalad-main.cc:81] Failed to get/create JVM
    + date
    + date
    Thu Jan 24 16:18:37 PST 2013
    + export IMPALA_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/impala-conf
    + IMPALA_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/impala-conf
    + export HADOOP_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/hadoop-conf
    + HADOOP_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/hadoop-conf
    + export HIVE_CONF_DIR=/var/run/**clouder**a-scm-agent/process/**197-**impala-IMPALAD/hive-conf
    + HIVE_CONF_DIR=/var/run/**clouder**a-scm-agent/process/**197-**impala-IMPALAD/hive-conf
    + export HBASE_CONF_DIR=/var/run/**cloude**ra-scm-agent/process/**197-**impala-IMPALAD/hbase-conf
    + HBASE_CONF_DIR=/var/run/**cloude**ra-scm-agent/process/**197-**impala-IMPALAD/hbase-conf
    + FLAG_FILE=/var/run/cloudera-**sc**m-agent/process/197-impala-**IMP**ALAD/impala-conf/impalad_**flags
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/**c**loudera-scm-agent/process/**197-**impala-IMPALAD#g' /var/run/cloudera-scm-agent/**pr**ocess/197-impala-IMPALAD/**impal**a-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad --flagfile=/var/run/cloudera-**s**cm-agent/process/197-impala-**IM**PALAD/impala-conf/impalad_**flag**s
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    E0124 16:18:37.914975 19591 impalad-main.cc:81] Failed to get/create JVM
    + date
    + date
    Thu Jan 24 16:18:39 PST 2013
    + export IMPALA_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/impala-conf
    + IMPALA_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/impala-conf
    + export HADOOP_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/hadoop-conf
    + HADOOP_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/hadoop-conf
    + export HIVE_CONF_DIR=/var/run/**clouder**a-scm-agent/process/**197-**impala-IMPALAD/hive-conf
    + HIVE_CONF_DIR=/var/run/**clouder**a-scm-agent/process/**197-**impala-IMPALAD/hive-conf
    + export HBASE_CONF_DIR=/var/run/**cloude**ra-scm-agent/process/**197-**impala-IMPALAD/hbase-conf
    + HBASE_CONF_DIR=/var/run/**cloude**ra-scm-agent/process/**197-**impala-IMPALAD/hbase-conf
    + FLAG_FILE=/var/run/cloudera-**sc**m-agent/process/197-impala-**IMP**ALAD/impala-conf/impalad_**flags
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/**c**loudera-scm-agent/process/**197-**impala-IMPALAD#g' /var/run/cloudera-scm-agent/**pr**ocess/197-impala-IMPALAD/**impal**a-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad --flagfile=/var/run/cloudera-**s**cm-agent/process/197-impala-**IM**PALAD/impala-conf/impalad_**flag**s
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    E0124 16:18:39.950352 19595 impalad-main.cc:81] Failed to get/create JVM
    + date
    + date
    Thu Jan 24 16:18:42 PST 2013
    + export IMPALA_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/impala-conf
    + IMPALA_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/impala-conf
    + export HADOOP_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/hadoop-conf
    + HADOOP_CONF_DIR=/var/run/**cloud**era-scm-agent/process/**197-**impala-IMPALAD/hadoop-conf
    + export HIVE_CONF_DIR=/var/run/**clouder**a-scm-agent/process/**197-**impala-IMPALAD/hive-conf
    + HIVE_CONF_DIR=/var/run/**clouder**a-scm-agent/process/**197-**impala-IMPALAD/hive-conf
    + export HBASE_CONF_DIR=/var/run/**cloude**ra-scm-agent/process/**197-**impala-IMPALAD/hbase-conf
    + HBASE_CONF_DIR=/var/run/**cloude**ra-scm-agent/process/**197-**impala-IMPALAD/hbase-conf
    + FLAG_FILE=/var/run/cloudera-**sc**m-agent/process/197-impala-**IMP**ALAD/impala-conf/impalad_**flags
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/**c**loudera-scm-agent/process/**197-**impala-IMPALAD#g' /var/run/cloudera-scm-agent/**pr**ocess/197-impala-IMPALAD/**impal**a-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad --flagfile=/var/run/cloudera-**s**cm-agent/process/197-impala-**IM**PALAD/impala-conf/impalad_**flag**s
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    E0124 16:18:42.983335 19599 impalad-main.cc:81] Failed to get/create JVM

    **/

    My install was working fine before I started building impalad - so I'm not sure whether build script somehow deletes existing



    impalad/statestored if it finds in /usr/bin folder?

    Now it is complaining about some missing environment variables - I wasn't setting these before as well. Did I lose a script file which is setting environment



    variable?

    Yes, I have not updated my CM yet. When I tried to udpate using the instructions here https://ccp.cloudera.com/**displ**ay/FREE41DOC/Upgrade+**from+**Cloudera+Manager+4+to+**the+**Latest+Version+of+**Cloudera+**Manager+4 <https://ccp.cloudera.com/display/FREE41DOC/Upgrade+from+Cloudera+Manager+4+to+the+Latest+Version+of+Cloudera+Manager+4>



    I got the following message -

    [root@khadi01-rhvm01 local]# sudo yum update --disablerepo='*' --enablerepo=cloudera-manager
    Loaded plugins: product-id, refresh-packagekit, rhnplugin, subscription-manager



    Updating certificate-based repositories.
    Unable to read consumer identity
    Repository 'dvd' is missing name in configuration, using id
    cloudera-manager | 951 B 00:00



    cloudera-manager/primary | 3.6 kB 00:00
    cloudera-manager 7/7
    Setting up Update Process
    No Packages marked for Update



    Now when I check "host inspector" result, I see that there exists mixed CDH versions. It seems my upgrade didn't go well. Correct?

    Thanks.
    -Dipesh


    Thanks.
    -Dipesh



    On Thursday, January 24, 2013 3:03:20 PM UTC-8, vikas wrote:

    Hi Dipesh,

    The binary is looked in the default PATH and impalad isn't present
    in it. You might want to symlink the build you created to /usr/bin.

    Also, it seems you aren't using the latest CM build, it has been
    updated to use full impala path when calling 'exec' instead of just
    'impalad'.

    Vikas

    On Thu, Jan 24, 2013 at 2:41 PM, Deepsoft wrote:

    I have impala and other services running on a node. I have built
    impalad on the same node - following the instructions.

    Now when I'm trying to start impala service along with other
    services through cloudera manager but it is not starting and logs are
    showing this error -

    /**
    + exec impalad --flagfile=/var/run/cloudera-**s****
    cm-agent/process/157-impala-**IM****PALAD/impala-conf/impalad_**
    flag****s
    /usr/lib64/cmf/service/impala/******impala.sh: line 23: exec:
    impalad: not found

    */

    When I search in my system I see that I have impalad at my
    installed location which is /usr/lib/imapla/sbin/impalad

    # find / -name impalad
    /var/log/impalad
    /usr/lib/impala/sbin/impalad
    /usr/lib64/cmf/agent/src/cmf/**m****onitor/impalad
    /home/src/impala/impala/be/**bui****ld/debug/service/impalad

    The highlighted location is the one where I have build impalad.

    Where is imapla.sh looking for impalad? I didn't modify anything
    - Does the build process delete impalad? Where should I copy impalad so
    that this script file finds it again and start the service?

    Thanks.

    --

    --
    --
  • Deepsoft at Jan 28, 2013 at 10:43 pm ⇧
    Vikas,

    I was following build/debug/test approach on the same machine with the
    assumption that these are totally isolated and won't cause any problem.

    I'm not sure whether you or anyone at cloudera will get time to reproduce
    it - if reproduced then it will let developer do the above on the same
    system otherwise this post will serve a reference to troubleshoot the
    problem.

    Thanks.
    -Dipesh
    On Friday, January 25, 2013 5:28:30 PM UTC-8, vikas wrote:

    Hi Dipesh,

    I am not sure that this is a bug. You are combining a
    production/deployment setup and development setup, two different phases of
    software lifecycle. CM and yum repositories are designed with out-of-the
    deployment experience in mind.

    Just curious why you picked this setup (i.e why aren't using your
    development box to build Impala and then copy the updated binaries to
    deployment cluster)?

    - Vikas


    On Fri, Jan 25, 2013 at 11:22 AM, Deepsoft <dipeshs...@gmail.com<javascript:>
    wrote:
    I meant jars in /usr/lib/imapla/lib directory

    On Friday, January 25, 2013 11:20:10 AM UTC-8, Deepsoft wrote:

    Vikas,

    After copying jar directory from Node2 to Node1, I'm able to start
    impalad service. Thanks for providing useful information about install
    locations.

    I will upgrade to the latest version.

    I'm not sure whether it is a bug that build scripts are deleting
    installled files/folders - please confirm and inform me I will file a jira
    for it.

    Thanks.
    -Dipesh

    On Friday, January 25, 2013 10:46:08 AM UTC-8, Deepsoft wrote:

    Vikas,

    Thanks. Hmnn...I had a working version of Impala so I'm not sure
    whether some jars got missing or got deleted. Here is the sequence of
    events -

    1. I have installed CDH 4.1.2 and impala 0.3 on Node1 and Nod2.
    2. Configured and ran impala on Node1.
    3. Built impala on Node1.
    4. Now when i tried to start impala on Node1, i found that following
    files were missing
    /usr/bin/imapad
    /usr/bin/statestored
    /etc/default/impala
    5. Copied these files from Node2.

    After this I'm getting CNFE (ClassNotFoundException) when I'm trying to
    start impala service.

    I didn't do cluster set up - wanted to make sure that it works on
    single node correctly then I was planning to add more nodes (well initially
    it happened that I had installed CDH and impala on two nodes). I'm trying
    to figure what went wrong after builiding and why the working state went
    berserk.

    Now I see that my /usr/lib/impala has only two folders -

    conf
    sbin

    When i compare it with my Node2, i see that number of directories have
    been nuked (most probably when I have built impala on this node??) and it
    is missing

    bin
    cloudera
    llvm-ir
    lib
    sbin-debug
    sbin-retail
    www

    Thanks.
    -Dipesh



    On Friday, January 25, 2013 10:25:01 AM UTC-8, vikas wrote:

    yes, /etc/default/impala is the script that sets the classpath, so you
    are probably missing the jars that come packaged with Impala. I think we
    need to know what you are trying to achieve here. That will help us provide
    a solution to the issue rather than responding to point questions. From the
    mail thread, here what I understand:

    1. You had working Impala installation with CM.
    2. On one of the nodes, you downloaded Impala source code and build it.
    3. As a result of that (or some command that got executed in that
    process), bunch of Impala installation files are missing
    4. Now Impala service will not start.

    Now are you trying to get the cluster back on track? Why not just
    uninstall Impala package and install it again? That will get all files back
    from yum in pristine state and it should work.

    - Vikas


    On Fri, Jan 25, 2013 at 10:11 AM, Deepsoft wrote:

    Vikas,

    Thanks. I will try upgrade using the below mentioned base url.

    I had same impala installation (CDH 4.1.2 and Imapla 0.3) on another
    node, so I took these bash files (impalad and statestored) and copied to my
    problematic node but then i go the following errors -

    /**

    + date
    + date
    Thu Jan 24 17:12:40 PST 2013
    + export IMPALA_CONF_DIR=/var/run/**cloudera-scm-agent/process/**202-impala-IMPALAD/impala-conf
    + IMPALA_CONF_DIR=/var/run/**cloudera-scm-agent/process/**202-impala-IMPALAD/impala-conf
    + export HADOOP_CONF_DIR=/var/run/**cloudera-scm-agent/process/**202-impala-IMPALAD/hadoop-conf
    + HADOOP_CONF_DIR=/var/run/**cloudera-scm-agent/process/**202-impala-IMPALAD/hadoop-conf
    + export HIVE_CONF_DIR=/var/run/**cloudera-scm-agent/process/**202-impala-IMPALAD/hive-conf
    + HIVE_CONF_DIR=/var/run/**cloudera-scm-agent/process/**202-impala-IMPALAD/hive-conf
    + export HBASE_CONF_DIR=/var/run/**cloudera-scm-agent/process/**202-impala-IMPALAD/hbase-conf
    + HBASE_CONF_DIR=/var/run/**cloudera-scm-agent/process/**202-impala-IMPALAD/hbase-conf
    + FLAG_FILE=/var/run/cloudera-**scm-agent/process/202-impala-**IMPALAD/impala-conf/impalad_**flags
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/**cloudera-scm-agent/process/**202-impala-IMPALAD#g' /var/run/cloudera-scm-agent/**process/202-impala-IMPALAD/**impala-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad --flagfile=/var/run/cloudera-**scm-agent/process/202-impala-**IMPALAD/impala-conf/impalad_**flags
    /usr/bin/impalad: line 2: /etc/default/impala: No such file or directory
    /usr/bin/impalad: line 16: /impalad: No such file or directory
    /usr/bin/impalad: line 16: exec: /impalad: cannot execute: No such file or directory

    **/

    After copying /etc/default/impala from my second node, i was able to start statestored service but impalad service is still not starting and now
    throwing following error



    /**

    + date
    + date
    Fri Jan 25 10:02:48 PST 2013
    + export IMPALA_CONF_DIR=/var/run/**cloudera-scm-agent/process/**
    204-impala-IMPALAD/impala-conf
    + IMPALA_CONF_DIR=/var/run/**cloudera-scm-agent/process/**
    204-impala-IMPALAD/impala-conf
    + export HADOOP_CONF_DIR=/var/run/**cloudera-scm-agent/process/**
    204-impala-IMPALAD/hadoop-conf
    + HADOOP_CONF_DIR=/var/run/**cloudera-scm-agent/process/**
    204-impala-IMPALAD/hadoop-conf
    + export HIVE_CONF_DIR=/var/run/**cloudera-scm-agent/process/**
    204-impala-IMPALAD/hive-conf
    + HIVE_CONF_DIR=/var/run/**cloudera-scm-agent/process/**
    204-impala-IMPALAD/hive-conf
    + export HBASE_CONF_DIR=/var/run/**cloudera-scm-agent/process/**
    204-impala-IMPALAD/hbase-conf
    + HBASE_CONF_DIR=/var/run/**cloudera-scm-agent/process/**
    204-impala-IMPALAD/hbase-conf
    + FLAG_FILE=/var/run/cloudera-**scm-agent/process/204-impala-**
    IMPALAD/impala-conf/impalad_**flags
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/**
    cloudera-scm-agent/process/**204-impala-IMPALAD#g'
    /var/run/cloudera-scm-agent/**process/204-impala-IMPALAD/**
    impala-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad --flagfile=/var/run/cloudera-**
    scm-agent/process/204-impala-**IMPALAD/impala-conf/impalad_**flags
    loadFileSystems error:
    (unable to get stack trace for java.lang.NoClassDefFoundError
    exception: ExceptionUtils::getStackTrace error.)
    hdfsBuilderConnect(**forceNewInstance=0, nn=default, port=0,
    kerbTicketCachePath=(NULL), userName=(NULL)) error:
    (unable to get stack trace for java.lang.NoClassDefFoundError
    exception: ExceptionUtils::getStackTrace error.)
    Exception in thread "main" java.lang.**NoClassDefFoundError:
    com/cloudera/impala/common/**JniUtil
    Caused by: java.lang.**ClassNotFoundException:
    com.cloudera.impala.common.**JniUtil
    at java.net.URLClassLoader$1.run(**URLClassLoader.java:202)
    at java.security.**AccessController.doPrivileged(**Native Method)
    at java.net.URLClassLoader.**findClass(URLClassLoader.java:**190)
    at java.lang.ClassLoader.**loadClass(ClassLoader.java:**306)
    at sun.misc.Launcher$**AppClassLoader.loadClass(**
    Launcher.java:301)
    at java.lang.ClassLoader.**loadClass(ClassLoader.java:**247)
    E0125 10:02:50.076205 29022 impalad-main.cc:81] Failed to find
    JniUtil class.

    */

    Seems it is a classpath issue now?

    Thanks.
    -Dipesh

    On Thursday, January 24, 2013 5:24:15 PM UTC-8, vikas wrote:

    Hi Dipesh,

    I think you need to change the baseurl to "
    http://archive.cloudera.com/**c**m4/redhat/6/x86_64/cm/4/<http://archive.cloudera.com/cm4/redhat/6/x86_64/cm/4/>",
    which is the url that will fetch the latest CM whenever you run "yum
    upgrade" command. For Imapla upgrade you may have to do something similar
    (depending on the content of your impala repo file)

    The file /usr/bin/impalad (which is a bash file) is the one that
    sets environment variables (and is missing in your setup).

    - Vikas

    On Thu, Jan 24, 2013 at 5:02 PM, Deepsoft wrote:

    Vikas,

    It is reporting version 4.1.2. I wanted to upgrade it to 4.1.3 so
    that I can upgrade to impala 0.4. Here is the content of my
    cloudera-manager.repo file

    [cloudera-manager]
    name = Cloudera Manager, Version 4.1.2
    baseurl = http://archive.cloudera.com/**cm**
    4/redhat/6/x86_64/cm/4.1.2/<http://archive.cloudera.com/cm4/redhat/6/x86_64/cm/4.1.2/>
    gpgkey = http://archive.cloudera.com/**re**dhat/cdh/RPM-GPG-KEY-**
    cloudera<http://archive.cloudera.com/redhat/cdh/RPM-GPG-KEY-cloudera>
    gpgcheck = 1

    Can you please confirm which file sets all the required enviornment
    variables? I have created a sym link to impalad in /usr/bin from the
    installed folder /usr/lib/impala/sbin/impalad.

    Thanks.
    -Dipesh


    On Thursday, January 24, 2013 4:47:22 PM UTC-8, vikas wrote:

    Hi Dipesh,

    I am actually not sure what happens if you build impala on a
    machine where you already have impala installed from package. If you
    install impala, the file '/usr/bin/impalad' is a bash file that takes care
    of setting all environment before it exec the actual 'impalad' binary. For
    some reason in your setup that file is missing and launching impalad
    directly fails because env insn't setup correctly.

    And upgrading CM shouldn't change CDH component versions, so mixed
    CDH versions were probably the case before too. What versions does it
    report?

    Vikas

    On Thu, Jan 24, 2013 at 4:30 PM, Deepsoft wrote:

    Vikas,

    Thanks for replying. After linking back impalad and statestored
    from /usr/lib/impala/sbin, I was able to start statestored but got some
    error while starting impalad

    /**

    + date
    + date
    Thu Jan 24 16:18:36 PST 2013
    + export IMPALA_CONF_DIR=/var/run/**cloud****era-scm-agent/process/**197-**impa**la-IMPALAD/impala-conf
    + IMPALA_CONF_DIR=/var/run/**cloud****era-scm-agent/process/**197-**impa**la-IMPALAD/impala-conf
    + export HADOOP_CONF_DIR=/var/run/**cloud****era-scm-agent/process/**197-**impa**la-IMPALAD/hadoop-conf
    + HADOOP_CONF_DIR=/var/run/**cloud****era-scm-agent/process/**197-**impa**la-IMPALAD/hadoop-conf
    + export HIVE_CONF_DIR=/var/run/**clouder****a-scm-agent/process/**197-**impala**-IMPALAD/hive-conf
    + HIVE_CONF_DIR=/var/run/**clouder****a-scm-agent/process/**197-**impala**-IMPALAD/hive-conf
    + export HBASE_CONF_DIR=/var/run/**cloude****ra-scm-agent/process/**197-**impal**a-IMPALAD/hbase-conf
    + HBASE_CONF_DIR=/var/run/**cloude****ra-scm-agent/process/**197-**impal**a-IMPALAD/hbase-conf
    + FLAG_FILE=/var/run/cloudera-**sc****m-agent/process/197-impala-**IMP****ALAD/impala-conf/impalad_**flags
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/**c****loudera-scm-agent/process/**197-****impala-IMPALAD#g' /var/run/cloudera-scm-agent/**pr****ocess/197-impala-IMPALAD/**impal****a-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad --flagfile=/var/run/cloudera-**s****cm-agent/process/197-impala-**IM****PALAD/impala-conf/impalad_**flag****s
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    E0124 16:18:36.749694 19556 impalad-main.cc:81] Failed to get/create JVM
    + date
    + date
    Thu Jan 24 16:18:37 PST 2013
    + export IMPALA_CONF_DIR=/var/run/**cloud****era-scm-agent/process/**197-**impa**la-IMPALAD/impala-conf
    + IMPALA_CONF_DIR=/var/run/**cloud****era-scm-agent/process/**197-**impa**la-IMPALAD/impala-conf
    + export HADOOP_CONF_DIR=/var/run/**cloud****era-scm-agent/process/**197-**impa**la-IMPALAD/hadoop-conf
    + HADOOP_CONF_DIR=/var/run/**cloud****era-scm-agent/process/**197-**impa**la-IMPALAD/hadoop-conf
    + export HIVE_CONF_DIR=/var/run/**clouder****a-scm-agent/process/**197-**impala**-IMPALAD/hive-conf
    + HIVE_CONF_DIR=/var/run/**clouder****a-scm-agent/process/**197-**impala**-IMPALAD/hive-conf
    + export HBASE_CONF_DIR=/var/run/**cloude****ra-scm-agent/process/**197-**impal**a-IMPALAD/hbase-conf
    + HBASE_CONF_DIR=/var/run/**cloude****ra-scm-agent/process/**197-**impal**a-IMPALAD/hbase-conf
    + FLAG_FILE=/var/run/cloudera-**sc****m-agent/process/197-impala-**IMP****ALAD/impala-conf/impalad_**flags
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/**c****loudera-scm-agent/process/**197-****impala-IMPALAD#g' /var/run/cloudera-scm-agent/**pr****ocess/197-impala-IMPALAD/**impal****a-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad --flagfile=/var/run/cloudera-**s****cm-agent/process/197-impala-**IM****PALAD/impala-conf/impalad_**flag****s
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    E0124 16:18:37.914975 19591 impalad-main.cc:81] Failed to get/create JVM
    + date
    + date
    Thu Jan 24 16:18:39 PST 2013
    + export IMPALA_CONF_DIR=/var/run/**cloud****era-scm-agent/process/**197-**impa**la-IMPALAD/impala-conf
    + IMPALA_CONF_DIR=/var/run/**cloud****era-scm-agent/process/**197-**impa**la-IMPALAD/impala-conf
    + export HADOOP_CONF_DIR=/var/run/**cloud****era-scm-agent/process/**197-**impa**la-IMPALAD/hadoop-conf
    + HADOOP_CONF_DIR=/var/run/**cloud****era-scm-agent/process/**197-**impa**la-IMPALAD/hadoop-conf
    + export HIVE_CONF_DIR=/var/run/**clouder****a-scm-agent/process/**197-**impala**-IMPALAD/hive-conf
    + HIVE_CONF_DIR=/var/run/**clouder****a-scm-agent/process/**197-**impala**-IMPALAD/hive-conf
    + export HBASE_CONF_DIR=/var/run/**cloude****ra-scm-agent/process/**197-**impal**a-IMPALAD/hbase-conf
    + HBASE_CONF_DIR=/var/run/**cloude****ra-scm-agent/process/**197-**impal**a-IMPALAD/hbase-conf
    + FLAG_FILE=/var/run/cloudera-**sc****m-agent/process/197-impala-**IMP****ALAD/impala-conf/impalad_**flags
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/**c****loudera-scm-agent/process/**197-****impala-IMPALAD#g' /var/run/cloudera-scm-agent/**pr****ocess/197-impala-IMPALAD/**impal****a-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad --flagfile=/var/run/cloudera-**s****cm-agent/process/197-impala-**IM****PALAD/impala-conf/impalad_**flag****s
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    E0124 16:18:39.950352 19595 impalad-main.cc:81] Failed to get/create JVM
    + date
    + date
    Thu Jan 24 16:18:42 PST 2013
    + export IMPALA_CONF_DIR=/var/run/**cloud****era-scm-agent/process/**197-**impa**la-IMPALAD/impala-conf
    + IMPALA_CONF_DIR=/var/run/**cloud****era-scm-agent/process/**197-**impa**la-IMPALAD/impala-conf
    + export HADOOP_CONF_DIR=/var/run/**cloud****era-scm-agent/process/**197-**impa**la-IMPALAD/hadoop-conf
    + HADOOP_CONF_DIR=/var/run/**cloud****era-scm-agent/process/**197-**impa**la-IMPALAD/hadoop-conf
    + export HIVE_CONF_DIR=/var/run/**clouder****a-scm-agent/process/**197-**impala**-IMPALAD/hive-conf
    + HIVE_CONF_DIR=/var/run/**clouder****a-scm-agent/process/**197-**impala**-IMPALAD/hive-conf
    + export HBASE_CONF_DIR=/var/run/**cloude****ra-scm-agent/process/**197-**impal**a-IMPALAD/hbase-conf
    + HBASE_CONF_DIR=/var/run/**cloude****ra-scm-agent/process/**197-**impal**a-IMPALAD/hbase-conf
    + FLAG_FILE=/var/run/cloudera-**sc****m-agent/process/197-impala-**IMP****ALAD/impala-conf/impalad_**flags
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/**c****loudera-scm-agent/process/**197-****impala-IMPALAD#g' /var/run/cloudera-scm-agent/**pr****ocess/197-impala-IMPALAD/**impal****a-conf/impalad_flags
    + '[' impalad = impalad ']'
    + exec impalad --flagfile=/var/run/cloudera-**s****cm-agent/process/197-impala-**IM****PALAD/impala-conf/impalad_**flag****s
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    Environment variable CLASSPATH not set!
    getJNIEnv: getGlobalJNIEnv failed
    E0124 16:18:42.983335 19599 impalad-main.cc:81] Failed to get/create JVM

    **/

    My install was working fine before I started building impalad - so I'm not sure whether build script somehow deletes existing




    impalad/statestored if it finds in /usr/bin folder?

    Now it is complaining about some missing environment variables - I wasn't setting these before as well. Did I lose a script file which is setting environment




    variable?

    Yes, I have not updated my CM yet. When I tried to udpate using the instructions here https://ccp.cloudera.com/**displ****ay/FREE41DOC/Upgrade+**from+**Clou**dera+Manager+4+to+**the+**Latest+**Version+of+**Cloudera+**Manager+4 <https://ccp.cloudera.com/display/FREE41DOC/Upgrade+from+Cloudera+Manager+4+to+the+Latest+Version+of+Cloudera+Manager+4>




    I got the following message -

    [root@khadi01-rhvm01 local]# sudo yum update --disablerepo='*' --enablerepo=cloudera-manager
    Loaded plugins: product-id, refresh-packagekit, rhnplugin, subscription-manager




    Updating certificate-based repositories.
    Unable to read consumer identity
    Repository 'dvd' is missing name in configuration, using id
    cloudera-manager | 951 B 00:00




    cloudera-manager/primary | 3.6 kB 00:00
    cloudera-manager 7/7
    Setting up Update Process
    No Packages marked for Update




    Now when I check "host inspector" result, I see that there exists mixed CDH versions. It seems my upgrade didn't go well. Correct?

    Thanks.
    -Dipesh


    Thanks.
    -Dipesh



    On Thursday, January 24, 2013 3:03:20 PM UTC-8, vikas wrote:

    Hi Dipesh,

    The binary is looked in the default PATH and impalad isn't
    present in it. You might want to symlink the build you created to /usr/bin.

    Also, it seems you aren't using the latest CM build, it has been
    updated to use full impala path when calling 'exec' instead of just
    'impalad'.

    Vikas

    On Thu, Jan 24, 2013 at 2:41 PM, Deepsoft wrote:

    I have impala and other services running on a node. I have
    built impalad on the same node - following the instructions.

    Now when I'm trying to start impala service along with other
    services through cloudera manager but it is not starting and logs are
    showing this error -

    /**
    + exec impalad --flagfile=/var/run/cloudera-**s******
    cm-agent/process/157-impala-**IM******
    PALAD/impala-conf/impalad_**flag******s
    /usr/lib64/cmf/service/impala/********impala.sh: line 23:
    exec: impalad: not found

    */

    When I search in my system I see that I have impalad at my
    installed location which is /usr/lib/imapla/sbin/impalad

    # find / -name impalad
    /var/log/impalad
    /usr/lib/impala/sbin/impalad
    /usr/lib64/cmf/agent/src/cmf/**m******onitor/impalad
    /home/src/impala/impala/be/**bui******ld/debug/service/impalad

    The highlighted location is the one where I have build impalad.

    Where is imapla.sh looking for impalad? I didn't modify
    anything - Does the build process delete impalad? Where should I copy
    impalad so that this script file finds it again and start the service?

    Thanks.

    --

    --
    --

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupimpala-user @
categorieshadoop
postedJan 24, '13 at 10:41p
activeJan 28, '13 at 10:43p
posts8
users2
websitecloudera.com
irc#hadoop

2 users in discussion

Deepsoft: 6 posts Vikas Singh: 2 posts

Content

People

Support

Translate

site design / logo © 2022 Grokbase