FAQ
Sorry - I should've specified that this is from the packages on ubuntu
precise.
On Tuesday, November 26, 2013 2:11:45 AM UTC-8, Colin Marc wrote:

I just installed impala 1.2, and I get this when I try to start the new
catalogd:

E1126 09:57:52.758210 11084 catalog.cc:65] java.lang.NoClassDefFoundError:
org/datanucleus/NucleusContext
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2534)
at java.lang.Class.getMethod0(Class.java:2777)
at java.lang.Class.getMethod(Class.java:1663)
at javax.jdo.JDOHelper$15.run(JDOHelper.java:1935)
at javax.jdo.JDOHelper$15.run(JDOHelper.java:1933)
at java.security.AccessController.doPrivileged(Native Method)
at javax.jdo.JDOHelper.getMethod(JDOHelper.java:1932)
at
javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1156)
at
javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:803)
at
javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:698)
at
org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:262)
at
org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:291)
at
org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:224)
at
org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:199)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
at
org.apache.hadoop.hive.metastore.RetryingRawStore.<init>(RetryingRawStore.java:62)
at
org.apache.hadoop.hive.metastore.RetryingRawStore.getProxy(RetryingRawStore.java:71)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:408)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:396)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:434)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:328)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.<init>(HiveMetaStore.java:289)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:56)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:61)
at
org.apache.hadoop.hive.metastore.HiveMetaStore.newHMSHandler(HiveMetaStore.java:4014)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:120)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:103)
at
com.cloudera.impala.catalog.MetaStoreClientPool$MetaStoreClient.<init>(MetaStoreClientPool.java:47)
at
com.cloudera.impala.catalog.MetaStoreClientPool$MetaStoreClient.<init>(MetaStoreClientPool.java:40)
at
com.cloudera.impala.catalog.MetaStoreClientPool.addClients(MetaStoreClientPool.java:105)
at com.cloudera.impala.catalog.Catalog.<init>(Catalog.java:112)
at
com.cloudera.impala.catalog.CatalogServiceCatalog.<init>(CatalogServiceCatalog.java:59)
at
com.cloudera.impala.catalog.CatalogServiceCatalog.<init>(CatalogServiceCatalog.java:50)
at
com.cloudera.impala.service.JniCatalog.<init>(JniCatalog.java:73)
Caused by: java.lang.ClassNotFoundException: org.datanucleus.NucleusContext
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
... 36 more

Any ideas?
To unsubscribe from this group and stop receiving emails from it, send an email to impala-user+unsubscribe@cloudera.org.

Search Discussions

  • Johannes Kirschnick at Nov 26, 2013 at 1:15 pm
    Hi,

    I saw exactly the same problem after rolling forward.
    We tracked that down to a version conflict in the impala lib directory.
    Basically it's hard coding version 2 but version 3 is needed.

    I manually went into the parcel directory to fix it - you need root for
    that.

    So on the node that hosts the catalog server role, find the unpacked parcel
    directory for impala.

    Something like
    /opt/cloudera/parcels/IMPALA

    in the original parcel directory, you'll find

    /opt/cloudera/parcels/IMPALA/lib/impala/lib/datanucleus-core-2.0.3.jar

    This is the conflict - you'll need version 3.

    You can just copy the correct version from the hive library

    See here

    /opt/cloudera/parcels/CDH/lib/hive/lib/datanucleus-core-3.2.2.jar

    Once this is done, you need to patch the impala start script to pick up the
    new datanucleus-core version.

    In /opt/cloudera/parcels/IMPALA/bin/catalogd

    Look for

    export CLASSPATH=

    and basically replace the datanucleus version from

    datanucleus-core-2.0.3.jar

    to

    datanucleus-core-3.2.2.jar


    Hope this helps

    Johannes
    On Tuesday, November 26, 2013 11:16:42 AM UTC+1, Colin Marc wrote:

    Sorry - I should've specified that this is from the packages on ubuntu
    precise.
    On Tuesday, November 26, 2013 2:11:45 AM UTC-8, Colin Marc wrote:

    I just installed impala 1.2, and I get this when I try to start the new
    catalogd:

    E1126 09:57:52.758210 11084 catalog.cc:65]
    java.lang.NoClassDefFoundError: org/datanucleus/NucleusContext
    at java.lang.Class.getDeclaredMethods0(Native Method)
    at java.lang.Class.privateGetDeclaredMethods(Class.java:2534)
    at java.lang.Class.getMethod0(Class.java:2777)
    at java.lang.Class.getMethod(Class.java:1663)
    at javax.jdo.JDOHelper$15.run(JDOHelper.java:1935)
    at javax.jdo.JDOHelper$15.run(JDOHelper.java:1933)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.jdo.JDOHelper.getMethod(JDOHelper.java:1932)
    at
    javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1156)
    at
    javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:803)
    at
    javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:698)
    at
    org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:262)
    at
    org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:291)
    at
    org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:224)
    at
    org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:199)
    at
    org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73)
    at
    org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
    at
    org.apache.hadoop.hive.metastore.RetryingRawStore.<init>(RetryingRawStore.java:62)
    at
    org.apache.hadoop.hive.metastore.RetryingRawStore.getProxy(RetryingRawStore.java:71)
    at
    org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:408)
    at
    org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:396)
    at
    org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:434)
    at
    org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:328)
    at
    org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.<init>(HiveMetaStore.java:289)
    at
    org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:56)
    at
    org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:61)
    at
    org.apache.hadoop.hive.metastore.HiveMetaStore.newHMSHandler(HiveMetaStore.java:4014)
    at
    org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:120)
    at
    org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:103)
    at
    com.cloudera.impala.catalog.MetaStoreClientPool$MetaStoreClient.<init>(MetaStoreClientPool.java:47)
    at
    com.cloudera.impala.catalog.MetaStoreClientPool$MetaStoreClient.<init>(MetaStoreClientPool.java:40)
    at
    com.cloudera.impala.catalog.MetaStoreClientPool.addClients(MetaStoreClientPool.java:105)
    at com.cloudera.impala.catalog.Catalog.<init>(Catalog.java:112)
    at
    com.cloudera.impala.catalog.CatalogServiceCatalog.<init>(CatalogServiceCatalog.java:59)
    at
    com.cloudera.impala.catalog.CatalogServiceCatalog.<init>(CatalogServiceCatalog.java:50)
    at
    com.cloudera.impala.service.JniCatalog.<init>(JniCatalog.java:73)
    Caused by: java.lang.ClassNotFoundException:
    org.datanucleus.NucleusContext
    at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
    ... 36 more

    Any ideas?
    To unsubscribe from this group and stop receiving emails from it, send an email to impala-user+unsubscribe@cloudera.org.
  • Colin Marc at Nov 26, 2013 at 8:12 pm
    Johannes, thanks for the tip. Weirdly, I don't see that jar (version 3) in
    the hive libs either. What version/package of hive do you have installed?
    On Tuesday, November 26, 2013 5:15:02 AM UTC-8, Johannes Kirschnick wrote:

    Hi,

    I saw exactly the same problem after rolling forward.
    We tracked that down to a version conflict in the impala lib directory.
    Basically it's hard coding version 2 but version 3 is needed.

    I manually went into the parcel directory to fix it - you need root for
    that.

    So on the node that hosts the catalog server role, find the unpacked
    parcel directory for impala.

    Something like
    /opt/cloudera/parcels/IMPALA

    in the original parcel directory, you'll find

    /opt/cloudera/parcels/IMPALA/lib/impala/lib/datanucleus-core-2.0.3.jar

    This is the conflict - you'll need version 3.

    You can just copy the correct version from the hive library

    See here

    /opt/cloudera/parcels/CDH/lib/hive/lib/datanucleus-core-3.2.2.jar

    Once this is done, you need to patch the impala start script to pick up
    the new datanucleus-core version.

    In /opt/cloudera/parcels/IMPALA/bin/catalogd

    Look for

    export CLASSPATH=

    and basically replace the datanucleus version from

    datanucleus-core-2.0.3.jar

    to

    datanucleus-core-3.2.2.jar


    Hope this helps

    Johannes
    On Tuesday, November 26, 2013 11:16:42 AM UTC+1, Colin Marc wrote:

    Sorry - I should've specified that this is from the packages on ubuntu
    precise.
    On Tuesday, November 26, 2013 2:11:45 AM UTC-8, Colin Marc wrote:

    I just installed impala 1.2, and I get this when I try to start the new
    catalogd:

    E1126 09:57:52.758210 11084 catalog.cc:65]
    java.lang.NoClassDefFoundError: org/datanucleus/NucleusContext
    at java.lang.Class.getDeclaredMethods0(Native Method)
    at java.lang.Class.privateGetDeclaredMethods(Class.java:2534)
    at java.lang.Class.getMethod0(Class.java:2777)
    at java.lang.Class.getMethod(Class.java:1663)
    at javax.jdo.JDOHelper$15.run(JDOHelper.java:1935)
    at javax.jdo.JDOHelper$15.run(JDOHelper.java:1933)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.jdo.JDOHelper.getMethod(JDOHelper.java:1932)
    at
    javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1156)
    at
    javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:803)
    at
    javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:698)
    at
    org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:262)
    at
    org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:291)
    at
    org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:224)
    at
    org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:199)
    at
    org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73)
    at
    org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
    at
    org.apache.hadoop.hive.metastore.RetryingRawStore.<init>(RetryingRawStore.java:62)
    at
    org.apache.hadoop.hive.metastore.RetryingRawStore.getProxy(RetryingRawStore.java:71)
    at
    org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:408)
    at
    org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:396)
    at
    org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:434)
    at
    org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:328)
    at
    org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.<init>(HiveMetaStore.java:289)
    at
    org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:56)
    at
    org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:61)
    at
    org.apache.hadoop.hive.metastore.HiveMetaStore.newHMSHandler(HiveMetaStore.java:4014)
    at
    org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:120)
    at
    org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:103)
    at
    com.cloudera.impala.catalog.MetaStoreClientPool$MetaStoreClient.<init>(MetaStoreClientPool.java:47)
    at
    com.cloudera.impala.catalog.MetaStoreClientPool$MetaStoreClient.<init>(MetaStoreClientPool.java:40)
    at
    com.cloudera.impala.catalog.MetaStoreClientPool.addClients(MetaStoreClientPool.java:105)
    at com.cloudera.impala.catalog.Catalog.<init>(Catalog.java:112)
    at
    com.cloudera.impala.catalog.CatalogServiceCatalog.<init>(CatalogServiceCatalog.java:59)
    at
    com.cloudera.impala.catalog.CatalogServiceCatalog.<init>(CatalogServiceCatalog.java:50)
    at
    com.cloudera.impala.service.JniCatalog.<init>(JniCatalog.java:73)
    Caused by: java.lang.ClassNotFoundException:
    org.datanucleus.NucleusContext
    at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
    ... 36 more

    Any ideas?
    To unsubscribe from this group and stop receiving emails from it, send an email to impala-user+unsubscribe@cloudera.org.
  • Henry Robinson at Nov 27, 2013 at 1:03 am
    Hi Colin -

    Thanks for the report, and sorry that you're having problems.

    We've been trying to reproduce what you've seen locally, and haven't yet
    done so. We are aware that there might be an issue with the versions of
    some jars in the shipped packages, but our testing hasn't yet revealed the
    exact nature of any problem.

    To confirm the package versions you have, could you run the following
    commands and let us have the output, on your CDH4.5 cluster?

    find /usr/lib/impala/lib -name *.jar | grep -i datanucleus
    find /usr/lib/hive/lib -name *.jar | grep -i datanucleus

    Thanks,
    Henry

    On 26 November 2013 12:12, Colin Marc wrote:

    Johannes, thanks for the tip. Weirdly, I don't see that jar (version 3) in
    the hive libs either. What version/package of hive do you have installed?

    On Tuesday, November 26, 2013 5:15:02 AM UTC-8, Johannes Kirschnick wrote:

    Hi,

    I saw exactly the same problem after rolling forward.
    We tracked that down to a version conflict in the impala lib directory.
    Basically it's hard coding version 2 but version 3 is needed.

    I manually went into the parcel directory to fix it - you need root for
    that.

    So on the node that hosts the catalog server role, find the unpacked
    parcel directory for impala.

    Something like
    /opt/cloudera/parcels/IMPALA

    in the original parcel directory, you'll find

    /opt/cloudera/parcels/IMPALA/lib/impala/lib/datanucleus-core-2.0.3.jar

    This is the conflict - you'll need version 3.

    You can just copy the correct version from the hive library

    See here

    /opt/cloudera/parcels/CDH/lib/hive/lib/datanucleus-core-3.2.2.jar

    Once this is done, you need to patch the impala start script to pick up
    the new datanucleus-core version.

    In /opt/cloudera/parcels/IMPALA/bin/catalogd

    Look for

    export CLASSPATH=

    and basically replace the datanucleus version from

    datanucleus-core-2.0.3.jar

    to

    datanucleus-core-3.2.2.jar


    Hope this helps

    Johannes
    On Tuesday, November 26, 2013 11:16:42 AM UTC+1, Colin Marc wrote:

    Sorry - I should've specified that this is from the packages on ubuntu
    precise.
    On Tuesday, November 26, 2013 2:11:45 AM UTC-8, Colin Marc wrote:

    I just installed impala 1.2, and I get this when I try to start the new
    catalogd:

    E1126 09:57:52.758210 11084 catalog.cc:65] java.lang.NoClassDefFoundError:
    org/datanucleus/NucleusContext
    at java.lang.Class.getDeclaredMethods0(Native Method)
    at java.lang.Class.privateGetDeclaredMethods(Class.java:2534)
    at java.lang.Class.getMethod0(Class.java:2777)
    at java.lang.Class.getMethod(Class.java:1663)
    at javax.jdo.JDOHelper$15.run(JDOHelper.java:1935)
    at javax.jdo.JDOHelper$15.run(JDOHelper.java:1933)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.jdo.JDOHelper.getMethod(JDOHelper.java:1932)
    at javax.jdo.JDOHelper.invokeGetPersistenceManagerFac
    toryOnImplementation(JDOHelper.java:1156)
    at javax.jdo.JDOHelper.getPersistenceManagerFactory(
    JDOHelper.java:803)
    at javax.jdo.JDOHelper.getPersistenceManagerFactory(
    JDOHelper.java:698)
    at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(
    ObjectStore.java:262)
    at org.apache.hadoop.hive.metastore.ObjectStore.
    getPersistenceManager(ObjectStore.java:291)
    at org.apache.hadoop.hive.metastore.ObjectStore.
    initialize(ObjectStore.java:224)
    at org.apache.hadoop.hive.metastore.ObjectStore.setConf(
    ObjectStore.java:199)
    at org.apache.hadoop.util.ReflectionUtils.setConf(
    ReflectionUtils.java:73)
    at org.apache.hadoop.util.ReflectionUtils.newInstance(
    ReflectionUtils.java:133)
    at org.apache.hadoop.hive.metastore.RetryingRawStore.<
    init>(RetryingRawStore.java:62)
    at org.apache.hadoop.hive.metastore.RetryingRawStore.
    getProxy(RetryingRawStore.java:71)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.newRawStore(HiveMetaStore.java:408)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.getMS(HiveMetaStore.java:396)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.createDefaultDB(HiveMetaStore.java:434)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.init(HiveMetaStore.java:328)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.<init>(HiveMetaStore.java:289)
    at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<
    init>(RetryingHMSHandler.java:56)
    at org.apache.hadoop.hive.metastore.RetryingHMSHandler.
    getProxy(RetryingHMSHandler.java:61)
    at org.apache.hadoop.hive.metastore.HiveMetaStore.
    newHMSHandler(HiveMetaStore.java:4014)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.
    <init>(HiveMetaStoreClient.java:120)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.
    <init>(HiveMetaStoreClient.java:103)
    at com.cloudera.impala.catalog.MetaStoreClientPool$
    MetaStoreClient.<init>(MetaStoreClientPool.java:47)
    at com.cloudera.impala.catalog.MetaStoreClientPool$
    MetaStoreClient.<init>(MetaStoreClientPool.java:40)
    at com.cloudera.impala.catalog.MetaStoreClientPool.addClients(
    MetaStoreClientPool.java:105)
    at com.cloudera.impala.catalog.Catalog.<init>(Catalog.java:112)
    at com.cloudera.impala.catalog.CatalogServiceCatalog.<init>(
    CatalogServiceCatalog.java:59)
    at com.cloudera.impala.catalog.CatalogServiceCatalog.<init>(
    CatalogServiceCatalog.java:50)
    at com.cloudera.impala.service.JniCatalog.<init>(JniCatalog.
    java:73)
    Caused by: java.lang.ClassNotFoundException:
    org.datanucleus.NucleusContext
    at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
    at sun.misc.Launcher$AppClassLoader.loadClass(
    Launcher.java:294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
    ... 36 more

    Any ideas?
    To unsubscribe from this group and stop receiving emails from it, send
    an email to impala-user+unsubscribe@cloudera.org.


    --
    Henry Robinson
    Software Engineer
    Cloudera
    415-994-6679

    To unsubscribe from this group and stop receiving emails from it, send an email to impala-user+unsubscribe@cloudera.org.
  • Henry Robinson at Nov 27, 2013 at 2:07 am
    Hi Colin -

    Thanks for the fast response!

    The version of Hive you have looks old to me - my installation of CDH4.5
    has datanucleus-core-3.2.2 for example. Can you confirm the installed
    version of CDH (e.g. via dpkg -l | grep -i cdh)?

    I believe that Impala 1.2.1 will only work on CDH4.5.

    Thanks,
    Henry

    On 26 November 2013 17:59, Colin Marc wrote:

    Hi Henry - thanks for following up. Here's what I have:

    $ find /usr/lib/impala/lib -name *.jar | grep -i datanucleus
    /usr/lib/impala/lib/datanucleus-enhancer-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-rdbms-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-connectionpool-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-api-jdo-3.2.1.jar
    /usr/lib/impala/lib/datanucleus-core-2.0.3.jar

    $ find /usr/lib/hive/lib -name *.jar | grep -i datanucleus
    /usr/lib/hive/lib/datanucleus-core-2.0.3-ZD5977-CDH5293.jar
    /usr/lib/hive/lib/datanucleus-enhancer-2.0.3.jar
    /usr/lib/hive/lib/datanucleus-rdbms-2.0.3.jar
    /usr/lib/hive/lib/datanucleus-connectionpool-2.0.3.jar


    On Tuesday, November 26, 2013 5:03:25 PM UTC-8, Henry wrote:

    Hi Colin -

    Thanks for the report, and sorry that you're having problems.

    We've been trying to reproduce what you've seen locally, and haven't yet
    done so. We are aware that there might be an issue with the versions of
    some jars in the shipped packages, but our testing hasn't yet revealed the
    exact nature of any problem.

    To confirm the package versions you have, could you run the following
    commands and let us have the output, on your CDH4.5 cluster?

    find /usr/lib/impala/lib -name *.jar | grep -i datanucleus
    find /usr/lib/hive/lib -name *.jar | grep -i datanucleus

    Thanks,
    Henry

    On 26 November 2013 12:12, Colin Marc wrote:

    Johannes, thanks for the tip. Weirdly, I don't see that jar (version 3)
    in the hive libs either. What version/package of hive do you have installed?


    On Tuesday, November 26, 2013 5:15:02 AM UTC-8, Johannes Kirschnick
    wrote:
    Hi,

    I saw exactly the same problem after rolling forward.
    We tracked that down to a version conflict in the impala lib directory.
    Basically it's hard coding version 2 but version 3 is needed.

    I manually went into the parcel directory to fix it - you need root for
    that.

    So on the node that hosts the catalog server role, find the unpacked
    parcel directory for impala.

    Something like
    /opt/cloudera/parcels/IMPALA

    in the original parcel directory, you'll find

    /opt/cloudera/parcels/IMPALA/lib/impala/lib/datanucleus-core-2.0.3.jar

    This is the conflict - you'll need version 3.

    You can just copy the correct version from the hive library

    See here

    /opt/cloudera/parcels/CDH/lib/hive/lib/datanucleus-core-3.2.2.jar

    Once this is done, you need to patch the impala start script to pick up
    the new datanucleus-core version.

    In /opt/cloudera/parcels/IMPALA/bin/catalogd

    Look for

    export CLASSPATH=

    and basically replace the datanucleus version from

    datanucleus-core-2.0.3.jar

    to

    datanucleus-core-3.2.2.jar


    Hope this helps

    Johannes
    On Tuesday, November 26, 2013 11:16:42 AM UTC+1, Colin Marc wrote:

    Sorry - I should've specified that this is from the packages on ubuntu
    precise.
    On Tuesday, November 26, 2013 2:11:45 AM UTC-8, Colin Marc wrote:

    I just installed impala 1.2, and I get this when I try to start the
    new catalogd:

    E1126 09:57:52.758210 11084 catalog.cc:65] java.lang.
    NoClassDefFoundError: org/datanucleus/NucleusContext
    at java.lang.Class.getDeclaredMethods0(Native Method)
    at java.lang.Class.privateGetDeclaredMethods(
    Class.java:2534)
    at java.lang.Class.getMethod0(Class.java:2777)
    at java.lang.Class.getMethod(Class.java:1663)
    at javax.jdo.JDOHelper$15.run(JDOHelper.java:1935)
    at javax.jdo.JDOHelper$15.run(JDOHelper.java:1933)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.jdo.JDOHelper.getMethod(JDOHelper.java:1932)
    at javax.jdo.JDOHelper.invokeGetPersistenceManagerFac
    toryOnImplementation(JDOHelper.java:1156)
    at javax.jdo.JDOHelper.getPersistenceManagerFactory(
    JDOHelper.java:803)
    at javax.jdo.JDOHelper.getPersistenceManagerFactory(
    JDOHelper.java:698)
    at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(
    ObjectStore.java:262)
    at org.apache.hadoop.hive.metastore.ObjectStore.
    getPersistenceManager(ObjectStore.java:291)
    at org.apache.hadoop.hive.metastore.ObjectStore.initialize(
    ObjectStore.java:224)
    at org.apache.hadoop.hive.metastore.ObjectStore.setConf(
    ObjectStore.java:199)
    at org.apache.hadoop.util.ReflectionUtils.setConf(
    ReflectionUtils.java:73)
    at org.apache.hadoop.util.ReflectionUtils.newInstance(
    ReflectionUtils.java:133)
    at org.apache.hadoop.hive.metastore.RetryingRawStore.<init>(
    RetryingRawStore.java:62)
    at org.apache.hadoop.hive.metastore.RetryingRawStore.
    getProxy(RetryingRawStore.java:71)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.
    newRawStore(HiveMetaStore.java:408)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.
    getMS(HiveMetaStore.java:396)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.
    createDefaultDB(HiveMetaStore.java:434)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.
    init(HiveMetaStore.java:328)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.<init>(HiveMetaStore.java:289)
    at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<
    init>(RetryingHMSHandler.java:56)
    at org.apache.hadoop.hive.metastore.RetryingHMSHandler.
    getProxy(RetryingHMSHandler.java:61)
    at org.apache.hadoop.hive.metastore.HiveMetaStore.
    newHMSHandler(HiveMetaStore.java:4014)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.
    <init>(HiveMetaStoreClient.java:120)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.
    <init>(HiveMetaStoreClient.java:103)
    at com.cloudera.impala.catalog.MetaStoreClientPool$
    MetaStoreClient.<init>(MetaStoreClientPool.java:47)
    at com.cloudera.impala.catalog.MetaStoreClientPool$
    MetaStoreClient.<init>(MetaStoreClientPool.java:40)
    at com.cloudera.impala.catalog.MetaStoreClientPool.
    addClients(MetaStoreClientPool.java:105)
    at com.cloudera.impala.catalog.Catalog.<init>(Catalog.java:
    112)
    at com.cloudera.impala.catalog.CatalogServiceCatalog.<init>(
    CatalogServiceCatalog.java:59)
    at com.cloudera.impala.catalog.CatalogServiceCatalog.<init>(
    CatalogServiceCatalog.java:50)
    at com.cloudera.impala.service.JniCatalog.<init>(JniCatalog.
    java:73)
    Caused by: java.lang.ClassNotFoundException:
    org.datanucleus.NucleusContext
    at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:
    294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
    ... 36 more

    Any ideas?
    To unsubscribe from this group and stop receiving emails from it,
    send an email to impala-user...@cloudera.org.


    --
    Henry Robinson
    Software Engineer
    Cloudera
    415-994-6679
    To unsubscribe from this group and stop receiving emails from it, send an
    email to impala-user+unsubscribe@cloudera.org.


    --
    Henry Robinson
    Software Engineer
    Cloudera
    415-994-6679

    To unsubscribe from this group and stop receiving emails from it, send an email to impala-user+unsubscribe@cloudera.org.
  • Henry Robinson at Nov 27, 2013 at 2:26 am
    Hi -

    You will need to upgrade to CDH4.5 to run Impala 1.2.1. (And if you're
    using CM, you'll need to upgrade to CM4.8). Hopefully that might resolve
    your classloading issue - let us know either way, and we'll keep working on
    any problems you encounter.

    Thanks,
    Henry

    On 26 November 2013 18:20, Colin Marc wrote:

    Hi Henry - I was looking at that, and realized the upgrade to 4.5 didn't
    completely work on the test cluster. I can work on that. Is upgrading the
    only way?

    On Tuesday, November 26, 2013 6:06:50 PM UTC-8, Henry wrote:

    Hi Colin -

    Thanks for the fast response!

    The version of Hive you have looks old to me - my installation of CDH4.5
    has datanucleus-core-3.2.2 for example. Can you confirm the installed
    version of CDH (e.g. via dpkg -l | grep -i cdh)?

    I believe that Impala 1.2.1 will only work on CDH4.5.

    Thanks,
    Henry

    On 26 November 2013 17:59, Colin Marc wrote:

    Hi Henry - thanks for following up. Here's what I have:

    $ find /usr/lib/impala/lib -name *.jar | grep -i datanucleus
    /usr/lib/impala/lib/datanucleus-enhancer-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-rdbms-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-connectionpool-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-api-jdo-3.2.1.jar
    /usr/lib/impala/lib/datanucleus-core-2.0.3.jar

    $ find /usr/lib/hive/lib -name *.jar | grep -i datanucleus
    /usr/lib/hive/lib/datanucleus-core-2.0.3-ZD5977-CDH5293.jar
    /usr/lib/hive/lib/datanucleus-enhancer-2.0.3.jar
    /usr/lib/hive/lib/datanucleus-rdbms-2.0.3.jar
    /usr/lib/hive/lib/datanucleus-connectionpool-2.0.3.jar


    On Tuesday, November 26, 2013 5:03:25 PM UTC-8, Henry wrote:

    Hi Colin -

    Thanks for the report, and sorry that you're having problems.

    We've been trying to reproduce what you've seen locally, and haven't
    yet done so. We are aware that there might be an issue with the versions of
    some jars in the shipped packages, but our testing hasn't yet revealed the
    exact nature of any problem.

    To confirm the package versions you have, could you run the following
    commands and let us have the output, on your CDH4.5 cluster?

    find /usr/lib/impala/lib -name *.jar | grep -i datanucleus
    find /usr/lib/hive/lib -name *.jar | grep -i datanucleus

    Thanks,
    Henry

    On 26 November 2013 12:12, Colin Marc wrote:

    Johannes, thanks for the tip. Weirdly, I don't see that jar (version
    3) in the hive libs either. What version/package of hive do you have
    installed?


    On Tuesday, November 26, 2013 5:15:02 AM UTC-8, Johannes Kirschnick
    wrote:
    Hi,

    I saw exactly the same problem after rolling forward.
    We tracked that down to a version conflict in the impala lib
    directory. Basically it's hard coding version 2 but version 3 is needed.

    I manually went into the parcel directory to fix it - you need root
    for that.

    So on the node that hosts the catalog server role, find the unpacked
    parcel directory for impala.

    Something like
    /opt/cloudera/parcels/IMPALA

    in the original parcel directory, you'll find

    /opt/cloudera/parcels/IMPALA/lib/impala/lib/datanucleus-core
    -2.0.3.jar

    This is the conflict - you'll need version 3.

    You can just copy the correct version from the hive library

    See here

    /opt/cloudera/parcels/CDH/lib/hive/lib/datanucleus-core-3.2.2.jar

    Once this is done, you need to patch the impala start script to pick
    up the new datanucleus-core version.

    In /opt/cloudera/parcels/IMPALA/bin/catalogd

    Look for

    export CLASSPATH=

    and basically replace the datanucleus version from

    datanucleus-core-2.0.3.jar

    to

    datanucleus-core-3.2.2.jar


    Hope this helps

    Johannes
    On Tuesday, November 26, 2013 11:16:42 AM UTC+1, Colin Marc wrote:

    Sorry - I should've specified that this is from the packages on
    ubuntu precise.
    On Tuesday, November 26, 2013 2:11:45 AM UTC-8, Colin Marc wrote:

    I just installed impala 1.2, and I get this when I try to start the
    new catalogd:

    E1126 09:57:52.758210 11084 catalog.cc:65] java.lang.
    NoClassDefFoundError: org/datanucleus/NucleusContext
    at java.lang.Class.getDeclaredMethods0(Native Method)
    at java.lang.Class.privateGetDeclaredMethods(
    Class.java:2534)
    at java.lang.Class.getMethod0(Class.java:2777)
    at java.lang.Class.getMethod(Class.java:1663)
    at javax.jdo.JDOHelper$15.run(JDOHelper.java:1935)
    at javax.jdo.JDOHelper$15.run(JDOHelper.java:1933)
    at java.security.AccessController.doPrivileged(Native
    Method)
    at javax.jdo.JDOHelper.getMethod(JDOHelper.java:1932)
    at javax.jdo.JDOHelper.invokeGetPersistenceManagerFac
    toryOnImplementation(JDOHelper.java:1156)
    at javax.jdo.JDOHelper.getPersistenceManagerFactory(
    JDOHelper.java:803)
    at javax.jdo.JDOHelper.getPersistenceManagerFactory(
    JDOHelper.java:698)
    at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(
    ObjectStore.java:262)
    at org.apache.hadoop.hive.metastore.ObjectStore.
    getPersistenceManager(ObjectStore.java:291)
    at org.apache.hadoop.hive.metastore.ObjectStore.initialize(
    ObjectStore.java:224)
    at org.apache.hadoop.hive.metastore.ObjectStore.setConf(
    ObjectStore.java:199)
    at org.apache.hadoop.util.ReflectionUtils.setConf(
    ReflectionUtils.java:73)
    at org.apache.hadoop.util.ReflectionUtils.newInstance(
    ReflectionUtils.java:133)
    at org.apache.hadoop.hive.metastore.RetryingRawStore.<
    init>(RetryingRawStore.java:62)
    at org.apache.hadoop.hive.metastore.RetryingRawStore.
    getProxy(RetryingRawStore.java:71)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.newRawStore(HiveMetaStore.java:408)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.getMS(HiveMetaStore.java:396)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.createDefaultDB(HiveMetaStore.java:434)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.init(HiveMetaStore.java:328)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.<init>(HiveMetaStore.java:289)
    at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<
    init>(RetryingHMSHandler.java:56)
    at org.apache.hadoop.hive.metastore.RetryingHMSHandler.
    getProxy(RetryingHMSHandler.java:61)
    at org.apache.hadoop.hive.metastore.HiveMetaStore.
    newHMSHandler(HiveMetaStore.java:4014)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.
    <init>(HiveMetaStoreClient.java:120)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.
    <init>(HiveMetaStoreClient.java:103)
    at com.cloudera.impala.catalog.MetaStoreClientPool$
    MetaStoreClient.<init>(MetaStoreClientPool.java:47)
    at com.cloudera.impala.catalog.MetaStoreClientPool$
    MetaStoreClient.<init>(MetaStoreClientPool.java:40)
    at com.cloudera.impala.catalog.MetaStoreClientPool.
    addClients(MetaStoreClientPool.java:105)
    at com.cloudera.impala.catalog.Catalog.<init>(Catalog.java:
    112)
    at com.cloudera.impala.catalog.Ca
    talogServiceCatalog.<init>(CatalogServiceCatalog.java:59)
    at com.cloudera.impala.catalog.Ca
    talogServiceCatalog.<init>(CatalogServiceCatalog.java:50)
    at com.cloudera.impala.service.Jn
    iCatalog.<init>(JniCatalog.java:73)
    Caused by: java.lang.ClassNotFoundException:
    org.datanucleus.NucleusContext
    at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native
    Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:
    205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
    at sun.misc.Launcher$AppClassLoader.loadClass(
    Launcher.java:294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
    ... 36 more

    Any ideas?
    To unsubscribe from this group and stop receiving emails from it,
    send an email to impala-user...@cloudera.org.


    --
    Henry Robinson
    Software Engineer
    Cloudera
    415-994-6679
    To unsubscribe from this group and stop receiving emails from it, send
    an email to impala-user...@cloudera.org.


    --
    Henry Robinson
    Software Engineer
    Cloudera
    415-994-6679
    To unsubscribe from this group and stop receiving emails from it, send an
    email to impala-user+unsubscribe@cloudera.org.


    --
    Henry Robinson
    Software Engineer
    Cloudera
    415-994-6679

    To unsubscribe from this group and stop receiving emails from it, send an email to impala-user+unsubscribe@cloudera.org.
  • Skye Wanderman-Milne at Nov 27, 2013 at 4:07 am
    Hey Colin, did you try the workaround suggested by Johannes? (Copying the
    jar from hive/lib to impala/lib and changing the CLASSPATH.)

    On Tue, Nov 26, 2013 at 7:45 PM, Colin Marc wrote:

    Hi Henry,

    I've upgraded to cdh4.5, and I'm still having the same problem. The hive
    directory now has the 3.x jars, but the CLASSPATH variable in
    /usr/bin/catalogd still points to the 2.x jars:

    $ dpkg -l | grep cdh
    ii bigtop-jsvc
    1.0.10-1.cdh4.5.0.p0.23~precise-cdh4.5.0 Application to launch
    java daemon
    ii bigtop-utils
    0.6.0+186-1.cdh4.5.0.p0.23~precise-cdh4.5.0 Collection of useful
    tools for Bigtop
    ii hadoop
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 A software platform for
    processing vast amounts of data
    ii hadoop-0.20-mapreduce
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 A software platform for
    processing vast amounts of data
    ii hadoop-0.20-mapreduce-tasktracker
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 Task Tracker for Hadoop
    ii hadoop-client
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 Hadoop client side
    dependencies
    ii hadoop-hdfs
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 The Hadoop Distributed
    File System
    ii hadoop-hdfs-datanode
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 Data Node for Hadoop
    ii hadoop-mapreduce
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 The Hadoop MapReduce
    (MRv2)
    ii hadoop-yarn
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 The Hadoop NextGen
    MapReduce (YARN)
    ii hbase
    0.94.6+165-1.cdh4.5.0.p0.21~precise-cdh4.5.0 HBase is the Hadoop
    database. Use it when you need random, realtime read/write access to your
    Big Data. This project's goal is the hosting of very large tables --
    billions of rows X millions of columns -- atop clusters of commodity
    hardware.
    ii hbase-regionserver
    0.94.6+165-1.cdh4.5.0.p0.21~precise-cdh4.5.0 The Hadoop HBase
    RegionServer server.
    ii hbase-rest
    0.94.6+165-1.cdh4.5.0.p0.21~precise-cdh4.5.0 The Apache HBase REST
    gateway
    ii hive
    0.10.0+214-1.cdh4.5.0.p0.25~precise-cdh4.5.0 Hive is a data
    warehouse infrastructure built on top of Hadoop
    ii hive-hbase
    0.10.0+214-1.cdh4.5.0.p0.25~precise-cdh4.5.0 Provides integration
    between Apache HBase and Apache Hive
    ii hive-jdbc
    0.10.0+214-1.cdh4.5.0.p0.25~precise-cdh4.5.0 Provides libraries
    necessary to connect to Apache Hive via JDBC
    ii parquet
    1.2.5-1.cdh4.5.0.p0.17~precise-cdh4.5.0 A columnar storage
    format for Hadoop.
    ii parquet-format
    1.0.0-1.cdh4.5.0.p0.20~precise-cdh4.5.0 Format definitions for
    Parquet
    ii zookeeper
    3.4.5+24-1.cdh4.5.0.p0.23~precise-cdh4.5.0 A high-performance
    coordination service for distributed applications.

    $ find /usr/lib/impala/lib -name *.jar | grep -i datanucleus
    /usr/lib/impala/lib/datanucleus-rdbms-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-connectionpool-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-enhancer-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-core-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-api-jdo-3.2.1.jar

    $ find /usr/lib/hive/lib -name *.jar | grep -i datanucleus
    /usr/lib/hive/lib/datanucleus-core-3.2.2.jar
    /usr/lib/hive/lib/datanucleus-rdbms-3.2.1.jar
    /usr/lib/hive/lib/datanucleus-api-jdo-3.2.1.jar

    $ grep datanucleus /usr/bin/catalogd | tr ':' '\n' | grep datanucleus
    ${IMPALA_HOME}/lib/datanucleus-api-jdo-3.2.1.jar
    ${IMPALA_HOME}/lib/datanucleus-connectionpool-2.0.3.jar
    ${IMPALA_HOME}/lib/datanucleus-core-2.0.3.jar
    ${IMPALA_HOME}/lib/datanucleus-enhancer-2.0.3.jar
    ${IMPALA_HOME}/lib/datanucleus-rdbms-2.0.3.jar
    On Tuesday, November 26, 2013 6:26:25 PM UTC-8, Henry wrote:

    Hi -

    You will need to upgrade to CDH4.5 to run Impala 1.2.1. (And if you're
    using CM, you'll need to upgrade to CM4.8). Hopefully that might resolve
    your classloading issue - let us know either way, and we'll keep working on
    any problems you encounter.

    Thanks,
    Henry

    On 26 November 2013 18:20, Colin Marc wrote:

    Hi Henry - I was looking at that, and realized the upgrade to 4.5 didn't
    completely work on the test cluster. I can work on that. Is upgrading the
    only way?

    On Tuesday, November 26, 2013 6:06:50 PM UTC-8, Henry wrote:

    Hi Colin -

    Thanks for the fast response!

    The version of Hive you have looks old to me - my installation of
    CDH4.5 has datanucleus-core-3.2.2 for example. Can you confirm the
    installed version of CDH (e.g. via dpkg -l | grep -i cdh)?

    I believe that Impala 1.2.1 will only work on CDH4.5.

    Thanks,
    Henry

    On 26 November 2013 17:59, Colin Marc wrote:

    Hi Henry - thanks for following up. Here's what I have:

    $ find /usr/lib/impala/lib -name *.jar | grep -i datanucleus
    /usr/lib/impala/lib/datanucleus-enhancer-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-rdbms-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-connectionpool-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-api-jdo-3.2.1.jar
    /usr/lib/impala/lib/datanucleus-core-2.0.3.jar

    $ find /usr/lib/hive/lib -name *.jar | grep -i datanucleus
    /usr/lib/hive/lib/datanucleus-core-2.0.3-ZD5977-CDH5293.jar
    /usr/lib/hive/lib/datanucleus-enhancer-2.0.3.jar
    /usr/lib/hive/lib/datanucleus-rdbms-2.0.3.jar
    /usr/lib/hive/lib/datanucleus-connectionpool-2.0.3.jar


    On Tuesday, November 26, 2013 5:03:25 PM UTC-8, Henry wrote:

    Hi Colin -

    Thanks for the report, and sorry that you're having problems.

    We've been trying to reproduce what you've seen locally, and haven't
    yet done so. We are aware that there might be an issue with the versions of
    some jars in the shipped packages, but our testing hasn't yet revealed the
    exact nature of any problem.

    To confirm the package versions you have, could you run the following
    commands and let us have the output, on your CDH4.5 cluster?

    find /usr/lib/impala/lib -name *.jar | grep -i datanucleus
    find /usr/lib/hive/lib -name *.jar | grep -i datanucleus

    Thanks,
    Henry

    On 26 November 2013 12:12, Colin Marc wrote:

    Johannes, thanks for the tip. Weirdly, I don't see that jar (version
    3) in the hive libs either. What version/package of hive do you have
    installed?


    On Tuesday, November 26, 2013 5:15:02 AM UTC-8, Johannes Kirschnick
    wrote:
    Hi,

    I saw exactly the same problem after rolling forward.
    We tracked that down to a version conflict in the impala lib
    directory. Basically it's hard coding version 2 but version 3 is needed.

    I manually went into the parcel directory to fix it - you need root
    for that.

    So on the node that hosts the catalog server role, find the
    unpacked parcel directory for impala.

    Something like
    /opt/cloudera/parcels/IMPALA

    in the original parcel directory, you'll find

    /opt/cloudera/parcels/IMPALA/lib/impala/lib/datanucleus-core
    -2.0.3.jar

    This is the conflict - you'll need version 3.

    You can just copy the correct version from the hive library

    See here

    /opt/cloudera/parcels/CDH/lib/hive/lib/datanucleus-core-3.2.2.jar

    Once this is done, you need to patch the impala start script to
    pick up the new datanucleus-core version.

    In /opt/cloudera/parcels/IMPALA/bin/catalogd

    Look for

    export CLASSPATH=

    and basically replace the datanucleus version from

    datanucleus-core-2.0.3.jar

    to

    datanucleus-core-3.2.2.jar


    Hope this helps

    Johannes
    On Tuesday, November 26, 2013 11:16:42 AM UTC+1, Colin Marc wrote:

    Sorry - I should've specified that this is from the packages on
    ubuntu precise.
    On Tuesday, November 26, 2013 2:11:45 AM UTC-8, Colin Marc wrote:

    I just installed impala 1.2, and I get this when I try to start
    the new catalogd:

    E1126 09:57:52.758210 11084 catalog.cc:65] java.lang.
    NoClassDefFoundError: org/datanucleus/NucleusContext
    at java.lang.Class.getDeclaredMethods0(Native Method)
    at java.lang.Class.privateGetDeclaredMethods(
    Class.java:2534)
    at java.lang.Class.getMethod0(Class.java:2777)
    at java.lang.Class.getMethod(Class.java:1663)
    at javax.jdo.JDOHelper$15.run(JDOHelper.java:1935)
    at javax.jdo.JDOHelper$15.run(JDOHelper.java:1933)
    at java.security.AccessController.doPrivileged(Native
    Method)
    at javax.jdo.JDOHelper.getMethod(JDOHelper.java:1932)
    at javax.jdo.JDOHelper.invokeGetPersistenceManagerFac
    toryOnImplementation(JDOHelper.java:1156)
    at javax.jdo.JDOHelper.getPersistenceManagerFactory(
    JDOHelper.java:803)
    at javax.jdo.JDOHelper.getPersistenceManagerFactory(
    JDOHelper.java:698)
    at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(
    ObjectStore.java:262)
    at org.apache.hadoop.hive.metastore.ObjectStore.
    getPersistenceManager(ObjectStore.java:291)
    at org.apache.hadoop.hive.metastore.ObjectStore.
    initialize(ObjectStore.java:224)
    at org.apache.hadoop.hive.metastore.ObjectStore.setConf(
    ObjectStore.java:199)
    at org.apache.hadoop.util.ReflectionUtils.setConf(
    ReflectionUtils.java:73)
    at org.apache.hadoop.util.ReflectionUtils.newInstance(
    ReflectionUtils.java:133)
    at org.apache.hadoop.hive.metastore.RetryingRawStore.<
    init>(RetryingRawStore.java:62)
    at org.apache.hadoop.hive.metastore.RetryingRawStore.
    getProxy(RetryingRawStore.java:71)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.newRawStore(HiveMetaStore.java:408)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.getMS(HiveMetaStore.java:396)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.createDefaultDB(HiveMetaStore.java:434)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.init(HiveMetaStore.java:328)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.<init>(HiveMetaStore.java:289)
    at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<
    init>(RetryingHMSHandler.java:56)
    at org.apache.hadoop.hive.metastore.RetryingHMSHandler.
    getProxy(RetryingHMSHandler.java:61)
    at org.apache.hadoop.hive.metastore.HiveMetaStore.
    newHMSHandler(HiveMetaStore.java:4014)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.
    <init>(HiveMetaStoreClient.java:120)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.
    <init>(HiveMetaStoreClient.java:103)
    at com.cloudera.impala.catalog.MetaStoreClientPool$
    MetaStoreClient.<init>(MetaStoreClientPool.java:47)
    at com.cloudera.impala.catalog.MetaStoreClientPool$
    MetaStoreClient.<init>(MetaStoreClientPool.java:40)
    at com.cloudera.impala.catalog.MetaStoreClientPool.
    addClients(MetaStoreClientPool.java:105)
    at com.cloudera.impala.catalog.Ca
    talog.<init>(Catalog.java:112)
    at com.cloudera.impala.catalog.Ca
    talogServiceCatalog.<init>(CatalogServiceCatalog.java:59)
    at com.cloudera.impala.catalog.Ca
    talogServiceCatalog.<init>(CatalogServiceCatalog.java:50)
    at com.cloudera.impala.service.Jn
    iCatalog.<init>(JniCatalog.java:73)
    Caused by: java.lang.ClassNotFoundException:
    org.datanucleus.NucleusContext
    at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native
    Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:
    205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
    at sun.misc.Launcher$AppClassLoader.loadClass(
    Launcher.java:294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
    ... 36 more

    Any ideas?
    To unsubscribe from this group and stop receiving emails from it,
    send an email to impala-user...@cloudera.org.


    --
    Henry Robinson
    Software Engineer
    Cloudera
    415-994-6679
    To unsubscribe from this group and stop receiving emails from it,
    send an email to impala-user...@cloudera.org.


    --
    Henry Robinson
    Software Engineer
    Cloudera
    415-994-6679
    To unsubscribe from this group and stop receiving emails from it, send
    an email to impala-user...@cloudera.org.


    --
    Henry Robinson
    Software Engineer
    Cloudera
    415-994-6679
    To unsubscribe from this group and stop receiving emails from it, send an
    email to impala-user+unsubscribe@cloudera.org.
    To unsubscribe from this group and stop receiving emails from it, send an email to impala-user+unsubscribe@cloudera.org.
  • Tony Li Xu at Nov 27, 2013 at 4:16 pm
    I am having exactly the same problem, impala-shell gives me ERROR:
    show tables;
    Query: show tables
    ERROR: AnalysisException: This Impala daemon is not ready to accept user
    requests. Status: Waiting for catalog update from the StateStore.

    also from the Cloudera Impala query UI, the database drop down menu is
    empty.
    [image: Inline image 1]

    I am using: CDH 4.5.0-1.cdh4.5.0.p0.30 (updated two days ago), IMPALA
    1.2.1-1.p0.850 and Cloudera Standard 4.7.0.

    The OS is CentOS 6.4.

    Roll back to IMPALA 1.1.1 doesn't have this issue. I also tried Johannes's
    suggestion, unfortunately it doesn't work for me.

    Please advice.

    Thanks

    --
    Tony

    On Tue, Nov 26, 2013 at 11:06 PM, Skye Wanderman-Milne wrote:

    Hey Colin, did you try the workaround suggested by Johannes? (Copying the
    jar from hive/lib to impala/lib and changing the CLASSPATH.)

    On Tue, Nov 26, 2013 at 7:45 PM, Colin Marc wrote:

    Hi Henry,

    I've upgraded to cdh4.5, and I'm still having the same problem. The hive
    directory now has the 3.x jars, but the CLASSPATH variable in
    /usr/bin/catalogd still points to the 2.x jars:

    $ dpkg -l | grep cdh
    ii bigtop-jsvc
    1.0.10-1.cdh4.5.0.p0.23~precise-cdh4.5.0 Application to launch
    java daemon
    ii bigtop-utils
    0.6.0+186-1.cdh4.5.0.p0.23~precise-cdh4.5.0 Collection of useful
    tools for Bigtop
    ii hadoop
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 A software platform for
    processing vast amounts of data
    ii hadoop-0.20-mapreduce
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 A software platform for
    processing vast amounts of data
    ii hadoop-0.20-mapreduce-tasktracker
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 Task Tracker for Hadoop
    ii hadoop-client
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 Hadoop client side
    dependencies
    ii hadoop-hdfs
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 The Hadoop Distributed
    File System
    ii hadoop-hdfs-datanode
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 Data Node for Hadoop
    ii hadoop-mapreduce
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 The Hadoop MapReduce
    (MRv2)
    ii hadoop-yarn
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 The Hadoop NextGen
    MapReduce (YARN)
    ii hbase
    0.94.6+165-1.cdh4.5.0.p0.21~precise-cdh4.5.0 HBase is the Hadoop
    database. Use it when you need random, realtime read/write access to your
    Big Data. This project's goal is the hosting of very large tables --
    billions of rows X millions of columns -- atop clusters of commodity
    hardware.
    ii hbase-regionserver
    0.94.6+165-1.cdh4.5.0.p0.21~precise-cdh4.5.0 The Hadoop HBase
    RegionServer server.
    ii hbase-rest
    0.94.6+165-1.cdh4.5.0.p0.21~precise-cdh4.5.0 The Apache HBase REST
    gateway
    ii hive
    0.10.0+214-1.cdh4.5.0.p0.25~precise-cdh4.5.0 Hive is a data
    warehouse infrastructure built on top of Hadoop
    ii hive-hbase
    0.10.0+214-1.cdh4.5.0.p0.25~precise-cdh4.5.0 Provides integration
    between Apache HBase and Apache Hive
    ii hive-jdbc
    0.10.0+214-1.cdh4.5.0.p0.25~precise-cdh4.5.0 Provides libraries
    necessary to connect to Apache Hive via JDBC
    ii parquet
    1.2.5-1.cdh4.5.0.p0.17~precise-cdh4.5.0 A columnar storage
    format for Hadoop.
    ii parquet-format
    1.0.0-1.cdh4.5.0.p0.20~precise-cdh4.5.0 Format definitions for
    Parquet
    ii zookeeper
    3.4.5+24-1.cdh4.5.0.p0.23~precise-cdh4.5.0 A high-performance
    coordination service for distributed applications.

    $ find /usr/lib/impala/lib -name *.jar | grep -i datanucleus
    /usr/lib/impala/lib/datanucleus-rdbms-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-connectionpool-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-enhancer-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-core-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-api-jdo-3.2.1.jar

    $ find /usr/lib/hive/lib -name *.jar | grep -i datanucleus
    /usr/lib/hive/lib/datanucleus-core-3.2.2.jar
    /usr/lib/hive/lib/datanucleus-rdbms-3.2.1.jar
    /usr/lib/hive/lib/datanucleus-api-jdo-3.2.1.jar

    $ grep datanucleus /usr/bin/catalogd | tr ':' '\n' | grep datanucleus
    ${IMPALA_HOME}/lib/datanucleus-api-jdo-3.2.1.jar
    ${IMPALA_HOME}/lib/datanucleus-connectionpool-2.0.3.jar
    ${IMPALA_HOME}/lib/datanucleus-core-2.0.3.jar
    ${IMPALA_HOME}/lib/datanucleus-enhancer-2.0.3.jar
    ${IMPALA_HOME}/lib/datanucleus-rdbms-2.0.3.jar
    On Tuesday, November 26, 2013 6:26:25 PM UTC-8, Henry wrote:

    Hi -

    You will need to upgrade to CDH4.5 to run Impala 1.2.1. (And if you're
    using CM, you'll need to upgrade to CM4.8). Hopefully that might resolve
    your classloading issue - let us know either way, and we'll keep working on
    any problems you encounter.

    Thanks,
    Henry

    On 26 November 2013 18:20, Colin Marc wrote:

    Hi Henry - I was looking at that, and realized the upgrade to 4.5
    didn't completely work on the test cluster. I can work on that. Is
    upgrading the only way?

    On Tuesday, November 26, 2013 6:06:50 PM UTC-8, Henry wrote:

    Hi Colin -

    Thanks for the fast response!

    The version of Hive you have looks old to me - my installation of
    CDH4.5 has datanucleus-core-3.2.2 for example. Can you confirm the
    installed version of CDH (e.g. via dpkg -l | grep -i cdh)?

    I believe that Impala 1.2.1 will only work on CDH4.5.

    Thanks,
    Henry

    On 26 November 2013 17:59, Colin Marc wrote:

    Hi Henry - thanks for following up. Here's what I have:

    $ find /usr/lib/impala/lib -name *.jar | grep -i datanucleus
    /usr/lib/impala/lib/datanucleus-enhancer-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-rdbms-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-connectionpool-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-api-jdo-3.2.1.jar
    /usr/lib/impala/lib/datanucleus-core-2.0.3.jar

    $ find /usr/lib/hive/lib -name *.jar | grep -i datanucleus
    /usr/lib/hive/lib/datanucleus-core-2.0.3-ZD5977-CDH5293.jar
    /usr/lib/hive/lib/datanucleus-enhancer-2.0.3.jar
    /usr/lib/hive/lib/datanucleus-rdbms-2.0.3.jar
    /usr/lib/hive/lib/datanucleus-connectionpool-2.0.3.jar


    On Tuesday, November 26, 2013 5:03:25 PM UTC-8, Henry wrote:

    Hi Colin -

    Thanks for the report, and sorry that you're having problems.

    We've been trying to reproduce what you've seen locally, and haven't
    yet done so. We are aware that there might be an issue with the versions of
    some jars in the shipped packages, but our testing hasn't yet revealed the
    exact nature of any problem.

    To confirm the package versions you have, could you run the
    following commands and let us have the output, on your CDH4.5 cluster?

    find /usr/lib/impala/lib -name *.jar | grep -i datanucleus
    find /usr/lib/hive/lib -name *.jar | grep -i datanucleus

    Thanks,
    Henry

    On 26 November 2013 12:12, Colin Marc wrote:

    Johannes, thanks for the tip. Weirdly, I don't see that jar
    (version 3) in the hive libs either. What version/package of hive do you
    have installed?


    On Tuesday, November 26, 2013 5:15:02 AM UTC-8, Johannes Kirschnick
    wrote:
    Hi,

    I saw exactly the same problem after rolling forward.
    We tracked that down to a version conflict in the impala lib
    directory. Basically it's hard coding version 2 but version 3 is needed.

    I manually went into the parcel directory to fix it - you need
    root for that.

    So on the node that hosts the catalog server role, find the
    unpacked parcel directory for impala.

    Something like
    /opt/cloudera/parcels/IMPALA

    in the original parcel directory, you'll find

    /opt/cloudera/parcels/IMPALA/lib/impala/lib/datanucleus-core
    -2.0.3.jar

    This is the conflict - you'll need version 3.

    You can just copy the correct version from the hive library

    See here

    /opt/cloudera/parcels/CDH/lib/hive/lib/datanucleus-core-3.2.2.jar

    Once this is done, you need to patch the impala start script to
    pick up the new datanucleus-core version.

    In /opt/cloudera/parcels/IMPALA/bin/catalogd

    Look for

    export CLASSPATH=

    and basically replace the datanucleus version from

    datanucleus-core-2.0.3.jar

    to

    datanucleus-core-3.2.2.jar


    Hope this helps

    Johannes
    On Tuesday, November 26, 2013 11:16:42 AM UTC+1, Colin Marc wrote:

    Sorry - I should've specified that this is from the packages on
    ubuntu precise.
    On Tuesday, November 26, 2013 2:11:45 AM UTC-8, Colin Marc wrote:

    I just installed impala 1.2, and I get this when I try to start
    the new catalogd:

    E1126 09:57:52.758210 11084 catalog.cc:65] java.lang.
    NoClassDefFoundError: org/datanucleus/NucleusContext
    at java.lang.Class.getDeclaredMethods0(Native Method)
    at java.lang.Class.privateGetDeclaredMethods(
    Class.java:2534)
    at java.lang.Class.getMethod0(Class.java:2777)
    at java.lang.Class.getMethod(Class.java:1663)
    at javax.jdo.JDOHelper$15.run(JDOHelper.java:1935)
    at javax.jdo.JDOHelper$15.run(JDOHelper.java:1933)
    at java.security.AccessController.doPrivileged(Native
    Method)
    at javax.jdo.JDOHelper.getMethod(JDOHelper.java:1932)
    at javax.jdo.JDOHelper.invokeGetPersistenceManagerFac
    toryOnImplementation(JDOHelper.java:1156)
    at javax.jdo.JDOHelper.getPersistenceManagerFactory(
    JDOHelper.java:803)
    at javax.jdo.JDOHelper.getPersistenceManagerFactory(
    JDOHelper.java:698)
    at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(
    ObjectStore.java:262)
    at org.apache.hadoop.hive.metastore.ObjectStore.
    getPersistenceManager(ObjectStore.java:291)
    at org.apache.hadoop.hive.metastore.ObjectStore.
    initialize(ObjectStore.java:224)
    at org.apache.hadoop.hive.metastore.ObjectStore.setConf(
    ObjectStore.java:199)
    at org.apache.hadoop.util.ReflectionUtils.setConf(
    ReflectionUtils.java:73)
    at org.apache.hadoop.util.ReflectionUtils.newInstance(
    ReflectionUtils.java:133)
    at org.apache.hadoop.hive.metastore.RetryingRawStore.<
    init>(RetryingRawStore.java:62)
    at org.apache.hadoop.hive.metastore.RetryingRawStore.
    getProxy(RetryingRawStore.java:71)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.newRawStore(HiveMetaStore.java:408)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.getMS(HiveMetaStore.java:396)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.createDefaultDB(HiveMetaStore.java:434)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.init(HiveMetaStore.java:328)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.<init>(HiveMetaStore.java:289)
    at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<
    init>(RetryingHMSHandler.java:56)
    at org.apache.hadoop.hive.metastore.RetryingHMSHandler.
    getProxy(RetryingHMSHandler.java:61)
    at org.apache.hadoop.hive.metastore.HiveMetaStore.
    newHMSHandler(HiveMetaStore.java:4014)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.
    <init>(HiveMetaStoreClient.java:120)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.
    <init>(HiveMetaStoreClient.java:103)
    at com.cloudera.impala.catalog.MetaStoreClientPool$
    MetaStoreClient.<init>(MetaStoreClientPool.java:47)
    at com.cloudera.impala.catalog.MetaStoreClientPool$
    MetaStoreClient.<init>(MetaStoreClientPool.java:40)
    at com.cloudera.impala.catalog.MetaStoreClientPool.
    addClients(MetaStoreClientPool.java:105)
    at com.cloudera.impala.catalog.Ca
    talog.<init>(Catalog.java:112)
    at com.cloudera.impala.catalog.Ca
    talogServiceCatalog.<init>(CatalogServiceCatalog.java:59)
    at com.cloudera.impala.catalog.Ca
    talogServiceCatalog.<init>(CatalogServiceCatalog.java:50)
    at com.cloudera.impala.service.Jn
    iCatalog.<init>(JniCatalog.java:73)
    Caused by: java.lang.ClassNotFoundException:
    org.datanucleus.NucleusContext
    at java.net.URLClassLoader$1.run(
    URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native
    Method)
    at java.net.URLClassLoader.findCl
    ass(URLClassLoader.java:205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:
    323)
    at sun.misc.Launcher$AppClassLoader.loadClass(
    Launcher.java:294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
    ... 36 more

    Any ideas?
    To unsubscribe from this group and stop receiving emails from
    it, send an email to impala-user...@cloudera.org.


    --
    Henry Robinson
    Software Engineer
    Cloudera
    415-994-6679
    To unsubscribe from this group and stop receiving emails from it,
    send an email to impala-user...@cloudera.org.


    --
    Henry Robinson
    Software Engineer
    Cloudera
    415-994-6679
    To unsubscribe from this group and stop receiving emails from it, send
    an email to impala-user...@cloudera.org.


    --
    Henry Robinson
    Software Engineer
    Cloudera
    415-994-6679
    To unsubscribe from this group and stop receiving emails from it, send
    an email to impala-user+unsubscribe@cloudera.org.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to impala-user+unsubscribe@cloudera.org.
    To unsubscribe from this group and stop receiving emails from it, send an email to impala-user+unsubscribe@cloudera.org.
  • Ricky Saltzer at Nov 27, 2013 at 4:36 pm
    Hi Tony -

    Please note that Impala 1.2.1 requires Cloudera Manager* 4.8 *due to the
    addition of the catalog server, did you confirm that you were receiving the
    same ClassNotFoundException as Colin and Johannes?

    Thanks!
    Ricky

    On Wed, Nov 27, 2013 at 11:15 AM, Tony Li Xu wrote:

    I am having exactly the same problem, impala-shell gives me ERROR:
    show tables;
    Query: show tables
    ERROR: AnalysisException: This Impala daemon is not ready to accept user
    requests. Status: Waiting for catalog update from the StateStore.

    also from the Cloudera Impala query UI, the database drop down menu is
    empty.
    [image: Inline image 1]

    I am using: CDH 4.5.0-1.cdh4.5.0.p0.30 (updated two days ago), IMPALA
    1.2.1-1.p0.850 and Cloudera Standard 4.7.0.

    The OS is CentOS 6.4.

    Roll back to IMPALA 1.1.1 doesn't have this issue. I also tried Johannes's
    suggestion, unfortunately it doesn't work for me.

    Please advice.

    Thanks

    --
    Tony

    On Tue, Nov 26, 2013 at 11:06 PM, Skye Wanderman-Milne wrote:

    Hey Colin, did you try the workaround suggested by Johannes? (Copying the
    jar from hive/lib to impala/lib and changing the CLASSPATH.)

    On Tue, Nov 26, 2013 at 7:45 PM, Colin Marc wrote:

    Hi Henry,

    I've upgraded to cdh4.5, and I'm still having the same problem. The hive
    directory now has the 3.x jars, but the CLASSPATH variable in
    /usr/bin/catalogd still points to the 2.x jars:

    $ dpkg -l | grep cdh
    ii bigtop-jsvc
    1.0.10-1.cdh4.5.0.p0.23~precise-cdh4.5.0 Application to launch
    java daemon
    ii bigtop-utils
    0.6.0+186-1.cdh4.5.0.p0.23~precise-cdh4.5.0 Collection of useful
    tools for Bigtop
    ii hadoop
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 A software platform for
    processing vast amounts of data
    ii hadoop-0.20-mapreduce
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 A software platform for
    processing vast amounts of data
    ii hadoop-0.20-mapreduce-tasktracker
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 Task Tracker for Hadoop
    ii hadoop-client
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 Hadoop client side
    dependencies
    ii hadoop-hdfs
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 The Hadoop Distributed
    File System
    ii hadoop-hdfs-datanode
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 Data Node for Hadoop
    ii hadoop-mapreduce
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 The Hadoop MapReduce
    (MRv2)
    ii hadoop-yarn
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 The Hadoop NextGen
    MapReduce (YARN)
    ii hbase
    0.94.6+165-1.cdh4.5.0.p0.21~precise-cdh4.5.0 HBase is the Hadoop
    database. Use it when you need random, realtime read/write access to your
    Big Data. This project's goal is the hosting of very large tables --
    billions of rows X millions of columns -- atop clusters of commodity
    hardware.
    ii hbase-regionserver
    0.94.6+165-1.cdh4.5.0.p0.21~precise-cdh4.5.0 The Hadoop HBase
    RegionServer server.
    ii hbase-rest
    0.94.6+165-1.cdh4.5.0.p0.21~precise-cdh4.5.0 The Apache HBase REST
    gateway
    ii hive
    0.10.0+214-1.cdh4.5.0.p0.25~precise-cdh4.5.0 Hive is a data
    warehouse infrastructure built on top of Hadoop
    ii hive-hbase
    0.10.0+214-1.cdh4.5.0.p0.25~precise-cdh4.5.0 Provides integration
    between Apache HBase and Apache Hive
    ii hive-jdbc
    0.10.0+214-1.cdh4.5.0.p0.25~precise-cdh4.5.0 Provides libraries
    necessary to connect to Apache Hive via JDBC
    ii parquet
    1.2.5-1.cdh4.5.0.p0.17~precise-cdh4.5.0 A columnar storage
    format for Hadoop.
    ii parquet-format
    1.0.0-1.cdh4.5.0.p0.20~precise-cdh4.5.0 Format definitions for
    Parquet
    ii zookeeper
    3.4.5+24-1.cdh4.5.0.p0.23~precise-cdh4.5.0 A high-performance
    coordination service for distributed applications.

    $ find /usr/lib/impala/lib -name *.jar | grep -i datanucleus
    /usr/lib/impala/lib/datanucleus-rdbms-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-connectionpool-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-enhancer-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-core-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-api-jdo-3.2.1.jar

    $ find /usr/lib/hive/lib -name *.jar | grep -i datanucleus
    /usr/lib/hive/lib/datanucleus-core-3.2.2.jar
    /usr/lib/hive/lib/datanucleus-rdbms-3.2.1.jar
    /usr/lib/hive/lib/datanucleus-api-jdo-3.2.1.jar

    $ grep datanucleus /usr/bin/catalogd | tr ':' '\n' | grep datanucleus
    ${IMPALA_HOME}/lib/datanucleus-api-jdo-3.2.1.jar
    ${IMPALA_HOME}/lib/datanucleus-connectionpool-2.0.3.jar
    ${IMPALA_HOME}/lib/datanucleus-core-2.0.3.jar
    ${IMPALA_HOME}/lib/datanucleus-enhancer-2.0.3.jar
    ${IMPALA_HOME}/lib/datanucleus-rdbms-2.0.3.jar
    On Tuesday, November 26, 2013 6:26:25 PM UTC-8, Henry wrote:

    Hi -

    You will need to upgrade to CDH4.5 to run Impala 1.2.1. (And if you're
    using CM, you'll need to upgrade to CM4.8). Hopefully that might resolve
    your classloading issue - let us know either way, and we'll keep working on
    any problems you encounter.

    Thanks,
    Henry

    On 26 November 2013 18:20, Colin Marc wrote:

    Hi Henry - I was looking at that, and realized the upgrade to 4.5
    didn't completely work on the test cluster. I can work on that. Is
    upgrading the only way?

    On Tuesday, November 26, 2013 6:06:50 PM UTC-8, Henry wrote:

    Hi Colin -

    Thanks for the fast response!

    The version of Hive you have looks old to me - my installation of
    CDH4.5 has datanucleus-core-3.2.2 for example. Can you confirm the
    installed version of CDH (e.g. via dpkg -l | grep -i cdh)?

    I believe that Impala 1.2.1 will only work on CDH4.5.

    Thanks,
    Henry

    On 26 November 2013 17:59, Colin Marc wrote:

    Hi Henry - thanks for following up. Here's what I have:

    $ find /usr/lib/impala/lib -name *.jar | grep -i datanucleus
    /usr/lib/impala/lib/datanucleus-enhancer-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-rdbms-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-connectionpool-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-api-jdo-3.2.1.jar
    /usr/lib/impala/lib/datanucleus-core-2.0.3.jar

    $ find /usr/lib/hive/lib -name *.jar | grep -i datanucleus
    /usr/lib/hive/lib/datanucleus-core-2.0.3-ZD5977-CDH5293.jar
    /usr/lib/hive/lib/datanucleus-enhancer-2.0.3.jar
    /usr/lib/hive/lib/datanucleus-rdbms-2.0.3.jar
    /usr/lib/hive/lib/datanucleus-connectionpool-2.0.3.jar


    On Tuesday, November 26, 2013 5:03:25 PM UTC-8, Henry wrote:

    Hi Colin -

    Thanks for the report, and sorry that you're having problems.

    We've been trying to reproduce what you've seen locally, and
    haven't yet done so. We are aware that there might be an issue with the
    versions of some jars in the shipped packages, but our testing hasn't yet
    revealed the exact nature of any problem.

    To confirm the package versions you have, could you run the
    following commands and let us have the output, on your CDH4.5 cluster?

    find /usr/lib/impala/lib -name *.jar | grep -i datanucleus
    find /usr/lib/hive/lib -name *.jar | grep -i datanucleus

    Thanks,
    Henry

    On 26 November 2013 12:12, Colin Marc wrote:

    Johannes, thanks for the tip. Weirdly, I don't see that jar
    (version 3) in the hive libs either. What version/package of hive do you
    have installed?


    On Tuesday, November 26, 2013 5:15:02 AM UTC-8, Johannes
    Kirschnick wrote:
    Hi,

    I saw exactly the same problem after rolling forward.
    We tracked that down to a version conflict in the impala lib
    directory. Basically it's hard coding version 2 but version 3 is needed.

    I manually went into the parcel directory to fix it - you need
    root for that.

    So on the node that hosts the catalog server role, find the
    unpacked parcel directory for impala.

    Something like
    /opt/cloudera/parcels/IMPALA

    in the original parcel directory, you'll find

    /opt/cloudera/parcels/IMPALA/lib/impala/lib/datanucleus-core
    -2.0.3.jar

    This is the conflict - you'll need version 3.

    You can just copy the correct version from the hive library

    See here

    /opt/cloudera/parcels/CDH/lib/hive/lib/datanucleus-core-3.2.2.jar

    Once this is done, you need to patch the impala start script to
    pick up the new datanucleus-core version.

    In /opt/cloudera/parcels/IMPALA/bin/catalogd

    Look for

    export CLASSPATH=

    and basically replace the datanucleus version from

    datanucleus-core-2.0.3.jar

    to

    datanucleus-core-3.2.2.jar


    Hope this helps

    Johannes
    On Tuesday, November 26, 2013 11:16:42 AM UTC+1, Colin Marc wrote:

    Sorry - I should've specified that this is from the packages on
    ubuntu precise.
    On Tuesday, November 26, 2013 2:11:45 AM UTC-8, Colin Marc wrote:

    I just installed impala 1.2, and I get this when I try to start
    the new catalogd:

    E1126 09:57:52.758210 11084 catalog.cc:65] java.lang.
    NoClassDefFoundError: org/datanucleus/NucleusContext
    at java.lang.Class.getDeclaredMethods0(Native Method)
    at java.lang.Class.privateGetDeclaredMethods(
    Class.java:2534)
    at java.lang.Class.getMethod0(Class.java:2777)
    at java.lang.Class.getMethod(Class.java:1663)
    at javax.jdo.JDOHelper$15.run(JDOHelper.java:1935)
    at javax.jdo.JDOHelper$15.run(JDOHelper.java:1933)
    at java.security.AccessController.doPrivileged(Native
    Method)
    at javax.jdo.JDOHelper.getMethod(JDOHelper.java:1932)
    at javax.jdo.JDOHelper.invokeGetPersistenceManagerFac
    toryOnImplementation(JDOHelper.java:1156)
    at javax.jdo.JDOHelper.getPersistenceManagerFactory(
    JDOHelper.java:803)
    at javax.jdo.JDOHelper.getPersistenceManagerFactory(
    JDOHelper.java:698)
    at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(
    ObjectStore.java:262)
    at org.apache.hadoop.hive.metastore.ObjectStore.
    getPersistenceManager(ObjectStore.java:291)
    at org.apache.hadoop.hive.metastore.ObjectStore.
    initialize(ObjectStore.java:224)
    at org.apache.hadoop.hive.metasto
    re.ObjectStore.setConf(ObjectStore.java:199)
    at org.apache.hadoop.util.ReflectionUtils.setConf(
    ReflectionUtils.java:73)
    at org.apache.hadoop.util.ReflectionUtils.newInstance(
    ReflectionUtils.java:133)
    at org.apache.hadoop.hive.metastore.RetryingRawStore.<
    init>(RetryingRawStore.java:62)
    at org.apache.hadoop.hive.metastore.RetryingRawStore.
    getProxy(RetryingRawStore.java:71)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.newRawStore(HiveMetaStore.java:408)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.getMS(HiveMetaStore.java:396)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.createDefaultDB(HiveMetaStore.java:434)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.init(HiveMetaStore.java:328)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.<init>(HiveMetaStore.java:289)
    at org.apache.hadoop.hive.metasto
    re.RetryingHMSHandler.<init>(RetryingHMSHandler.java:56)
    at org.apache.hadoop.hive.metastore.RetryingHMSHandler.
    getProxy(RetryingHMSHandler.java:61)
    at org.apache.hadoop.hive.metastore.HiveMetaStore.
    newHMSHandler(HiveMetaStore.java:4014)
    at org.apache.hadoop.hive.metasto
    re.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:120)
    at org.apache.hadoop.hive.metasto
    re.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:103)
    at com.cloudera.impala.catalog.MetaStoreClientPool$
    MetaStoreClient.<init>(MetaStoreClientPool.java:47)
    at com.cloudera.impala.catalog.MetaStoreClientPool$
    MetaStoreClient.<init>(MetaStoreClientPool.java:40)
    at com.cloudera.impala.catalog.MetaStoreClientPool.
    addClients(MetaStoreClientPool.java:105)
    at com.cloudera.impala.catalog.Ca
    talog.<init>(Catalog.java:112)
    at com.cloudera.impala.catalog.Ca
    talogServiceCatalog.<init>(CatalogServiceCatalog.java:59)
    at com.cloudera.impala.catalog.Ca
    talogServiceCatalog.<init>(CatalogServiceCatalog.java:50)
    at com.cloudera.impala.service.Jn
    iCatalog.<init>(JniCatalog.java:73)
    Caused by: java.lang.ClassNotFoundException:
    org.datanucleus.NucleusContext
    at java.net.URLClassLoader$1.run(
    URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native
    Method)
    at java.net.URLClassLoader.findCl
    ass(URLClassLoader.java:205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:
    323)
    at sun.misc.Launcher$AppClassLoader.loadClass(
    Launcher.java:294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:
    268)
    ... 36 more

    Any ideas?
    To unsubscribe from this group and stop receiving emails from
    it, send an email to impala-user...@cloudera.org.


    --
    Henry Robinson
    Software Engineer
    Cloudera
    415-994-6679
    To unsubscribe from this group and stop receiving emails from it,
    send an email to impala-user...@cloudera.org.


    --
    Henry Robinson
    Software Engineer
    Cloudera
    415-994-6679
    To unsubscribe from this group and stop receiving emails from it,
    send an email to impala-user...@cloudera.org.


    --
    Henry Robinson
    Software Engineer
    Cloudera
    415-994-6679
    To unsubscribe from this group and stop receiving emails from it, send
    an email to impala-user+unsubscribe@cloudera.org.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to impala-user+unsubscribe@cloudera.org.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to impala-user+unsubscribe@cloudera.org.


    --
    Ricky Saltzer
    http://www.cloudera.com

    To unsubscribe from this group and stop receiving emails from it, send an email to impala-user+unsubscribe@cloudera.org.
  • Tony Li Xu at Nov 27, 2013 at 4:50 pm
    Hi Ricky:

    Sorry didn't read the Impala1.2.1 requirement. It was showed up in "New
    parcels" page and I just went ahead download and installed it.

    I will upgrade our CM to 4.8.0.

    Thanks for pointing that out.

    --
    Tony

    On Wed, Nov 27, 2013 at 11:36 AM, Ricky Saltzer wrote:

    Hi Tony -

    Please note that Impala 1.2.1 requires Cloudera Manager* 4.8 *due to the
    addition of the catalog server, did you confirm that you were receiving the
    same ClassNotFoundException as Colin and Johannes?

    Thanks!
    Ricky

    On Wed, Nov 27, 2013 at 11:15 AM, Tony Li Xu wrote:

    I am having exactly the same problem, impala-shell gives me ERROR:
    show tables;
    Query: show tables
    ERROR: AnalysisException: This Impala daemon is not ready to accept user
    requests. Status: Waiting for catalog update from the StateStore.

    also from the Cloudera Impala query UI, the database drop down menu is
    empty.
    [image: Inline image 1]

    I am using: CDH 4.5.0-1.cdh4.5.0.p0.30 (updated two days ago), IMPALA
    1.2.1-1.p0.850 and Cloudera Standard 4.7.0.

    The OS is CentOS 6.4.

    Roll back to IMPALA 1.1.1 doesn't have this issue. I also tried Johannes's
    suggestion, unfortunately it doesn't work for me.

    Please advice.

    Thanks

    --
    Tony


    On Tue, Nov 26, 2013 at 11:06 PM, Skye Wanderman-Milne <skye@cloudera.com
    wrote:
    Hey Colin, did you try the workaround suggested by Johannes? (Copying
    the jar from hive/lib to impala/lib and changing the CLASSPATH.)

    On Tue, Nov 26, 2013 at 7:45 PM, Colin Marc wrote:

    Hi Henry,

    I've upgraded to cdh4.5, and I'm still having the same problem. The
    hive directory now has the 3.x jars, but the CLASSPATH variable in
    /usr/bin/catalogd still points to the 2.x jars:

    $ dpkg -l | grep cdh
    ii bigtop-jsvc
    1.0.10-1.cdh4.5.0.p0.23~precise-cdh4.5.0 Application to launch
    java daemon
    ii bigtop-utils
    0.6.0+186-1.cdh4.5.0.p0.23~precise-cdh4.5.0 Collection of useful
    tools for Bigtop
    ii hadoop
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 A software platform for
    processing vast amounts of data
    ii hadoop-0.20-mapreduce
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 A software platform for
    processing vast amounts of data
    ii hadoop-0.20-mapreduce-tasktracker
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 Task Tracker for Hadoop
    ii hadoop-client
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 Hadoop client side
    dependencies
    ii hadoop-hdfs
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 The Hadoop Distributed
    File System
    ii hadoop-hdfs-datanode
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 Data Node for Hadoop
    ii hadoop-mapreduce
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 The Hadoop MapReduce
    (MRv2)
    ii hadoop-yarn
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 The Hadoop NextGen
    MapReduce (YARN)
    ii hbase
    0.94.6+165-1.cdh4.5.0.p0.21~precise-cdh4.5.0 HBase is the Hadoop
    database. Use it when you need random, realtime read/write access to your
    Big Data. This project's goal is the hosting of very large tables --
    billions of rows X millions of columns -- atop clusters of commodity
    hardware.
    ii hbase-regionserver
    0.94.6+165-1.cdh4.5.0.p0.21~precise-cdh4.5.0 The Hadoop HBase
    RegionServer server.
    ii hbase-rest
    0.94.6+165-1.cdh4.5.0.p0.21~precise-cdh4.5.0 The Apache HBase REST
    gateway
    ii hive
    0.10.0+214-1.cdh4.5.0.p0.25~precise-cdh4.5.0 Hive is a data
    warehouse infrastructure built on top of Hadoop
    ii hive-hbase
    0.10.0+214-1.cdh4.5.0.p0.25~precise-cdh4.5.0 Provides integration
    between Apache HBase and Apache Hive
    ii hive-jdbc
    0.10.0+214-1.cdh4.5.0.p0.25~precise-cdh4.5.0 Provides libraries
    necessary to connect to Apache Hive via JDBC
    ii parquet
    1.2.5-1.cdh4.5.0.p0.17~precise-cdh4.5.0 A columnar storage
    format for Hadoop.
    ii parquet-format
    1.0.0-1.cdh4.5.0.p0.20~precise-cdh4.5.0 Format definitions for
    Parquet
    ii zookeeper
    3.4.5+24-1.cdh4.5.0.p0.23~precise-cdh4.5.0 A high-performance
    coordination service for distributed applications.

    $ find /usr/lib/impala/lib -name *.jar | grep -i datanucleus
    /usr/lib/impala/lib/datanucleus-rdbms-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-connectionpool-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-enhancer-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-core-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-api-jdo-3.2.1.jar

    $ find /usr/lib/hive/lib -name *.jar | grep -i datanucleus
    /usr/lib/hive/lib/datanucleus-core-3.2.2.jar
    /usr/lib/hive/lib/datanucleus-rdbms-3.2.1.jar
    /usr/lib/hive/lib/datanucleus-api-jdo-3.2.1.jar

    $ grep datanucleus /usr/bin/catalogd | tr ':' '\n' | grep datanucleus
    ${IMPALA_HOME}/lib/datanucleus-api-jdo-3.2.1.jar
    ${IMPALA_HOME}/lib/datanucleus-connectionpool-2.0.3.jar
    ${IMPALA_HOME}/lib/datanucleus-core-2.0.3.jar
    ${IMPALA_HOME}/lib/datanucleus-enhancer-2.0.3.jar
    ${IMPALA_HOME}/lib/datanucleus-rdbms-2.0.3.jar
    On Tuesday, November 26, 2013 6:26:25 PM UTC-8, Henry wrote:

    Hi -

    You will need to upgrade to CDH4.5 to run Impala 1.2.1. (And if you're
    using CM, you'll need to upgrade to CM4.8). Hopefully that might resolve
    your classloading issue - let us know either way, and we'll keep working on
    any problems you encounter.

    Thanks,
    Henry

    On 26 November 2013 18:20, Colin Marc wrote:

    Hi Henry - I was looking at that, and realized the upgrade to 4.5
    didn't completely work on the test cluster. I can work on that. Is
    upgrading the only way?

    On Tuesday, November 26, 2013 6:06:50 PM UTC-8, Henry wrote:

    Hi Colin -

    Thanks for the fast response!

    The version of Hive you have looks old to me - my installation of
    CDH4.5 has datanucleus-core-3.2.2 for example. Can you confirm the
    installed version of CDH (e.g. via dpkg -l | grep -i cdh)?

    I believe that Impala 1.2.1 will only work on CDH4.5.

    Thanks,
    Henry

    On 26 November 2013 17:59, Colin Marc wrote:

    Hi Henry - thanks for following up. Here's what I have:

    $ find /usr/lib/impala/lib -name *.jar | grep -i datanucleus
    /usr/lib/impala/lib/datanucleus-enhancer-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-rdbms-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-connectionpool-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-api-jdo-3.2.1.jar
    /usr/lib/impala/lib/datanucleus-core-2.0.3.jar

    $ find /usr/lib/hive/lib -name *.jar | grep -i datanucleus
    /usr/lib/hive/lib/datanucleus-core-2.0.3-ZD5977-CDH5293.jar
    /usr/lib/hive/lib/datanucleus-enhancer-2.0.3.jar
    /usr/lib/hive/lib/datanucleus-rdbms-2.0.3.jar
    /usr/lib/hive/lib/datanucleus-connectionpool-2.0.3.jar


    On Tuesday, November 26, 2013 5:03:25 PM UTC-8, Henry wrote:

    Hi Colin -

    Thanks for the report, and sorry that you're having problems.

    We've been trying to reproduce what you've seen locally, and
    haven't yet done so. We are aware that there might be an issue with the
    versions of some jars in the shipped packages, but our testing hasn't yet
    revealed the exact nature of any problem.

    To confirm the package versions you have, could you run the
    following commands and let us have the output, on your CDH4.5 cluster?

    find /usr/lib/impala/lib -name *.jar | grep -i datanucleus
    find /usr/lib/hive/lib -name *.jar | grep -i datanucleus

    Thanks,
    Henry

    On 26 November 2013 12:12, Colin Marc wrote:

    Johannes, thanks for the tip. Weirdly, I don't see that jar
    (version 3) in the hive libs either. What version/package of hive do you
    have installed?


    On Tuesday, November 26, 2013 5:15:02 AM UTC-8, Johannes
    Kirschnick wrote:
    Hi,

    I saw exactly the same problem after rolling forward.
    We tracked that down to a version conflict in the impala lib
    directory. Basically it's hard coding version 2 but version 3 is needed.

    I manually went into the parcel directory to fix it - you need
    root for that.

    So on the node that hosts the catalog server role, find the
    unpacked parcel directory for impala.

    Something like
    /opt/cloudera/parcels/IMPALA

    in the original parcel directory, you'll find

    /opt/cloudera/parcels/IMPALA/lib/impala/lib/datanucleus-core
    -2.0.3.jar

    This is the conflict - you'll need version 3.

    You can just copy the correct version from the hive library

    See here

    /opt/cloudera/parcels/CDH/lib/hive/lib/datanucleus-core-3.2.
    2.jar

    Once this is done, you need to patch the impala start script to
    pick up the new datanucleus-core version.

    In /opt/cloudera/parcels/IMPALA/bin/catalogd

    Look for

    export CLASSPATH=

    and basically replace the datanucleus version from

    datanucleus-core-2.0.3.jar

    to

    datanucleus-core-3.2.2.jar


    Hope this helps

    Johannes

    On Tuesday, November 26, 2013 11:16:42 AM UTC+1, Colin Marc
    wrote:
    Sorry - I should've specified that this is from the packages on
    ubuntu precise.

    On Tuesday, November 26, 2013 2:11:45 AM UTC-8, Colin Marc
    wrote:
    I just installed impala 1.2, and I get this when I try to
    start the new catalogd:

    E1126 09:57:52.758210 11084 catalog.cc:65] java.lang.
    NoClassDefFoundError: org/datanucleus/NucleusContext
    at java.lang.Class.getDeclaredMethods0(Native Method)
    at java.lang.Class.privateGetDeclaredMethods(
    Class.java:2534)
    at java.lang.Class.getMethod0(Class.java:2777)
    at java.lang.Class.getMethod(Class.java:1663)
    at javax.jdo.JDOHelper$15.run(JDOHelper.java:1935)
    at javax.jdo.JDOHelper$15.run(JDOHelper.java:1933)
    at java.security.AccessController.doPrivileged(Native
    Method)
    at javax.jdo.JDOHelper.getMethod(JDOHelper.java:1932)
    at javax.jdo.JDOHelper.invokeGetPersistenceManagerFac
    toryOnImplementation(JDOHelper.java:1156)
    at javax.jdo.JDOHelper.getPersistenceManagerFactory(
    JDOHelper.java:803)
    at javax.jdo.JDOHelper.getPersistenceManagerFactory(
    JDOHelper.java:698)
    at org.apache.hadoop.hive.metasto
    re.ObjectStore.getPMF(ObjectStore.java:262)
    at org.apache.hadoop.hive.metastore.ObjectStore.
    getPersistenceManager(ObjectStore.java:291)
    at org.apache.hadoop.hive.metastore.ObjectStore.
    initialize(ObjectStore.java:224)
    at org.apache.hadoop.hive.metasto
    re.ObjectStore.setConf(ObjectStore.java:199)
    at org.apache.hadoop.util.ReflectionUtils.setConf(
    ReflectionUtils.java:73)
    at org.apache.hadoop.util.ReflectionUtils.newInstance(
    ReflectionUtils.java:133)
    at org.apache.hadoop.hive.metasto
    re.RetryingRawStore.<init>(RetryingRawStore.java:62)
    at org.apache.hadoop.hive.metastore.RetryingRawStore.
    getProxy(RetryingRawStore.java:71)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.newRawStore(HiveMetaStore.java:408)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.getMS(HiveMetaStore.java:396)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.createDefaultDB(HiveMetaStore.java:434)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.init(HiveMetaStore.java:328)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.<init>(HiveMetaStore.java:289)
    at org.apache.hadoop.hive.metasto
    re.RetryingHMSHandler.<init>(RetryingHMSHandler.java:56)
    at org.apache.hadoop.hive.metasto
    re.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:61)
    at org.apache.hadoop.hive.metastore.HiveMetaStore.
    newHMSHandler(HiveMetaStore.java:4014)
    at org.apache.hadoop.hive.metasto
    re.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:120)
    at org.apache.hadoop.hive.metasto
    re.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:103)
    at com.cloudera.impala.catalog.MetaStoreClientPool$
    MetaStoreClient.<init>(MetaStoreClientPool.java:47)
    at com.cloudera.impala.catalog.MetaStoreClientPool$
    MetaStoreClient.<init>(MetaStoreClientPool.java:40)
    at com.cloudera.impala.catalog.MetaStoreClientPool.
    addClients(MetaStoreClientPool.java:105)
    at com.cloudera.impala.catalog.Ca
    talog.<init>(Catalog.java:112)
    at com.cloudera.impala.catalog.Ca
    talogServiceCatalog.<init>(CatalogServiceCatalog.java:59)
    at com.cloudera.impala.catalog.Ca
    talogServiceCatalog.<init>(CatalogServiceCatalog.java:50)
    at com.cloudera.impala.service.Jn
    iCatalog.<init>(JniCatalog.java:73)
    Caused by: java.lang.ClassNotFoundException:
    org.datanucleus.NucleusContext
    at java.net.URLClassLoader$1.run(
    URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native
    Method)
    at java.net.URLClassLoader.findCl
    ass(URLClassLoader.java:205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:
    323)
    at sun.misc.Launcher$AppClassLoader.loadClass(
    Launcher.java:294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:
    268)
    ... 36 more

    Any ideas?
    To unsubscribe from this group and stop receiving emails from
    it, send an email to impala-user...@cloudera.org.


    --
    Henry Robinson
    Software Engineer
    Cloudera
    415-994-6679
    To unsubscribe from this group and stop receiving emails from it,
    send an email to impala-user...@cloudera.org.


    --
    Henry Robinson
    Software Engineer
    Cloudera
    415-994-6679
    To unsubscribe from this group and stop receiving emails from it,
    send an email to impala-user...@cloudera.org.


    --
    Henry Robinson
    Software Engineer
    Cloudera
    415-994-6679
    To unsubscribe from this group and stop receiving emails from it, send
    an email to impala-user+unsubscribe@cloudera.org.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to impala-user+unsubscribe@cloudera.org.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to impala-user+unsubscribe@cloudera.org.


    --
    Ricky Saltzer
    http://www.cloudera.com


    To unsubscribe from this group and stop receiving emails from it, send
    an email to impala-user+unsubscribe@cloudera.org.
    To unsubscribe from this group and stop receiving emails from it, send an email to impala-user+unsubscribe@cloudera.org.
  • Ricky Saltzer at Nov 27, 2013 at 4:52 pm
    No problem, after you upgrade CM to 4.8.0 you should be able to upgrade
    Impala to 1.2.1. If you encounter any errors please let us know, but if the
    error is different than that of Colins, be sure to make a new post so we
    can track them separately.

    Thanks!
    Ricky

    On Wed, Nov 27, 2013 at 11:49 AM, Tony Li Xu wrote:

    Hi Ricky:

    Sorry didn't read the Impala1.2.1 requirement. It was showed up in "New
    parcels" page and I just went ahead download and installed it.

    I will upgrade our CM to 4.8.0.

    Thanks for pointing that out.

    --
    Tony

    On Wed, Nov 27, 2013 at 11:36 AM, Ricky Saltzer wrote:

    Hi Tony -

    Please note that Impala 1.2.1 requires Cloudera Manager* 4.8 *due to the
    addition of the catalog server, did you confirm that you were receiving the
    same ClassNotFoundException as Colin and Johannes?

    Thanks!
    Ricky

    On Wed, Nov 27, 2013 at 11:15 AM, Tony Li Xu wrote:

    I am having exactly the same problem, impala-shell gives me ERROR:
    show tables;
    Query: show tables
    ERROR: AnalysisException: This Impala daemon is not ready to accept user
    requests. Status: Waiting for catalog update from the StateStore.

    also from the Cloudera Impala query UI, the database drop down menu is
    empty.
    [image: Inline image 1]

    I am using: CDH 4.5.0-1.cdh4.5.0.p0.30 (updated two days ago), IMPALA
    1.2.1-1.p0.850 and Cloudera Standard 4.7.0.

    The OS is CentOS 6.4.

    Roll back to IMPALA 1.1.1 doesn't have this issue. I also tried Johannes's
    suggestion, unfortunately it doesn't work for me.

    Please advice.

    Thanks

    --
    Tony


    On Tue, Nov 26, 2013 at 11:06 PM, Skye Wanderman-Milne <
    skye@cloudera.com> wrote:
    Hey Colin, did you try the workaround suggested by Johannes? (Copying
    the jar from hive/lib to impala/lib and changing the CLASSPATH.)

    On Tue, Nov 26, 2013 at 7:45 PM, Colin Marc wrote:

    Hi Henry,

    I've upgraded to cdh4.5, and I'm still having the same problem. The
    hive directory now has the 3.x jars, but the CLASSPATH variable in
    /usr/bin/catalogd still points to the 2.x jars:

    $ dpkg -l | grep cdh
    ii bigtop-jsvc
    1.0.10-1.cdh4.5.0.p0.23~precise-cdh4.5.0 Application to launch
    java daemon
    ii bigtop-utils
    0.6.0+186-1.cdh4.5.0.p0.23~precise-cdh4.5.0 Collection of useful
    tools for Bigtop
    ii hadoop
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 A software platform for
    processing vast amounts of data
    ii hadoop-0.20-mapreduce
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 A software platform for
    processing vast amounts of data
    ii hadoop-0.20-mapreduce-tasktracker
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 Task Tracker for Hadoop
    ii hadoop-client
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 Hadoop client side
    dependencies
    ii hadoop-hdfs
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 The Hadoop Distributed
    File System
    ii hadoop-hdfs-datanode
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 Data Node for Hadoop
    ii hadoop-mapreduce
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 The Hadoop MapReduce
    (MRv2)
    ii hadoop-yarn
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 The Hadoop NextGen
    MapReduce (YARN)
    ii hbase
    0.94.6+165-1.cdh4.5.0.p0.21~precise-cdh4.5.0 HBase is the Hadoop
    database. Use it when you need random, realtime read/write access to your
    Big Data. This project's goal is the hosting of very large tables --
    billions of rows X millions of columns -- atop clusters of commodity
    hardware.
    ii hbase-regionserver
    0.94.6+165-1.cdh4.5.0.p0.21~precise-cdh4.5.0 The Hadoop HBase
    RegionServer server.
    ii hbase-rest
    0.94.6+165-1.cdh4.5.0.p0.21~precise-cdh4.5.0 The Apache HBase REST
    gateway
    ii hive
    0.10.0+214-1.cdh4.5.0.p0.25~precise-cdh4.5.0 Hive is a data
    warehouse infrastructure built on top of Hadoop
    ii hive-hbase
    0.10.0+214-1.cdh4.5.0.p0.25~precise-cdh4.5.0 Provides integration
    between Apache HBase and Apache Hive
    ii hive-jdbc
    0.10.0+214-1.cdh4.5.0.p0.25~precise-cdh4.5.0 Provides libraries
    necessary to connect to Apache Hive via JDBC
    ii parquet
    1.2.5-1.cdh4.5.0.p0.17~precise-cdh4.5.0 A columnar storage
    format for Hadoop.
    ii parquet-format
    1.0.0-1.cdh4.5.0.p0.20~precise-cdh4.5.0 Format definitions for
    Parquet
    ii zookeeper
    3.4.5+24-1.cdh4.5.0.p0.23~precise-cdh4.5.0 A high-performance
    coordination service for distributed applications.

    $ find /usr/lib/impala/lib -name *.jar | grep -i datanucleus
    /usr/lib/impala/lib/datanucleus-rdbms-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-connectionpool-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-enhancer-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-core-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-api-jdo-3.2.1.jar

    $ find /usr/lib/hive/lib -name *.jar | grep -i datanucleus
    /usr/lib/hive/lib/datanucleus-core-3.2.2.jar
    /usr/lib/hive/lib/datanucleus-rdbms-3.2.1.jar
    /usr/lib/hive/lib/datanucleus-api-jdo-3.2.1.jar

    $ grep datanucleus /usr/bin/catalogd | tr ':' '\n' | grep datanucleus
    ${IMPALA_HOME}/lib/datanucleus-api-jdo-3.2.1.jar
    ${IMPALA_HOME}/lib/datanucleus-connectionpool-2.0.3.jar
    ${IMPALA_HOME}/lib/datanucleus-core-2.0.3.jar
    ${IMPALA_HOME}/lib/datanucleus-enhancer-2.0.3.jar
    ${IMPALA_HOME}/lib/datanucleus-rdbms-2.0.3.jar
    On Tuesday, November 26, 2013 6:26:25 PM UTC-8, Henry wrote:

    Hi -

    You will need to upgrade to CDH4.5 to run Impala 1.2.1. (And if
    you're using CM, you'll need to upgrade to CM4.8). Hopefully that might
    resolve your classloading issue - let us know either way, and we'll keep
    working on any problems you encounter.

    Thanks,
    Henry

    On 26 November 2013 18:20, Colin Marc wrote:

    Hi Henry - I was looking at that, and realized the upgrade to 4.5
    didn't completely work on the test cluster. I can work on that. Is
    upgrading the only way?

    On Tuesday, November 26, 2013 6:06:50 PM UTC-8, Henry wrote:

    Hi Colin -

    Thanks for the fast response!

    The version of Hive you have looks old to me - my installation of
    CDH4.5 has datanucleus-core-3.2.2 for example. Can you confirm the
    installed version of CDH (e.g. via dpkg -l | grep -i cdh)?

    I believe that Impala 1.2.1 will only work on CDH4.5.

    Thanks,
    Henry

    On 26 November 2013 17:59, Colin Marc wrote:

    Hi Henry - thanks for following up. Here's what I have:

    $ find /usr/lib/impala/lib -name *.jar | grep -i datanucleus
    /usr/lib/impala/lib/datanucleus-enhancer-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-rdbms-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-connectionpool-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-api-jdo-3.2.1.jar
    /usr/lib/impala/lib/datanucleus-core-2.0.3.jar

    $ find /usr/lib/hive/lib -name *.jar | grep -i datanucleus
    /usr/lib/hive/lib/datanucleus-core-2.0.3-ZD5977-CDH5293.jar
    /usr/lib/hive/lib/datanucleus-enhancer-2.0.3.jar
    /usr/lib/hive/lib/datanucleus-rdbms-2.0.3.jar
    /usr/lib/hive/lib/datanucleus-connectionpool-2.0.3.jar


    On Tuesday, November 26, 2013 5:03:25 PM UTC-8, Henry wrote:

    Hi Colin -

    Thanks for the report, and sorry that you're having problems.

    We've been trying to reproduce what you've seen locally, and
    haven't yet done so. We are aware that there might be an issue with the
    versions of some jars in the shipped packages, but our testing hasn't yet
    revealed the exact nature of any problem.

    To confirm the package versions you have, could you run the
    following commands and let us have the output, on your CDH4.5 cluster?

    find /usr/lib/impala/lib -name *.jar | grep -i datanucleus
    find /usr/lib/hive/lib -name *.jar | grep -i datanucleus

    Thanks,
    Henry

    On 26 November 2013 12:12, Colin Marc wrote:

    Johannes, thanks for the tip. Weirdly, I don't see that jar
    (version 3) in the hive libs either. What version/package of hive do you
    have installed?


    On Tuesday, November 26, 2013 5:15:02 AM UTC-8, Johannes
    Kirschnick wrote:
    Hi,

    I saw exactly the same problem after rolling forward.
    We tracked that down to a version conflict in the impala lib
    directory. Basically it's hard coding version 2 but version 3 is needed.

    I manually went into the parcel directory to fix it - you need
    root for that.

    So on the node that hosts the catalog server role, find the
    unpacked parcel directory for impala.

    Something like
    /opt/cloudera/parcels/IMPALA

    in the original parcel directory, you'll find

    /opt/cloudera/parcels/IMPALA/lib/impala/lib/datanucleus-core
    -2.0.3.jar

    This is the conflict - you'll need version 3.

    You can just copy the correct version from the hive library

    See here

    /opt/cloudera/parcels/CDH/lib/hive/lib/datanucleus-core-3.2.
    2.jar

    Once this is done, you need to patch the impala start script to
    pick up the new datanucleus-core version.

    In /opt/cloudera/parcels/IMPALA/bin/catalogd

    Look for

    export CLASSPATH=

    and basically replace the datanucleus version from

    datanucleus-core-2.0.3.jar

    to

    datanucleus-core-3.2.2.jar


    Hope this helps

    Johannes

    On Tuesday, November 26, 2013 11:16:42 AM UTC+1, Colin Marc
    wrote:
    Sorry - I should've specified that this is from the packages
    on ubuntu precise.

    On Tuesday, November 26, 2013 2:11:45 AM UTC-8, Colin Marc
    wrote:
    I just installed impala 1.2, and I get this when I try to
    start the new catalogd:

    E1126 09:57:52.758210 11084 catalog.cc:65] java.lang.
    NoClassDefFoundError: org/datanucleus/NucleusContext
    at java.lang.Class.getDeclaredMethods0(Native Method)
    at java.lang.Class.privateGetDeclaredMethods(
    Class.java:2534)
    at java.lang.Class.getMethod0(Class.java:2777)
    at java.lang.Class.getMethod(Class.java:1663)
    at javax.jdo.JDOHelper$15.run(JDOHelper.java:1935)
    at javax.jdo.JDOHelper$15.run(JDOHelper.java:1933)
    at java.security.AccessController.doPrivileged(Native
    Method)
    at javax.jdo.JDOHelper.getMethod(JDOHelper.java:1932)
    at javax.jdo.JDOHelper.invokeGetPersistenceManagerFac
    toryOnImplementation(JDOHelper.java:1156)
    at javax.jdo.JDOHelper.getPersistenceManagerFactory(
    JDOHelper.java:803)
    at javax.jdo.JDOHelper.getPersistenceManagerFactory(
    JDOHelper.java:698)
    at org.apache.hadoop.hive.metasto
    re.ObjectStore.getPMF(ObjectStore.java:262)
    at org.apache.hadoop.hive.metastore.ObjectStore.
    getPersistenceManager(ObjectStore.java:291)
    at org.apache.hadoop.hive.metastore.ObjectStore.
    initialize(ObjectStore.java:224)
    at org.apache.hadoop.hive.metasto
    re.ObjectStore.setConf(ObjectStore.java:199)
    at org.apache.hadoop.util.ReflectionUtils.setConf(
    ReflectionUtils.java:73)
    at org.apache.hadoop.util.Reflect
    ionUtils.newInstance(ReflectionUtils.java:133)
    at org.apache.hadoop.hive.metasto
    re.RetryingRawStore.<init>(RetryingRawStore.java:62)
    at org.apache.hadoop.hive.metastore.RetryingRawStore.
    getProxy(RetryingRawStore.java:71)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.newRawStore(HiveMetaStore.java:408)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.getMS(HiveMetaStore.java:396)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.createDefaultDB(HiveMetaStore.java:434)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.init(HiveMetaStore.java:328)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.<init>(HiveMetaStore.java:289)
    at org.apache.hadoop.hive.metasto
    re.RetryingHMSHandler.<init>(RetryingHMSHandler.java:56)
    at org.apache.hadoop.hive.metasto
    re.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:61)
    at org.apache.hadoop.hive.metastore.HiveMetaStore.
    newHMSHandler(HiveMetaStore.java:4014)
    at org.apache.hadoop.hive.metasto
    re.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:120)
    at org.apache.hadoop.hive.metasto
    re.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:103)
    at com.cloudera.impala.catalog.MetaStoreClientPool$
    MetaStoreClient.<init>(MetaStoreClientPool.java:47)
    at com.cloudera.impala.catalog.MetaStoreClientPool$
    MetaStoreClient.<init>(MetaStoreClientPool.java:40)
    at com.cloudera.impala.catalog.MetaStoreClientPool.
    addClients(MetaStoreClientPool.java:105)
    at com.cloudera.impala.catalog.Ca
    talog.<init>(Catalog.java:112)
    at com.cloudera.impala.catalog.Ca
    talogServiceCatalog.<init>(CatalogServiceCatalog.java:59)
    at com.cloudera.impala.catalog.Ca
    talogServiceCatalog.<init>(CatalogServiceCatalog.java:50)
    at com.cloudera.impala.service.Jn
    iCatalog.<init>(JniCatalog.java:73)
    Caused by: java.lang.ClassNotFoundException:
    org.datanucleus.NucleusContext
    at java.net.URLClassLoader$1.run(
    URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native
    Method)
    at java.net.URLClassLoader.findCl
    ass(URLClassLoader.java:205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:
    323)
    at sun.misc.Launcher$AppClassLoader.loadClass(
    Launcher.java:294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:
    268)
    ... 36 more

    Any ideas?
    To unsubscribe from this group and stop receiving emails from
    it, send an email to impala-user...@cloudera.org.


    --
    Henry Robinson
    Software Engineer
    Cloudera
    415-994-6679
    To unsubscribe from this group and stop receiving emails from it,
    send an email to impala-user...@cloudera.org.


    --
    Henry Robinson
    Software Engineer
    Cloudera
    415-994-6679
    To unsubscribe from this group and stop receiving emails from it,
    send an email to impala-user...@cloudera.org.


    --
    Henry Robinson
    Software Engineer
    Cloudera
    415-994-6679
    To unsubscribe from this group and stop receiving emails from it,
    send an email to impala-user+unsubscribe@cloudera.org.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to impala-user+unsubscribe@cloudera.org.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to impala-user+unsubscribe@cloudera.org.


    --
    Ricky Saltzer
    http://www.cloudera.com


    To unsubscribe from this group and stop receiving emails from it, send
    an email to impala-user+unsubscribe@cloudera.org.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to impala-user+unsubscribe@cloudera.org.


    --
    Ricky Saltzer
    http://www.cloudera.com

    To unsubscribe from this group and stop receiving emails from it, send an email to impala-user+unsubscribe@cloudera.org.
  • Colin Marc at Nov 27, 2013 at 7:26 pm
    Hi Skye,

    I did try that workaround on one of the servers, and got catalogd to run
    there. I don't want to encode that in my puppet configuration, though -
    I'll wait for the package to be fixed before upgrading.
    On Nov 26, 2013 8:07 PM, "Skye Wanderman-Milne" wrote:

    Hey Colin, did you try the workaround suggested by Johannes? (Copying the
    jar from hive/lib to impala/lib and changing the CLASSPATH.)

    On Tue, Nov 26, 2013 at 7:45 PM, Colin Marc wrote:

    Hi Henry,

    I've upgraded to cdh4.5, and I'm still having the same problem. The hive
    directory now has the 3.x jars, but the CLASSPATH variable in
    /usr/bin/catalogd still points to the 2.x jars:

    $ dpkg -l | grep cdh
    ii bigtop-jsvc
    1.0.10-1.cdh4.5.0.p0.23~precise-cdh4.5.0 Application to launch
    java daemon
    ii bigtop-utils
    0.6.0+186-1.cdh4.5.0.p0.23~precise-cdh4.5.0 Collection of useful
    tools for Bigtop
    ii hadoop
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 A software platform for
    processing vast amounts of data
    ii hadoop-0.20-mapreduce
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 A software platform for
    processing vast amounts of data
    ii hadoop-0.20-mapreduce-tasktracker
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 Task Tracker for Hadoop
    ii hadoop-client
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 Hadoop client side
    dependencies
    ii hadoop-hdfs
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 The Hadoop Distributed
    File System
    ii hadoop-hdfs-datanode
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 Data Node for Hadoop
    ii hadoop-mapreduce
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 The Hadoop MapReduce
    (MRv2)
    ii hadoop-yarn
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 The Hadoop NextGen
    MapReduce (YARN)
    ii hbase
    0.94.6+165-1.cdh4.5.0.p0.21~precise-cdh4.5.0 HBase is the Hadoop
    database. Use it when you need random, realtime read/write access to your
    Big Data. This project's goal is the hosting of very large tables --
    billions of rows X millions of columns -- atop clusters of commodity
    hardware.
    ii hbase-regionserver
    0.94.6+165-1.cdh4.5.0.p0.21~precise-cdh4.5.0 The Hadoop HBase
    RegionServer server.
    ii hbase-rest
    0.94.6+165-1.cdh4.5.0.p0.21~precise-cdh4.5.0 The Apache HBase REST
    gateway
    ii hive
    0.10.0+214-1.cdh4.5.0.p0.25~precise-cdh4.5.0 Hive is a data
    warehouse infrastructure built on top of Hadoop
    ii hive-hbase
    0.10.0+214-1.cdh4.5.0.p0.25~precise-cdh4.5.0 Provides integration
    between Apache HBase and Apache Hive
    ii hive-jdbc
    0.10.0+214-1.cdh4.5.0.p0.25~precise-cdh4.5.0 Provides libraries
    necessary to connect to Apache Hive via JDBC
    ii parquet
    1.2.5-1.cdh4.5.0.p0.17~precise-cdh4.5.0 A columnar storage
    format for Hadoop.
    ii parquet-format
    1.0.0-1.cdh4.5.0.p0.20~precise-cdh4.5.0 Format definitions for
    Parquet
    ii zookeeper
    3.4.5+24-1.cdh4.5.0.p0.23~precise-cdh4.5.0 A high-performance
    coordination service for distributed applications.

    $ find /usr/lib/impala/lib -name *.jar | grep -i datanucleus
    /usr/lib/impala/lib/datanucleus-rdbms-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-connectionpool-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-enhancer-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-core-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-api-jdo-3.2.1.jar

    $ find /usr/lib/hive/lib -name *.jar | grep -i datanucleus
    /usr/lib/hive/lib/datanucleus-core-3.2.2.jar
    /usr/lib/hive/lib/datanucleus-rdbms-3.2.1.jar
    /usr/lib/hive/lib/datanucleus-api-jdo-3.2.1.jar

    $ grep datanucleus /usr/bin/catalogd | tr ':' '\n' | grep datanucleus
    ${IMPALA_HOME}/lib/datanucleus-api-jdo-3.2.1.jar
    ${IMPALA_HOME}/lib/datanucleus-connectionpool-2.0.3.jar
    ${IMPALA_HOME}/lib/datanucleus-core-2.0.3.jar
    ${IMPALA_HOME}/lib/datanucleus-enhancer-2.0.3.jar
    ${IMPALA_HOME}/lib/datanucleus-rdbms-2.0.3.jar
    On Tuesday, November 26, 2013 6:26:25 PM UTC-8, Henry wrote:

    Hi -

    You will need to upgrade to CDH4.5 to run Impala 1.2.1. (And if you're
    using CM, you'll need to upgrade to CM4.8). Hopefully that might resolve
    your classloading issue - let us know either way, and we'll keep working on
    any problems you encounter.

    Thanks,
    Henry

    On 26 November 2013 18:20, Colin Marc wrote:

    Hi Henry - I was looking at that, and realized the upgrade to 4.5
    didn't completely work on the test cluster. I can work on that. Is
    upgrading the only way?

    On Tuesday, November 26, 2013 6:06:50 PM UTC-8, Henry wrote:

    Hi Colin -

    Thanks for the fast response!

    The version of Hive you have looks old to me - my installation of
    CDH4.5 has datanucleus-core-3.2.2 for example. Can you confirm the
    installed version of CDH (e.g. via dpkg -l | grep -i cdh)?

    I believe that Impala 1.2.1 will only work on CDH4.5.

    Thanks,
    Henry

    On 26 November 2013 17:59, Colin Marc wrote:

    Hi Henry - thanks for following up. Here's what I have:

    $ find /usr/lib/impala/lib -name *.jar | grep -i datanucleus
    /usr/lib/impala/lib/datanucleus-enhancer-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-rdbms-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-connectionpool-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-api-jdo-3.2.1.jar
    /usr/lib/impala/lib/datanucleus-core-2.0.3.jar

    $ find /usr/lib/hive/lib -name *.jar | grep -i datanucleus
    /usr/lib/hive/lib/datanucleus-core-2.0.3-ZD5977-CDH5293.jar
    /usr/lib/hive/lib/datanucleus-enhancer-2.0.3.jar
    /usr/lib/hive/lib/datanucleus-rdbms-2.0.3.jar
    /usr/lib/hive/lib/datanucleus-connectionpool-2.0.3.jar


    On Tuesday, November 26, 2013 5:03:25 PM UTC-8, Henry wrote:

    Hi Colin -

    Thanks for the report, and sorry that you're having problems.

    We've been trying to reproduce what you've seen locally, and haven't
    yet done so. We are aware that there might be an issue with the versions of
    some jars in the shipped packages, but our testing hasn't yet revealed the
    exact nature of any problem.

    To confirm the package versions you have, could you run the
    following commands and let us have the output, on your CDH4.5 cluster?

    find /usr/lib/impala/lib -name *.jar | grep -i datanucleus
    find /usr/lib/hive/lib -name *.jar | grep -i datanucleus

    Thanks,
    Henry

    On 26 November 2013 12:12, Colin Marc wrote:

    Johannes, thanks for the tip. Weirdly, I don't see that jar
    (version 3) in the hive libs either. What version/package of hive do you
    have installed?


    On Tuesday, November 26, 2013 5:15:02 AM UTC-8, Johannes Kirschnick
    wrote:
    Hi,

    I saw exactly the same problem after rolling forward.
    We tracked that down to a version conflict in the impala lib
    directory. Basically it's hard coding version 2 but version 3 is needed.

    I manually went into the parcel directory to fix it - you need
    root for that.

    So on the node that hosts the catalog server role, find the
    unpacked parcel directory for impala.

    Something like
    /opt/cloudera/parcels/IMPALA

    in the original parcel directory, you'll find

    /opt/cloudera/parcels/IMPALA/lib/impala/lib/datanucleus-core
    -2.0.3.jar

    This is the conflict - you'll need version 3.

    You can just copy the correct version from the hive library

    See here

    /opt/cloudera/parcels/CDH/lib/hive/lib/datanucleus-core-3.2.2.jar

    Once this is done, you need to patch the impala start script to
    pick up the new datanucleus-core version.

    In /opt/cloudera/parcels/IMPALA/bin/catalogd

    Look for

    export CLASSPATH=

    and basically replace the datanucleus version from

    datanucleus-core-2.0.3.jar

    to

    datanucleus-core-3.2.2.jar


    Hope this helps

    Johannes
    On Tuesday, November 26, 2013 11:16:42 AM UTC+1, Colin Marc wrote:

    Sorry - I should've specified that this is from the packages on
    ubuntu precise.
    On Tuesday, November 26, 2013 2:11:45 AM UTC-8, Colin Marc wrote:

    I just installed impala 1.2, and I get this when I try to start
    the new catalogd:

    E1126 09:57:52.758210 11084 catalog.cc:65] java.lang.
    NoClassDefFoundError: org/datanucleus/NucleusContext
    at java.lang.Class.getDeclaredMethods0(Native Method)
    at java.lang.Class.privateGetDeclaredMethods(
    Class.java:2534)
    at java.lang.Class.getMethod0(Class.java:2777)
    at java.lang.Class.getMethod(Class.java:1663)
    at javax.jdo.JDOHelper$15.run(JDOHelper.java:1935)
    at javax.jdo.JDOHelper$15.run(JDOHelper.java:1933)
    at java.security.AccessController.doPrivileged(Native
    Method)
    at javax.jdo.JDOHelper.getMethod(JDOHelper.java:1932)
    at javax.jdo.JDOHelper.invokeGetPersistenceManagerFac
    toryOnImplementation(JDOHelper.java:1156)
    at javax.jdo.JDOHelper.getPersistenceManagerFactory(
    JDOHelper.java:803)
    at javax.jdo.JDOHelper.getPersistenceManagerFactory(
    JDOHelper.java:698)
    at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(
    ObjectStore.java:262)
    at org.apache.hadoop.hive.metastore.ObjectStore.
    getPersistenceManager(ObjectStore.java:291)
    at org.apache.hadoop.hive.metastore.ObjectStore.
    initialize(ObjectStore.java:224)
    at org.apache.hadoop.hive.metastore.ObjectStore.setConf(
    ObjectStore.java:199)
    at org.apache.hadoop.util.ReflectionUtils.setConf(
    ReflectionUtils.java:73)
    at org.apache.hadoop.util.ReflectionUtils.newInstance(
    ReflectionUtils.java:133)
    at org.apache.hadoop.hive.metastore.RetryingRawStore.<
    init>(RetryingRawStore.java:62)
    at org.apache.hadoop.hive.metastore.RetryingRawStore.
    getProxy(RetryingRawStore.java:71)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.newRawStore(HiveMetaStore.java:408)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.getMS(HiveMetaStore.java:396)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.createDefaultDB(HiveMetaStore.java:434)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.init(HiveMetaStore.java:328)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.<init>(HiveMetaStore.java:289)
    at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<
    init>(RetryingHMSHandler.java:56)
    at org.apache.hadoop.hive.metastore.RetryingHMSHandler.
    getProxy(RetryingHMSHandler.java:61)
    at org.apache.hadoop.hive.metastore.HiveMetaStore.
    newHMSHandler(HiveMetaStore.java:4014)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.
    <init>(HiveMetaStoreClient.java:120)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.
    <init>(HiveMetaStoreClient.java:103)
    at com.cloudera.impala.catalog.MetaStoreClientPool$
    MetaStoreClient.<init>(MetaStoreClientPool.java:47)
    at com.cloudera.impala.catalog.MetaStoreClientPool$
    MetaStoreClient.<init>(MetaStoreClientPool.java:40)
    at com.cloudera.impala.catalog.MetaStoreClientPool.
    addClients(MetaStoreClientPool.java:105)
    at com.cloudera.impala.catalog.Ca
    talog.<init>(Catalog.java:112)
    at com.cloudera.impala.catalog.Ca
    talogServiceCatalog.<init>(CatalogServiceCatalog.java:59)
    at com.cloudera.impala.catalog.Ca
    talogServiceCatalog.<init>(CatalogServiceCatalog.java:50)
    at com.cloudera.impala.service.Jn
    iCatalog.<init>(JniCatalog.java:73)
    Caused by: java.lang.ClassNotFoundException:
    org.datanucleus.NucleusContext
    at java.net.URLClassLoader$1.run(
    URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native
    Method)
    at java.net.URLClassLoader.findCl
    ass(URLClassLoader.java:205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:
    323)
    at sun.misc.Launcher$AppClassLoader.loadClass(
    Launcher.java:294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
    ... 36 more

    Any ideas?
    To unsubscribe from this group and stop receiving emails from
    it, send an email to impala-user...@cloudera.org.


    --
    Henry Robinson
    Software Engineer
    Cloudera
    415-994-6679
    To unsubscribe from this group and stop receiving emails from it,
    send an email to impala-user...@cloudera.org.


    --
    Henry Robinson
    Software Engineer
    Cloudera
    415-994-6679
    To unsubscribe from this group and stop receiving emails from it, send
    an email to impala-user...@cloudera.org.


    --
    Henry Robinson
    Software Engineer
    Cloudera
    415-994-6679
    To unsubscribe from this group and stop receiving emails from it, send
    an email to impala-user+unsubscribe@cloudera.org.
    To unsubscribe from this group and stop receiving emails from it, send an email to impala-user+unsubscribe@cloudera.org.
  • Colin Marc at Dec 2, 2013 at 10:05 pm
    Hi guys - any update on this?
    On Wednesday, November 27, 2013 12:25:56 PM UTC-7, Colin Marc wrote:

    Hi Skye,

    I did try that workaround on one of the servers, and got catalogd to run
    there. I don't want to encode that in my puppet configuration, though -
    I'll wait for the package to be fixed before upgrading.
    On Nov 26, 2013 8:07 PM, "Skye Wanderman-Milne" wrote:

    Hey Colin, did you try the workaround suggested by Johannes? (Copying the
    jar from hive/lib to impala/lib and changing the CLASSPATH.)

    On Tue, Nov 26, 2013 at 7:45 PM, Colin Marc wrote:

    Hi Henry,

    I've upgraded to cdh4.5, and I'm still having the same problem. The hive
    directory now has the 3.x jars, but the CLASSPATH variable in
    /usr/bin/catalogd still points to the 2.x jars:

    $ dpkg -l | grep cdh
    ii bigtop-jsvc
    1.0.10-1.cdh4.5.0.p0.23~precise-cdh4.5.0 Application to launch
    java daemon
    ii bigtop-utils
    0.6.0+186-1.cdh4.5.0.p0.23~precise-cdh4.5.0 Collection of useful
    tools for Bigtop
    ii hadoop
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 A software platform for
    processing vast amounts of data
    ii hadoop-0.20-mapreduce
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 A software platform for
    processing vast amounts of data
    ii hadoop-0.20-mapreduce-tasktracker
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 Task Tracker for Hadoop
    ii hadoop-client
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 Hadoop client side
    dependencies
    ii hadoop-hdfs
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 The Hadoop Distributed
    File System
    ii hadoop-hdfs-datanode
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 Data Node for Hadoop
    ii hadoop-mapreduce
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 The Hadoop MapReduce
    (MRv2)
    ii hadoop-yarn
    2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0 The Hadoop NextGen
    MapReduce (YARN)
    ii hbase
    0.94.6+165-1.cdh4.5.0.p0.21~precise-cdh4.5.0 HBase is the Hadoop
    database. Use it when you need random, realtime read/write access to your
    Big Data. This project's goal is the hosting of very large tables --
    billions of rows X millions of columns -- atop clusters of commodity
    hardware.
    ii hbase-regionserver
    0.94.6+165-1.cdh4.5.0.p0.21~precise-cdh4.5.0 The Hadoop HBase
    RegionServer server.
    ii hbase-rest
    0.94.6+165-1.cdh4.5.0.p0.21~precise-cdh4.5.0 The Apache HBase REST
    gateway
    ii hive
    0.10.0+214-1.cdh4.5.0.p0.25~precise-cdh4.5.0 Hive is a data
    warehouse infrastructure built on top of Hadoop
    ii hive-hbase
    0.10.0+214-1.cdh4.5.0.p0.25~precise-cdh4.5.0 Provides integration
    between Apache HBase and Apache Hive
    ii hive-jdbc
    0.10.0+214-1.cdh4.5.0.p0.25~precise-cdh4.5.0 Provides libraries
    necessary to connect to Apache Hive via JDBC
    ii parquet
    1.2.5-1.cdh4.5.0.p0.17~precise-cdh4.5.0 A columnar storage
    format for Hadoop.
    ii parquet-format
    1.0.0-1.cdh4.5.0.p0.20~precise-cdh4.5.0 Format definitions for
    Parquet
    ii zookeeper
    3.4.5+24-1.cdh4.5.0.p0.23~precise-cdh4.5.0 A high-performance
    coordination service for distributed applications.

    $ find /usr/lib/impala/lib -name *.jar | grep -i datanucleus
    /usr/lib/impala/lib/datanucleus-rdbms-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-connectionpool-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-enhancer-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-core-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-api-jdo-3.2.1.jar

    $ find /usr/lib/hive/lib -name *.jar | grep -i datanucleus
    /usr/lib/hive/lib/datanucleus-core-3.2.2.jar
    /usr/lib/hive/lib/datanucleus-rdbms-3.2.1.jar
    /usr/lib/hive/lib/datanucleus-api-jdo-3.2.1.jar

    $ grep datanucleus /usr/bin/catalogd | tr ':' '\n' | grep datanucleus
    ${IMPALA_HOME}/lib/datanucleus-api-jdo-3.2.1.jar
    ${IMPALA_HOME}/lib/datanucleus-connectionpool-2.0.3.jar
    ${IMPALA_HOME}/lib/datanucleus-core-2.0.3.jar
    ${IMPALA_HOME}/lib/datanucleus-enhancer-2.0.3.jar
    ${IMPALA_HOME}/lib/datanucleus-rdbms-2.0.3.jar
    On Tuesday, November 26, 2013 6:26:25 PM UTC-8, Henry wrote:

    Hi -

    You will need to upgrade to CDH4.5 to run Impala 1.2.1. (And if you're
    using CM, you'll need to upgrade to CM4.8). Hopefully that might resolve
    your classloading issue - let us know either way, and we'll keep working on
    any problems you encounter.

    Thanks,
    Henry

    On 26 November 2013 18:20, Colin Marc wrote:

    Hi Henry - I was looking at that, and realized the upgrade to 4.5
    didn't completely work on the test cluster. I can work on that. Is
    upgrading the only way?

    On Tuesday, November 26, 2013 6:06:50 PM UTC-8, Henry wrote:

    Hi Colin -

    Thanks for the fast response!

    The version of Hive you have looks old to me - my installation of
    CDH4.5 has datanucleus-core-3.2.2 for example. Can you confirm the
    installed version of CDH (e.g. via dpkg -l | grep -i cdh)?

    I believe that Impala 1.2.1 will only work on CDH4.5.

    Thanks,
    Henry

    On 26 November 2013 17:59, Colin Marc wrote:

    Hi Henry - thanks for following up. Here's what I have:

    $ find /usr/lib/impala/lib -name *.jar | grep -i datanucleus
    /usr/lib/impala/lib/datanucleus-enhancer-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-rdbms-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-connectionpool-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-api-jdo-3.2.1.jar
    /usr/lib/impala/lib/datanucleus-core-2.0.3.jar

    $ find /usr/lib/hive/lib -name *.jar | grep -i datanucleus
    /usr/lib/hive/lib/datanucleus-core-2.0.3-ZD5977-CDH5293.jar
    /usr/lib/hive/lib/datanucleus-enhancer-2.0.3.jar
    /usr/lib/hive/lib/datanucleus-rdbms-2.0.3.jar
    /usr/lib/hive/lib/datanucleus-connectionpool-2.0.3.jar


    On Tuesday, November 26, 2013 5:03:25 PM UTC-8, Henry wrote:

    Hi Colin -

    Thanks for the report, and sorry that you're having problems.

    We've been trying to reproduce what you've seen locally, and
    haven't yet done so. We are aware that there might be an issue with the
    versions of some jars in the shipped packages, but our testing hasn't yet
    revealed the exact nature of any problem.

    To confirm the package versions you have, could you run the
    following commands and let us have the output, on your CDH4.5 cluster?

    find /usr/lib/impala/lib -name *.jar | grep -i datanucleus
    find /usr/lib/hive/lib -name *.jar | grep -i datanucleus

    Thanks,
    Henry

    On 26 November 2013 12:12, Colin Marc wrote:

    Johannes, thanks for the tip. Weirdly, I don't see that jar
    (version 3) in the hive libs either. What version/package of hive do you
    have installed?


    On Tuesday, November 26, 2013 5:15:02 AM UTC-8, Johannes
    Kirschnick wrote:
    Hi,

    I saw exactly the same problem after rolling forward.
    We tracked that down to a version conflict in the impala lib
    directory. Basically it's hard coding version 2 but version 3 is needed.

    I manually went into the parcel directory to fix it - you need
    root for that.

    So on the node that hosts the catalog server role, find the
    unpacked parcel directory for impala.

    Something like
    /opt/cloudera/parcels/IMPALA

    in the original parcel directory, you'll find

    /opt/cloudera/parcels/IMPALA/lib/impala/lib/datanucleus-core
    -2.0.3.jar

    This is the conflict - you'll need version 3.

    You can just copy the correct version from the hive library

    See here

    /opt/cloudera/parcels/CDH/lib/hive/lib/datanucleus-core-3.2.2.jar

    Once this is done, you need to patch the impala start script to
    pick up the new datanucleus-core version.

    In /opt/cloudera/parcels/IMPALA/bin/catalogd

    Look for

    export CLASSPATH=

    and basically replace the datanucleus version from

    datanucleus-core-2.0.3.jar

    to

    datanucleus-core-3.2.2.jar


    Hope this helps

    Johannes
    On Tuesday, November 26, 2013 11:16:42 AM UTC+1, Colin Marc wrote:

    Sorry - I should've specified that this is from the packages on
    ubuntu precise.
    On Tuesday, November 26, 2013 2:11:45 AM UTC-8, Colin Marc wrote:

    I just installed impala 1.2, and I get this when I try to start
    the new catalogd:

    E1126 09:57:52.758210 11084 catalog.cc:65] java.lang.
    NoClassDefFoundError: org/datanucleus/NucleusContext
    at java.lang.Class.getDeclaredMethods0(Native Method)
    at java.lang.Class.privateGetDeclaredMethods(
    Class.java:2534)
    at java.lang.Class.getMethod0(Class.java:2777)
    at java.lang.Class.getMethod(Class.java:1663)
    at javax.jdo.JDOHelper$15.run(JDOHelper.java:1935)
    at javax.jdo.JDOHelper$15.run(JDOHelper.java:1933)
    at java.security.AccessController.doPrivileged(Native
    Method)
    at javax.jdo.JDOHelper.getMethod(JDOHelper.java:1932)
    at javax.jdo.JDOHelper.invokeGetPersistenceManagerFac
    toryOnImplementation(JDOHelper.java:1156)
    at javax.jdo.JDOHelper.getPersistenceManagerFactory(
    JDOHelper.java:803)
    at javax.jdo.JDOHelper.getPersistenceManagerFactory(
    JDOHelper.java:698)
    at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(
    ObjectStore.java:262)
    at org.apache.hadoop.hive.metastore.ObjectStore.
    getPersistenceManager(ObjectStore.java:291)
    at org.apache.hadoop.hive.metastore.ObjectStore.
    initialize(ObjectStore.java:224)
    at org.apache.hadoop.hive.metasto
    re.ObjectStore.setConf(ObjectStore.java:199)
    at org.apache.hadoop.util.ReflectionUtils.setConf(
    ReflectionUtils.java:73)
    at org.apache.hadoop.util.ReflectionUtils.newInstance(
    ReflectionUtils.java:133)
    at org.apache.hadoop.hive.metastore.RetryingRawStore.<
    init>(RetryingRawStore.java:62)
    at org.apache.hadoop.hive.metastore.RetryingRawStore.
    getProxy(RetryingRawStore.java:71)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.newRawStore(HiveMetaStore.java:408)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.getMS(HiveMetaStore.java:396)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.createDefaultDB(HiveMetaStore.java:434)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.init(HiveMetaStore.java:328)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.<init>(HiveMetaStore.java:289)
    at org.apache.hadoop.hive.metasto
    re.RetryingHMSHandler.<init>(RetryingHMSHandler.java:56)
    at org.apache.hadoop.hive.metastore.RetryingHMSHandler.
    getProxy(RetryingHMSHandler.java:61)
    at org.apache.hadoop.hive.metastore.HiveMetaStore.
    newHMSHandler(HiveMetaStore.java:4014)
    at org.apache.hadoop.hive.metasto
    re.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:120)
    at org.apache.hadoop.hive.metasto
    re.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:103)
    at com.cloudera.impala.catalog.MetaStoreClientPool$
    MetaStoreClient.<init>(MetaStoreClientPool.java:47)
    at com.cloudera.impala.catalog.MetaStoreClientPool$
    MetaStoreClient.<init>(MetaStoreClientPool.java:40)
    at com.cloudera.impala.catalog.MetaStoreClientPool.
    addClients(MetaStoreClientPool.java:105)
    at com.cloudera.impala.catalog.Ca
    talog.<init>(Catalog.java:112)
    at com.cloudera.impala.catalog.Ca
    talogServiceCatalog.<init>(CatalogServiceCatalog.java:59)
    at com.cloudera.impala.catalog.Ca
    talogServiceCatalog.<init>(CatalogServiceCatalog.java:50)
    at com.cloudera.impala.service.Jn
    iCatalog.<init>(JniCatalog.java:73)
    Caused by: java.lang.ClassNotFoundException:
    org.datanucleus.NucleusContext
    at java.net.URLClassLoader$1.run(
    URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native
    Method)
    at java.net.URLClassLoader.findCl
    ass(URLClassLoader.java:205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:
    323)
    at sun.misc.Launcher$AppClassLoader.loadClass(
    Launcher.java:294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:
    268)
    ... 36 more

    Any ideas?
    To unsubscribe from this group and stop receiving emails from
    it, send an email to impala-user...@cloudera.org.


    --
    Henry Robinson
    Software Engineer
    Cloudera
    415-994-6679
    To unsubscribe from this group and stop receiving emails from it,
    send an email to impala-user...@cloudera.org.


    --
    Henry Robinson
    Software Engineer
    Cloudera
    415-994-6679
    To unsubscribe from this group and stop receiving emails from it,
    send an email to impala-user...@cloudera.org.


    --
    Henry Robinson
    Software Engineer
    Cloudera
    415-994-6679
    To unsubscribe from this group and stop receiving emails from it, send
    an email to impala-user+unsubscribe@cloudera.org.
    To unsubscribe from this group and stop receiving emails from it, send an email to impala-user+unsubscribe@cloudera.org.
  • Anson Abraham at Dec 26, 2013 at 10:21 pm
    How long will the release be for the fix. I'm experiencing same exact
    issue but on CentOS. But in my case, I'm using a separate hive metastore
    service.
    On Monday, December 2, 2013 8:45:22 PM UTC-5, Skye Wanderman-Milne wrote:

    Hi Colin,

    We found the source of the problem, and are targeting another point
    release for next week that should fix the issue. Thanks for your patience
    and feedback!

    Are you running a Hive metastore service? So far we've only seen this
    issue come up when not using a separate metastore service, which is not
    recommended when using Impala; see Impala's metastore requirements here<http://www.google.com/url?q=http%3A%2F%2Fwww.cloudera.com%2Fcontent%2Fcloudera-content%2Fcloudera-docs%2FImpala%2Flatest%2FInstalling-and-Using-Impala%2Fciiu_prereqs.html%3Fscroll%3Dprereqs_hive_unique_1&sa=D&sntz=1&usg=AFQjCNEMicEE2raoN-DQe1KcWW9VYkqUCA>
    .

    Skye


    On Mon, Dec 2, 2013 at 2:05 PM, Colin Marc <coli...@gmail.com<javascript:>
    wrote:
    Hi guys - any update on this?
    On Wednesday, November 27, 2013 12:25:56 PM UTC-7, Colin Marc wrote:

    Hi Skye,

    I did try that workaround on one of the servers, and got catalogd to run
    there. I don't want to encode that in my puppet configuration, though -
    I'll wait for the package to be fixed before upgrading.
    On Nov 26, 2013 8:07 PM, "Skye Wanderman-Milne" <sk...@cloudera.com<javascript:>>
    wrote:
    Hey Colin, did you try the workaround suggested by Johannes? (Copying
    the jar from hive/lib to impala/lib and changing the CLASSPATH.)


    On Tue, Nov 26, 2013 at 7:45 PM, Colin Marc <coli...@gmail.com<javascript:>
    wrote:
    Hi Henry,

    I've upgraded to cdh4.5, and I'm still having the same problem. The
    hive directory now has the 3.x jars, but the CLASSPATH variable in
    /usr/bin/catalogd still points to the 2.x jars:

    $ dpkg -l | grep cdh
    ii bigtop-jsvc 1.0.10-1.cdh4.5.0.p0.23~precise-cdh4.5.0
    Application to launch java daemon
    ii bigtop-utils 0.6.0+186-1.cdh4.5.0.p0.23~precise-cdh4.5.0
    Collection of useful tools for Bigtop
    ii hadoop 2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0
    A software platform for processing vast amounts of data
    ii hadoop-0.20-mapreduce 2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0
    A software platform for processing vast amounts of data
    ii hadoop-0.20-mapreduce-tasktracker 2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0
    Task Tracker for Hadoop
    ii hadoop-client 2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0
    Hadoop client side dependencies
    ii hadoop-hdfs 2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0
    The Hadoop Distributed File System
    ii hadoop-hdfs-datanode 2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0
    Data Node for Hadoop
    ii hadoop-mapreduce 2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0
    The Hadoop MapReduce (MRv2)
    ii hadoop-yarn 2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0
    The Hadoop NextGen MapReduce (YARN)
    ii hbase 0.94.6+165-1.cdh4.5.0.p0.21~precise-cdh4.5.0
    HBase is the Hadoop database. Use it when you need random, realtime
    read/write access to your Big Data. This project's goal is the hosting of
    very large tables -- billions of rows X millions of columns -- atop
    clusters of commodity hardware.
    ii hbase-regionserver 0.94.6+165-1.cdh4.5.0.p0.21~precise-cdh4.5.0
    The Hadoop HBase RegionServer server.
    ii hbase-rest 0.94.6+165-1.cdh4.5.0.p0.21~precise-cdh4.5.0
    The Apache HBase REST gateway
    ii hive 0.10.0+214-1.cdh4.5.0.p0.25~precise-cdh4.5.0
    Hive is a data warehouse infrastructure built on top of Hadoop
    ii hive-hbase 0.10.0+214-1.cdh4.5.0.p0.25~precise-cdh4.5.0
    Provides integration between Apache HBase and Apache Hive
    ii hive-jdbc 0.10.0+214-1.cdh4.5.0.p0.25~precise-cdh4.5.0
    Provides libraries necessary to connect to Apache Hive via JDBC
    ii parquet 1.2.5-1.cdh4.5.0.p0.17~precise-cdh4.5.0
    A columnar storage format for Hadoop.
    ii parquet-format 1.0.0-1.cdh4.5.0.p0.20~precise-cdh4.5.0
    Format definitions for Parquet
    ii zookeeper 3.4.5+24-1.cdh4.5.0.p0.23~precise-cdh4.5.0
    A high-performance coordination service for distributed
    applications.

    $ find /usr/lib/impala/lib -name *.jar | grep -i datanucleus
    /usr/lib/impala/lib/datanucleus-rdbms-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-connectionpool-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-enhancer-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-core-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-api-jdo-3.2.1.jar

    $ find /usr/lib/hive/lib -name *.jar | grep -i datanucleus
    /usr/lib/hive/lib/datanucleus-core-3.2.2.jar
    /usr/lib/hive/lib/datanucleus-rdbms-3.2.1.jar
    /usr/lib/hive/lib/datanucleus-api-jdo-3.2.1.jar

    $ grep datanucleus /usr/bin/catalogd | tr ':' '\n' | grep datanucleus
    ${IMPALA_HOME}/lib/datanucleus-api-jdo-3.2.1.jar
    ${IMPALA_HOME}/lib/datanucleus-connectionpool-2.0.3.jar
    ${IMPALA_HOME}/lib/datanucleus-core-2.0.3.jar
    ${IMPALA_HOME}/lib/datanucleus-enhancer-2.0.3.jar
    ${IMPALA_HOME}/lib/datanucleus-rdbms-2.0.3.jar
    On Tuesday, November 26, 2013 6:26:25 PM UTC-8, Henry wrote:

    Hi -

    You will need to upgrade to CDH4.5 to run Impala 1.2.1. (And if
    you're using CM, you'll need to upgrade to CM4.8). Hopefully that might
    resolve your classloading issue - let us know either way, and we'll keep
    working on any problems you encounter.

    Thanks,
    Henry

    On 26 November 2013 18:20, Colin Marc wrote:

    Hi Henry - I was looking at that, and realized the upgrade to 4.5
    didn't completely work on the test cluster. I can work on that. Is
    upgrading the only way?

    On Tuesday, November 26, 2013 6:06:50 PM UTC-8, Henry wrote:

    Hi Colin -

    Thanks for the fast response!

    The version of Hive you have looks old to me - my installation of
    CDH4.5 has datanucleus-core-3.2.2 for example. Can you confirm the
    installed version of CDH (e.g. via dpkg -l | grep -i cdh)?

    I believe that Impala 1.2.1 will only work on CDH4.5.

    Thanks,
    Henry

    On 26 November 2013 17:59, Colin Marc wrote:

    Hi Henry - thanks for following up. Here's what I have:

    $ find /usr/lib/impala/lib -name *.jar | grep -i datanucleus
    /usr/lib/impala/lib/datanucleus-enhancer-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-rdbms-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-connectionpool-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-api-jdo-3.2.1.jar
    /usr/lib/impala/lib/datanucleus-core-2.0.3.jar

    $ find /usr/lib/hive/lib -name *.jar | grep -i datanucleus
    /usr/lib/hive/lib/datanucleus-core-2.0.3-ZD5977-CDH5293.jar
    /usr/lib/hive/lib/datanucleus-enhancer-2.0.3.jar
    /usr/lib/hive/lib/datanucleus-rdbms-2.0.3.jar
    /usr/lib/hive/lib/datanucleus-connectionpool-2.0.3.jar


    On Tuesday, November 26, 2013 5:03:25 PM UTC-8, Henry wrote:

    Hi Colin -

    Thanks for the report, and sorry that you're having problems.

    We've been trying to reproduce what you've seen locally, and
    haven't yet done so. We are aware that there might be an issue with the
    versions of some jars in the shipped packages, but our testing hasn't yet
    revealed the exact nature of any problem.

    To confirm the package versions you have, could you run the
    following commands and let us have the output, on your CDH4.5 cluster?

    find /usr/lib/impala/lib -name *.jar | grep -i datanucleus
    find /usr/lib/hive/lib -name *.jar | grep -i datanucleus

    Thanks,
    Henry

    On 26 November 2013 12:12, Colin Marc wrote:

    Johannes, thanks for the tip. Weirdly, I don't see that jar
    (version 3) in the hive libs either. What version/package of hive do you
    have installed?


    On Tuesday, November 26, 2013 5:15:02 AM UTC-8, Johannes
    Kirschnick wrote:
    Hi,

    I saw exactly the same problem after rolling forward.
    We tracked that down to a version conflict in the impala lib
    directory. Basically it's hard coding version 2 but version 3 is needed.

    I manually went into the parcel directory to fix it - you need
    root for that.

    So on the node that hosts the catalog server role, find the
    unpacked parcel directory for impala.

    Something like
    /opt/cloudera/parcels/IMPALA

    in the original parcel directory, you'll find

    /opt/cloudera/parcels/IMPALA/lib/impala/lib/datanucleus-core
    -2.0.3.jar

    This is the conflict - you'll need version 3.

    You can just copy the correct version from the hive library

    See here

    /opt/cloudera/parcels/CDH/lib/hive/lib/datanucleus-core-3.2.
    2.jar

    Once this is done, you need to patch the impala start script to
    pick up the new datanucleus-core version.

    In /opt/cloudera/parcels/IMPALA/bin/catalogd

    Look for

    export CLASSPATH=

    and basically replace the datanucleus version from

    datanucleus-core-2.0.3.jar

    to

    datanucleus-core-3.2.2.jar


    Hope this helps

    Johannes

    On Tuesday, November 26, 2013 11:16:42 AM UTC+1, Colin Marc
    wrote:
    Sorry - I should've specified that this is from the packages
    on ubuntu precise.

    On Tuesday, November 26, 2013 2:11:45 AM UTC-8, Colin Marc
    wrote:
    I just installed impala 1.2, and I get this when I try to
    start the new catalogd:

    E1126 09:57:52.758210 11084 catalog.cc:65] java.lang.
    NoClassDefFoundError: org/datanucleus/NucleusContext
    at java.lang.Class.getDeclaredMethods0(Native Method)
    at java.lang.Class.privateGetDeclaredMethods(
    Class.java:2534)
    at java.lang.Class.getMethod0(Class.java:2777)
    at java.lang.Class.getMethod(Class.java:1663)
    at javax.jdo.JDOHelper$15.run(JDOHelper.java:1935)
    at javax.jdo.JDOHelper$15.run(JDOHelper.java:1933)
    at java.security.AccessController.doPrivileged(Native
    Method)
    at javax.jdo.JDOHelper.getMethod(JDOHelper.java:1932)
    at javax.jdo.JDOHelper.invokeGetPersistenceManagerFac
    toryOnImplementation(JDOHelper.java:1156)
    at javax.jdo.JDOHelper.getPersistenceManagerFactory(
    JDOHelper.java:803)
    at javax.jdo.JDOHelper.getPersistenceManagerFactory(
    JDOHelper.java:698)
    at org.apache.hadoop.hive.metasto
    re.ObjectStore.getPMF(ObjectStore.java:262)
    at org.apache.hadoop.hive.metastore.ObjectStore.
    getPersistenceManager(ObjectStore.java:291)
    at org.apache.hadoop.hive.metastore.ObjectStore.
    initialize(ObjectStore.java:224)
    at org.apache.hadoop.hive.metasto
    re.ObjectStore.setConf(ObjectStore.java:199)
    at org.apache.hadoop.util.ReflectionUtils.setConf(
    ReflectionUtils.java:73)
    at org.apache.hadoop.util.Reflect
    ionUtils.newInstance(ReflectionUtils.java:133)
    at org.apache.hadoop.hive.metasto
    re.RetryingRawStore.<init>(RetryingRawStore.java:62)
    at org.apache.hadoop.hive.metastore.RetryingRawStore.
    getProxy(RetryingRawStore.java:71)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.newRawStore(HiveMetaStore.java:408)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.getMS(HiveMetaStore.java:396)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.createDefaultDB(HiveMetaStore.java:434)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.init(HiveMetaStore.java:328)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.<init>(HiveMetaStore.java:289)
    at org.apache.hadoop.hive.metasto
    re.RetryingHMSHandler.<init>(RetryingHMSHandler.java:56)
    at org.apache.hadoop.hive.metasto
    re.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:61)
    at org.apache.hadoop.hive.metastore.HiveMetaStore.
    newHMSHandler(HiveMetaStore.java:4014)
    at org.apache.hadoop.hive.metasto
    re.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:120)
    at org.apache.hadoop.hive.metasto
    re.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:103)
    at com.cloudera.impala.catalog.MetaStoreClientPool$
    MetaStoreClient.<init>(MetaStoreClientPool.java:47)
    at com.cloudera.impala.catalog.MetaStoreClientPool$
    MetaStoreClient.<init>(MetaStoreClientPool.java:40)
    at com.cloudera.impala.catalog.MetaStoreClientPool.
    addClients(MetaStoreClientPool.java:105)
    at com.cloudera.impala.catalog.Ca
    talog.<init>(Catalog.java:112)
    at com.cloudera.impala.catalog.Ca
    talogServiceCatalog.<init>(CatalogServiceCatalog.java:59)
    at com.cloudera.impala.catalog.Ca
    talogServiceCatalog.<init>(CatalogServiceCatalog.java:50)
    at com.cloudera.impala.service.Jn
    iCatalog.<init>(JniCatalog.java:73)
    Caused by: java.lang.ClassNotFoundException:
    org.datanucleus.NucleusContext
    at java.net.URLClassLoader$1.run(
    URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native
    Method)
    at java.net.URLClassLoader.findCl
    ass(URLClassLoader.java:205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:
    323)
    at sun.misc.Launcher$AppClassLoader.loadClass(
    Launcher.java:294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:
    268)
    ... 36 more

    Any ideas?
    To unsubscribe from this group and stop receiving emails from
    it, send an email to impala-user...@cloudera.org.


    --
    Henry Robinson
    Software Engineer
    Cloudera
    415-994-6679
    To unsubscribe from this group and stop receiving emails from it,
    send an email to impala-user...@cloudera.org.


    --
    Henry Robinson
    Software Engineer
    Cloudera
    415-994-6679
    To unsubscribe from this group and stop receiving emails from it,
    send an email to impala-user...@cloudera.org.


    --
    Henry Robinson
    Software Engineer
    Cloudera
    415-994-6679
    To unsubscribe from this group and stop receiving emails from it,
    send an email to impala-user...@cloudera.org <javascript:>.
    To unsubscribe from this group and stop receiving emails from it,
    send an email to impala-user...@cloudera.org <javascript:>.
    To unsubscribe from this group and stop receiving emails from it, send an email to impala-user+unsubscribe@cloudera.org.
  • Alex Behm at Dec 30, 2013 at 8:48 pm
    Hi Anson,

    are you running Impala 1.2.1? The latest version of Impala (1.2.3) should
    not have this issue anymore.

    Alex

    On Thu, Dec 26, 2013 at 2:20 PM, Anson Abraham wrote:

    How long will the release be for the fix. I'm experiencing same exact
    issue but on CentOS. But in my case, I'm using a separate hive metastore
    service.

    On Monday, December 2, 2013 8:45:22 PM UTC-5, Skye Wanderman-Milne wrote:

    Hi Colin,

    We found the source of the problem, and are targeting another point
    release for next week that should fix the issue. Thanks for your patience
    and feedback!

    Are you running a Hive metastore service? So far we've only seen this
    issue come up when not using a separate metastore service, which is not
    recommended when using Impala; see Impala's metastore requirements here<http://www.google.com/url?q=http%3A%2F%2Fwww.cloudera.com%2Fcontent%2Fcloudera-content%2Fcloudera-docs%2FImpala%2Flatest%2FInstalling-and-Using-Impala%2Fciiu_prereqs.html%3Fscroll%3Dprereqs_hive_unique_1&sa=D&sntz=1&usg=AFQjCNEMicEE2raoN-DQe1KcWW9VYkqUCA>
    .

    Skye

    On Mon, Dec 2, 2013 at 2:05 PM, Colin Marc wrote:

    Hi guys - any update on this?
    On Wednesday, November 27, 2013 12:25:56 PM UTC-7, Colin Marc wrote:

    Hi Skye,

    I did try that workaround on one of the servers, and got catalogd to
    run there. I don't want to encode that in my puppet configuration, though -
    I'll wait for the package to be fixed before upgrading.
    On Nov 26, 2013 8:07 PM, "Skye Wanderman-Milne" <sk...@cloudera.com>
    wrote:
    Hey Colin, did you try the workaround suggested by Johannes? (Copying
    the jar from hive/lib to impala/lib and changing the CLASSPATH.)

    On Tue, Nov 26, 2013 at 7:45 PM, Colin Marc wrote:

    Hi Henry,

    I've upgraded to cdh4.5, and I'm still having the same problem. The
    hive directory now has the 3.x jars, but the CLASSPATH variable in
    /usr/bin/catalogd still points to the 2.x jars:

    $ dpkg -l | grep cdh
    ii bigtop-jsvc 1.0.10-1.cdh4.5.0.p0.23~precise-cdh4.5.0
    Application to launch java daemon
    ii bigtop-utils 0.6.0+186-1.cdh4.5.0.p0.23~precise-cdh4.5.0
    Collection of useful tools for Bigtop
    ii hadoop 2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0
    A software platform for processing vast amounts of data
    ii hadoop-0.20-mapreduce 2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0
    A software platform for processing vast amounts of data
    ii hadoop-0.20-mapreduce-tasktracker 2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0
    Task Tracker for Hadoop
    ii hadoop-client 2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0
    Hadoop client side dependencies
    ii hadoop-hdfs 2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0
    The Hadoop Distributed File System
    ii hadoop-hdfs-datanode 2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0
    Data Node for Hadoop
    ii hadoop-mapreduce 2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0
    The Hadoop MapReduce (MRv2)
    ii hadoop-yarn 2.0.0+1518-1.cdh4.5.0.p0.24~precise-cdh4.5.0
    The Hadoop NextGen MapReduce (YARN)
    ii hbase 0.94.6+165-1.cdh4.5.0.p0.21~precise-cdh4.5.0
    HBase is the Hadoop database. Use it when you need random, realtime
    read/write access to your Big Data. This project's goal is the hosting of
    very large tables -- billions of rows X millions of columns -- atop
    clusters of commodity hardware.
    ii hbase-regionserver 0.94.6+165-1.cdh4.5.0.p0.21~precise-cdh4.5.0
    The Hadoop HBase RegionServer server.
    ii hbase-rest 0.94.6+165-1.cdh4.5.0.p0.21~precise-cdh4.5.0
    The Apache HBase REST gateway
    ii hive 0.10.0+214-1.cdh4.5.0.p0.25~precise-cdh4.5.0
    Hive is a data warehouse infrastructure built on top of Hadoop
    ii hive-hbase 0.10.0+214-1.cdh4.5.0.p0.25~precise-cdh4.5.0
    Provides integration between Apache HBase and Apache Hive
    ii hive-jdbc 0.10.0+214-1.cdh4.5.0.p0.25~precise-cdh4.5.0
    Provides libraries necessary to connect to Apache Hive via JDBC
    ii parquet 1.2.5-1.cdh4.5.0.p0.17~precise-cdh4.5.0
    A columnar storage format for Hadoop.
    ii parquet-format 1.0.0-1.cdh4.5.0.p0.20~precise-cdh4.5.0
    Format definitions for Parquet
    ii zookeeper 3.4.5+24-1.cdh4.5.0.p0.23~precise-cdh4.5.0
    A high-performance coordination service for distributed
    applications.

    $ find /usr/lib/impala/lib -name *.jar | grep -i datanucleus
    /usr/lib/impala/lib/datanucleus-rdbms-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-connectionpool-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-enhancer-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-core-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-api-jdo-3.2.1.jar

    $ find /usr/lib/hive/lib -name *.jar | grep -i datanucleus
    /usr/lib/hive/lib/datanucleus-core-3.2.2.jar
    /usr/lib/hive/lib/datanucleus-rdbms-3.2.1.jar
    /usr/lib/hive/lib/datanucleus-api-jdo-3.2.1.jar

    $ grep datanucleus /usr/bin/catalogd | tr ':' '\n' | grep datanucleus
    ${IMPALA_HOME}/lib/datanucleus-api-jdo-3.2.1.jar
    ${IMPALA_HOME}/lib/datanucleus-connectionpool-2.0.3.jar
    ${IMPALA_HOME}/lib/datanucleus-core-2.0.3.jar
    ${IMPALA_HOME}/lib/datanucleus-enhancer-2.0.3.jar
    ${IMPALA_HOME}/lib/datanucleus-rdbms-2.0.3.jar
    On Tuesday, November 26, 2013 6:26:25 PM UTC-8, Henry wrote:

    Hi -

    You will need to upgrade to CDH4.5 to run Impala 1.2.1. (And if
    you're using CM, you'll need to upgrade to CM4.8). Hopefully that might
    resolve your classloading issue - let us know either way, and we'll keep
    working on any problems you encounter.

    Thanks,
    Henry

    On 26 November 2013 18:20, Colin Marc wrote:

    Hi Henry - I was looking at that, and realized the upgrade to 4.5
    didn't completely work on the test cluster. I can work on that. Is
    upgrading the only way?

    On Tuesday, November 26, 2013 6:06:50 PM UTC-8, Henry wrote:

    Hi Colin -

    Thanks for the fast response!

    The version of Hive you have looks old to me - my installation of
    CDH4.5 has datanucleus-core-3.2.2 for example. Can you confirm the
    installed version of CDH (e.g. via dpkg -l | grep -i cdh)?

    I believe that Impala 1.2.1 will only work on CDH4.5.

    Thanks,
    Henry

    On 26 November 2013 17:59, Colin Marc wrote:

    Hi Henry - thanks for following up. Here's what I have:

    $ find /usr/lib/impala/lib -name *.jar | grep -i datanucleus
    /usr/lib/impala/lib/datanucleus-enhancer-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-rdbms-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-connectionpool-2.0.3.jar
    /usr/lib/impala/lib/datanucleus-api-jdo-3.2.1.jar
    /usr/lib/impala/lib/datanucleus-core-2.0.3.jar

    $ find /usr/lib/hive/lib -name *.jar | grep -i datanucleus
    /usr/lib/hive/lib/datanucleus-core-2.0.3-ZD5977-CDH5293.jar
    /usr/lib/hive/lib/datanucleus-enhancer-2.0.3.jar
    /usr/lib/hive/lib/datanucleus-rdbms-2.0.3.jar
    /usr/lib/hive/lib/datanucleus-connectionpool-2.0.3.jar


    On Tuesday, November 26, 2013 5:03:25 PM UTC-8, Henry wrote:

    Hi Colin -

    Thanks for the report, and sorry that you're having problems.

    We've been trying to reproduce what you've seen locally, and
    haven't yet done so. We are aware that there might be an issue with the
    versions of some jars in the shipped packages, but our testing hasn't yet
    revealed the exact nature of any problem.

    To confirm the package versions you have, could you run the
    following commands and let us have the output, on your CDH4.5 cluster?

    find /usr/lib/impala/lib -name *.jar | grep -i datanucleus
    find /usr/lib/hive/lib -name *.jar | grep -i datanucleus

    Thanks,
    Henry

    On 26 November 2013 12:12, Colin Marc wrote:

    Johannes, thanks for the tip. Weirdly, I don't see that jar
    (version 3) in the hive libs either. What version/package of hive do you
    have installed?


    On Tuesday, November 26, 2013 5:15:02 AM UTC-8, Johannes
    Kirschnick wrote:
    Hi,

    I saw exactly the same problem after rolling forward.
    We tracked that down to a version conflict in the impala lib
    directory. Basically it's hard coding version 2 but version 3 is needed.

    I manually went into the parcel directory to fix it - you need
    root for that.

    So on the node that hosts the catalog server role, find the
    unpacked parcel directory for impala.

    Something like
    /opt/cloudera/parcels/IMPALA

    in the original parcel directory, you'll find

    /opt/cloudera/parcels/IMPALA/lib/impala/lib/datanucleus-core
    -2.0.3.jar

    This is the conflict - you'll need version 3.

    You can just copy the correct version from the hive library

    See here

    /opt/cloudera/parcels/CDH/lib/hive/lib/datanucleus-core-3.2.
    2.jar

    Once this is done, you need to patch the impala start script
    to pick up the new datanucleus-core version.

    In /opt/cloudera/parcels/IMPALA/bin/catalogd

    Look for

    export CLASSPATH=

    and basically replace the datanucleus version from

    datanucleus-core-2.0.3.jar

    to

    datanucleus-core-3.2.2.jar


    Hope this helps

    Johannes

    On Tuesday, November 26, 2013 11:16:42 AM UTC+1, Colin Marc
    wrote:
    Sorry - I should've specified that this is from the packages
    on ubuntu precise.

    On Tuesday, November 26, 2013 2:11:45 AM UTC-8, Colin Marc
    wrote:
    I just installed impala 1.2, and I get this when I try to
    start the new catalogd:

    E1126 09:57:52.758210 11084 catalog.cc:65] java.lang.
    NoClassDefFoundError: org/datanucleus/NucleusContext
    at java.lang.Class.getDeclaredMethods0(Native
    Method)
    at java.lang.Class.privateGetDeclaredMethods(
    Class.java:2534)
    at java.lang.Class.getMethod0(Class.java:2777)
    at java.lang.Class.getMethod(Class.java:1663)
    at javax.jdo.JDOHelper$15.run(JDOHelper.java:1935)
    at javax.jdo.JDOHelper$15.run(JDOHelper.java:1933)
    at java.security.AccessController.doPrivileged(Native
    Method)
    at javax.jdo.JDOHelper.getMethod(
    JDOHelper.java:1932)
    at javax.jdo.JDOHelper.invokeGetP
    ersistenceManagerFactoryOnImplementation(JDOHelper.java:1156
    )
    at javax.jdo.JDOHelper.getPersistenceManagerFactory(
    JDOHelper.java:803)
    at javax.jdo.JDOHelper.getPersistenceManagerFactory(
    JDOHelper.java:698)
    at org.apache.hadoop.hive.metasto
    re.ObjectStore.getPMF(ObjectStore.java:262)
    at org.apache.hadoop.hive.metastore.ObjectStore.
    getPersistenceManager(ObjectStore.java:291)
    at org.apache.hadoop.hive.metastore.ObjectStore.
    initialize(ObjectStore.java:224)
    at org.apache.hadoop.hive.metasto
    re.ObjectStore.setConf(ObjectStore.java:199)
    at org.apache.hadoop.util.ReflectionUtils.setConf(
    ReflectionUtils.java:73)
    at org.apache.hadoop.util.Reflect
    ionUtils.newInstance(ReflectionUtils.java:133)
    at org.apache.hadoop.hive.metasto
    re.RetryingRawStore.<init>(RetryingRawStore.java:62)
    at org.apache.hadoop.hive.metasto
    re.RetryingRawStore.getProxy(RetryingRawStore.java:71)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.newRawStore(HiveMetaStore.java:408)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.getMS(HiveMetaStore.java:396)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.createDefaultDB(HiveMetaStore.java:434)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.init(HiveMetaStore.java:328)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$
    HMSHandler.<init>(HiveMetaStore.java:289)
    at org.apache.hadoop.hive.metasto
    re.RetryingHMSHandler.<init>(RetryingHMSHandler.java:56)
    at org.apache.hadoop.hive.metasto
    re.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:61)
    at org.apache.hadoop.hive.metastore.HiveMetaStore.
    newHMSHandler(HiveMetaStore.java:4014)
    at org.apache.hadoop.hive.metasto
    re.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:120)
    at org.apache.hadoop.hive.metasto
    re.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:103)
    at com.cloudera.impala.catalog.MetaStoreClientPool$
    MetaStoreClient.<init>(MetaStoreClientPool.java:47)
    at com.cloudera.impala.catalog.MetaStoreClientPool$
    MetaStoreClient.<init>(MetaStoreClientPool.java:40)
    at com.cloudera.impala.catalog.MetaStoreClientPool.
    addClients(MetaStoreClientPool.java:105)
    at com.cloudera.impala.catalog.Ca
    talog.<init>(Catalog.java:112)
    at com.cloudera.impala.catalog.Ca
    talogServiceCatalog.<init>(CatalogServiceCatalog.java:59)
    at com.cloudera.impala.catalog.Ca
    talogServiceCatalog.<init>(CatalogServiceCatalog.java:50)
    at com.cloudera.impala.service.Jn
    iCatalog.<init>(JniCatalog.java:73)
    Caused by: java.lang.ClassNotFoundException:
    org.datanucleus.NucleusContext
    at java.net.URLClassLoader$1.run(
    URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native
    Method)
    at java.net.URLClassLoader.findCl
    ass(URLClassLoader.java:205)
    at java.lang.ClassLoader.loadClas
    s(ClassLoader.java:323)
    at sun.misc.Launcher$AppClassLoader.loadClass(
    Launcher.java:294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:
    268)
    ... 36 more

    Any ideas?
    To unsubscribe from this group and stop receiving emails
    from it, send an email to impala-user...@cloudera.org.


    --
    Henry Robinson
    Software Engineer
    Cloudera
    415-994-6679
    To unsubscribe from this group and stop receiving emails from
    it, send an email to impala-user...@cloudera.org.


    --
    Henry Robinson
    Software Engineer
    Cloudera
    415-994-6679
    To unsubscribe from this group and stop receiving emails from it,
    send an email to impala-user...@cloudera.org.


    --
    Henry Robinson
    Software Engineer
    Cloudera
    415-994-6679
    To unsubscribe from this group and stop receiving emails from it,
    send an email to impala-user...@cloudera.org.
    To unsubscribe from this group and stop receiving emails from it,
    send an email to impala-user...@cloudera.org.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to impala-user+unsubscribe@cloudera.org.
    To unsubscribe from this group and stop receiving emails from it, send an email to impala-user+unsubscribe@cloudera.org.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupimpala-user @
categorieshadoop
postedNov 26, '13 at 10:16a
activeDec 30, '13 at 8:48p
posts15
users8
websitecloudera.com
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase