FAQ
Yes, I see .index file for all table files except last file which is 42k in size.


Sent via the Samsung Galaxy S™III, an AT&T 4G LTE smartphone

-------- Original message --------
From: Miklos Christine <mwc@cloudera.com>
Date:
To: impala-user@cloudera.org
Subject: Re: Impala 0.7 indexer error with lzo compressed table

Hello Kashif,

Within HDFS, do you see the files with the corresponding .index extension for your table?
To check you can run "hadoop fs -ls " on the location of your table to see if they exist.
Instructions are located here:
http://www.cloudera.com/content/cloudera-content/cloudera-docs/ImpalaBeta/0.7/Installing-and-Using-Impala/ciiu_topic_7_2.html

Thanks,
Miklos


On Thu, Apr 18, 2013 at 2:38 PM, Kashif Khan wrote:
I am getting some wierd error when trying to run coun(*) on table having lzo compressed data and index built. Same table can be queried from hive. I am also able to query same data that is uncompressed in other table.

All I did is
created a table with INPUTFORMAT as lzo, inserted the data into this table from other table in hive. Ran indexer for this table directory so that index is created on all .lzo files in hdfs. When running count(*) on this table getting below error:

Backend 2:No block index for hdfs://xxx:8020/user/beeswax/warehouse/kk_test_event_lzo/000007_0.lzo after offset: 142742390
No block index for hdfs://xxx:8020/user/beeswax/warehouse/kk_test_event_lzo/000006_0.lzo after offset: 137878948
No block index for hdfs://xxx:8020/user/beeswax/warehouse/kk_test_event_lzo/000009_0.lzo after offset: 79791501

Would appreciate any help. Data delimiter is hive default.

Search Discussions

  • Skye Wanderman-Milne at Apr 21, 2013 at 11:44 pm
    The "No block index" errors you're seeing are likely a bug and can be
    ignored -- it looks like Impala is trying to read past the end of the .lzo
    files, and giving you an error when it fails (but will still process the
    the files correctly). I've filed a JIRA to track the issue:
    https://issues.cloudera.org/browse/IMPALA-298.

    Thanks,
    Skye

    On Thu, Apr 18, 2013 at 3:00 PM, rafzain wrote:

    Yes, I see .index file for all table files except last file which is 42k
    in size.


    Sent via the Samsung Galaxy S™III, an AT&T 4G LTE smartphone



    -------- Original message --------
    From: Miklos Christine <mwc@cloudera.com>
    Date:
    To: impala-user@cloudera.org
    Subject: Re: Impala 0.7 indexer error with lzo compressed table


    Hello Kashif,

    Within HDFS, do you see the files with the corresponding .index extension
    for your table?
    To check you can run "hadoop fs -ls " on the location of your table to see
    if they exist.
    Instructions are located here:

    http://www.cloudera.com/content/cloudera-content/cloudera-docs/ImpalaBeta/0.7/Installing-and-Using-Impala/ciiu_topic_7_2.html

    Thanks,
    Miklos

    On Thu, Apr 18, 2013 at 2:38 PM, Kashif Khan wrote:

    I am getting some wierd error when trying to run coun(*) on table having
    lzo compressed data and index built. Same table can be queried from hive. I
    am also able to query same data that is uncompressed in other table.

    All I did is
    created a table with INPUTFORMAT as lzo, inserted the data into this
    table from other table in hive. Ran indexer for this table directory so
    that index is created on all .lzo files in hdfs. When running count(*) on
    this table getting below error:

    Backend 2:No block index for
    hdfs://xxx:8020/user/beeswax/warehouse/kk_test_event_lzo/000007_0.lzo after
    offset: 142742390
    No block index for
    hdfs://xxx:8020/user/beeswax/warehouse/kk_test_event_lzo/000006_0.lzo after
    offset: 137878948
    No block index for
    hdfs://xxx:8020/user/beeswax/warehouse/kk_test_event_lzo/000009_0.lzo after
    offset: 79791501

    Would appreciate any help. Data delimiter is hive default.
  • Kashif Khan at Apr 22, 2013 at 12:22 am
    Thanks for the update. In my case, query was just hanging. Once I removed
    the unindexed file, it was working fine.

    On Sun, Apr 21, 2013 at 7:43 PM, Skye Wanderman-Milne wrote:

    The "No block index" errors you're seeing are likely a bug and can be
    ignored -- it looks like Impala is trying to read past the end of the .lzo
    files, and giving you an error when it fails (but will still process the
    the files correctly). I've filed a JIRA to track the issue:
    https://issues.cloudera.org/browse/IMPALA-298.

    Thanks,
    Skye

    On Thu, Apr 18, 2013 at 3:00 PM, rafzain wrote:

    Yes, I see .index file for all table files except last file which is 42k
    in size.


    Sent via the Samsung Galaxy S™III, an AT&T 4G LTE smartphone



    -------- Original message --------
    From: Miklos Christine <mwc@cloudera.com>
    Date:
    To: impala-user@cloudera.org
    Subject: Re: Impala 0.7 indexer error with lzo compressed table


    Hello Kashif,

    Within HDFS, do you see the files with the corresponding .index
    extension for your table?
    To check you can run "hadoop fs -ls " on the location of your table to
    see if they exist.
    Instructions are located here:

    http://www.cloudera.com/content/cloudera-content/cloudera-docs/ImpalaBeta/0.7/Installing-and-Using-Impala/ciiu_topic_7_2.html

    Thanks,
    Miklos

    On Thu, Apr 18, 2013 at 2:38 PM, Kashif Khan wrote:

    I am getting some wierd error when trying to run coun(*) on table having
    lzo compressed data and index built. Same table can be queried from hive. I
    am also able to query same data that is uncompressed in other table.

    All I did is
    created a table with INPUTFORMAT as lzo, inserted the data into this
    table from other table in hive. Ran indexer for this table directory so
    that index is created on all .lzo files in hdfs. When running count(*) on
    this table getting below error:

    Backend 2:No block index for
    hdfs://xxx:8020/user/beeswax/warehouse/kk_test_event_lzo/000007_0.lzo after
    offset: 142742390
    No block index for
    hdfs://xxx:8020/user/beeswax/warehouse/kk_test_event_lzo/000006_0.lzo after
    offset: 137878948
    No block index for
    hdfs://xxx:8020/user/beeswax/warehouse/kk_test_event_lzo/000009_0.lzo after
    offset: 79791501

    Would appreciate any help. Data delimiter is hive default.

    --
    Cheers,
    Kashif

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupimpala-user @
categorieshadoop
postedApr 18, '13 at 9:59p
activeApr 22, '13 at 12:22a
posts3
users2
websitecloudera.com
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase