Grokbase Groups HBase user July 2010
FAQ
Classification: UNCLASSIFIED
Caveats: NONE

I'm having a strange issue that I need some help with.

I'm running a hadoop cluster with 1 name server, 26 data nodes, 1
zookeeper server, 1 hmaster, and 26 region servers with Hadoop 0.20.1
and Hbase 0.20.4.

I have a java program inserting rows from raw xml files into HBase. I've
run it a few times and I keep running into this issue. My program
reports no errors when inserting the rows and column families. No
exceptions are raised. However, when I go into the hbase shell and count
the rows I've just inserted, it reports 0 rows.

My program is not complicated. It's just opening an parsing an xml file,
looping through using the same HTable object and creating Put objects
and adding the data, and putting it in. I'm also using Get objects to
check for duplicates before doing the Put, all using the same HTable.
Seems straightforward to me.

Last night I imported 246K rows, and this morning the shell says there
are still 0 rows. I've used hadoop to browse the Hbase folder and the
folder for my table exists and definitely does contain the folders for
my column families and there are large files there which would indicate
that the data is present in Hadoop, but HBase is still reporting no
data. I've run major_compact on .META. and my table to no avail.

I've looked at the hmaster log, and I think I tracked down the right
hregion log (based on what the hbase web page says is running for that
table) and I see no exceptions. I'm still on the default log levels of
info and debug as installed by Hbase.

I'm totally mystified. Has anyone seen anything similar to this and has
an idea what to look for? I haven't posted the logs anywhere yet because
that's a bit tricky for me to get them out of the lab (we're running
standalone, not connected to the net).

Thanks for any help in advance

Chris Schlesiger
cschles@arl.army.mil
Electronics/Software Engineer
U.S. Army Research Lab

Classification: UNCLASSIFIED
Caveats: NONE

Search Discussions

  • Ted Yu at Jul 1, 2010 at 4:29 pm
    HBase 0.20.5 has been released.
    You can try to reproduce the problem for 0.20.5
    On Thu, Jul 1, 2010 at 7:06 AM, Schlesiger, Chris (Civ, ARL/CISD) wrote:

    Classification: UNCLASSIFIED
    Caveats: NONE

    I'm having a strange issue that I need some help with.

    I'm running a hadoop cluster with 1 name server, 26 data nodes, 1
    zookeeper server, 1 hmaster, and 26 region servers with Hadoop 0.20.1
    and Hbase 0.20.4.

    I have a java program inserting rows from raw xml files into HBase. I've
    run it a few times and I keep running into this issue. My program
    reports no errors when inserting the rows and column families. No
    exceptions are raised. However, when I go into the hbase shell and count
    the rows I've just inserted, it reports 0 rows.

    My program is not complicated. It's just opening an parsing an xml file,
    looping through using the same HTable object and creating Put objects
    and adding the data, and putting it in. I'm also using Get objects to
    check for duplicates before doing the Put, all using the same HTable.
    Seems straightforward to me.

    Last night I imported 246K rows, and this morning the shell says there
    are still 0 rows. I've used hadoop to browse the Hbase folder and the
    folder for my table exists and definitely does contain the folders for
    my column families and there are large files there which would indicate
    that the data is present in Hadoop, but HBase is still reporting no
    data. I've run major_compact on .META. and my table to no avail.

    I've looked at the hmaster log, and I think I tracked down the right
    hregion log (based on what the hbase web page says is running for that
    table) and I see no exceptions. I'm still on the default log levels of
    info and debug as installed by Hbase.

    I'm totally mystified. Has anyone seen anything similar to this and has
    an idea what to look for? I haven't posted the logs anywhere yet because
    that's a bit tricky for me to get them out of the lab (we're running
    standalone, not connected to the net).

    Thanks for any help in advance

    Chris Schlesiger
    cschles@arl.army.mil
    Electronics/Software Engineer
    U.S. Army Research Lab

    Classification: UNCLASSIFIED
    Caveats: NONE

  • Andrew Purtell at Jul 2, 2010 at 8:13 pm
    Chris,

    We'd love to help the Army use HBase effectively.

    In the beginning stages of your project it would be really useful if there is some way you can simulate (enough) the behavior of the classified application on an unclassified testbed. That would be very helpful. Just about all troubleshooting support the HBase community can provide relies on being able to examine log files, perhaps all of them over a period of a day or two. If I recall correctly, you can get the logs out of the SCIF, but the paperwork would be a pain.

    - Andy

    --- On Thu, 7/1/10, Schlesiger, Chris (Civ, ARL/CISD) wrote:

    [...]
    I'm totally mystified. Has anyone seen anything similar to
    this and has
    an idea what to look for? I haven't posted the logs
    anywhere yet because
    that's a bit tricky for me to get them out of the lab
    (we're running
    standalone, not connected to the net).

    Thanks for any help in advance

    Chris Schlesiger
    cschles@arl.army.mil
    Electronics/Software Engineer
    U.S. Army Research Lab

    Classification: UNCLASSIFIED
    Caveats: NONE

  • Stack at Jul 3, 2010 at 2:03 am

    On Thu, Jul 1, 2010 at 7:06 AM, Schlesiger, Chris (Civ, ARL/CISD) wrote:
    I'm running a hadoop cluster with 1 name server, 26 data nodes, 1
    zookeeper server, 1 hmaster, and 26 region servers with Hadoop 0.20.1
    and Hbase 0.20.4.
    Yeah, update to 0.20.5 though probably for reasons unrelated to this issue.

    I have a java program inserting rows from raw xml files into HBase. I've
    run it a few times and I keep running into this issue. My program
    reports no errors when inserting the rows and column families. No
    exceptions are raised. However, when I go into the hbase shell and count
    the rows I've just inserted, it reports 0 rows.

    Very strange.

    My program is not complicated. It's just opening an parsing an xml file,
    looping through using the same HTable object and creating Put objects
    and adding the data, and putting it in. I'm also using Get objects to
    check for duplicates before doing the Put, all using the same HTable.
    Seems straightforward to me.

    Can you try one entry only? Does checking and putting one entry not
    work? If so, paste your code (if you can).

    In the past, weird stuff around bulk uploads and subsequent scans
    reporting 0 rows have sometimes revolved around timestamp. Are you
    specifying timestamp on your uploads or letting the system do it for
    you? If former, any chance the ts could be in the future?


    St.Ack


    Last night I imported 246K rows, and this morning the shell says there
    are still 0 rows. I've used hadoop to browse the Hbase folder and the
    folder for my table exists and definitely does contain the folders for
    my column families and there are large files there which would indicate
    that the data is present in Hadoop, but HBase is still reporting no
    data. I've run major_compact on .META. and my table to no avail.

    I've looked at the hmaster log, and I think I tracked down the right
    hregion log (based on what the hbase web page says is running for that
    table) and I see no exceptions. I'm still on the default log levels of
    info and debug as installed by Hbase.

    I'm totally mystified. Has anyone seen anything similar to this and has
    an idea what to look for? I haven't posted the logs anywhere yet because
    that's a bit tricky for me to get them out of the lab (we're running
    standalone, not connected to the net).

    Thanks for any help in advance

    Chris Schlesiger
    cschles@arl.army.mil
    Electronics/Software Engineer
    U.S. Army Research Lab

    Classification: UNCLASSIFIED
    Caveats: NONE

  • Michael Segel at Jul 3, 2010 at 6:31 pm
    I'm not sure of the problem.
    We're doing the same thing on 20.3.

    I know you're doing a count, but what about a scan of the table?


    Date: Fri, 2 Jul 2010 19:02:44 -0700
    Subject: Re: Rows added but not showing up (UNCLASSIFIED)
    From: stack@duboce.net
    To: user@hbase.apache.org

    On Thu, Jul 1, 2010 at 7:06 AM, Schlesiger, Chris (Civ, ARL/CISD)
    wrote:
    I'm running a hadoop cluster with 1 name server, 26 data nodes, 1
    zookeeper server, 1 hmaster, and 26 region servers with Hadoop 0.20.1
    and Hbase 0.20.4.
    Yeah, update to 0.20.5 though probably for reasons unrelated to this issue.

    I have a java program inserting rows from raw xml files into HBase. I've
    run it a few times and I keep running into this issue. My program
    reports no errors when inserting the rows and column families. No
    exceptions are raised. However, when I go into the hbase shell and count
    the rows I've just inserted, it reports 0 rows.

    Very strange.

    My program is not complicated. It's just opening an parsing an xml file,
    looping through using the same HTable object and creating Put objects
    and adding the data, and putting it in. I'm also using Get objects to
    check for duplicates before doing the Put, all using the same HTable.
    Seems straightforward to me.

    Can you try one entry only? Does checking and putting one entry not
    work? If so, paste your code (if you can).

    In the past, weird stuff around bulk uploads and subsequent scans
    reporting 0 rows have sometimes revolved around timestamp. Are you
    specifying timestamp on your uploads or letting the system do it for
    you? If former, any chance the ts could be in the future?


    St.Ack


    Last night I imported 246K rows, and this morning the shell says there
    are still 0 rows. I've used hadoop to browse the Hbase folder and the
    folder for my table exists and definitely does contain the folders for
    my column families and there are large files there which would indicate
    that the data is present in Hadoop, but HBase is still reporting no
    data. I've run major_compact on .META. and my table to no avail.

    I've looked at the hmaster log, and I think I tracked down the right
    hregion log (based on what the hbase web page says is running for that
    table) and I see no exceptions. I'm still on the default log levels of
    info and debug as installed by Hbase.

    I'm totally mystified. Has anyone seen anything similar to this and has
    an idea what to look for? I haven't posted the logs anywhere yet because
    that's a bit tricky for me to get them out of the lab (we're running
    standalone, not connected to the net).

    Thanks for any help in advance

    Chris Schlesiger
    cschles@arl.army.mil
    Electronics/Software Engineer
    U.S. Army Research Lab

    Classification: UNCLASSIFIED
    Caveats: NONE

    _________________________________________________________________
    Hotmail is redefining busy with tools for the New Busy. Get more from your inbox.
    http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_2

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categorieshbase, hadoop
postedJul 1, '10 at 2:06p
activeJul 3, '10 at 6:31p
posts5
users5
websitehbase.apache.org

People

Translate

site design / logo © 2022 Grokbase