Grokbase Groups HBase user June 2016
FAQ
Wondering if anyone knows the cause of messages like :

"MemstoreFlusherChore requesting flush of
<TABLE>,,1465289533949.94286e3c616b8b4635c9f0665e424a05. because
<COLUMN_FAMILY> has an old edit so flush to free WALs after random delay
265357ms"

My log files seem to be filled with them, I think they account for over 50%
of the lines in my log files.

--
Ted.

Search Discussions

  • Vladimir Rodionov at Jun 17, 2016 at 7:49 pm
    has an old edit
    Yes, this is what PeriodicMemstoreFlusher does: it checks if there are
    edits older than (1h, but configurable) in memstore
    and flushes this memstore if find old edits.

    -Vlad
    On Fri, Jun 17, 2016 at 11:49 AM, Ted wrote:

    Wondering if anyone knows the cause of messages like :

    "MemstoreFlusherChore requesting flush of
    <TABLE>,,1465289533949.94286e3c616b8b4635c9f0665e424a05. because
    <COLUMN_FAMILY> has an old edit so flush to free WALs after random delay
    265357ms"

    My log files seem to be filled with them, I think they account for over 50%
    of the lines in my log files.

    --
    Ted.
  • Vladimir Rodionov at Jun 17, 2016 at 7:52 pm
    This is done to prevent runaway WAL files. WAL file can't be deleted if
      some unflushed edits from this file exist in RS memstore.
    In case of a high load this may lead to accumulation of a large number of
    WAL files in a file system.

    -Vlad
    On Fri, Jun 17, 2016 at 12:49 PM, Vladimir Rodionov wrote:

    has an old edit
    Yes, this is what PeriodicMemstoreFlusher does: it checks if there are
    edits older than (1h, but configurable) in memstore
    and flushes this memstore if find old edits.

    -Vlad
    On Fri, Jun 17, 2016 at 11:49 AM, Ted wrote:

    Wondering if anyone knows the cause of messages like :

    "MemstoreFlusherChore requesting flush of
    <TABLE>,,1465289533949.94286e3c616b8b4635c9f0665e424a05. because
    <COLUMN_FAMILY> has an old edit so flush to free WALs after random delay
    265357ms"

    My log files seem to be filled with them, I think they account for over
    50%
    of the lines in my log files.

    --
    Ted.
  • Ted at Jun 17, 2016 at 9:30 pm
    so what you're saying is that everyone will see at least 1 of those lines
    roughly every hour no matter what, and under higher load you'll see it when
    ever limits are hit for either WAL or Memstore causing flushes.

    So by the sounds of it, it's perfectly normal to be seeing a slew of those
    in the log files and they can more or less be safely ignore.


    thanks.
    On Fri, Jun 17, 2016 at 12:52 PM, Vladimir Rodionov wrote:

    This is done to prevent runaway WAL files. WAL file can't be deleted if
    some unflushed edits from this file exist in RS memstore.
    In case of a high load this may lead to accumulation of a large number of
    WAL files in a file system.

    -Vlad

    On Fri, Jun 17, 2016 at 12:49 PM, Vladimir Rodionov <
    vladrodionov@gmail.com>
    wrote:
    has an old edit
    Yes, this is what PeriodicMemstoreFlusher does: it checks if there are
    edits older than (1h, but configurable) in memstore
    and flushes this memstore if find old edits.

    -Vlad
    On Fri, Jun 17, 2016 at 11:49 AM, Ted wrote:

    Wondering if anyone knows the cause of messages like :

    "MemstoreFlusherChore requesting flush of
    <TABLE>,,1465289533949.94286e3c616b8b4635c9f0665e424a05. because
    <COLUMN_FAMILY> has an old edit so flush to free WALs after random delay
    265357ms"

    My log files seem to be filled with them, I think they account for over
    50%
    of the lines in my log files.

    --
    Ted.


    --
    Ted.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categorieshbase, hadoop
postedJun 17, '16 at 6:50p
activeJun 17, '16 at 9:30p
posts4
users2
websitehbase.apache.org

2 users in discussion

Ted: 2 posts Vladimir Rodionov: 2 posts

People

Translate

site design / logo © 2018 Grokbase