FAQ
Hi all,

This is a potential memory error in nodeSubplan.c or execGrouping.c
Using select '1'::TEXT IN ((SELECT '1'::NAME) UNION ALL SELECT '1'::NAME);
to reproduce this bug.

You may see the memory content that slot1's tts_values[0] point to before
and after the statement : MemoryContextReset(evalContext) of
execTuplesMatch() in PG version with --enable-cassert switch on. You will
find that the memory is set to 0x7f7f7f7f.

Here are my code analysis:
For the executor node SubPlanState, The node has a ExprContext named
'innerecontext' that is created by itself in function ExecInitSubPlan().
Also, the 'innerecontext' is used to build a project info, 'projRight', of
SubPlanState node.

When the SubPlanState node is executed, a hash table of subplan will be
built, so buildSubPlanHash() will be called. In buildSubPlanHash function,
'hashtable' of SubPlanState will be initialized by calling
BuildTupleHashTable(), and the code passes the
'ecxt_per_tuple_memory' of 'innerecontext' to BuildTupleHashTable() as the
'hashtable''s tempcxt.

At this point, we can conclude that the 'projRight' and 'hashtable''s
'tempcxt' of SubPlanState node are all using the same memory context, that
is 'innerecontext' in SubPlanState node. So:
1) The memory of all the fetched tuples from 'projRight' will be located in
'innerecontext' of SubPlanState node.
2) All other intermediate result that is located in 'hashtable''s tempcxt,
also in fact, will reside in the 'innerecontext' of SubPlanState node.

Now next:
In buildSubPlanHash(), we will fetch tuple from 'projRight' to fill the
hash table of SubPlanState node. As we known, all the fetched tuples are
located in the 'innerecontext'. When filling the tuple hash table, if the
tuple already exists in the hash table of SubPlanState node, the match
function execTuplesMatch() will be called by tuples matching, but in this
function, the statement:
MemoryContextReset(evalContext);
will be reset 'evalContext', to free some memory usage. In fact, the
'evalContext' is equal to 'innerecontext' that the fetched tuples are
located in, and actually this statement will reset the 'innerecontext'. So
the fetched tuple's memory are all lost. That's the problem.

In fact, this error only in debug version, but not in release version. In
debug using --enable-cassert to configure, the memory will be set to
0x7f7f7f7f, but not for release. For this error memory usage, the pg does
not always report error because the 0x7f7f7f in testing Macro
VARATT_IS_COMPRESSED() and VARATT_IS_EXTERNAL() are always false in
!WORDS_BIGENDIAN platform such as i386 GNU Linux and in WORDS_BIGENDIAN, the
testing Macro VARATT_IS_COMPRESSED() will be true and error may be reported
such as AIX(Power 6 AIX V6.1).

To fix this problem, we can use another memory context to passin
BuildTupleHashTable() as the hashtable's tempcxt, or use other memory
context as hash table's tempcxt or other ways.


Any questions, please contact me.
Regards

Search Discussions

  • Tom Lane at Jul 28, 2010 at 3:19 am

    "Tao Ma" <feng_eden@163.com> writes:
    This is a potential memory error in nodeSubplan.c or execGrouping.c
    Using select '1'::TEXT IN ((SELECT '1'::NAME) UNION ALL SELECT '1'::NAME);
    to reproduce this bug.
    ...
    To fix this problem, we can use another memory context to passin
    BuildTupleHashTable() as the hashtable's tempcxt, or use other memory
    context as hash table's tempcxt or other ways.
    Yeah, I think you're right --- we can't get away with reusing the
    innerecontext's per-tuple context for the hashtable temp contexts.
    The best solution is probably to make an additional context that
    does nothing but act as the hashtable temp context.

    This bug's been there a long time :-(. Surprising that no one found it
    before. It would be mostly masked in a non-assert build, but not
    entirely, I think.

    regards, tom lane
  • Tom Lane at Jul 28, 2010 at 4:55 am

    I wrote:
    "Tao Ma" <feng_eden@163.com> writes:
    This is a potential memory error in nodeSubplan.c or execGrouping.c
    Using select '1'::TEXT IN ((SELECT '1'::NAME) UNION ALL SELECT '1'::NAME);
    to reproduce this bug.
    ...
    To fix this problem, we can use another memory context to passin
    BuildTupleHashTable() as the hashtable's tempcxt, or use other memory
    context as hash table's tempcxt or other ways.
    Yeah, I think you're right --- we can't get away with reusing the
    innerecontext's per-tuple context for the hashtable temp contexts.
    The best solution is probably to make an additional context that
    does nothing but act as the hashtable temp context.
    I've committed a fix along those lines. Thanks for the report!

    regards, tom lane

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppgsql-hackers @
categoriespostgresql
postedJul 26, '10 at 10:26a
activeJul 28, '10 at 4:55a
posts3
users2
websitepostgresql.org...
irc#postgresql

2 users in discussion

Tom Lane: 2 posts Tao Ma: 1 post

People

Translate

site design / logo © 2022 Grokbase