I need suggestion about how region based memory management is done in
postgres. I know the concept of region based memory management and also
know about the functions like memorycontextswitch().

But I am not understanding how Postgres uses hierarchical, region-based
memory management. That is I am not getting the inner idea or meaning of
the code.

Currently, my project topic is "Parallelizing the spatial join" using
POSIX threads, so I have to understand the inner details and meanings
of the code. I am using "ddd" to step through its code, from there I am
getting the flow of the code but not understanding the semantics of its
data-structures and its region based memory management.

Kindly, if anybody can give some Ideas or Suggestions.


Thank You,
Subham Roy,
CSE,
IIT Bombay.

Search Discussions

  • Tom Lane at Aug 5, 2010 at 4:43 pm

    subham@cse.iitb.ac.in writes:
    I need suggestion about how region based memory management is done in
    postgres.
    Have you read src/backend/utils/mmgr/README ? It's old but still
    reasonably accurate.

    regards, tom lane
  • Subham at Aug 30, 2010 at 3:12 pm
    Can anyone explain this ?

    My question is -
    What is lnext:; , lreplace:; in postgres code ?
    I found lnext:; in 1501 and lreplace:; in 2065 in execMain.c file.


    --

    Thank You,
    Subham Roy.
    CSE,
    IIT Bombay.
  • Robert Haas at Sep 2, 2010 at 8:25 pm

    On Mon, Aug 30, 2010 at 11:12 AM, wrote:
    Can anyone explain this ?

    My question is -
    What is lnext:; , lreplace:; in postgres code ?
    I found lnext:; in 1501 and lreplace:; in 2065 in execMain.c file.
    It's a label.

    http://www.lysator.liu.se/c/bwk-tutor.html#goto

    --
    Robert Haas
    EnterpriseDB: http://www.enterprisedb.com
    The Enterprise Postgres Company
  • Subham at Sep 22, 2010 at 8:14 am
    How can I increase the stack depth limit ? Is it only by modifying the
    postgres.conf file, but there I cannot increase the stack depth beyond 4
    MB.

    Actually, my problem is that, when I set the stack base address of the
    child thread using the POSIX library function "pthread_setstackaddr()", I
    am unable to access the memory contents of its parent. The data-structures
    in the parent are either getting destroyed or unaccessible when moving to
    the context of the child thread.
    But when I create the child thread without setting its stack base address
    then it is able to access the parent's memory contents. But if the child
    thread's code size is large then it is getting stack fault and displaying
    the message "MAX stack depth limit reached ...".
    Can anyone give any reason why is that so ?
    OR
    If I cannot set the child thread's stack base address. Then How can I
    increase the stack depth limit to a large value ?

    --

    Thank You,
    Subham Roy.
  • Heikki Linnakangas at Sep 22, 2010 at 8:22 am

    On 22/09/10 11:14, subham@cse.iitb.ac.in wrote:
    How can I increase the stack depth limit ? Is it only by modifying the
    postgres.conf file, but there I cannot increase the stack depth beyond 4
    MB.

    Actually, my problem is that, when I set the stack base address of the
    child thread using the POSIX library function "pthread_setstackaddr()", I
    am unable to access the memory contents of its parent. The data-structures
    in the parent are either getting destroyed or unaccessible when moving to
    the context of the child thread.
    It is not a good idea to use threads in server code. PostgreSQL server
    code is not thread-safe, things will break.

    Assuming that you're not actually doing that but using threads in the
    client instead, max_stack_depth should have no effect on you as it only
    affects the server.

    But you really should find another way to communicate between threads.
    Stacks should be treated as thread-private. Use malloc() or something
    and global variables.

    --
    Heikki Linnakangas
    EnterpriseDB http://www.enterprisedb.com
  • Subham at Sep 22, 2010 at 9:04 am
    Actually, I used palloc() to set the stack base address.
    And I am trying to create only a single thread, then also it is causing
    problem.
    Actually, I created all the data-structures using palloc(), then I am
    passing these to the child thread. Even if I make these variables global
    then also it is not working.
    But if I don't set its stack address then it is working. But I need to do
    that because when my thread body is big then it is causing stack fault.
    So if I cannot set its stack address then Can I increase the stack depth
    limit to a large value ?

    --

    Thank You,
    Subham Roy.
    On 22/09/10 11:14, subham@cse.iitb.ac.in wrote:

    How can I increase the stack depth limit ? Is it only by modifying the
    postgres.conf file, but there I cannot increase the stack depth beyond 4
    MB.

    Actually, my problem is that, when I set the stack base address of the
    child thread using the POSIX library function "pthread_setstackaddr()",
    I
    am unable to access the memory contents of its parent. The
    data-structures
    in the parent are either getting destroyed or unaccessible when moving
    to
    the context of the child thread.
    It is not a good idea to use threads in server code. PostgreSQL server
    code is not thread-safe, things will break.

    Assuming that you're not actually doing that but using threads in the
    client instead, max_stack_depth should have no effect on you as it only
    affects the server.

    But you really should find another way to communicate between threads.
    Stacks should be treated as thread-private. Use malloc() or something
    and global variables.

    --
    Heikki Linnakangas
    EnterpriseDB http://www.enterprisedb.com
  • Heikki Linnakangas at Sep 22, 2010 at 9:10 am

    On 22/09/10 12:03, subham@cse.iitb.ac.in wrote:
    Actually, I used palloc() to set the stack base address.
    And I am trying to create only a single thread, then also it is causing
    problem.
    Actually, I created all the data-structures using palloc(), then I am
    passing these to the child thread. Even if I make these variables global
    then also it is not working.
    But if I don't set its stack address then it is working. But I need to do
    that because when my thread body is big then it is causing stack fault.
    So if I cannot set its stack address then Can I increase the stack depth
    limit to a large value ?
    It's not clear what you're trying to do, but it's just not going to
    work. The backend code is not thread-safe, so you can't safely create
    any threads in server code. Not even a single one. And even if you
    could, you should not mess with stack base addresses.

    --
    Heikki Linnakangas
    EnterpriseDB http://www.enterprisedb.com
  • Subham at Sep 22, 2010 at 9:30 am
    So, Can I increase the stack depth limit to a large value ?

    On 22/09/10 12:03, subham@cse.iitb.ac.in wrote:
    Actually, I used palloc() to set the stack base address.
    And I am trying to create only a single thread, then also it is causing
    problem.
    Actually, I created all the data-structures using palloc(), then I am
    passing these to the child thread. Even if I make these variables global
    then also it is not working.
    But if I don't set its stack address then it is working. But I need to
    do
    that because when my thread body is big then it is causing stack fault.
    So if I cannot set its stack address then Can I increase the stack depth
    limit to a large value ?
    It's not clear what you're trying to do, but it's just not going to
    work. The backend code is not thread-safe, so you can't safely create
    any threads in server code. Not even a single one. And even if you
    could, you should not mess with stack base addresses.

    --
    Heikki Linnakangas
    EnterpriseDB http://www.enterprisedb.com

    --

    Thank You,
    Subham Roy.
  • Andrew Dunstan at Sep 23, 2010 at 12:40 pm

    On 09/22/2010 05:03 AM, subham@cse.iitb.ac.in wrote:
    Actually, I used palloc() to set the stack base address.
    And I am trying to create only a single thread, then also it is causing
    problem.
    Did you not understand when people told you this wasn't going to work?
    Don't create any threads.

    cheers

    andrew
  • Tom Lane at Sep 22, 2010 at 2:34 pm

    subham@cse.iitb.ac.in writes:
    How can I increase the stack depth limit ? Is it only by modifying the
    postgres.conf file, but there I cannot increase the stack depth beyond 4
    MB.
    Actually, my problem is that, when I set the stack base address of the
    child thread using the POSIX library function "pthread_setstackaddr()", I
    am unable to access the memory contents of its parent.
    You've got worse problems than the stack depth limit. Creating multiple
    threads inside a backend process is entirely unsupported and just about
    guaranteed to result in massive breakage. Don't do it.

    regards, tom lane

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppgsql-hackers @
categoriespostgresql
postedAug 4, '10 at 6:35p
activeSep 23, '10 at 12:40p
posts11
users5
websitepostgresql.org...
irc#postgresql

People

Translate

site design / logo © 2022 Grokbase