FAQ
Hi all,

I've been working with Go for some time and I'm looking at ways to
checkpoint/backup the state of a running program at certain intervals. The
domain would be long-running scientific computations.

My current solution is a simple language level checkpointing mechanisms
that relies on bson, encoding/json and encoding/gob to essentially
serialise a state defined by the programmer (requiring fields of structs to
be exported etc).

What I'm not quite clear on, would be the best approach to do incremental
checkpointing in go. It's currently very easy and nice to just restore a
gob into the object provided, but my use-case only changes a small set of
the objects, so incremental checkpointing would reduce the disk writes
tremendously. Would a simply binary diff, overwrite etc. be the best
approach?

Also, classical checkpointing mechanisms rely on either OS level knowledge
of changed pages/words (by locking and unlocking pages before they can be
written to etc.). Is if there's anything in the Go runtime that I can
leverage this, or some OS project I can look at?

Thanks a bunch!

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Search Discussions

  • Sugu Sougoumarane at Nov 27, 2013 at 11:58 pm
    Have you looked at http://golang.org/pkg/expvar/?

    On Wednesday, November 27, 2013 11:50:33 AM UTC-8, Patrick-Ranjit D. Madsen
    wrote:
    Hi all,

    I've been working with Go for some time and I'm looking at ways to
    checkpoint/backup the state of a running program at certain intervals. The
    domain would be long-running scientific computations.

    My current solution is a simple language level checkpointing mechanisms
    that relies on bson, encoding/json and encoding/gob to essentially
    serialise a state defined by the programmer (requiring fields of structs to
    be exported etc).

    What I'm not quite clear on, would be the best approach to do incremental
    checkpointing in go. It's currently very easy and nice to just restore a
    gob into the object provided, but my use-case only changes a small set of
    the objects, so incremental checkpointing would reduce the disk writes
    tremendously. Would a simply binary diff, overwrite etc. be the best
    approach?

    Also, classical checkpointing mechanisms rely on either OS level knowledge
    of changed pages/words (by locking and unlocking pages before they can be
    written to etc.). Is if there's anything in the Go runtime that I can
    leverage this, or some OS project I can look at?

    Thanks a bunch!
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Elias Naur at Nov 28, 2013 at 12:00 am
    Hi,

    Perhaps the linux specific Check-Restore would fit well to your needs:

    http://criu.org/ <http://criu.org/Main_Page>

    I haven't used it myself, but it appears useful and it recently reached v
    1.0.

      - elias

    On Wednesday, November 27, 2013 8:50:33 PM UTC+1, Patrick-Ranjit D. Madsen
    wrote:
    Hi all,

    I've been working with Go for some time and I'm looking at ways to
    checkpoint/backup the state of a running program at certain intervals. The
    domain would be long-running scientific computations.

    My current solution is a simple language level checkpointing mechanisms
    that relies on bson, encoding/json and encoding/gob to essentially
    serialise a state defined by the programmer (requiring fields of structs to
    be exported etc).

    What I'm not quite clear on, would be the best approach to do incremental
    checkpointing in go. It's currently very easy and nice to just restore a
    gob into the object provided, but my use-case only changes a small set of
    the objects, so incremental checkpointing would reduce the disk writes
    tremendously. Would a simply binary diff, overwrite etc. be the best
    approach?

    Also, classical checkpointing mechanisms rely on either OS level knowledge
    of changed pages/words (by locking and unlocking pages before they can be
    written to etc.). Is if there's anything in the Go runtime that I can
    leverage this, or some OS project I can look at?

    Thanks a bunch!
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedNov 27, '13 at 7:50p
activeNov 28, '13 at 12:00a
posts3
users3
websitegolang.org

People

Translate

site design / logo © 2021 Grokbase