FAQ
HI all,

I am trying to learn go so I thought I would write a password cracker to
test out go's concurrency. My design for this was to have a loop that
creates all the possible combination of password and for each one fire off
a new goroutine that hashes the password and compared it against the hash I
already had. A simple example for the characters A - Z and 2 bytes long
would be http://play.golang.org/p/lYVKF8GG3l

This works perfectly well for up to 4 characters but when I get into 5
characters that includes A-Z, a-z,0-9 and special characters this is trying
to create over a billion goroutines which ends up using all the ram on the
machine and the program terminates with "signal 9" and I get the below in
/var/log/messages

Sep 28 18:03:29 localhost kernel: Out of memory: Kill process 2050 (a.out)
score 947 or sacrifice child
Sep 28 18:03:29 localhost kernel: Killed process 2050, UID 501, (a.out)
total-vm:4725900kB, anon-rss:3685824kB, file-rss:20kB

I would like to just let the Waitgroup work out how many threads it can
handle at once without getting killed but I doubt this is possible.

How would I go about limiting this?
What would be a reasonable number to limit it to ?
Would I be able to make this adjustable by amount of Ram or is this a
stupid thing to try to do?

Jeff

--

Search Discussions

  • Rémy Oudompheng at Sep 28, 2012 at 7:26 pm

    On 2012/9/28 wrote:
    HI all,

    I am trying to learn go so I thought I would write a password cracker to
    test out go's concurrency. My design for this was to have a loop that
    creates all the possible combination of password and for each one fire off a
    new goroutine that hashes the password and compared it against the hash I
    already had. A simple example for the characters A - Z and 2 bytes long
    would be http://play.golang.org/p/lYVKF8GG3l

    This works perfectly well for up to 4 characters but when I get into 5
    characters that includes A-Z, a-z,0-9 and special characters this is trying
    to create over a billion goroutines which ends up using all the ram on the
    machine and the program terminates with "signal 9" and I get the below in
    /var/log/messages

    Sep 28 18:03:29 localhost kernel: Out of memory: Kill process 2050 (a.out)
    score 947 or sacrifice child
    Sep 28 18:03:29 localhost kernel: Killed process 2050, UID 501, (a.out)
    total-vm:4725900kB, anon-rss:3685824kB, file-rss:20kB

    I would like to just let the Waitgroup work out how many threads it can
    handle at once without getting killed but I doubt this is possible.
    The WaitGroup cannot do anything. It is too late if you created the goroutines.
    How would I go about limiting this?
    Creating a goroutine (using the "go" word) uses a bit of memory for
    the goroutines. You usually cannot have billions of that. Millions is
    already difficult. You can limit the number of goroutines by not
    creating them. The threadpool "pattern" is a possibility: create a
    given number of goroutines and feed jobs to them using a channel.
    What would be a reasonable number to limit it to ?
    Would I be able to make this adjustable by amount of Ram or is this a stupid
    thing to try to do?
    I usually find that thousands is usually ok. But either you don't
    limit at all but know that the number will be reasonable. If you want
    to fix some limit, I would say a few dozens/hundreds will be enough
    for most purposes.

    Rémy.

    --
  • Andrew Csillag at Sep 28, 2012 at 7:37 pm
    One way to do it is to just have a fixed (possibly large) pool of
    goroutines and have them reading passwords from a channel that gets filled
    by the password generator loop.

    Something like this:
    http://play.golang.org/p/Dr6xHNJPu6

    There may be a better idiom to send a "kill" signal to a worker goroutine,
    but I'm still fairly new at go.
    On Fri, Sep 28, 2012 at 2:15 PM, wrote:

    HI all,

    I am trying to learn go so I thought I would write a password cracker to
    test out go's concurrency. My design for this was to have a loop that
    creates all the possible combination of password and for each one fire off
    a new goroutine that hashes the password and compared it against the hash I
    already had. A simple example for the characters A - Z and 2 bytes long
    would be http://play.golang.org/p/lYVKF8GG3l

    This works perfectly well for up to 4 characters but when I get into 5
    characters that includes A-Z, a-z,0-9 and special characters this is trying
    to create over a billion goroutines which ends up using all the ram on the
    machine and the program terminates with "signal 9" and I get the below in
    /var/log/messages

    Sep 28 18:03:29 localhost kernel: Out of memory: Kill process 2050 (a.out)
    score 947 or sacrifice child
    Sep 28 18:03:29 localhost kernel: Killed process 2050, UID 501, (a.out)
    total-vm:4725900kB, anon-rss:3685824kB, file-rss:20kB

    I would like to just let the Waitgroup work out how many threads it can
    handle at once without getting killed but I doubt this is possible.

    How would I go about limiting this?
    What would be a reasonable number to limit it to ?
    Would I be able to make this adjustable by amount of Ram or is this a
    stupid thing to try to do?

    Jeff

    --

    --
  • Rémy Oudompheng at Sep 28, 2012 at 7:33 pm

    On 2012/9/28 Andrew Csillag wrote:
    One way to do it is to just have a fixed (possibly large) pool of goroutines
    and have them reading passwords from a channel that gets filled by the
    password generator loop.

    Something like this:
    http://play.golang.org/p/Dr6xHNJPu6

    There may be a better idiom to send a "kill" signal to a worker goroutine,
    but I'm still fairly new at go.
    A more usual idiom is to sue a range loop on the channel and close the
    channel when all values are sent.

    Rémy.

    --

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedSep 28, '12 at 7:26p
activeSep 28, '12 at 7:37p
posts4
users3
websitegolang.org

People

Translate

site design / logo © 2021 Grokbase