FAQ
Hey all! Thought I'd share the Go project I've been working on over at
CoreOS for a while:

https://coreos.com/blog/torus-distributed-storage-by-coreos.html

The short version is, use etcd as the consistent metadata store, and build
the basics around a gRPC storage protocol, a rebalancer and garbage
collector for an append-only data store. Mount these distributed "files"
(analogous to loopback) as block devices, but plan also for object storage.
Would love feedback and contributors!

--Barak

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

  • Nate Finch at Jun 2, 2016 at 2:23 pm
    Not to be a debbie downer, but what is up with these panics?

    https://github.com/coreos/torus/blob/master/inode.go#L47

    func (b *INodeStore) WriteINode(ctx context.Context, i INodeRef, inode
    *models.INode) error {

    if i.INode == 0 {

    panic("Writing zero inode")

    }


    The function returns an error, why on earth would you panic here?

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Chris dollin at Jun 2, 2016 at 2:35 pm

    On 2 June 2016 at 15:23, Nate Finch wrote:
    Not to be a debbie downer, but what is up with these panics?

    https://github.com/coreos/torus/blob/master/inode.go#L47

    func (b *INodeStore) WriteINode(ctx context.Context, i INodeRef, inode
    *models.INode) error {

    if i.INode == 0 {

    panic("Writing zero inode")

    }


    The function returns an error, why on earth would you panic here?
    Probably because the program is broken if its ever asked to
    write a zero inode?

    Precondition violation is a good reason to panic.

    Chris

    --
    Chris "allusive" Dollin

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Nate Finch at Jun 2, 2016 at 3:43 pm

    On Thursday, June 2, 2016 at 10:35:59 AM UTC-4, ehedgehog wrote:
    Probably because the program is broken if its ever asked to
    write a zero inode?

    Precondition violation is a good reason to panic.
    There's no precondition here. It's an exported method. If this were an
    internal method that has some guarantee from the rest of the local code
    that the inode can't be zero, then maybe... but even then, I'd prefer to
    just return an error. Panic = whole application goes down, period. Error
    = some possibility to react, retry, fail gracefully, etc.

    From effective Go: "library functions should avoid panic". The convention
    is, libraries should never let panics escape the bounds of the package,
    unless requested by the caller (e.g. MustParse).

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Manlio Perillo at Jun 2, 2016 at 6:37 pm
    Il giorno giovedì 2 giugno 2016 17:43:27 UTC+2, Nate Finch ha scritto:
    On Thursday, June 2, 2016 at 10:35:59 AM UTC-4, ehedgehog wrote:


    Probably because the program is broken if its ever asked to
    write a zero inode?

    Precondition violation is a good reason to panic.
    There's no precondition here. It's an exported method. If this were an
    internal method that has some guarantee from the rest of the local code
    that the inode can't be zero, then maybe... but even then, I'd prefer to
    just return an error. Panic = whole application goes down, period. Error
    = some possibility to react, retry, fail gracefully, etc.

    From effective Go: "library functions should avoid panic". The convention
    is, libraries should never let panics escape the bounds of the package,
    unless requested by the caller (e.g. MustParse).
    However when a function is called with an invalid parameter, panic is a
    valid choice.
    There are a lot of examples in the standard library. The only difference
    with the code in the torus package, is that the functions do not also
    return an error value, AFAIK.


    Manlio

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Nate Finch at Jun 2, 2016 at 7:45 pm
    As far as I can tell, very few exported functions in the stdlib will panic
    if you pass them invalid data. base64.NewEncoding will... but that
    probably *should* be returning an error instead of panicking if you do the
    wrong thing.. maybe the reasoning was that it'll almost always be called
    during application startup with a constant string, and so in practice
    should never panic in production. A few things that take an int but expect
    a non-negative integer will panic if the integer is negative....

    There's really very few places in the stdlib where panics occur in exported
    functions as parameter validation. If you have specific examples beyond
    what I've mentioned above, I'd be interested to see them.
    On Thursday, June 2, 2016 at 2:37:16 PM UTC-4, Manlio Perillo wrote:


    However when a function is called with an invalid parameter, panic is a
    valid choice.
    There are a lot of examples in the standard library. The only difference
    with the code in the torus package, is that the functions do not also
    return an error value, AFAIK.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Tyler Compton at Jun 2, 2016 at 8:25 pm
    It looks like there's a lot of disagreement about whether panics should be
    used for precondition violation or not. Here's another discussion I've
    found which is also inconclusive[1].

    Personally, I'm not sure what the best choice is here. I've panicked on
    precondition violations in the past, but I'm not 100% convinced on that
    method. One thing to consider is that in languages like Java (where
    forcefully aborting on a precondition violation is common),
    RuntimeExceptions don't entirely kill Tomcat servers, but panics can
    entirely kill Go servers (IIRC), so the cost of a panic is higher in Go.
    That may not be entirely correct as I'm nowhere near an expert in Java
    Servlets or Tomcat.

    [1] https://groups.google.com/forum/embed/#!topic/golang-nuts/P-_HtvxhVT4
    On Thursday, June 2, 2016 at 12:45:49 PM UTC-7, Nate Finch wrote:

    As far as I can tell, very few exported functions in the stdlib will panic
    if you pass them invalid data. base64.NewEncoding will... but that
    probably *should* be returning an error instead of panicking if you do the
    wrong thing.. maybe the reasoning was that it'll almost always be called
    during application startup with a constant string, and so in practice
    should never panic in production. A few things that take an int but expect
    a non-negative integer will panic if the integer is negative....

    There's really very few places in the stdlib where panics occur in
    exported functions as parameter validation. If you have specific examples
    beyond what I've mentioned above, I'd be interested to see them.
    On Thursday, June 2, 2016 at 2:37:16 PM UTC-4, Manlio Perillo wrote:


    However when a function is called with an invalid parameter, panic is a
    valid choice.
    There are a lot of examples in the standard library. The only difference
    with the code in the torus package, is that the functions do not also
    return an error value, AFAIK.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Axel Wagner at Jun 2, 2016 at 8:30 pm
    panic's won't kill a go HTTP server, as net/http recover()s. But I wish it
    didn't or at least I could turn it off. I want my failure model to be
    fail-stop, not byzantine, because byzantine failures aren't any fun.
    On Thu, Jun 2, 2016 at 10:25 PM, Tyler Compton wrote:

    It looks like there's a lot of disagreement about whether panics should be
    used for precondition violation or not. Here's another discussion I've
    found which is also inconclusive[1].

    Personally, I'm not sure what the best choice is here. I've panicked on
    precondition violations in the past, but I'm not 100% convinced on that
    method. One thing to consider is that in languages like Java (where
    forcefully aborting on a precondition violation is common),
    RuntimeExceptions don't entirely kill Tomcat servers, but panics can
    entirely kill Go servers (IIRC), so the cost of a panic is higher in Go.
    That may not be entirely correct as I'm nowhere near an expert in Java
    Servlets or Tomcat.

    [1] https://groups.google.com/forum/embed/#!topic/golang-nuts/P-_HtvxhVT4
    On Thursday, June 2, 2016 at 12:45:49 PM UTC-7, Nate Finch wrote:

    As far as I can tell, very few exported functions in the stdlib will
    panic if you pass them invalid data. base64.NewEncoding will... but that
    probably *should* be returning an error instead of panicking if you do the
    wrong thing.. maybe the reasoning was that it'll almost always be called
    during application startup with a constant string, and so in practice
    should never panic in production. A few things that take an int but expect
    a non-negative integer will panic if the integer is negative....

    There's really very few places in the stdlib where panics occur in
    exported functions as parameter validation. If you have specific examples
    beyond what I've mentioned above, I'd be interested to see them.
    On Thursday, June 2, 2016 at 2:37:16 PM UTC-4, Manlio Perillo wrote:


    However when a function is called with an invalid parameter, panic is a
    valid choice.
    There are a lot of examples in the standard library. The only
    difference with the code in the torus package, is that the functions do not
    also return an error value, AFAIK.

    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Tyler Compton at Jun 2, 2016 at 9:23 pm
    Oh, my mistake. I thought that behavior was something special with app
    engine, not net/http itself. I should probably research a little more
    before posting in the future :D

    I suppose it does make sense that net/http would recover, though. I would
    suspect that a DOS attacker or a very persistent user would be able to take
    down every instance of service pretty quickly and repeatedly if that
    weren't the case, assuming they found that a certain action caused a panic.
    On Thursday, June 2, 2016 at 1:31:03 PM UTC-7, Axel Wagner wrote:

    panic's won't kill a go HTTP server, as net/http recover()s. But I wish it
    didn't or at least I could turn it off. I want my failure model to be
    fail-stop, not byzantine, because byzantine failures aren't any fun.

    On Thu, Jun 2, 2016 at 10:25 PM, Tyler Compton <xav...@gmail.com
    <javascript:>> wrote:
    It looks like there's a lot of disagreement about whether panics should
    be used for precondition violation or not. Here's another discussion I've
    found which is also inconclusive[1].

    Personally, I'm not sure what the best choice is here. I've panicked on
    precondition violations in the past, but I'm not 100% convinced on that
    method. One thing to consider is that in languages like Java (where
    forcefully aborting on a precondition violation is common),
    RuntimeExceptions don't entirely kill Tomcat servers, but panics can
    entirely kill Go servers (IIRC), so the cost of a panic is higher in Go.
    That may not be entirely correct as I'm nowhere near an expert in Java
    Servlets or Tomcat.

    [1] https://groups.google.com/forum/embed/#!topic/golang-nuts/P-_HtvxhVT4
    On Thursday, June 2, 2016 at 12:45:49 PM UTC-7, Nate Finch wrote:

    As far as I can tell, very few exported functions in the stdlib will
    panic if you pass them invalid data. base64.NewEncoding will... but that
    probably *should* be returning an error instead of panicking if you do the
    wrong thing.. maybe the reasoning was that it'll almost always be called
    during application startup with a constant string, and so in practice
    should never panic in production. A few things that take an int but expect
    a non-negative integer will panic if the integer is negative....

    There's really very few places in the stdlib where panics occur in
    exported functions as parameter validation. If you have specific examples
    beyond what I've mentioned above, I'd be interested to see them.
    On Thursday, June 2, 2016 at 2:37:16 PM UTC-4, Manlio Perillo wrote:


    However when a function is called with an invalid parameter, panic is a
    valid choice.
    There are a lot of examples in the standard library. The only
    difference with the code in the torus package, is that the functions do not
    also return an error value, AFAIK.

    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts...@googlegroups.com <javascript:>.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Dave Cheney at Jun 2, 2016 at 9:34 pm
    I think net/http's recover behaviour is a mistake and would like to think
    if this package was written today it wouldn't be added at all.
    On Friday, 3 June 2016 07:23:52 UTC+10, Tyler Compton wrote:

    Oh, my mistake. I thought that behavior was something special with app
    engine, not net/http itself. I should probably research a little more
    before posting in the future :D

    I suppose it does make sense that net/http would recover, though. I would
    suspect that a DOS attacker or a very persistent user would be able to take
    down every instance of service pretty quickly and repeatedly if that
    weren't the case, assuming they found that a certain action caused a panic.
    On Thursday, June 2, 2016 at 1:31:03 PM UTC-7, Axel Wagner wrote:

    panic's won't kill a go HTTP server, as net/http recover()s. But I wish
    it didn't or at least I could turn it off. I want my failure model to be
    fail-stop, not byzantine, because byzantine failures aren't any fun.
    On Thu, Jun 2, 2016 at 10:25 PM, Tyler Compton wrote:

    It looks like there's a lot of disagreement about whether panics should
    be used for precondition violation or not. Here's another discussion I've
    found which is also inconclusive[1].

    Personally, I'm not sure what the best choice is here. I've panicked on
    precondition violations in the past, but I'm not 100% convinced on that
    method. One thing to consider is that in languages like Java (where
    forcefully aborting on a precondition violation is common),
    RuntimeExceptions don't entirely kill Tomcat servers, but panics can
    entirely kill Go servers (IIRC), so the cost of a panic is higher in Go.
    That may not be entirely correct as I'm nowhere near an expert in Java
    Servlets or Tomcat.

    [1]
    https://groups.google.com/forum/embed/#!topic/golang-nuts/P-_HtvxhVT4
    On Thursday, June 2, 2016 at 12:45:49 PM UTC-7, Nate Finch wrote:

    As far as I can tell, very few exported functions in the stdlib will
    panic if you pass them invalid data. base64.NewEncoding will... but that
    probably *should* be returning an error instead of panicking if you do the
    wrong thing.. maybe the reasoning was that it'll almost always be called
    during application startup with a constant string, and so in practice
    should never panic in production. A few things that take an int but expect
    a non-negative integer will panic if the integer is negative....

    There's really very few places in the stdlib where panics occur in
    exported functions as parameter validation. If you have specific examples
    beyond what I've mentioned above, I'd be interested to see them.
    On Thursday, June 2, 2016 at 2:37:16 PM UTC-4, Manlio Perillo wrote:


    However when a function is called with an invalid parameter, panic is
    a valid choice.
    There are a lot of examples in the standard library. The only
    difference with the code in the torus package, is that the functions do not
    also return an error value, AFAIK.

    --
    You received this message because you are subscribed to the Google
    Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to golang-nuts...@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Nigel Tao at Jun 3, 2016 at 12:30 am

    On Fri, Jun 3, 2016 at 7:34 AM, Dave Cheney wrote:
    I think net/http's recover behaviour is a mistake and would like to think if
    this package was written today it wouldn't be added at all.
    So says https://github.com/golang/go/issues/5465#issuecomment-66079372

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • James Aguilar at Jun 3, 2016 at 4:09 am

    On Thursday, June 2, 2016 at 5:30:32 PM UTC-7, Nigel Tao wrote:
    On Fri, Jun 3, 2016 at 7:34 AM, Dave Cheney <da...@cheney.net
    <javascript:>> wrote:
    I think net/http's recover behaviour is a mistake and would like to think if
    this package was written today it wouldn't be added at all.
    So says https://github.com/golang/go/issues/5465#issuecomment-66079372
    Also don't forget, the http server's recover only works in the handler
    goroutine. Any helper routines *it* spawns will crash the program if they
    panic.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Manlio Perillo at Jun 4, 2016 at 9:35 am
    Il giorno venerdì 3 giugno 2016 02:30:32 UTC+2, Nigel Tao ha scritto:
    On Fri, Jun 3, 2016 at 7:34 AM, Dave Cheney <da...@cheney.net
    <javascript:>> wrote:
    I think net/http's recover behaviour is a mistake and would like to think if
    this package was written today it wouldn't be added at all.
    So says https://github.com/golang/go/issues/5465#issuecomment-66079372
    Any hopes of seeing the proposed changes in something like
    golang.org/2/net/http ?


    Manlio

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Nigel Tao at Jun 7, 2016 at 12:13 pm

    On Sat, Jun 4, 2016 at 7:35 PM, Manlio Perillo wrote:
    Il giorno venerdì 3 giugno 2016 02:30:32 UTC+2, Nigel Tao ha scritto:
    Any hopes of seeing the proposed changes in something like
    golang.org/2/net/http ?
    I suppose that that's possible, but the whole idea of a golang.org/2
    is a separate, and much larger, discussion.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Manlio Perillo at Jun 4, 2016 at 8:06 am
    Il giorno giovedì 2 giugno 2016 23:34:14 UTC+2, Dave Cheney ha scritto:
    I think net/http's recover behaviour is a mistake and would like to think
    if this package was written today it wouldn't be added at all.
    Isn't it possible to change the default behaviour with some additional API?


    Thanks Manlio

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Manlio Perillo at Jun 3, 2016 at 9:55 am
    Il giorno giovedì 2 giugno 2016 21:45:49 UTC+2, Nate Finch ha scritto:
    As far as I can tell, very few exported functions in the stdlib will panic
    if you pass them invalid data. base64.NewEncoding will...
    A search with grep reveals that *many* functions in the stdlib panic with
    invalid data.
    [...]

    Manlio

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Matt Harden at Jun 4, 2016 at 3:14 am
    So the way to opt-out of the HTTP server's recovering from panics is to
    spawn another goroutine and handle the request there? Yuck!
    On Fri, Jun 3, 2016 at 2:55 AM Manlio Perillo wrote:

    Il giorno giovedì 2 giugno 2016 21:45:49 UTC+2, Nate Finch ha scritto:
    As far as I can tell, very few exported functions in the stdlib will
    panic if you pass them invalid data. base64.NewEncoding will...
    A search with grep reveals that *many* functions in the stdlib panic with
    invalid data.
    [...]

    Manlio

    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • As Utf8 at Jun 4, 2016 at 6:39 am
    What constitutes a "precondition violation"? I think the answer depends on the developer. I would prefer to call that event a fault. A panic is not fault-tolerant, it steps on the stack, executing defers in the process. I wouldn't expect to recover() from a function returning an error unless it was stated in the package documentation, and even in that case it would feel unwholesome since the flow control after the recovery isn't as obvious as handing an error.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedJun 1, '16 at 7:58p
activeJun 7, '16 at 12:13p
posts18
users11
websitegolang.org

People

Translate

site design / logo © 2021 Grokbase