FAQ
Hi:

I should properly explain what I am trying to accomplish. Perhaps I'm
approaching this the wrong way. I need to do the equivalent of a
popen("/bin/zcat filename"). The file is likely to be fairly large, and I
will usually only need to examine the first 10-20 lines of the file and
then I can shut down the command. I'm essentially just examining the
comment block at the top of the file.

What I'm interested in catching is if zcat runs into an error due to a
missing file. When I use:

package main

import (
         "fmt"
         "log"
         "os/exec"
         "os"
         "bufio"
)

func main() {

     cmd := exec.Command("/bin/zcat", "foo.gz")

     stdout, err1 := cmd.StdoutPipe()

     if err1 = cmd.Start(); err1 != nil {
         log.Fatal(err1)
     }
...

If foo.gz doesn't exist, err1 is not populated with a bad exit status as I
would have expected. The code ultimately gets down to when it's starting to
try reading from the pipe and gets "reading file: EOF".

I can attempt to use StderrPipe() as well to get the details from the
failed call to zcat, but reading from that will block if nothing is writing
to it, so that's not right, either.

Is the only way to safely do this to test for the existence of "foo.gz"
before I try running this snippet?

I know it seems like I'm being pedantic about this, but I am just trying to
understand the failure cases, and I would like the error messages to be at
least well-formed so that an end-user can figure out what's wrong.

Thanks,

-Clint





--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

  • Nick Craig-Wood at Feb 11, 2015 at 1:40 pm

    On 11/02/15 08:41, Clint O wrote:
    I should properly explain what I am trying to accomplish. Perhaps I'm
    approaching this the wrong way. I need to do the equivalent of a
    popen("/bin/zcat filename"). The file is likely to be fairly large, and
    I will usually only need to examine the first 10-20 lines of the file
    and then I can shut down the command. I'm essentially just examining the
    comment block at the top of the file.

    What I'm interested in catching is if zcat runs into an error due to a
    missing file. When I use:

    package main

    import (
    "fmt"
    "log"
    "os/exec"
    "os"
    "bufio"
    )

    func main() {

    cmd := exec.Command("/bin/zcat", "foo.gz")

    stdout, err1 := cmd.StdoutPipe()

    if err1 = cmd.Start(); err1 != nil {
    log.Fatal(err1)
    }
    ...

    If foo.gz doesn't exist, err1 is not populated with a bad exit status as
    I would have expected. The code ultimately gets down to when it's
    starting to try reading from the pipe and gets "reading file: EOF".

    I can attempt to use StderrPipe() as well to get the details from the
    failed call to zcat, but reading from that will block if nothing is
    writing to it, so that's not right, either.
    You need to run cmd.Wait() to get the exit status of the cmd - that will
    give you errors returned by zcat if any.

    Not sure why you aren't using http://golang.org/pkg/compress/gzip/ for
    this though...


    --
    Nick Craig-Wood <nick@craig-wood.com> -- http://www.craig-wood.com/nick

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Clint O at Feb 11, 2015 at 6:11 pm

    On Wednesday, February 11, 2015 at 5:41:47 AM UTC-8, Nick Craig-Wood wrote:

    You need to run cmd.Wait() to get the exit status of the cmd - that will
    give you errors returned by zcat if any.

    Not sure why you aren't using http://golang.org/pkg/compress/gzip/ for
    this though...


    --
    Nick Craig-Wood <ni...@craig-wood.com <javascript:>> --
    http://www.craig-wood.com/nick

    Thanks. I wasn't aware of the existence of this package. But in this case I
    would consider this an educational exercise anyway. Of course, based on the
    constraints I listed originally, I cannot use cmd.Wait() because I never
    expect the command to complete because it's a large file, right? It's going
    to fill a buffer an then block on it's standard output.

    Thanks,

    -Clint

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Justin Israel at Feb 11, 2015 at 6:28 pm
    But Start() is only going to tell you if the command failed to start. In
    which case, it does start fine, but does not succeed when it processes the
    argument for the missing file.
    I would think you would want to read the pipe until EOF and then call
    Wait() to see if there is an error.
    On Thu, 12 Feb 2015 7:11 AM Clint O wrote:
    On Wednesday, February 11, 2015 at 5:41:47 AM UTC-8, Nick Craig-Wood wrote:


    You need to run cmd.Wait() to get the exit status of the cmd - that will
    give you errors returned by zcat if any.

    Not sure why you aren't using http://golang.org/pkg/compress/gzip/ for
    this though...


    --
    Nick Craig-Wood <ni...@craig-wood.com> -- http://www.craig-wood.com/nick


    Thanks. I wasn't aware of the existence of this package. But in this case
    I would consider this an educational exercise anyway. Of course, based on
    the constraints I listed originally, I cannot use cmd.Wait() because I
    never expect the command to complete because it's a large file, right? It's
    going to fill a buffer an then block on it's standard output.

    Thanks,

    -Clint

    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Tamás Gulácsi at Feb 11, 2015 at 9:23 pm
    Check out github.com/kmanley/gexec - it starts a goroutine which sends back what Wait returns, so it can timeout waiting for result with select.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Lars Seipel at Feb 12, 2015 at 12:40 pm

    On Wed, Feb 11, 2015 at 10:11:25AM -0800, Clint O wrote:
    constraints I listed originally, I cannot use cmd.Wait() because I never
    expect the command to complete because it's a large file, right? It's going
    to fill a buffer an then block on it's standard output.
    You read as much as you need and then *close* the pipe. This will cause
    the command to unblock writing to its stdout so you can wait for it.
    Meanwhile, you let stderr go to a buffer and examine it after the
    command exits with a non-zero status.

    As long as you don't also want to feed something to the command's stdin
    there are no extra goroutines required.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Kevin Malachowski at Feb 12, 2015 at 3:21 am
    Just like you realized, there are multiple things you need to do: read the
    output and check the exit status, each of which are blocking operations.
    Using goroutines to delegate these tasks simultaneously is an elegant way
    to handle it: http://play.golang.org/p/izD1rjVtZ_
    On Wednesday, February 11, 2015 at 3:41:25 AM UTC-5, Clint O wrote:

    Hi:

    I should properly explain what I am trying to accomplish. Perhaps I'm
    approaching this the wrong way. I need to do the equivalent of a
    popen("/bin/zcat filename"). The file is likely to be fairly large, and I
    will usually only need to examine the first 10-20 lines of the file and
    then I can shut down the command. I'm essentially just examining the
    comment block at the top of the file.

    What I'm interested in catching is if zcat runs into an error due to a
    missing file. When I use:

    package main

    import (
    "fmt"
    "log"
    "os/exec"
    "os"
    "bufio"
    )

    func main() {

    cmd := exec.Command("/bin/zcat", "foo.gz")

    stdout, err1 := cmd.StdoutPipe()

    if err1 = cmd.Start(); err1 != nil {
    log.Fatal(err1)
    }
    ...

    If foo.gz doesn't exist, err1 is not populated with a bad exit status as I
    would have expected. The code ultimately gets down to when it's starting to
    try reading from the pipe and gets "reading file: EOF".

    I can attempt to use StderrPipe() as well to get the details from the
    failed call to zcat, but reading from that will block if nothing is writing
    to it, so that's not right, either.

    Is the only way to safely do this to test for the existence of "foo.gz"
    before I try running this snippet?

    I know it seems like I'm being pedantic about this, but I am just trying
    to understand the failure cases, and I would like the error messages to be
    at least well-formed so that an end-user can figure out what's wrong.

    Thanks,

    -Clint




    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Kevin Malachowski at Feb 12, 2015 at 3:28 am
    Whoops, I take that back. I forgot that you aren't allowed to use Run with
    StdoutPipe (or StderrPipe).

    But one thing: you should *always* check the return value from the process
    anyway, even if the pipe from StdoutPipe lets you read something from it.
    What if your process starts and runs find for a few seconds but then fails
    due to a disk being randomly unmounted or some other random IO error? If
    you just trust that the end of the pipe is the end of data you might miss
    something at the end.

    Try something like this:

    Start the process
    Read from stdout until you get EOF. This will always happen when the
    process exits (I believe)
    Check the result from Wait: if nil, everything was ok. Otherwise the
    process exited for some bad reason and either some or no data was processed
    from it.

    You might want to do something different if some versus no data was read,
    but in general I would consider both the same sort of error to be reported.

    On Wednesday, February 11, 2015 at 10:21:09 PM UTC-5, Kevin Malachowski
    wrote:
    Just like you realized, there are multiple things you need to do: read the
    output and check the exit status, each of which are blocking operations.
    Using goroutines to delegate these tasks simultaneously is an elegant way
    to handle it: http://play.golang.org/p/izD1rjVtZ_
    On Wednesday, February 11, 2015 at 3:41:25 AM UTC-5, Clint O wrote:

    Hi:

    I should properly explain what I am trying to accomplish. Perhaps I'm
    approaching this the wrong way. I need to do the equivalent of a
    popen("/bin/zcat filename"). The file is likely to be fairly large, and I
    will usually only need to examine the first 10-20 lines of the file and
    then I can shut down the command. I'm essentially just examining the
    comment block at the top of the file.

    What I'm interested in catching is if zcat runs into an error due to a
    missing file. When I use:

    package main

    import (
    "fmt"
    "log"
    "os/exec"
    "os"
    "bufio"
    )

    func main() {

    cmd := exec.Command("/bin/zcat", "foo.gz")

    stdout, err1 := cmd.StdoutPipe()

    if err1 = cmd.Start(); err1 != nil {
    log.Fatal(err1)
    }
    ...

    If foo.gz doesn't exist, err1 is not populated with a bad exit status as
    I would have expected. The code ultimately gets down to when it's starting
    to try reading from the pipe and gets "reading file: EOF".

    I can attempt to use StderrPipe() as well to get the details from the
    failed call to zcat, but reading from that will block if nothing is writing
    to it, so that's not right, either.

    Is the only way to safely do this to test for the existence of "foo.gz"
    before I try running this snippet?

    I know it seems like I'm being pedantic about this, but I am just trying
    to understand the failure cases, and I would like the error messages to be
    at least well-formed so that an end-user can figure out what's wrong.

    Thanks,

    -Clint




    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedFeb 11, '15 at 8:41a
activeFeb 12, '15 at 12:40p
posts8
users6
websitegolang.org

People

Translate

site design / logo © 2022 Grokbase