[Perl-beginners] Most dependable way to run system commands

Well, that's pragmatic. :-) But it's generally the best way: Find the
answer first, improve upon it second.
You don't mean eval. You think you mean exec, but you don't mean that
either, probably.
I think you're looking for the program's exit status. Traditionally on
Unix and many similar systems, the exit status is an integer, with 0
meaning "normal exit" and anything else meaning that something went
wrong. That's the value that's used by programs that run other
programs (such as the 'make' system utility) and need to know when a
step has failed. In Perl, you can access the exit status with the $?
variable after running a command via system or backticks. Beware: If
your command is run by the shell, you'll get the shell's exit status,
which may not be the exit status you were looking for.

You can also consider capturing the STDERR output of the external
program. Since the exit status doesn't say much, most programs cough
out some last words when something goes wrong. In fact, if the program
outputs anything to STDERR, even if it exited with "success", the
error text may be important news you'd want to treat as a failure. One
common way to capture STDERR when running an external program is with
IPC::Open3.

I must admit to a philosophical problem with the idea of a program
that sends email when something goes wrong. How, I ask myself, does
such a program report that it cannot send mail? That 100% is looking
more and more elusive....

Hope this helps!

--Tom Phoenix
Stonehenge Perl Training

Generally, it's always my approach, but I'm sure as any other person
here, we've all had to deal with instances where this is but a lucky
circumstance. IANWNAPP (I am no where near a professional programmer).
I was in a hurry when I wrote that, so just like many of the statistics
we hear from politicians or some organizations, it came quickly out of
the wrong spot.
Being honest, I don't know what I meant. I truthfully didn't have time
to read through the docs for all the comparisons (although I looked at
eval in particular: perldoc -f eval which didn't seem proper for my
specific execution issue (it dealt more with executing Perl code)).
Wow, thank you for such a thoughtful and detailed explanation! I don't
think I've ever run across the $? before (forgive me if it was noted in
the ref, obj, mod book).

Personally, I cringe whenever I see `$command` or system "$command"
inside of anything I have to work on (albeit I have several scripts I've
done this in myself, and still use). Everyone has their own way of
coding, and someone's code always looks worse to someone else. After
all, we are all only learning, no matter what we know...right?
You are speaking of Unix (under shell) STDERR here, correct?

Your understanding and diligence in providing useful, specific and very
detailed responses amazes me. You get paid for this, right ;)
The email thing was an example. I won't get into the drawbacks on
relying on SMTP as an error messaging protocol, as it's not documented
in any of the RFC's I've read ;)

100%, I'd rather not state where I got that figure. Given that I run a
network and do much of the automation programming, I'd like to restate
that number to perhaps something that any network op could agree upon
regarding reliability of perfection...

Thanks Tom,

I think I'll check manually the next couple days for the success of my
system routines, then early next week take the time to get them Perl
lexical-ized.

Steve

>>
[snip]
[snip]

Tom was probably thinking of Unix, but same principles apply to other
platforms.

It's also worth noting that whatever application you're using to update
the database may well output something useful to STDOUT and it may be a
simple strategy to examine this output to determine the status of an
operation. If you assign the result of a backtick string to a scalar
variable

my $status = `command`;

then the entire output of the command will be stored as a single string.
If you assign it to an array

my @status = `command`;

then the output will be stored as one line per array element.

HTH,

Rob

[snipped due to excessive content]
...
The problem here is that I will be dealing with > 500k elements per
cycle (day).

I essentially need to know that ALL 500k elements were successful,
otherwise, I need to know where it broke and where it stopped.

This is a billing situation so it has to be accurate. I'd rather ensure
(by doing manual double-entry checking) accuracy then having to go
through potentially 500,000 lines in a log to see what ones went wrong,
while trying to eliminate the ones that went 'right'.

Or perhaps I understood you wrong.

Tks for the reply,

Steve

I think you did. Unfortunately we are guessing in the dark about what
this external program of yours does when when it fails (or indeed when
it succeeds). Tom suggested checking the exit status or the STDERR
output. I am saying that there's probably also something useful on
STDOUT, which is easier to examine.

But each of these is a way to check whether a single run of your
application succeeded or failed. Look at what you see on the screen when
you run the command manually. Is there anything in that text that your
program could check for after each run? I would be surprised if there
wasn't.

I hope that's clearer.

Rob

[snip]
The external program is the MySQL client binary. I know the queries are
clean (it will run manually with no problems).

Here is what I'm doing:

- taking all rows for $today from table1
- deleting any possible erroneous rows from $today from table2 & table3
- grouping by fieldX and fieldY and inserting into table2
- updating (renaming) fieldY values based on it's content in table2.
fieldY's content will be renamed into one of three names, based on the
number range it contains (208.70% == 'dialup' 65.70 == 'dsl' etc)
- taking all rows from table2, grouping them based on fieldY (name), and
inserting into table3
Indeed. More importantly as stated above, I need to be sure that all
records (or a random statistical subset) succeeded. mysql will succeed
even if there were a large chunk of records that had erroneous entries
in the field I am looking at with a WHERE clause.

I know if I Perl-ize this, I'll be able to produce myself a report of
any sort of statistics I want from the run 7 ways from Sunday.

That is exactly what I plan to do instead of worrying about it.

Thanks for the replies. All the thought that went into this got me
thinking on other tracks for other code, but that's for another day.

Steve