eichin at metacarta.com wrote:
One of my recent projects has involved taking an accretion of sh and
perl scripts and "doing them right" - making them modular, improving
the error reporting, making it easier to add even more features to
them. "Of course," I'm redoing them in python - much of the cut&paste
reuse has become common functions, which then get made more robust and
have a common style and are callable from other (python) tools
directly, instead of having to exec scripts to get at them. The usual
"glorious refactoring." <<SNIP>>
Implementing pipelines takes rather a bit more work, and one might
(not unreasonably) throw up one's hands and just use os.system and
some re.sub's to do the quoting. However, I had enough cases where
the goal really was to run a complex shell pipeline (I also had cases
where the pipeline converted nicely to some inline python code,
especially with the help of the gzip module) that I sat down and
cooked up a pipeline class.

The interface I ended up with is pretty simple:
g_pipe = pipeline()
g_pipe.stdin(open("blort.gz", "r"))
g_pipe.append(["sort", "-u"])
g_pipe.append(["wc", "-l"])
g_pipe.stdout(open("blort.count", "w"))
print g_pipe.run()

is equivalent to the sh:
gunzip < blort.gz | sort -u | wc -l > blort.count <<SNIP>>
_Mark_ <eichin at metacarta.com>

[1] in the Invader Zim sense :)
I think that your pipeline code looks nothing like the original sh
script pipeline which to me counts heavily against it.
Just playing at the cygwin prompt...
$ ls -l|wc -l > /tmp/lines_in_dir
$ cat /tmp/lines_in_dir
$ python
from os import system
system(r'''/bin/ls -l|/bin/wc -l > /tmp/lines_in_dir2''')
system(r'''/bin/cat /tmp/lines_in_dir2''')

I prefer the above because it looks like the original sh command.
Of course, if script security is very important then you may want to
change the way things are implemented again.

Cheers, Paddy.

Search Discussions

Discussion Posts


Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 2 of 3 | next ›
Discussion Overview
grouppython-list @
postedOct 2, '03 at 4:20a
activeOct 2, '03 at 5:43a



site design / logo © 2022 Grokbase