[Python] Creating a function for a directory

Its better in the future to cut and paste the traceback rather than
paraphrase...


That being said, it can't find the in_file. Maybe you aren't in the
directory above desktop? Try specifying a complete path, instead of
relative. Once you solve that, be sure the ouput path is specificed
completely and that it exists.

You're using the % operator, which does your interpolations, at the
wrong point. You want to be adjusting the file name, not adjusting the
opened file:


         in_file = open("desktop/%s.txt" % file)
         out_file = open("desktop/newfolder/%s.txt" % file, 'w')


ChrisA

The problem is the above line. Rewrite is thus:


  in_file = open("desktop/%s.txt" % file)

Sorry for incorect answer. Those guys nailed it
On Nov 11, 2013 5:43 PM, "bob gailer" wrote:

Your answer wasn't incorrect, because it didn't give any false
information. Bob and I saw the problem itself and gave advice, but you
gave useful general advice on how to find the problem, which is worth
bearing in mind. Matt, read Joel's response for useful tips on how to
figure out what might have gone wrong. :)


ChrisA

in_file = open("desktop/%s.txt" % file)

1. i believe win32 file paths require a qualifying volume
letter.


2. Never, ever, *EVER* write data to disc before confirming
the paths your passing are pointing to the location you
intended to write the data. Use os.path.exists(path) to test
your paths BEFORE trying to write data.


3. Be sure your variables names are both "self documenting"
and "non clobbering". psst: "file" is a builtin! Using
"filename" would be a far wiser choice for a variable
containing a filename. When i see "file", i think of a "file
object"


4. When dealing with files you must be sure that exceptions
are handled cleanly. You don't want open file objects
floating aimlessly around in memory because your naive code
blew chunks.


5. Remember, you cannot write a file into a directory that
does not exist.


6 For OS compatibility always use os.path.join() to join
path parts into a whole. This method will insert the proper
separator for you depending on the OS.

They do not; omitting the drive letter makes the path relative to the
current drive (and since it doesn't start with a directory specifier,
the current directory).


Why? Why, oh why? If there's a problem, it'll be signalled with an
exception. Testing that the path exists opens you up to race problems,
so you won't see anything now, but some day your code will be in a
concurrent situation and you'll get unexpected exceptions. Why not
just expect the exception?


This one's arguable. How often do you use the 'file' builtin? I've
worked with files in Python innumerable times, and I don't remember
the last time I used 'file'. Yes, avoid shadowing important builtins
like 'list' and 'int', but 'id' and 'file' aren't that big a deal.


They won't float around forever. The garbage collector will get to
them. If you're advocating use of 'with', that's only going to be an
issue if the code's called in a loop AND if it throws before the
close() calls come through.


So? Exception thrown, traceback printed to console, process terminated
cleanly. I'm not seeing a problem here.


Technically true. However, most modern OSes will accept a slash.
There'll be a few situations where that's not true, but the OP an
happily just use slashes for simplicity. Remember the zen:
Practicality beats purity.


ChrisA

This is subject to a race condition, which opens you to a security
vulnerability: "time of check to time of use" bugs.


If you follow Rick's advice, and write code like this:


if os.path.exists(path):
     write_to(path) # custom function to open and write to the file
else:
     handle_missing_file()




your code is doubly buggy. First, os.path.exists only tells you if the
path exists, not whether it is writable. Perhaps it is on read-only
media, or you don't have permission to open it, or it's a directory.


Secondly, even if the file exists at the moment you call os.path.exists,
there is no guarantee that it will still exist a moment later when you
try writing to it. Another process may delete or rename the file, or
change permissions in the meantime. So you have to write:


if os.path.exists(path):
     try:
         write_to(path)
     except (IOError, OSError):
         handle_missing_or_locked_file()
else:
     handle_missing_or_locked_file()




But now your test doesn't actually accomplish anything. Worse, just
because the path *doesn't* exist when you check using exists, that
doesn't mean it won't exist by the time you call write_to!


Using os.path.exists before opening a file is, for the most part, a waste
of time.

I use it in conjuction with file creation times to check that I
didn't forget to update/create one of the data files a process
might need.


Since I have to create the data file manually, race conditions
seem unlikely.

Hmm. Let's confirm:

     True
     True


Yep, it's official. "Implicit File Path Resolution"
I need to author a PyWart on this soon!


Except when there's a problem that won't be signaled by an
EXCEPTION, but will be signaled by EMOTIONS; like for
instance removing the wrong directory or truncating the
wrong file because of a typo in your source code.


  OPPS! :-'(


Because today i'm not facing a concurrent situation, so i'm
not going to bother and protect from it. Just like i'm not
facing a snake bite, so i won't bother to lug around a vial
of antidote. Your attempts to discredit me via hypothetical
scenarios is entertaining however.


My first concern is with the OP adopting "self documenting"
names. Shadowing file is merely ensuring that the OP knows
"file" is a builtin. Shadowing any builtin can create very
difficult bugs to track. This is the point the OP should
remember.


That depends on how they are stored. You make too many
assumptions Chris. I don't have an problems with GC's, but
i'm not about to write sloppy code and "assume" the GC is
going to swoop in and save me like a feline trapped in a
tree.


Now you're just trolling!


     def firstdev(file):
         in_file = open("desktop/%s.txt") % file
         indata = in_file.read()
         out_file = open("desktop/newfolder/%s.txt", 'w') % file


Just from reading that code NO ONE could know for sure if
"newfolder" even existed BEFORE the OP tried to open the
"out_file". Maybe the OP thinks that missing sub-directories
are auto-created by the open function, but there's no way to
know for sure, hence my comment.


Many a developer have lived to regret those words.


PS: I thought Steven was the official "devils advocate"
around here? Hmm. I guess everyone needs a day off.

That's not a Python issue at all. And are you seriously going to decry
relative pathnames?!?


Which os.path.exists will not protect you from. So?


(Reordered to bring together two comments getting one response) Do you
care about what you currently are seeing, or what you might some day
see? Concurrency is a common problem on many platforms; platforms not
supporting slashes between paths are rare, and most of them will have
other porting issues. Why are you ignoring the likely, while trying to
prevent the unlikely?


Ah, actually yes you are. When have you ever explicitly disposed of an
object in Python? You just let the GC deal with it for you.


Mercurial signals some errors by dumping an exception to stderr. This
works quite nicely, even in production. If your script can't recover
from being given a non-writeable directory, don't catch the exception.


And the OP might have thought that it would automatically publish that
to the world, tweet the URL, and create a Facebook status saying "just
opened a file lolol" and get five people to Like it. Doesn't mean we
need to probe before doing things.


ChrisA

Thank you guys so much. Brain fart moment. I appreciate it

would it not be more efficient and less error prone to use the os module
to copy the file rather than manually reading & re-writing it (unless of-
course you intend to expand this in future to process the data first)?

Hmm, I don't see a suitable function in os -- but there is


shutil.copy()

that's what i get for posting on the fly without checking first but i
think the principle holds.