FAQ
Hi,



I am getting following error message while unziping a .zip file. Any
help or idea is highly appreciated.



Error message>>>

Traceback (most recent call last):

File "C:\Zip_Process\py\test2_new.py", line 15, in <module>

outfile.write(z.read(name))

IOError: (22, 'Invalid argument')





The script is here:

*****************************

fh = open('T:\\test\\*.zip', 'rb')

z = zipfile.ZipFile(fh)

for name in z.namelist():

outfile = open(name, 'wb')



outfile.write(z.read(name))

print z

print outfile

outfile.close()



fh.close()

********************************



-------------- next part --------------
A non-text attachment was scrubbed...
Name: winmail.dat
Type: application/ms-tnef
Size: 4958 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-list/attachments/20110624/d78fe7a2/attachment-0001.bin>

Search Discussions

  • Philip Semanchuk at Jun 24, 2011 at 3:18 pm

    On Jun 24, 2011, at 10:55 AM, Ahmed, Shakir wrote:

    Hi,



    I am getting following error message while unziping a .zip file. Any
    help or idea is highly appreciated.



    Error message>>>

    Traceback (most recent call last):

    File "C:\Zip_Process\py\test2_new.py", line 15, in <module>

    outfile.write(z.read(name))

    IOError: (22, 'Invalid argument')

    Start debugging with these two steps --
    1) Add this just after "for name in z.namelist():"
    print name

    That way you can tell which file is failing.

    2) You can't tell whether you're getting an error on the write or the read because you've got two statements combined into one line. Change this --
    outfile.write(z.read(name))
    to this --
    data = z.read(name)
    outfile.write(data)


    Good luck
    Philip





    The script is here:

    *****************************

    fh = open('T:\\test\\*.zip', 'rb')

    z = zipfile.ZipFile(fh)

    for name in z.namelist():

    outfile = open(name, 'wb')



    outfile.write(z.read(name))

    print z

    print outfile

    outfile.close()



    fh.close()

    ********************************



    <winmail.dat>--
    http://mail.python.org/mailman/listinfo/python-list
  • Ahmed, Shakir at Jun 24, 2011 at 5:31 pm
    -----Original Message-----
    From: python-list-bounces+shahmed=sfwmd.gov at python.org
    [mailto:python-list-bounces+shahmed=sfwmd.gov at python.org] On Behalf Of
    Philip Semanchuk
    Sent: Friday, June 24, 2011 11:18 AM
    To: Lista-Comp-Lang-Python list
    Subject: Re: unzip problem

    On Jun 24, 2011, at 10:55 AM, Ahmed, Shakir wrote:

    Hi,



    I am getting following error message while unziping a .zip file. Any
    help or idea is highly appreciated.



    Error message>>>

    Traceback (most recent call last):

    File "C:\Zip_Process\py\test2_new.py", line 15, in <module>

    outfile.write(z.read(name))

    IOError: (22, 'Invalid argument')

    Start debugging with these two steps --
    1) Add this just after "for name in z.namelist():"
    print name

    That way you can tell which file is failing.

    2) You can't tell whether you're getting an error on the write or the
    read because you've got two statements combined into one line. Change
    this --
    outfile.write(z.read(name))
    to this --
    data = z.read(name)
    outfile.write(data)


    Good luck
    Philip




    The script is here:

    *****************************

    fh = open('T:\\test\\*.zip', 'rb')

    z = zipfile.ZipFile(fh)

    for name in z.namelist():

    outfile = open(name, 'wb')



    outfile.write(z.read(name))

    print z

    print outfile

    outfile.close()



    fh.close()

    ********************************

    The problem found in outfile.Write. The error code is here and is
    happening when few of the files are already unzipped from the zip file
    T:\applications\tst\py\Zip_Process
    drg100.aux
    <zipfile.ZipFile instance at 0x00E2F648>
    <open file 'drg100.aux', mode 'wb' at 0x00E12608>
    drg100.fgdc.htm
    <zipfile.ZipFile instance at 0x00E2F648>
    <open file 'drg100.fgdc.htm', mode 'wb' at 0x00E128D8>
    drg100.htm
    <zipfile.ZipFile instance at 0x00E2F648>
    <open file 'drg100.htm', mode 'wb' at 0x00E12608>
    drg100.sdw
    <zipfile.ZipFile instance at 0x00E2F648>
    <open file 'drg100.sdw', mode 'wb' at 0x00E128D8>
    drg100.sid
    Unhandled exception while debugging...
    Traceback (most recent call last):
    File "C:\Zip_Process\py\test2_new.py", line 16, in <module>
    outfile.write(data)
    IOError: (22, 'Invalid argument')
  • Steven D'Aprano at Jun 24, 2011 at 3:33 pm

    On Fri, 24 Jun 2011 10:55:52 -0400, Ahmed, Shakir wrote:

    Hi,



    I am getting following error message while unziping a .zip file. Any
    help or idea is highly appreciated.
    How do you know it is when unzipping the file? Maybe it is, or maybe it
    isn't. The line giving the error has *two* IO operations, a read and a
    write. Either one could give IOError.

    outfile.write(z.read(name))
    IOError: (22, 'Invalid argument')

    The first thing is to isolate what is causing the error:

    data = z.read(name)
    outfile.write(data)

    Now you can be sure whether it is the read or the write which causes the
    error.

    Secondly, find out what the argument is. Assuming it is the read, try
    this:

    print repr(name)

    Can you open the zip file in Winzip or some other zip utility? Does it
    need a password?

    I see you do this:

    fh = open('T:\\test\\*.zip', 'rb')

    Do you really have a file called *.zip? I find that very hard to
    believe... as I understand it, * is a reserved character in Windows and
    you cannot have a file with that name.

    My guess is that the actual error is when you try to open the file in the
    first place, before any unzipping takes place, but you've messed up the
    line numbers (probably by editing the file after importing it).


    --
    Steven
  • Ahmed, Shakir at Jun 24, 2011 at 5:23 pm

    On Fri, 24 Jun 2011 10:55:52 -0400, Ahmed, Shakir wrote:

    Hi,



    I am getting following error message while unziping a .zip file. Any
    help or idea is highly appreciated.
    How do you know it is when unzipping the file? Maybe it is, or maybe it
    isn't. The line giving the error has *two* IO operations, a read and a
    write. Either one could give IOError.

    outfile.write(z.read(name))
    IOError: (22, 'Invalid argument')

    The first thing is to isolate what is causing the error:

    data = z.read(name)
    outfile.write(data)

    Now you can be sure whether it is the read or the write which causes
    the
    error.

    Secondly, find out what the argument is. Assuming it is the read, try
    this:

    print repr(name)

    Can you open the zip file in Winzip or some other zip utility? Does it
    need a password?

    I see you do this:

    fh = open('T:\\test\\*.zip', 'rb')

    Do you really have a file called *.zip? I find that very hard to
    believe... as I understand it, * is a reserved character in Windows and
    you cannot have a file with that name.

    My guess is that the actual error is when you try to open the file in
    the
    first place, before any unzipping takes place, but you've messed up the
    line numbers (probably by editing the file after importing it).


    --
    Steven

    >>
    The problem is happening when it is coming to write option.

    The * is not the real name of the zip file, I just hide the name.
  • Steve+Comp Lang Python at Jun 24, 2011 at 5:55 pm

    Ahmed, Shakir wrote:

    The problem is happening when it is coming to write option.

    The * is not the real name of the zip file, I just hide the name.
    Please don't waste our time by showing us fake code that doesn't do what you
    say it does.

    You said that "The script is here", but that was not true, was it? What you
    showed us was not the script you were running. What other changes,
    deliberate or accidental, did you make?

    If you are going to edit the script to make it simpler (a good idea!),
    please run it first to make sure you still get the same error.



    --
    Steven
  • Ethan Furman at Jun 24, 2011 at 6:00 pm

    Ahmed, Shakir wrote:
    Hi,



    I am getting following error message while unziping a .zip file. Any
    help or idea is highly appreciated.



    Error message>>>

    Traceback (most recent call last):

    File "C:\Zip_Process\py\test2_new.py", line 15, in <module>

    outfile.write(z.read(name))

    IOError: (22, 'Invalid argument')
    I just dealt with this problem on my system -- only pops up when writing
    large files (just over 50Mb for me) to a network drive; no problems when
    writing to a local drive.

    Here's how I get around it:

    CHUNK_SIZE = 10 * 1024 * 1024

    fn = open(uncompressed_file, 'wb')
    ptr = 0
    size = len(data)
    while ptr < size:
    fn.write(data[ptr:ptr+CHUNK_SIZE])
    ptr += CHUNK_SIZE
    fn.close()


    ~Ethan~
  • Ethan Furman at Jun 24, 2011 at 6:55 pm

    Ahmed, Shakir wrote:
    Thanks for your help and really appreciate your time.

    I changed the code as you mentioned and here it is:


    fn = open('T:\\applications\\tst\\py\\Zip_Process\\Zip\\myzip.zip',
    'rb')
    z = zipfile.ZipFile(fn)
    for name in z.namelist():
    data = z.read(name)
    ptr = 0
    size = len(name) # change this back to data
    ^- No. You need the size of the data read in from the
    zipfile for that name, not the length of the name. Sorry for the omission.
    print size
    print ptr
    while ptr < size:

    fn.write(name[ptr:ptr+CHUNK_SIZE]) # change name to data
    ptr += CHUNK_SIZE

    fn.close()
    ~Ethan~
  • Ethan Furman at Jun 24, 2011 at 7:46 pm

    Ahmed, Shakir wrote:
    Thanks once again and you are right I am trying to unzip in the network
    share drive. here is the script now: If I am doing any wrong. :
    ## code start here

    import zipfile
    import os
    import time
    dir1 = "T:\\applications\\tst\\py\\Zip_Process"
    test = '%s/shp'%dir1
    os.chdir(test)
    cwd = os.getcwd()
    print cwd


    CHUNK_SIZE = 10 * 1024 * 1024


    fn = open('T:\\applications\\tst\\py\\Zip_Process\\Zip\\myzip.zip',
    'rb')
    z = zipfile.ZipFile(fn)
    for name in z.namelist():
    ptr = 0
    data = z.read(name)
    size = len(data)
    print size
    print ptr
    while ptr < size:

    fn.write(data[ptr:ptr+CHUNK_SIZE])
    ptr += CHUNK_SIZE

    fn.close()

    #Code done here.

    But got error as follows:
    T:\applications\tst\py\Zip_Process\shp
    59160

    Traceback (most recent call last):
    File
    "C:\Python25\Lib\site-packages\pythonwin\pywin\framework\scriptutils.py"
    , line 325, in RunScript
    exec codeObject in __main__.__dict__
    File "t:\scratch\shahmed\test2_new.py", line 24, in <module>
    fn.write(data[ptr:ptr+CHUNK_SIZE])
    IOError: [Errno 9] Bad file descriptor
    I didn't notice this in your last post, but you are using fn as both the
    zipfile name and the name of the file you are writing to. You'll need
    to create the file you want to write before you write to it (plus any
    intermediate directories).


    Here's the (slightly stripped) version I actually use:

    8<--------------------------------------------------------------------
    import os
    from zipfile import ZipFile

    def retrieve_files(zipped, destination, files=None):
    "retrieves files from <zipped>"
    CHUNK_SIZE = 10 * 1024 * 1024
    try:
    os.makedirs(destination)
    except WindowsError:
    pass
    target = ZipFile(zipped, 'r')
    stored = dict([(k.filename.lower(), k.filename) for k in
    target.infolist()])
    if files is None:
    files = [member.filename.lower() for member in target.infolist()]
    elif isinstance(files, (str, unicode)):
    files = [files.lower()]
    else:
    files = [f.lower() for f in files]
    for compressed_file in files:
    uncompressed_file = os.path.join(destination, compressed_file)
    path, filename = os.path.split(uncompressed_file)
    if not os.path.exists(path):
    os.makedirs(path)
    if filename == '__empty__':
    continue
    data = target.read(stored[compressed_file])
    fn = open(uncompressed_file, 'wb')
    ptr = 0
    size = len(data)
    while ptr < size:
    fn.write(data[ptr:ptr+CHUNK_SIZE])
    ptr += CHUNK_SIZE
    fn.close()
    target.close()
    8<--------------------------------------------------------------------

    I convert all filenames to lower case since MS Windows doesn't care, but
    Python does.

    The test for filename == '__empty__': when I create the zipfile, if the
    directory is empty I store a 0-length file named '__empty__' in that
    subdirectory (actually, it only happens in the zipfile) so that I can
    get the directory back later.

    Hope this helps.

    ~Ethan~
  • Ahmed, Shakir at Jun 24, 2011 at 8:04 pm
    -----Original Message-----
    From: Ethan Furman [mailto:ethan at stoneleaf.us]
    Sent: Friday, June 24, 2011 3:47 PM
    To: Ahmed, Shakir
    Cc: Python
    Subject: Re: unzip problem

    Ahmed, Shakir wrote:
    Thanks once again and you are right I am trying to unzip in the network
    share drive. here is the script now: If I am doing any wrong. :
    ## code start here

    import zipfile
    import os
    import time
    dir1 = "T:\\applications\\tst\\py\\Zip_Process"
    test = '%s/shp'%dir1
    os.chdir(test)
    cwd = os.getcwd()
    print cwd


    CHUNK_SIZE = 10 * 1024 * 1024


    fn = open('T:\\applications\\tst\\py\\Zip_Process\\Zip\\myzip.zip',
    'rb')
    z = zipfile.ZipFile(fn)
    for name in z.namelist():
    ptr = 0
    data = z.read(name)
    size = len(data)
    print size
    print ptr
    while ptr < size:

    fn.write(data[ptr:ptr+CHUNK_SIZE])
    ptr += CHUNK_SIZE

    fn.close()

    #Code done here.

    But got error as follows:
    T:\applications\tst\py\Zip_Process\shp
    59160

    Traceback (most recent call last):
    File
    "C:\Python25\Lib\site-packages\pythonwin\pywin\framework\scriptutils.py"
    , line 325, in RunScript
    exec codeObject in __main__.__dict__
    File "t:\scratch\shahmed\test2_new.py", line 24, in <module>
    fn.write(data[ptr:ptr+CHUNK_SIZE])
    IOError: [Errno 9] Bad file descriptor
    I didn't notice this in your last post, but you are using fn as both the

    zipfile name and the name of the file you are writing to. You'll need
    to create the file you want to write before you write to it (plus any
    intermediate directories).


    Here's the (slightly stripped) version I actually use:

    8<--------------------------------------------------------------------
    import os
    from zipfile import ZipFile

    def retrieve_files(zipped, destination, files=None):
    "retrieves files from <zipped>"
    CHUNK_SIZE = 10 * 1024 * 1024
    try:
    os.makedirs(destination)
    except WindowsError:
    pass
    target = ZipFile(zipped, 'r')
    stored = dict([(k.filename.lower(), k.filename) for k in
    target.infolist()])
    if files is None:
    files = [member.filename.lower() for member in
    target.infolist()]
    elif isinstance(files, (str, unicode)):
    files = [files.lower()]
    else:
    files = [f.lower() for f in files]
    for compressed_file in files:
    uncompressed_file = os.path.join(destination, compressed_file)
    path, filename = os.path.split(uncompressed_file)
    if not os.path.exists(path):
    os.makedirs(path)
    if filename == '__empty__':
    continue
    data = target.read(stored[compressed_file])
    fn = open(uncompressed_file, 'wb')
    ptr = 0
    size = len(data)
    while ptr < size:
    fn.write(data[ptr:ptr+CHUNK_SIZE])
    ptr += CHUNK_SIZE
    fn.close()
    target.close()
    8<--------------------------------------------------------------------

    I convert all filenames to lower case since MS Windows doesn't care, but

    Python does.

    The test for filename == '__empty__': when I create the zipfile, if the
    directory is empty I store a 0-length file named '__empty__' in that
    subdirectory (actually, it only happens in the zipfile) so that I can
    get the directory back later.

    Hope this helps.

    ~Ethan~

    Thanks a lot to Ethan who really helped to find out the clue in the
    code.
    Here is the final code that worked to unzip a large file in the network
    drive.

    CHUNK_SIZE = 10 * 1024 * 1024


    fh = open('T:\\applications\\tst\\py\\Zip_Process\\Zip\\myzip.zip',
    'rb')
    z = zipfile.ZipFile(fh)
    for name in z.namelist():
    fn = open(name, 'wb')
    ptr = 0
    data = z.read(name)
    size = len(name)
    print size
    print ptr
    while ptr < size:
    #fn.write(data)
    fn.write(data[ptr:ptr+CHUNK_SIZE])
    ptr += CHUNK_SIZE
    fn.close()
    fh.close()
  • Ethan Furman at Jun 24, 2011 at 8:21 pm

    Ahmed, Shakir wrote:
    Here is the final code that worked to unzip a large file in the network
    drive.

    CHUNK_SIZE = 10 * 1024 * 1024


    fh = open('T:\\applications\\tst\\py\\Zip_Process\\Zip\\myzip.zip',
    'rb')
    z = zipfile.ZipFile(fh)
    for name in z.namelist():
    fn = open(name, 'wb')
    ptr = 0
    data = z.read(name)
    size = len(name)
    print size
    print ptr
    while ptr < size:
    #fn.write(data)
    fn.write(data[ptr:ptr+CHUNK_SIZE])
    ptr += CHUNK_SIZE
    fn.close()
    fh.close()
    The 'size = len(name)' should be 'size = len(data)'.

    ~Ethan~

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppython-list @
categoriespython
postedJun 24, '11 at 2:55p
activeJun 24, '11 at 8:21p
posts11
users4
websitepython.org

People

Translate

site design / logo © 2022 Grokbase