[Image-SIG] ANN: PIL 1.1.6 final (december 3, 2006)

- A week or so ago, I reported that PIL loads multi-byte (e.g. 16-
and 32-bit raster data) TIFF files incorrectly if the files were
written as big-endian. (I also and provided a test case and patch.)
Now, I just discovered a minor problem in my patch relating to 32-bit
images, so I will send a new one in a few hours. This problem has
been in PIL for some time, and has basically disqualified it for use
in scientific imaging (which uses 16-bit data all of the time). I'd
love to get that changed.

- There is a buffer-overflow problem in 'struct
ImagingMemoryInstance' defined in 'Imaging.h'. Specifically, this
struct has a field defined as follows:
char mode[4+1]; /* Band names ("1", "L", "P", "RGB", "RGBA",
"CMYK") */
However, mode strings with more than 4 characters are copied into
this buffer via strcpy, clobbering other data fields in the struct. I
will provide a patch that makes the buffer large enough to hold all
mode strings that might be used, and changes the strcpy calls to a
strncpy. This patch also I will try to provide in a few hours.

- The 'fromarray' command is a bit broken in Image.py:
Specifically, the following stanza is incorrect --
if mode is None:
typestr = arr['typestr']
if not (typestr[0] == '|' or typestr[0] == _ENDIAN or
typestr[1:] not in ['u1', 'b1', 'i4', 'f4']):
raise TypeError("cannot handle data-type")
typestr = typestr[:2]
if typestr == 'i4':
...

The error is that 'typestr = typestr[:2]' should instead be 'typestr
= typestr[1:]'

- Also, the to/from array commands don't handle 16-bit images, though
it could be accomplished easily. I will provide a patch for this too.

- I still think that the handling of 16-bit images itself represents
a major bug (see my previous email to the list, but in short, having
to deal with big/little endian issues at the high-level API for 16-
bit images but no other image types is a bug and causes other bugs in
the code). I've been working on a patch to fix this, as described in
that earlier email. Probably this patch is too big a change to apply
to this release, but I would REALLY like to talk to someone about the
chances for 1.7 (or whatever) accepting changes that make 16-bit
image handling less vexing.

Thanks,

Zach Pincus

Program in Biomedical Informatics and Department of Biochemistry
Stanford University School of Medicine

I noticed the post, but haven't had time to look at it; given my current
schedule, it's been hard enough to get 1.1.6 out of the door (and in
case you wonder, the 1.1.6 feature set has been frozen for quite some
time; the last beta was basically a release candidate).
that's a known minor issue, and the struct is intentionally left as is
for compatibility reasons (see the mailing list archives for details).
this will be fixed in 1.2, where the internal storage model.
that code was contributed by the numpy developers, but it sure looks
broken. hmm.
as you might have noticed, the I16 support is only documented in the
CHANGES file, where it is labeled as experimental and incomplete; most
of it is based on contributed code from a small number of projects that
rely on "zero copy" access to underlying data.

under PIL 1.2's storage model, all I;16 images will most likely always
look like "I" images for ordinary users; if you need to get at the "raw"
data, you'd need to explicitly specify that when opening/loading the image.

as for the 1.1 branch, we'd like to keep it relatively stable, but
"low-hanging fruit" patches are of course still welcome (and nothing
prevents us from doing a 1.1.7 maintenance release).

</F>

Thanks for the information, Fredrik.
No problem. I'll try to provide a minimally-invasive patch to address
this issue (it's really simple to do -- the code just needs to pass
the correct byte order to Unpack.c, so hopefully this will count as
'low-hanging fruit').
Thanks for the info. I'm glad to hear about the upcoming changes for
1.2, and would be happy to help with some of that if I can...
I'll also try to provide a minimally-invasive patch here too.
Does this count as low-hanging? I'd be happy to send a patch if it
would be useful; otherwise I'll just put it into my code where it
interfaces with PIL.
Great -- I'm looking forward to 1.2!

Thanks again,

Zach

Hello all,

Here is a simple patch to make PIL read TIFFs in big-endian format
with multibyte pixels properly.

The changes are that the 'pack' and 'unpack' raw modes are now keyed
to the byte order in the TIFF file, which they were not in the past.
I also removed attempts to read and write TIFFs with signed 16-bit
integer pixels into 'I;16S' files, because Unpack.c and Pack.c don't
have any way to convert rawmode 'I;16S' files to/from user-mode 'I;
16S'. Now these files are properly read in as 'I'-mode files.

Will this patch make it into a maintenance release 1.1.7? I'd like to
know whether I should keep a fork of PIL until 1.2 comes out, so that
people in my lab and others can read image from our microscopes into
and out of python/numpy.

Zach

Hi,
Congrats on the new release ... one question/comment:

A few weeks back I was having problems "painting images" pixel by
pixel ... it was coming out very poorly running PIL using OS X.

You found the problem, however, in PIL/Image.py ~ line 2061:

elif sys.platform == "darwin":
format = "JPEG"
if not command:
command = "open -a /Applications/Preview.app"

changing JPEG to PNG turned out to solve the problem and the images
were being rendered as expected.

I just d/led the 1.1.6 and realized that it still defaults to JPEG
instead of PNG and was wondering if this was an oversight or not (I'm
changing mine back to PNG, though :-).

-steve

it's a damned-if-change-it-damned-if-you-don't issue ;-), and was too
late for 1.1.6 anyway. hopefully, 1.2 will provide more flexible image
display facilities.

</F>

Completely understandable ...

Just out of curiosity, and so I know what to keep my eyes open for
since I changed it to PNG in my compile, what puts it in the "damned -
if-you-do" scenario? Is there something I should expect to get hosed?

Thanks,
-steve

performance, mostly. doing lossless compression is a lot more costly
than doing lossy compression, or not doing compression at all.

</F>