[Python] dictionary size changed during iteration

The keys() method which used to return a list in 2.x was changed in 3.x to
return a view object and to become more or less the equivalent of the old
dict.iterkeys():
dict_keys(['a'])
... d["b"] = 42
...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
RuntimeError: dictionary changed size during iteration
dict_keys(['a', 'b'])

You now have to create the list explicitly to avoid the error:
... d["b"] = 42
...
{'a': 1, 'b': 42}
['a']

Hm, I see you are using has_key() which is 2.x-only and invalidates my
explanation :(

`ukeys` isn't a different dictionary from `self.updates.keys` I'ts merely
another name referring to the same dict object. I think

ukeys = dict (self.updates.keys)

would do what you want.

Mel.

Sorry. Belay that. Thought I'd had enough coffee.

Mel.

I think we need to see some more code here.

Also, what version of Python are you running?

http://docs.python.org/library/stdtypes.html

says "keys()
Return a copy of the dictionary?s list of keys."

So "ukeys" should be a unique list, not some part of
self.updates or a generator.

At least in Python 2.6, keys() does return a unique list. Each
call to "keys()" returns a new list object unconnected to the
dictionary from which the keys were extracted. But someone may
have decided in a later version to return a generator, as
an optimization. Did that happen?

John Nagle

As of Python 3.x (which I suspect you are running):

"The objects returned by dict.keys(), dict.values() and dict.items() are view objects. They provide a dynamic view on the dictionary?s entries, which means that when the dictionary changes, the view reflects these changes.", and

"Iterating views while adding or deleting entries in the dictionary may raise a RuntimeError or fail to iterate over all entries."

see: http://docs.python.org/release/3.1.3/library/stdtypes.html#dict-views

On my system:
ActivePython 3.2.0.0 (ActiveState Software Inc.) based on
Python 3.2 (r32:88445, Feb 21 2011, 11:25:33)
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type "copyright", "credits" or "license()" for more information.
<class 'dict_keys'>
>>>

So, to achieve your objective, one solution could be to change the ukeys assignment statement to:

ukeys = list(self.updates.keys())



Hope this helps,

Mark N.

In article <mailman.644.1303306435.9059.python-list at python.org>,
Peter Otten wrote:
That works, but if d is large, it won't be very efficient because it has
to generate a large list.

If d is large, and the number of keys to be mutated is relatively small,
a better solution may be to do it in two passes. The first loop
traverses the iterator and builds a list of things to be changed. The
second loop changes them.

changes = [ ]
for key in d.iterkeys():
if is_bad(key):
changes.append(key)
for key in changes:
d[key] = "I'm not dead yet"

Both solutions are O(n), but the second may run significantly faster and
use less memory.

It is not large. But I'm using Python 2.6 , not Python 3.

I did not get this error again in the last two days. I'll post a new
reply if I encounter it again. (It happened just a few times out of many
thousand program invocations)

changes = list(k for k in d if is_bad(k))

is a little bit more direct.

In article <7xd3jukyn9.fsf at ruckus.brouhaha.com>,
Paul Rubin wrote:
This is true. I still file list comprehensions under "new fangled
toys". While I use them, and appreciate their value, I admit they're
not always the first thing that comes to my mind.

OBTW,
is even more direct :-)

How about:

changes = filter(is_bad, d)

Or would that be too compact?

-- HansM

I thought of writing something like that but filter in python 3 creates
an iterator that would have the same issue of walking the dictionary
while the dictionary is mutating.

changes = list(filter(is_bad, d))

should work.