FAQ
Dear all,
I'm trying to use the shove module (http://pypi.python.org/pypi/shove)
for a simple script. The script read a CSV file ad store the data.
When I check the content of the "store" object (instance of Shove)
*before* I close it, the data are all there but when I close and re-
open it some data are lost. How it is possible? There is something
wrong in my code or I didn't understand how shove works?

Thanks in advance.

Here is a sample of my code:

DBNAME = "file://store.db"

csv_file ="test.csv"
cities = csv.reader(open(csv_file), delimiter=";")
store = Shove(DBNAME, compress=True)
for city,region,code in cities:
entry_list = store.setdefault(region, [])
data = 'just a sample'
entry = {city:data}
entry_list.append(entry)

print "----Before closing----"
for k,v in store.iteritems():
print "Key->", k
print "Value->", v
print "----"
store.close()

store = Shove(DBNAME, compress=True)
print "----After closing and re-opening----"
for k,v in store.iteritems():
print "Key->", k
print "Value->", v
print "----"

Here is the output

----Before closing----
Key-> Marche
Value-> [{'Ancona': 'just a sample'}, {'Ascoli Piceno': 'just a
sample'}]
Key-> Piemonte
Value-> [{'Alessandria': 'just a sample'}, {'Asti': 'just a sample'}]
Key-> Puglia
Value-> [{'Bari': 'just a sample'}, {'Barletta-Andria-Trani': 'just a
sample'}]
Key-> Valle d'Aosta
Value-> [{'Aosta': 'just a sample'}]
----
----After closing and re-opening----
Key-> Marche
Value-> [{'Ancona': 'just a sample'}]
Key-> Piemonte
Value-> [{'Alessandria': 'just a sample'}]
Key-> Puglia
Value-> [{'Bari': 'just a sample'}, {'Barletta-Andria-Trani': 'just a
sample'}]
Key-> Valle d'Aosta
Value-> [{'Aosta': 'just a sample'}]
----

Here is the content of the CSV file:

Alessandria;Piemonte;AL
Ancona;Marche;AN
Aosta;Valle d'Aosta;AO
Ascoli Piceno;Marche;AP
Asti;Piemonte;AT
Bari;Puglia;BA
Barletta-Andria-Trani;Puglia;BT

Search Discussions

  • Chris Rebert at Apr 21, 2010 at 10:36 am

    On Wed, Apr 21, 2010 at 2:51 AM, Alex wrote:
    I'm trying to use the shove module (http://pypi.python.org/pypi/shove)
    for a simple script. The script read a CSV file ad store the data.
    When I check the content of the "store" object (instance of Shove)
    *before* I close it, the data are all there but when I close and re-
    open it some data are lost. How it is possible? There is something
    wrong in my code or I didn't understand how shove works?

    Thanks in advance.

    Here is a sample of my code:

    DBNAME = "file://store.db"

    ? ?csv_file ="test.csv"
    ? ?cities = csv.reader(open(csv_file), delimiter=";")
    ? ?store = Shove(DBNAME, compress=True)
    ? ?for city,region,code in cities:
    ? ? ? ?entry_list = store.setdefault(region, [])
    ? ? ? ?data = 'just a sample'
    ? ? ? ?entry = {city:data}
    ? ? ? ?entry_list.append(entry)
    If `shove` is like the std lib `shelve` module, the following might
    fix the problem:

    #untested
    for city,region,code in cities:
    entry_list = store.get(region, [])
    data = 'just a sample'
    entry = {city:data}
    entry_list.append(entry)
    store[region] = entry_list

    Explanation:
    The explicit assignment back to the `store` pseudo-dictionary lets it
    properly update its internal state to reflect the change to the value
    (in this case, the list) associated with the region key. In your
    original version, you merely modified the list in-place, and (due to
    the way Python works) `store` has no way of knowing that you've done
    that and thus doesn't do the necessary bookkeeping, hence the behavior
    you're observing.

    See also: http://docs.python.org/library/shelve.html#example

    And further note that shove seems to be beta and (apart from
    docstrings in the source code) undocumented.

    Cheers,
    Chris
  • Alex at Apr 21, 2010 at 11:37 am
    On 21 Apr, 12:36, Chris Rebert wrote:
    [cut]
    Explanation:
    The explicit assignment back to the `store` pseudo-dictionary lets it
    properly update its internal state to reflect the change to the value
    (in this case, the list) associated with the region key. In your
    original version, you merely modified the list in-place, and (due to
    the way Python works) `store` has no way of knowing that you've done
    that and thus doesn't do the necessary bookkeeping, hence the behavior
    you're observing.

    See also:http://docs.python.org/library/shelve.html#example

    And further note that shove seems to be beta and (apart from
    docstrings in the source code) undocumented.
    Thanks a lot for the clear explanation. It works!
    I will read the docs more carefully next time :-)

    Alex

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppython-list @
categoriespython
postedApr 21, '10 at 9:51a
activeApr 21, '10 at 11:37a
posts3
users2
websitepython.org

2 users in discussion

Alex: 2 posts Chris Rebert: 1 post

People

Translate

site design / logo © 2023 Grokbase