I think my question is a little bit off topic... sorry.
I'm working with a large file of network logs (a file of 25Mb or 150000
lines). What I do is parse the file, identify the fields that interest
me most, put them on lists and then play with the lists to print out the
information I want.
This takes 5 minutes and eats a lot of RAM (150000 list elements is
quite a lot :-) so I though about creating a database with all the fiels
I want. Thinking about the database I'm studying the possibility of
using a classical MySQL database or using an XML document as a database
using XPath. The XML possibility would allow me to run my program in any
computer with python installed and eliminates the need of database
installation and management.
My questions are:
1- Would I gain in speed and RAM with the use of a database?
2- If it's the case, would XML resist the search over 150000 entries?
3- How much would I loose in speed with an XML document compared to the
This is, of course, implemented in python :-)
I'm very new to databases and XMl.