Jesper Krogh (j...@krogh.cc)

This is excactly what I do. Can you elaborate a bit more on this?

I currently do put in the database-id into the document, but it doesn't
serve any purpose. I do put in an unique term into the term-set of the
documents that make replace-by-term work flawlessly. Can I preserve this
during a merge (that I have a "unique-term" in the documents)?

> You may want to look at:
> <http://xapian.org/docs/admin_notes.html#backup-strategies>
>
> and
>
> <http://xapian.org/docs/admin_notes.html#merging-databases>
> for some other notes that may be of use here.

Thanks alot for the hints.

Hi.

This i a small feature request for Xapian. Currently I have a
xapian-database with >5m records, the files fills around 124GB in the
Xapian catalog. With a few "quite large" files:

# du -sh *
0       flintlock
4.0K    iamflint
1000K   position.baseA
63G     position.DB
716K    postlist.baseA
624K    postlist.baseB
45G     postlist.DB
8.0K    record.baseA
385M    record.DB
240K    termlist.baseA
15G     termlist.DB
12K     value.baseB
696M    value.DB

(And it is my impression that I have a quite small record.DB-file)
The idea comes from PostgreSQL's filesystem layout, it has a (probably
historic) filesize of 2GB, but it helps the backup significantly.

This layout, gives some "challenges" to backup systems since the daily
incremental runs basically now has to backup the complete set => 124GB
even if only a single new document has been merged.

The suggesting would be to split the files in several smaller files. I
know that the algorithms for searching the binary trees probably would
be a bit more complex, but it could result in that changes only touches
a subset of the files, thus letting the backup proceed easier.

Another solution could be to let Xapian query several databases and
"merge" the result. Then I could make a new database each day and merge
once a week (or another timepattern that would fit the purpose).

Other suggestions are welcome.

Thanks.

Jesper

Thats a downright bogus argument..  the archive should be properly 
digitally signed to provide the real security.

Not that I have a solution, but I have a similar problem with my Xapian
database. (doccount 8millions) flushtime is fairly long (over 10 minutes
on a 16 SAS disk array for 1000 documents added) and monitoring vmstat
(and top) I can see that it neither saturates 1 cpu or anything near the
block input/output that the disk can deliver (uses around 5MB/s in
block/in and out), viewing "top" only around 8-12% IO wait.

All of above is measured when Xapian is "flushing".

Still running xapian 1.0.4 (with perl-bindings)

Hi

Can anyone help me find out what my problem is here?

$ perl -MDevel::SimpleTrace script/efam_server.pl
Undefined subroutine &Catalyst::Utils::env_value called
         at 
Catalyst::Plugin::ConfigLoader::get_config_path(/usr/share/perl5/Catalyst/Plugin/ConfigLoader.pm:165)
         at 
Catalyst::Plugin::ConfigLoader::find_files(/usr/share/perl5/Catalyst/Plugin/ConfigLoader.pm:109)
         at 
Catalyst::Plugin::ConfigLoader::setup(/usr/share/perl5/Catalyst/Plugin/ConfigLoader.pm:52)
         at Catalyst::setup(/usr/share/perl5/Catalyst.pm:851)
         at 
<eval>(/net/atlas.nzcorp.net/z/fx1200/bio/home/jk/NzDB/devel/script/../lib/Efam.pm:49)
         at main::(script/efam_server.pl:59)
Compilation failed in require at script/efam_server.pl line 59.

Efam.pm:49 is
__PACKAGE__->setup();


Jesper