[Xapian-discuss] strange problem with number value range in 1.0.4

NumberValueRangeProcessor expects the value to be encoded with
Xapian::sortable_serialise(). Otherwise the string compare which
is used means that 123 < 20 < 3!

In 1.0.0, NumberValueRangeProcessor had a broken design where it just
used a string compare like this, but we did quickly realise and warn
people not to use it until it was fixed.
I don't think scriptindex supports encoding with sortable_serialise()
currently. But if your timestamps are always fixed length strings then
you could just use StringValueRangeProcessor.

Cheers,
Olly

StringValueRangeProcessor works for me!

Thanks again Olly.

Is Xapian::sortable_serialise() wrapped in Search::Xapian?

It didn't seem so when I tried it just now (1.0.4.0-2 from Debian
unstable) - which means that NumberValueRangeProcessor is not that
usable if you index from Perl, right?
I was trying to use StringValueRangeProcessor, but the constructor does
not take a prefix (as opposed to NumberVRP), so I can only have one
"rangeable" value, which is too little in my case.

Should I try figuring out how to wrap sortable_serialise, or should I
try to provide a patch for StringVRP that adds the str and prefix
arguments, similar to NumberVRP?


Best regards,

Adam

Odd, Debian only seems to have -1 currently:

http://packages.qa.debian.org/libs/libsearch-xapian-perl.html

Perhaps you're using a privately updated package? But these issues
affect vanilla 1.0.4.0 anyway.
StringVRP really ought to support a prefix/suffix like NumberVRP does.
There's already a wishlist bug for adding this feature to DateVRP, so
I've updated that to cover StringVRP too:

http://www.xapian.org/cgi-bin/bugzilla/show_bug.cgi?id"0
Patches for either or both would certainly be welcome. I'm currently
busy trying to get a 1.0.5 release together.

Cheers,
Olly

Ok - I couldn't quite figure out where it would go, by my patented
monkey-see - monkey-copy method.
Ah - I accidentally bumped the packaging number to -2 when I built a
local backport to Ubuntu gutsy; I should have said 1.0.4.0-1, sorry.
Yes - the only thing I updated was the debian/changelog (so we can
easily distinguish locally built packages).
Cool.
My C is rusty and my C++ has never really been shining, but I will see
if I can find time to have a go at the latter tomorrow; I think we need
it for our app, so I am motivated :-)


Best regards,

Adam

We wrapped it for our own needs for Perl usage.

Do the following:

in the XS directory, there is a file called Database.xs, towards the end
of it add the following :

string
Database::sortable_serialise(double value)
CODE:
try {
RETVAL = sortable_serialise(value);
}
catch (const Error &error) {
croak( "Exception: %s", error.get_msg().c_str() );
}
OUTPUT:
RETVAL

double
Database::sortable_unserialise(string value)
CODE:
try {
RETVAL = sortable_unserialise(value);
}
catch (const Error &error) {
croak( "Exception: %s", error.get_msg().c_str() );
}
OUTPUT:
RETVAL

Note: Place this code just BEFORE

void
Database::DESTROY()


Then recompile the Perl-XS (follow the readme file)..

You will now be able to call sortable_serialise (and
sortable_unserialise) from perl, via any database object.


For example: (just a general example)

my $XAP_DB Search::Xapian::WritableDatabase->new("/test_database/",DB_CREATE_OR_OPEN);


my $TimeStamp = 1035122221;

my $TimeStamp_Serialized = $XAP_DB->sortable_serialise($TimeStamp);


This will return the funky string thingy you saw in the query, so you
can now store those as values and use as intended..

In this example we use an actual WritableDatabase, but it can be any
database.

Note that we did this per our need, and it may very well be not the
perfect/ideal wrapping, but it works perfectly for us. I think the best
place to wrap it is part of the actual top Xapian object, but we didn't
succeed with it right away and it was easier (and enough) for us to do
on the Database object, so thats what we kept for now...


Hope it helps. Good luck..


Ron


Adam Sj?gren wrote:

[... nice description elided ...]

Cool - thanks! How about contributing a patch, so it can be integrated
into Xapian proper?


Best regards,

Adam