Does QueryParser uses Analyzer ?

look at the getFieldQuery method. It uses it to extract the tokens
from each part of the query (phrases and stand-alone terms).
What Analyzer are you using? My guess is that your analyzer is what
did the splitting, though it could be something fishy in how you got
the string into QueryParser in the first place?

Erik


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org

i was using an adaptation of the SearchFiles class distibuted in the
demo (demo.org.apache.lucene.demo.SearchFiles)
The Analyzer is the BrazilianAnalyzer avaliable in the sandbox
(org.apache.lucene.analysis.br.BrazilianAnalyzer)**
After looker with more attetion to the code i found that the tokenStream
method in the BrazilianAnalyzer calls the StandardTokenizer and is this
the one that split the search string, is there a simple way of subclass
the tokenizer to avoid splitting those characters or do i have make a
custom implementation of that class.

though it could be something fishy in how you got the string into
QueryParser in the first place?

As this only happends when i make a search (during indexing the
splitting of those characters doesn't happend) i thought that i had to
do with the QueryParser, but it seems that the problem is with the
StandardTokenizer.

Thanks

Erik Hatcher wrote:
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org

You can verify this by using the AnalysisDemo referenced here:

http://wiki.apache.org/jakarta-lucene/AnalysisParalysis

Or use Luke - http://www.getopt.org/luke/ - which has a nice plugin
page that can do this type of analysis inspection (you'll have to add
the sandbox analyzer JAR to the classpath when launching Luke).

As for subclassing StandardTokenizer - no, you won't have much luck
there. StandardTokenizer is a JavaCC-based tokenizer and is not
designed for subclassing to control this sort of thing.
Are you sure that splitting is not happening during indexing? If the
AnalysisDemo (or Luke) run on your string splits then it is splitting
at indexing time too. Keep in mind that looking at a field's value is
showing you the stored *original* value, not the tokenized values.
I'm not sure - I haven't tried that string with the analyzer you
provided. If it was with StandardTokenizer and you're using the same
analyzer for indexing and searching, you'd have the values split in
both places - which is actually fine as searches would match what was
indexed :)

Erik


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org

I tried luke and is great, i don't like the code but the tool is really
good.

I found the problem, but i don't understand why.

This was the old code (doesn't work) :

--------//-----------------
IndexSearcher searcher = new IndexSearcher("data");
BrazilianAnalyzer analyzer = new BrazilianAnalyzer();

// begin of code block that doesn't work
BufferedReader in = new BufferedReader(new InputStreamReader(System.in));
System.out.print("Query: ");
String line = in.readLine();
// end of code that doesn't work

Query query = QueryParser.parse(line, "contents", analyzer);
Hits hits = searcher.search(query);
System.out.println(hits.length() + " total matching documents");

-------------//------------
but if i replace the above code that doesn't work by this it works fine:

String line = "text to search";

This doesn't have anything to do with lucene, but why does it work if i
supply the code directly into the string and doesn't work usign the
inputstream?
Does it has something to do with the encoding or something like that ?
is it problem of the windows shell that passes the punctuation
characters in an incorrect way?

Thanks for your help

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org