Position increments are for relative token positions. A position
increment of zero means that a token is logically at the same position
as the previous token. A position increment of one means that a token
immediately follows the preceding token in the stream, it's the next
token to the right (in a left-to-right language). A position increment
of two means that it is two tokens past the previous token, that there's
a "phantom" token between them, inhibiting exact phrase matches.

You're setting the position increment to things based on the number of
characters in the token's text. That makes no sense. Token positions
are not character positions. I think what you want to do is use a
positionIncrement of zero, so the tokens lie at the same position.


Allen Atamer wrote:
The FAQ describes implementing a TokenFilter for applying aliases. I have a
trouble accomplishing this.

This is the code that I have so far for the next Method within AliasFilter.
After reading some posts, I also got the idea to call
setPositionIncrement(). Neither way works, because when I search for the
alias, no search results come back.

Thank you for your help,

Allen Atamer


public Token next() throws java.io.IOException {
Token token = tokenStream.next();

if (aliasMap == null || token == null) {
return token;

TermData t = (TermData)aliasMap.get(token.termText());

if (t == null) {
return token;

String tokenText = AliasManager.replaceIgnoreCase(
token.termText(), t.getTerm(), t.getTeach());

int increment = tokenText.length() - token.termText().length();
if (increment > 0) {

return new Token(tokenText, token.startOffset(), token.endOffset());

To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org

Search Discussions

Discussion Posts


Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 6 of 6 | next ›
Discussion Overview
groupjava-user @
postedDec 4, '03 at 9:58p
activeDec 5, '03 at 6:47p



site design / logo © 2021 Grokbase