FAQ
Position increments are for relative token positions. A position
increment of zero means that a token is logically at the same position
as the previous token. A position increment of one means that a token
immediately follows the preceding token in the stream, it's the next
token to the right (in a left-to-right language). A position increment
of two means that it is two tokens past the previous token, that there's
a "phantom" token between them, inhibiting exact phrase matches.

You're setting the position increment to things based on the number of
characters in the token's text. That makes no sense. Token positions
are not character positions. I think what you want to do is use a
positionIncrement of zero, so the tokens lie at the same position.

Doug

Allen Atamer wrote:
The FAQ describes implementing a TokenFilter for applying aliases. I have a
trouble accomplishing this.

This is the code that I have so far for the next Method within AliasFilter.
After reading some posts, I also got the idea to call
setPositionIncrement(). Neither way works, because when I search for the
alias, no search results come back.

Thank you for your help,

Allen Atamer

----

public Token next() throws java.io.IOException {
Token token = tokenStream.next();

if (aliasMap == null || token == null) {
return token;
}

TermData t = (TermData)aliasMap.get(token.termText());

if (t == null) {
return token;
}

String tokenText = AliasManager.replaceIgnoreCase(
token.termText(), t.getTerm(), t.getTeach());

int increment = tokenText.length() - token.termText().length();
if (increment > 0) {
token.setPositionIncrement(increment);
}

return new Token(tokenText, token.startOffset(), token.endOffset());
}





---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org

Search Discussions

Discussion Posts

Previous

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 6 of 6 | next ›
Discussion Overview
groupjava-user @
categorieslucene
postedDec 4, '03 at 9:58p
activeDec 5, '03 at 6:47p
posts6
users3
websitelucene.apache.org

People

Translate

site design / logo © 2021 Grokbase