help customfilter with incrementToken() and AttributeSource APIs

The source code for LowerCaseFilter or StopFilter can be a good starting point.
DIGY

-----Original Message-----
From: maxSchlein
Sent: Thursday, December 24, 2009 7:10 PM
To: java-user@lucene.apache.org
Subject: help customfilter with incrementToken() and AttributeSource APIs


In the current version of lucene, 3.0 the following methods are no longer
available.
- TokenStream.next()
- TokenStream.next(Token).
- Token.setTermText()
- Token.termText().

The newer versions says to use, incrementToken() and AttributeSource APIs.
But I cannot find much help using these in this way. Any help again is
appreciated. If anyone has a basic example, or can point me to something
useful that would be awesome. Thanx.

Merry Christmas too.

public class CustomAnalyzer extends Analyzer
{
@Override
public TokenStream tokenStream(final String fieldName, final Reader
reader)
{
TokenStream ts = new WhitespaceTokenizer(reader);
ts = new StopFilter(false, ts, StopAnalyzer.ENGLISH_STOP_WORDS_SET);
ts = new LowerCaseFilter(ts);
ts = new CustomFilter(ts);

return ts;
}

}

public class CustomFilter extends TokenFilter
{
protected CustomFilter(TokenStream tokenStream)
{
super(tokenStream);
}
@Override
public Token next(final Token reusableToken) throws IOException
{
Token nextToken = input.next(reusableToken);

if(nextToken != null)
{

nextToken.setTermText(nextToken.termText().replaceAll(":|,|\\(|\\)|“|~|;|&|\\.",""));
}
return nextToken;
}
}