FAQ
hello;

i want to filter my tokens and keep only string tokens ( remove numbers
ect).
i sue this :

public TokenStream tokenStream(String fieldName, Reader reader) {
return new PorterStemFilter(
new StopFilter(
new LowerCaseFilter(
new StandardFilter(
new StandardTokenizer(reader))), stopset));
}



thanks

--
View this message in context: http://old.nabble.com/how-to-filter-numeric-values--tp27989882p27989882.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Search Discussions

  • Ahmet Arslan at Mar 22, 2010 at 5:48 pm

    hello;

    i want to filter my tokens and keep only string tokens (
    remove numbers
    ect).
    i sue this :

    public TokenStream tokenStream(String fieldName, Reader
    reader) {
    return new PorterStemFilter(
    new StopFilter(
    new LowerCaseFilter(
    new StandardFilter(
    new
    StandardTokenizer(reader))), stopset));
    }

    Why not use LowerCaseTokenizer [1] instead of StandardTokenizer + StandardFilter + LowerCaseFilter.

    [1]http://lucene.apache.org/java/2_9_2/api/core/org/apache/lucene/analysis/LowerCaseTokenizer.html




    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Juniol at Mar 22, 2010 at 6:09 pm
    hello thanks about the reply
    i found another solution:

    StopAnalyzer std1 = new StopAnalyzer(Version.LUCENE_CURRENT);
    PorterStemFilter std =new PorterStemFilter(std1.tokenStream("field",
    reader));




    juniol wrote:
    hello;

    i want to filter my tokens and keep only string tokens ( remove numbers
    ect).
    i use this :

    public TokenStream tokenStream(String fieldName, Reader reader) {
    return new PorterStemFilter(
    new StopFilter(
    new LowerCaseFilter(
    new StandardFilter(
    new StandardTokenizer(reader))), stopset));
    }



    thanks
    --
    View this message in context: http://old.nabble.com/how-to-filter-numeric-values--tp27989882p27990352.html
    Sent from the Lucene - Java Users mailing list archive at Nabble.com.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Erick Erickson at Mar 22, 2010 at 7:15 pm
    Why not just use SimpleAnalyzer? From the javadocs:
    An Analyzer<http://lucene.apache.org/java/3_0_1/api/all/org/apache/lucene/analysis/Analyzer.html>
    that
    filters LetterTokenizer<http://lucene.apache.org/java/3_0_1/api/all/org/apache/lucene/analysis/LetterTokenizer.html>
    with LowerCaseFilter<http://lucene.apache.org/java/3_0_1/api/all/org/apache/lucene/analysis/LowerCaseFilter.html>

    <http://lucene.apache.org/java/3_0_1/api/all/org/apache/lucene/analysis/LowerCaseFilter.html>
    Erick
    On Mon, Mar 22, 2010 at 2:09 PM, juniol wrote:


    hello thanks about the reply
    i found another solution:

    StopAnalyzer std1 = new StopAnalyzer(Version.LUCENE_CURRENT);
    PorterStemFilter std =new PorterStemFilter(std1.tokenStream("field",
    reader));




    juniol wrote:
    hello;

    i want to filter my tokens and keep only string tokens ( remove numbers
    ect).
    i use this :

    public TokenStream tokenStream(String fieldName, Reader reader) {
    return new PorterStemFilter(
    new StopFilter(
    new LowerCaseFilter(
    new StandardFilter(
    new StandardTokenizer(reader))), stopset));
    }



    thanks
    --
    View this message in context:
    http://old.nabble.com/how-to-filter-numeric-values--tp27989882p27990352.html
    Sent from the Lucene - Java Users mailing list archive at Nabble.com.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupjava-user @
categorieslucene
postedMar 22, '10 at 5:37p
activeMar 22, '10 at 7:15p
posts4
users3
websitelucene.apache.org

People

Translate

site design / logo © 2022 Grokbase