FAQ
Catchall field has its own disadvantages of increasing index size.
MultiFieldQueryParser has to know the field names before hand.

How do I do a multi field search
- without knowing fields in the index
- without having a CatchAll field


PS : I went through the Lucene in Action book, and in chapter 13 ( SIREn
case study ), he says SIREn does it. I also wanted to know how can I do the
same with similar efficiency ?

--
View this message in context: http://lucene.472066.n3.nabble.com/How-to-do-an-all-field-search-without-using-a-catchall-field-or-MultiFieldQueryParser-tp2681163p2681163.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Search Discussions

  • Ian Lea at Mar 15, 2011 at 11:15 am
    IndexReader.getFieldNames()?

    public abstract Collection<String>
    getFieldNames(IndexReader.FieldOption fldOption)

    Use that to get indexed fields and pass that list to MultiFieldQueryParser.


    --
    Ian.
    On Tue, Mar 15, 2011 at 10:46 AM, shrinath.m wrote:
    Catchall field has its own disadvantages of increasing index size.
    MultiFieldQueryParser has to know the field names before hand.

    How do I do a multi field search
    - without knowing fields in the index
    - without having a CatchAll field


    PS : I went through the Lucene in Action book, and in chapter 13 ( SIREn
    case study ), he says SIREn does it. I also wanted to know how can I do the
    same with similar efficiency ?

    --
    View this message in context: http://lucene.472066.n3.nabble.com/How-to-do-an-all-field-search-without-using-a-catchall-field-or-MultiFieldQueryParser-tp2681163p2681163.html
    Sent from the Lucene - Java Users mailing list archive at Nabble.com.

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Shrinath.m at Mar 16, 2011 at 2:05 am
    Thanks Ian,

    I'd have to create a similarly sized array of Boolean.Occur too, isn't it?

    1. Is that how SIREn implements it?
    2. Is that optimal solution if I'm going to have an index of a billion docs with varying number of fields?


    --
    View this message in context: http://lucene.472066.n3.nabble.com/How-to-do-an-all-field-search-without-using-a-catchall-field-or-MultiFieldQueryParser-tp2681163p2685072.html
    Sent from the Lucene - Java Users mailing list archive at Nabble.com.
  • Ian Lea at Mar 17, 2011 at 11:18 am

    I'd have to create a similarly sized array of Boolean.Occur too, isn't it? What for?
    1. Is that how SIREn implements it? No idea.
    2. Is that optimal solution if I'm going to have an index of a billion docs with varying number of fields?
    Probably not. I always use a catchall field. I don't claim that is
    optimal, certainly not for your circumstances which I know nothing
    about, but it works for me. Sure, it makes the index bigger but not
    dramatically so, for my data anyway. There is no need to store it,
    just index it. And unless you need to search on the individual fields
    that contribute to the catchall you don't need them indexed.

    Disk space is cheap and I assume you have suitable hardware to support
    your billion doc index/sharded indexes.



    --
    Ian.

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Shrinath.m at Mar 17, 2011 at 11:52 am

    On Thu, Mar 17, 2011 at 4:48 PM, Ian Lea [via Lucene] wrote:

    I'd have to create a similarly sized array of Boolean.Occur too, isn't
    it?

    What for?

    http://lucene.apache.org/java/2_3_1/api/org/apache/lucene/queryParser/MultiFieldQueryParser.html

    <http://lucene.apache.org/java/2_3_1/api/org/apache/lucene/queryParser/MultiFieldQueryParser.html>Look
    at the parse method.

    --
    Regards
    Shrinath.M


    --
    View this message in context: http://lucene.472066.n3.nabble.com/How-to-do-an-all-field-search-without-using-a-catchall-field-or-MultiFieldQueryParser-tp2681163p2693112.html
    Sent from the Lucene - Java Users mailing list archive at Nabble.com.
  • Ian Lea at Mar 17, 2011 at 12:10 pm
    OK, yes, if you do need to "specify certain fields as required, and
    others as prohibited" then you'll need to build and populate that
    array. There is also a parse method that doesn't require the array if
    you just want to search all the fields.

    I hope that the 2_3_1 links don't indicate that you are using that version.


    --
    Ian.

    On Thu, Mar 17, 2011 at 11:51 AM, shrinath.m wrote:
    On Thu, Mar 17, 2011 at 4:48 PM, Ian Lea [via Lucene] <
    ml-node+2693036-1270863785-376162@n3.nabble.com> wrote:
    I'd have to create a similarly sized array of Boolean.Occur too, isn't
    it?

    What for?

    http://lucene.apache.org/java/2_3_1/api/org/apache/lucene/queryParser/MultiFieldQueryParser.html

    <http://lucene.apache.org/java/2_3_1/api/org/apache/lucene/queryParser/MultiFieldQueryParser.html>Look
    at the parse method.

    --
    Regards
    Shrinath.M


    --
    View this message in context: http://lucene.472066.n3.nabble.com/How-to-do-an-all-field-search-without-using-a-catchall-field-or-MultiFieldQueryParser-tp2681163p2693112.html
    Sent from the Lucene - Java Users mailing list archive at Nabble.com.
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Shrinath.m at Mar 17, 2011 at 12:30 pm

    On Thu, Mar 17, 2011 at 5:40 PM, Ian Lea [via Lucene] wrote:

    OK, yes, if you do need to "specify certain fields as required, and
    others as prohibited" then you'll need to build and populate that
    array. There is also a parse method that doesn't require the array if
    you just want to search all the fields.
    what ? where ?
    I don't see any such parse method that does a all field search !!
    I am seriously hoping there exists such a method and I might have missed it
    !! :)


    I hope that the 2_3_1 links don't indicate that you are using that version.

    yeah, my bad... I am using 3.0.3...
    http://lucene.apache.org/java/3_0_3/api/all/org/apache/lucene/queryParser/MultiFieldQueryParser.html

    <http://lucene.apache.org/java/3_0_3/api/all/org/apache/lucene/queryParser/MultiFieldQueryParser.html>


    --
    Regards
    Shrinath.M


    --
    View this message in context: http://lucene.472066.n3.nabble.com/How-to-do-an-all-field-search-without-using-a-catchall-field-or-MultiFieldQueryParser-tp2681163p2693226.html
    Sent from the Lucene - Java Users mailing list archive at Nabble.com.
  • Ian Lea at Mar 17, 2011 at 1:37 pm
    I was referring to the MultiFieldQueryParser.parse method that doesn't
    require an array of BooleanClause.Occur flags.

    static Query parse(Version matchVersion, String[] queries, String[]
    fields, Analyzer analyzer)
    Parses a query which searches on the fields specified.

    Good to hear you're on 3.0.3!


    --
    Ian.

    On Thu, Mar 17, 2011 at 12:30 PM, shrinath.m wrote:
    On Thu, Mar 17, 2011 at 5:40 PM, Ian Lea [via Lucene] <
    ml-node+2693169-1481141743-376162@n3.nabble.com> wrote:
    OK, yes, if you do need to "specify certain fields as required, and
    others as prohibited" then you'll need to build and populate that
    array.  There is also a parse method that doesn't require the array if
    you just want to search all the fields.
    what ? where ?
    I don't see any such parse method that does a all field search !!
    I am seriously hoping there exists such a method and I might have missed it
    !! :)


    I hope that the 2_3_1 links don't indicate that you are using that version.

    yeah, my bad... I am using 3.0.3...
    http://lucene.apache.org/java/3_0_3/api/all/org/apache/lucene/queryParser/MultiFieldQueryParser.html

    <http://lucene.apache.org/java/3_0_3/api/all/org/apache/lucene/queryParser/MultiFieldQueryParser.html>


    --
    Regards
    Shrinath.M


    --
    View this message in context: http://lucene.472066.n3.nabble.com/How-to-do-an-all-field-search-without-using-a-catchall-field-or-MultiFieldQueryParser-tp2681163p2693226.html
    Sent from the Lucene - Java Users mailing list archive at Nabble.com.
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Shrinath.m at Mar 17, 2011 at 1:46 pm

    On Thu, Mar 17, 2011 at 7:07 PM, Ian Lea [via Lucene] wrote:

    static Query parse(Version matchVersion, String[] queries, String[]
    fields, Analyzer analyzer)
    Parses a query which searches on the fields specified.

    Well, at the end of the day its either I make an array of Boolean.Occur
    or I make an array of Queries to go with the fields right ?
    Nothing helps me from creating an unnecessary array of repetitive elements ?
    :(

    So if I have 2 fields "name" and "address",
    and if the query is "shrinath",

    then there should be arrays like this :
    private static String[] fields = {"name", "address"};
    private static BooleanClause.Occur[] flags= {BooleanClause.Occur.SHOULD,
    BooleanClause.Occur.SHOULD};
    and the parse method should go like this :
    MultiFieldQueryParser.parse(Version.LUCENE_CURRENT, s, fields, flags, new
    StandardAnalyzer(Version.LUCENE_CURRENT, Collections.emptySet()));

    :(
    --
    Regards
    Shrinath.M


    --
    View this message in context: http://lucene.472066.n3.nabble.com/How-to-do-an-all-field-search-without-using-a-catchall-field-or-MultiFieldQueryParser-tp2681163p2693475.html
    Sent from the Lucene - Java Users mailing list archive at Nabble.com.
  • Ian Lea at Mar 17, 2011 at 2:11 pm
    if you go the MultiFieldQueryParser route you are going to need some
    arrays. You specifically asked if you needed an array of
    BooleanClause.Occur and the answer to that specific question is still
    no.

    I can't tell you any more on this than you can get from reading the javadocs.


    --
    Ian.


    On Thu, Mar 17, 2011 at 1:45 PM, shrinath.m wrote:
    On Thu, Mar 17, 2011 at 7:07 PM, Ian Lea [via Lucene] <
    ml-node+2693447-1032232993-376162@n3.nabble.com> wrote:
    static Query parse(Version matchVersion, String[] queries, String[]
    fields, Analyzer analyzer)
    Parses a query which searches on the fields specified.

    Well, at the end of the day its either I make an array of Boolean.Occur
    or I make an array of Queries to go with the fields right ?
    Nothing helps me from creating an unnecessary array of repetitive elements ?
    :(

    So if I have 2 fields "name" and "address",
    and if the query is "shrinath",

    then there should be arrays like this :
    private static String[] fields = {"name", "address"};
    private static BooleanClause.Occur[] flags= {BooleanClause.Occur.SHOULD,
    BooleanClause.Occur.SHOULD};
    and the parse method should go like this :
    MultiFieldQueryParser.parse(Version.LUCENE_CURRENT, s, fields, flags, new
    StandardAnalyzer(Version.LUCENE_CURRENT, Collections.emptySet()));

    :(
    --
    Regards
    Shrinath.M


    --
    View this message in context: http://lucene.472066.n3.nabble.com/How-to-do-an-all-field-search-without-using-a-catchall-field-or-MultiFieldQueryParser-tp2681163p2693475.html
    Sent from the Lucene - Java Users mailing list archive at Nabble.com.
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Shrinath.m at Mar 17, 2011 at 2:22 pm

    On Thu, Mar 17, 2011 at 7:42 PM, Ian Lea [via Lucene] wrote:

    if you go the MultiFieldQueryParser route you are going to need some
    arrays.
    Which other route can I take to do a all field search ?
    (Keeping in mind that I don't know what fields are in document)


    --
    Regards
    Shrinath.M


    --
    View this message in context: http://lucene.472066.n3.nabble.com/How-to-do-an-all-field-search-without-using-a-catchall-field-or-MultiFieldQueryParser-tp2681163p2693635.html
    Sent from the Lucene - Java Users mailing list archive at Nabble.com.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupjava-user @
categorieslucene
postedMar 15, '11 at 10:46a
activeMar 17, '11 at 2:22p
posts11
users2
websitelucene.apache.org

2 users in discussion

Shrinath.m: 6 posts Ian Lea: 5 posts

People

Translate

site design / logo © 2022 Grokbase