FAQ
On the link
http://lucene.apache.org/java/3_0_3/queryparsersyntax.html#Range%20Searches


There is ranged searched, how do I specify everything above a date from date 20020101 to end of time?



Next, I am temporarily using lucene in a noSQL solution(to switch to Solr later after prototype) and

So I am just indexing basic columns..no need for "top search results", etc.



When I look at the IndexSearcher and it's list of methods I am not sure how I can grab the first 100

Results, then the second 100 results(that is if I need them), then the third 100 results (again if needed)



I see a TopScoreDocCollector.create method but the IndexSearcher.search(Query, Collector) method states only to call that method if you need ALL the results. I definitely don't need all but need to page through the

Results and typically exit out around the third page. This is not a web app, so ideally I want a reference held into the indexed tree so it can keep giving me the next 100 results.



Thanks,

Dean

This message and any attachments are intended only for the use of the addressee and
may contain information that is privileged and confidential. If the reader of the
message is not the intended recipient or an authorized representative of the
intended recipient, you are hereby notified that any dissemination of this
communication is strictly prohibited. If you have received this communication in
error, please notify us immediately by e-mail and delete the message and any
attachments from your system.

Search Discussions

  • Simon Willnauer at Jun 19, 2011 at 5:48 pm

    On Sun, Jun 19, 2011 at 7:29 PM, Hiller, Dean x66079 wrote:
    On the link
    http://lucene.apache.org/java/3_0_3/queryparsersyntax.html#Range%20Searches


    There is ranged searched, how do I specify everything above a date from date 20020101  to end of time?
    here you can simply go for field:[20020101 TO ] and leave the end
    blank. If you want to do fast numeric searches you should use
    NumericRangeQuery instead.


    Next, I am temporarily using lucene in a noSQL solution(to switch to Solr later after prototype) and

    So I am just indexing basic columns..no need for "top search results", etc.



    When I look at the IndexSearcher and it's list of methods I am not sure how I can grab the first 100

    Results, then the second 100 results(that is if I need them), then the third 100 results (again if needed)
    so what you do here is basically requesting as many documents as you
    need lets say 100, then you display it. Once you need the next hundred
    you search again requesting 200 results and once the search returns
    simply discard the first 100
    use this as the basic method if you simply use a query without filters
    or anything.

    public TopDocs search(Query query, int n)


    I see a TopScoreDocCollector.create method but the IndexSearcher.search(Query, Collector) method states only to call that method if you need ALL the results.  I definitely don't need all but need to page through the

    Results and typically exit out around the third page.  This is not a web app, so ideally I want a reference held into the indexed tree so it can keep giving me the next 100 results.
    in lucene you must search again to the the next 100 but in general the
    search should be very fast.

    lemme know if you have more quesitons.

    simon


    Thanks,

    Dean

    This message and any attachments are intended only for the use of the addressee and
    may contain information that is privileged and confidential. If the reader of the
    message is not the intended recipient or an authorized representative of the
    intended recipient, you are hereby notified that any dissemination of this
    communication is strictly prohibited. If you have received this communication in
    error, please notify us immediately by e-mail and delete the message and any
    attachments from your system.
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Hiller, Dean x66079 at Jun 19, 2011 at 6:46 pm
    Hmmm, maybe I am using the wrong library?

    See the post I just sent especially on the hibernate section where in hibernate
    you can do select * from xxx where yyy and page the results(gets slower and slower
    as you go to the nth page) vs. using ScrollableResultSet in hibernate which does
    not get any slower as you move towards the nth page.

    I am not close to a web app search at all. More of a noSQL environment that I need indexing on.

    Thanks,
    Dean

    -----Original Message-----
    From: Simon Willnauer
    Sent: Sunday, June 19, 2011 11:48 AM
    To: java-user@lucene.apache.org
    Subject: Re: how to do simple search paging results of 100 each? and query syntax question

    On Sun, Jun 19, 2011 at 7:29 PM, Hiller, Dean x66079
    wrote:
    On the link
    http://lucene.apache.org/java/3_0_3/queryparsersyntax.html#Range%20Searches


    There is ranged searched, how do I specify everything above a date from date 20020101  to end of time?
    here you can simply go for field:[20020101 TO ] and leave the end
    blank. If you want to do fast numeric searches you should use
    NumericRangeQuery instead.


    Next, I am temporarily using lucene in a noSQL solution(to switch to Solr later after prototype) and

    So I am just indexing basic columns..no need for "top search results", etc.



    When I look at the IndexSearcher and it's list of methods I am not sure how I can grab the first 100

    Results, then the second 100 results(that is if I need them), then the third 100 results (again if needed)
    so what you do here is basically requesting as many documents as you
    need lets say 100, then you display it. Once you need the next hundred
    you search again requesting 200 results and once the search returns
    simply discard the first 100
    use this as the basic method if you simply use a query without filters
    or anything.

    public TopDocs search(Query query, int n)


    I see a TopScoreDocCollector.create method but the IndexSearcher.search(Query, Collector) method states only to call that method if you need ALL the results.  I definitely don't need all but need to page through the

    Results and typically exit out around the third page.  This is not a web app, so ideally I want a reference held into the indexed tree so it can keep giving me the next 100 results.
    in lucene you must search again to the the next 100 but in general the
    search should be very fast.

    lemme know if you have more quesitons.

    simon


    Thanks,

    Dean

    This message and any attachments are intended only for the use of the addressee and
    may contain information that is privileged and confidential. If the reader of the
    message is not the intended recipient or an authorized representative of the
    intended recipient, you are hereby notified that any dissemination of this
    communication is strictly prohibited. If you have received this communication in
    error, please notify us immediately by e-mail and delete the message and any
    attachments from your system.
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org


    This message and any attachments are intended only for the use of the addressee and
    may contain information that is privileged and confidential. If the reader of the
    message is not the intended recipient or an authorized representative of the
    intended recipient, you are hereby notified that any dissemination of this
    communication is strictly prohibited. If you have received this communication in
    error, please notify us immediately by e-mail and delete the message and any
    attachments from your system.
  • Simon Willnauer at Jun 19, 2011 at 7:48 pm
    So do you need to score the documents or can they be in arbitrary order?

    On Sun, Jun 19, 2011 at 8:45 PM, Hiller, Dean x66079
    wrote:
    Hmmm, maybe I am using the wrong library?

    See the post I just sent especially on the hibernate section where in hibernate
    you can do select * from xxx where yyy and page the results(gets slower and slower
    as you go to the nth page) vs. using ScrollableResultSet in hibernate which does
    not get any slower as you move towards the nth page.

    I am not close to a web app search at all.  More of a noSQL environment that I need indexing on.

    Thanks,
    Dean

    -----Original Message-----
    From: Simon Willnauer
    Sent: Sunday, June 19, 2011 11:48 AM
    To: java-user@lucene.apache.org
    Subject: Re: how to do simple search paging results of 100 each? and query syntax question

    On Sun, Jun 19, 2011 at 7:29 PM, Hiller, Dean  x66079
    wrote:
    On the link
    http://lucene.apache.org/java/3_0_3/queryparsersyntax.html#Range%20Searches


    There is ranged searched, how do I specify everything above a date from date 20020101  to end of time?
    here you can simply go for field:[20020101 TO ] and leave the end
    blank. If you want to do fast numeric searches you should use
    NumericRangeQuery instead.


    Next, I am temporarily using lucene in a noSQL solution(to switch to Solr later after prototype) and

    So I am just indexing basic columns..no need for "top search results", etc.



    When I look at the IndexSearcher and it's list of methods I am not sure how I can grab the first 100

    Results, then the second 100 results(that is if I need them), then the third 100 results (again if needed)
    so what you do here is basically requesting as many documents as you
    need lets say 100, then you display it. Once you need the next hundred
    you search again requesting 200 results and once the search returns
    simply discard the first 100
    use this as the basic method if you simply use a query without filters
    or anything.

    public TopDocs search(Query query, int n)


    I see a TopScoreDocCollector.create method but the IndexSearcher.search(Query, Collector) method states only to call that method if you need ALL the results.  I definitely don't need all but need to page through the

    Results and typically exit out around the third page.  This is not a web app, so ideally I want a reference held into the indexed tree so it can keep giving me the next 100 results.
    in lucene you must search again to the the next 100 but in general the
    search should be very fast.

    lemme know if you have more quesitons.

    simon


    Thanks,

    Dean

    This message and any attachments are intended only for the use of the addressee and
    may contain information that is privileged and confidential. If the reader of the
    message is not the intended recipient or an authorized representative of the
    intended recipient, you are hereby notified that any dissemination of this
    communication is strictly prohibited. If you have received this communication in
    error, please notify us immediately by e-mail and delete the message and any
    attachments from your system.
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org


    This message and any attachments are intended only for the use of the addressee and
    may contain information that is privileged and confidential. If the reader of the
    message is not the intended recipient or an authorized representative of the
    intended recipient, you are hereby notified that any dissemination of this
    communication is strictly prohibited. If you have received this communication in
    error, please notify us immediately by e-mail and delete the message and any
    attachments from your system.
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Hiller, Dean x66079 at Jun 19, 2011 at 8:11 pm
    No need to score at all. Just need paging and typically it is a loop since this is an overnight batch job pairing up a trade with one or many trades on a nosql system. We do need a sorted order as well so we kind of want to

    1. have something like ScrollableResultSet
    2. be able to pass in order by xxx
    3. page over the results without releasing the cursor until we are all matched up(which can typically be the third page once in a while is the 80th page).

    BIG NOTE: Our first page even with indexing is slow because the index is HUGE. The second page incurs that same hit if it starts over which is why ScrollableResultSet is very desired in the noSQL world. Ideally, we only want the first page to have that hit, and the second page picks up in the tree where the first one left off.

    I did post on the hbase list as I am curious if other noSQL users are starting to see this need yet. I am sure people well just as ScrollableResultSet was eventually added into hibernate.

    Thanks,
    Dean

    -----Original Message-----
    From: Simon Willnauer
    Sent: Sunday, June 19, 2011 1:48 PM
    To: Hiller, Dean x66079
    Cc: java-user@lucene.apache.org
    Subject: Re: how to do simple search paging results of 100 each? and query syntax question

    So do you need to score the documents or can they be in arbitrary order?

    On Sun, Jun 19, 2011 at 8:45 PM, Hiller, Dean x66079
    wrote:
    Hmmm, maybe I am using the wrong library?

    See the post I just sent especially on the hibernate section where in hibernate
    you can do select * from xxx where yyy and page the results(gets slower and slower
    as you go to the nth page) vs. using ScrollableResultSet in hibernate which does
    not get any slower as you move towards the nth page.

    I am not close to a web app search at all.  More of a noSQL environment that I need indexing on.

    Thanks,
    Dean

    -----Original Message-----
    From: Simon Willnauer
    Sent: Sunday, June 19, 2011 11:48 AM
    To: java-user@lucene.apache.org
    Subject: Re: how to do simple search paging results of 100 each? and query syntax question

    On Sun, Jun 19, 2011 at 7:29 PM, Hiller, Dean  x66079
    wrote:
    On the link
    http://lucene.apache.org/java/3_0_3/queryparsersyntax.html#Range%20Searches


    There is ranged searched, how do I specify everything above a date from date 20020101  to end of time?
    here you can simply go for field:[20020101 TO ] and leave the end
    blank. If you want to do fast numeric searches you should use
    NumericRangeQuery instead.


    Next, I am temporarily using lucene in a noSQL solution(to switch to Solr later after prototype) and

    So I am just indexing basic columns..no need for "top search results", etc.



    When I look at the IndexSearcher and it's list of methods I am not sure how I can grab the first 100

    Results, then the second 100 results(that is if I need them), then the third 100 results (again if needed)
    so what you do here is basically requesting as many documents as you
    need lets say 100, then you display it. Once you need the next hundred
    you search again requesting 200 results and once the search returns
    simply discard the first 100
    use this as the basic method if you simply use a query without filters
    or anything.

    public TopDocs search(Query query, int n)


    I see a TopScoreDocCollector.create method but the IndexSearcher.search(Query, Collector) method states only to call that method if you need ALL the results.  I definitely don't need all but need to page through the

    Results and typically exit out around the third page.  This is not a web app, so ideally I want a reference held into the indexed tree so it can keep giving me the next 100 results.
    in lucene you must search again to the the next 100 but in general the
    search should be very fast.

    lemme know if you have more quesitons.

    simon


    Thanks,

    Dean

    This message and any attachments are intended only for the use of the addressee and
    may contain information that is privileged and confidential. If the reader of the
    message is not the intended recipient or an authorized representative of the
    intended recipient, you are hereby notified that any dissemination of this
    communication is strictly prohibited. If you have received this communication in
    error, please notify us immediately by e-mail and delete the message and any
    attachments from your system.
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org


    This message and any attachments are intended only for the use of the addressee and
    may contain information that is privileged and confidential. If the reader of the
    message is not the intended recipient or an authorized representative of the
    intended recipient, you are hereby notified that any dissemination of this
    communication is strictly prohibited. If you have received this communication in
    error, please notify us immediately by e-mail and delete the message and any
    attachments from your system.
    This message and any attachments are intended only for the use of the addressee and
    may contain information that is privileged and confidential. If the reader of the
    message is not the intended recipient or an authorized representative of the
    intended recipient, you are hereby notified that any dissemination of this
    communication is strictly prohibited. If you have received this communication in
    error, please notify us immediately by e-mail and delete the message and any
    attachments from your system.
  • Sanne Grinovero at Jul 14, 2011 at 1:49 pm
    Hello,
    sorry for the late reply.
    I don't think that generally noSQL users need a ScrollableResult as
    usually NoSQL is being used in big data environments, in which case
    it's preferred to send your computation and data crunching to the data
    as with Map/Reduce operations (but not limited to) rather than
    fetching all the data locally.
    This is just my opinion in the general case. With Lucene specifically
    you can implement something similar, and in fact we implemented
    exactly Hibernate's interface in Hibernate Search, which is providing
    the full JPA api to a Lucene index; feel free to have a look:

    Implementation:
    https://github.com/hibernate/hibernate-search/blob/master/hibernate-search/src/main/java/org/hibernate/search/query/hibernate/impl/ScrollableResultsImpl.java

    Tests:
    https://github.com/hibernate/hibernate-search/blob/master/hibernate-search/src/test/java/org/hibernate/search/test/query/ScrollableResultsTest.java

    This code is implementing the full Hibernate semantics (including
    keeping loaded entities attached to the session), so if you don't need
    that you could extract the logic into something much simpler - or use
    it directly and clear the Session regularly, as with all batch
    operations.

    Generally I think (and hope!) this implementation makes sense as the
    goal is to facilitate developers having a JPA or Hibernate
    applications to get started with Lucene.

    Regards,
    Sanne

    2011/6/19 Hiller, Dean x66079 <dean.hiller@broadridge.com>:
    No need to score at all.  Just need paging and typically it is a loop since this is an overnight batch job pairing up a trade with one or many trades on a nosql system.  We do need a sorted order as well so we kind of want to

    1. have something like ScrollableResultSet
    2. be able to pass in order by xxx
    3. page over the results without releasing the cursor until we are all matched up(which can typically be the third page once in a while is the 80th page).

    BIG NOTE: Our first page even with indexing is slow because the index is HUGE.  The second page incurs that same hit if it starts over which is why ScrollableResultSet is very desired in the noSQL world.  Ideally, we only want the first page to have that hit, and the second page picks up in the tree where the first one left off.

    I did post on the hbase list as I am curious if other noSQL users are starting to see this need yet.  I am sure people well just as ScrollableResultSet was eventually added into hibernate.

    Thanks,
    Dean

    -----Original Message-----
    From: Simon Willnauer
    Sent: Sunday, June 19, 2011 1:48 PM
    To: Hiller, Dean x66079
    Cc: java-user@lucene.apache.org
    Subject: Re: how to do simple search paging results of 100 each? and query syntax question

    So do you need to score the documents or can they be in arbitrary order?

    On Sun, Jun 19, 2011 at 8:45 PM, Hiller, Dean  x66079
    wrote:
    Hmmm, maybe I am using the wrong library?

    See the post I just sent especially on the hibernate section where in hibernate
    you can do select * from xxx where yyy and page the results(gets slower and slower
    as you go to the nth page) vs. using ScrollableResultSet in hibernate which does
    not get any slower as you move towards the nth page.

    I am not close to a web app search at all.  More of a noSQL environment that I need indexing on.

    Thanks,
    Dean

    -----Original Message-----
    From: Simon Willnauer
    Sent: Sunday, June 19, 2011 11:48 AM
    To: java-user@lucene.apache.org
    Subject: Re: how to do simple search paging results of 100 each? and query syntax question

    On Sun, Jun 19, 2011 at 7:29 PM, Hiller, Dean  x66079
    wrote:
    On the link
    http://lucene.apache.org/java/3_0_3/queryparsersyntax.html#Range%20Searches


    There is ranged searched, how do I specify everything above a date from date 20020101  to end of time?
    here you can simply go for field:[20020101 TO ] and leave the end
    blank. If you want to do fast numeric searches you should use
    NumericRangeQuery instead.


    Next, I am temporarily using lucene in a noSQL solution(to switch to Solr later after prototype) and

    So I am just indexing basic columns..no need for "top search results", etc.



    When I look at the IndexSearcher and it's list of methods I am not sure how I can grab the first 100

    Results, then the second 100 results(that is if I need them), then the third 100 results (again if needed)
    so what you do here is basically requesting as many documents as you
    need lets say 100, then you display it. Once you need the next hundred
    you search again requesting 200 results and once the search returns
    simply discard the first 100
    use this as the basic method if you simply use a query without filters
    or anything.

    public TopDocs search(Query query, int n)


    I see a TopScoreDocCollector.create method but the IndexSearcher.search(Query, Collector) method states only to call that method if you need ALL the results.  I definitely don't need all but need to page through the

    Results and typically exit out around the third page.  This is not a web app, so ideally I want a reference held into the indexed tree so it can keep giving me the next 100 results.
    in lucene you must search again to the the next 100 but in general the
    search should be very fast.

    lemme know if you have more quesitons.

    simon


    Thanks,

    Dean

    This message and any attachments are intended only for the use of the addressee and
    may contain information that is privileged and confidential. If the reader of the
    message is not the intended recipient or an authorized representative of the
    intended recipient, you are hereby notified that any dissemination of this
    communication is strictly prohibited. If you have received this communication in
    error, please notify us immediately by e-mail and delete the message and any
    attachments from your system.
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org


    This message and any attachments are intended only for the use of the addressee and
    may contain information that is privileged and confidential. If the reader of the
    message is not the intended recipient or an authorized representative of the
    intended recipient, you are hereby notified that any dissemination of this
    communication is strictly prohibited. If you have received this communication in
    error, please notify us immediately by e-mail and delete the message and any
    attachments from your system.
    This message and any attachments are intended only for the use of the addressee and
    may contain information that is privileged and confidential. If the reader of the
    message is not the intended recipient or an authorized representative of the
    intended recipient, you are hereby notified that any dissemination of this
    communication is strictly prohibited. If you have received this communication in
    error, please notify us immediately by e-mail and delete the message and any
    attachments from your system.
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Hiller, Dean x66079 at Jun 19, 2011 at 10:19 pm
    "here you can simply go for field:[20020101 TO ] and leave the end
    blank. If you want to do fast numeric searches you should use
    NumericRangeQuery instead."

    I tried that and got an exception...

    I need to do something like String queryStr = "author:dean and date:[" + date1 + " TO ]"; so can I do a combination of NumericRangeQuery and text query and is there any example on this?

    org.apache.lucene.queryParser.ParseException: Cannot parse 'author:dean and date:[20110622 TO ]': Encountered " "]" "] "" at line 1, column 34.
    Was expecting one of:
    <RANGEIN_QUOTED> ...
    <RANGEIN_GOOP> ...

    at org.apache.lucene.queryParser.QueryParser.parse(QueryParser.java:211)
    at com.broadridge.papr1.test.other.PlayWithLuceneTest.runSearch(PlayWithLuceneTest.java:82)
    at com.broadridge.papr1.test.other.PlayWithLuceneTest.playWithLucene(PlayWithLuceneTest.java:59)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
    at org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4ClassRunner.java:79)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:71)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:49)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
    at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:49)
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
    Caused by: org.apache.lucene.queryParser.ParseException: Encountered " "]" "] "" at line 1, column 34.
    Was expecting one of:
    <RANGEIN_QUOTED> ...
    <RANGEIN_GOOP> ...

    at org.apache.lucene.queryParser.QueryParser.generateParseException(QueryParser.java:1818)
    at org.apache.lucene.queryParser.QueryParser.jj_consume_token(QueryParser.java:1700)
    at org.apache.lucene.queryParser.QueryParser.Term(QueryParser.java:1455)
    at org.apache.lucene.queryParser.QueryParser.Clause(QueryParser.java:1309)
    at org.apache.lucene.queryParser.QueryParser.Query(QueryParser.java:1266)
    at org.apache.lucene.queryParser.QueryParser.TopLevelQuery(QueryParser.java:1226)
    at org.apache.lucene.queryParser.QueryParser.parse(QueryParser.java:206)
    ... 25 more


    -----Original Message-----
    From: Simon Willnauer
    Sent: Sunday, June 19, 2011 11:48 AM
    To: java-user@lucene.apache.org
    Subject: Re: how to do simple search paging results of 100 each? and query syntax question

    On Sun, Jun 19, 2011 at 7:29 PM, Hiller, Dean x66079
    wrote:
    On the link
    http://lucene.apache.org/java/3_0_3/queryparsersyntax.html#Range%20Searches


    There is ranged searched, how do I specify everything above a date from date 20020101  to end of time?


    Next, I am temporarily using lucene in a noSQL solution(to switch to Solr later after prototype) and

    So I am just indexing basic columns..no need for "top search results", etc.



    When I look at the IndexSearcher and it's list of methods I am not sure how I can grab the first 100

    Results, then the second 100 results(that is if I need them), then the third 100 results (again if needed)
    so what you do here is basically requesting as many documents as you
    need lets say 100, then you display it. Once you need the next hundred
    you search again requesting 200 results and once the search returns
    simply discard the first 100
    use this as the basic method if you simply use a query without filters
    or anything.

    public TopDocs search(Query query, int n)


    I see a TopScoreDocCollector.create method but the IndexSearcher.search(Query, Collector) method states only to call that method if you need ALL the results.  I definitely don't need all but need to page through the

    Results and typically exit out around the third page.  This is not a web app, so ideally I want a reference held into the indexed tree so it can keep giving me the next 100 results.
    in lucene you must search again to the the next 100 but in general the
    search should be very fast.

    lemme know if you have more quesitons.

    simon


    Thanks,

    Dean

    This message and any attachments are intended only for the use of the addressee and
    may contain information that is privileged and confidential. If the reader of the
    message is not the intended recipient or an authorized representative of the
    intended recipient, you are hereby notified that any dissemination of this
    communication is strictly prohibited. If you have received this communication in
    error, please notify us immediately by e-mail and delete the message and any
    attachments from your system.
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org


    This message and any attachments are intended only for the use of the addressee and
    may contain information that is privileged and confidential. If the reader of the
    message is not the intended recipient or an authorized representative of the
    intended recipient, you are hereby notified that any dissemination of this
    communication is strictly prohibited. If you have received this communication in
    error, please notify us immediately by e-mail and delete the message and any
    attachments from your system.
  • Michael Sokolov at Jun 19, 2011 at 10:26 pm
    I think you need

    field:[20020101 TO *]

    although the "*" option isn't available in some versions (pre 3.1?) and
    you just have to supply a big value:

    field:[20020101 TO 99999999]

    -Mike
    On 6/19/2011 6:18 PM, Hiller, Dean x66079 wrote:
    "here you can simply go for field:[20020101 TO ] and leave the end
    blank. If you want to do fast numeric searches you should use
    NumericRangeQuery instead."

    I tried that and got an exception...

    I need to do something like String queryStr = "author:dean and date:[" + date1 + " TO ]"; so can I do a combination of NumericRangeQuery and text query and is there any example on this?

    org.apache.lucene.queryParser.ParseException: Cannot parse 'author:dean and date:[20110622 TO ]': Encountered " "]" "] "" at line 1, column 34.
    Was expecting one of:
    <RANGEIN_QUOTED> ...
    <RANGEIN_GOOP> ...

    at org.apache.lucene.queryParser.QueryParser.parse(QueryParser.java:211)
    at com.broadridge.papr1.test.other.PlayWithLuceneTest.runSearch(PlayWithLuceneTest.java:82)
    at com.broadridge.papr1.test.other.PlayWithLuceneTest.playWithLucene(PlayWithLuceneTest.java:59)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
    at org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4ClassRunner.java:79)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:71)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:49)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
    at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:49)
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
    Caused by: org.apache.lucene.queryParser.ParseException: Encountered " "]" "] "" at line 1, column 34.
    Was expecting one of:
    <RANGEIN_QUOTED> ...
    <RANGEIN_GOOP> ...

    at org.apache.lucene.queryParser.QueryParser.generateParseException(QueryParser.java:1818)
    at org.apache.lucene.queryParser.QueryParser.jj_consume_token(QueryParser.java:1700)
    at org.apache.lucene.queryParser.QueryParser.Term(QueryParser.java:1455)
    at org.apache.lucene.queryParser.QueryParser.Clause(QueryParser.java:1309)
    at org.apache.lucene.queryParser.QueryParser.Query(QueryParser.java:1266)
    at org.apache.lucene.queryParser.QueryParser.TopLevelQuery(QueryParser.java:1226)
    at org.apache.lucene.queryParser.QueryParser.parse(QueryParser.java:206)
    ... 25 more


    -----Original Message-----
    From: Simon Willnauer
    Sent: Sunday, June 19, 2011 11:48 AM
    To: java-user@lucene.apache.org
    Subject: Re: how to do simple search paging results of 100 each? and query syntax question

    On Sun, Jun 19, 2011 at 7:29 PM, Hiller, Dean x66079
    wrote:
    On the link
    http://lucene.apache.org/java/3_0_3/queryparsersyntax.html#Range%20Searches


    There is ranged searched, how do I specify everything above a date from date 20020101 to end of time?


    Next, I am temporarily using lucene in a noSQL solution(to switch to Solr later after prototype) and

    So I am just indexing basic columns..no need for "top search results", etc.



    When I look at the IndexSearcher and it's list of methods I am not sure how I can grab the first 100

    Results, then the second 100 results(that is if I need them), then the third 100 results (again if needed)
    so what you do here is basically requesting as many documents as you
    need lets say 100, then you display it. Once you need the next hundred
    you search again requesting 200 results and once the search returns
    simply discard the first 100
    use this as the basic method if you simply use a query without filters
    or anything.

    public TopDocs search(Query query, int n)

    I see a TopScoreDocCollector.create method but the IndexSearcher.search(Query, Collector) method states only to call that method if you need ALL the results. I definitely don't need all but need to page through the

    Results and typically exit out around the third page. This is not a web app, so ideally I want a reference held into the indexed tree so it can keep giving me the next 100 results.
    in lucene you must search again to the the next 100 but in general the
    search should be very fast.

    lemme know if you have more quesitons.

    simon

    Thanks,

    Dean

    This message and any attachments are intended only for the use of the addressee and
    may contain information that is privileged and confidential. If the reader of the
    message is not the intended recipient or an authorized representative of the
    intended recipient, you are hereby notified that any dissemination of this
    communication is strictly prohibited. If you have received this communication in
    error, please notify us immediately by e-mail and delete the message and any
    attachments from your system.
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org


    This message and any attachments are intended only for the use of the addressee and
    may contain information that is privileged and confidential. If the reader of the
    message is not the intended recipient or an authorized representative of the
    intended recipient, you are hereby notified that any dissemination of this
    communication is strictly prohibited. If you have received this communication in
    error, please notify us immediately by e-mail and delete the message and any
    attachments from your system.

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Hiller, Dean x66079 at Jun 19, 2011 at 10:37 pm
    Sweeeet, * worked, but SHUCKS.....query completely returns no results. I don't get it...it gets into the lucene code and since there is no "Scorer" returned, it seems to skip over any results that would be valid :(....any ideas what is wrong with this program....

    MyCollector.collect(int docId) never gets called :(.

    package com.broadridge.papr1.test.other;

    import java.io.File;
    import java.io.IOException;

    import org.apache.lucene.analysis.standard.StandardAnalyzer;
    import org.apache.lucene.document.Document;
    import org.apache.lucene.document.Field;
    import org.apache.lucene.index.IndexReader;
    import org.apache.lucene.index.IndexWriter;
    import org.apache.lucene.index.IndexWriterConfig;
    import org.apache.lucene.queryParser.ParseException;
    import org.apache.lucene.queryParser.QueryParser;
    import org.apache.lucene.search.Collector;
    import org.apache.lucene.search.IndexSearcher;
    import org.apache.lucene.search.Query;
    import org.apache.lucene.search.Scorer;
    import org.apache.lucene.store.Directory;
    import org.apache.lucene.store.FSDirectory;
    import org.apache.lucene.util.Version;
    import org.joda.time.LocalDate;
    import org.joda.time.format.DateTimeFormat;
    import org.joda.time.format.DateTimeFormatter;
    import org.junit.Test;
    import org.slf4j.Logger;
    import org.slf4j.LoggerFactory;

    public class PlayWithLuceneTest {

    private final static Logger log = LoggerFactory
    .getLogger(PlayWithLuceneTest.class);

    private static final String INDEX_DIR_PREFIX = "output/lucene/lucene-index";
    private static final DateTimeFormatter FMT2 = DateTimeFormat
    .forPattern("yyyyMMdd");

    @Test
    public void playWithLucene() throws Exception {
    IndexWriterConfig conf = new IndexWriterConfig(Version.LUCENE_32,
    new StandardAnalyzer(Version.LUCENE_32));

    Directory dir = FSDirectory.open(new File(INDEX_DIR_PREFIX));
    IndexWriter writer = new IndexWriter(dir, conf);

    LocalDate date = new LocalDate();

    createAndIndex(writer, "title1", "dean", date, "entry1");
    createAndIndex(writer, "title1", "dean2", date.plusDays(8), "entry2");
    createAndIndex(writer, "title1", "dean", date.plusDays(4), "entry3");
    createAndIndex(writer, "title1", "dean2", date.plusDays(5), "entry4");
    createAndIndex(writer, "title1", "dean2", date.plusDays(2), "entry5");
    createAndIndex(writer, "title1", "dean2", date.plusDays(6), "entry6");
    createAndIndex(writer, "title1", "dean", date.plusDays(9), "entry7");
    createAndIndex(writer, "title1", "dean2", date.plusDays(5), "entry8");

    writer.optimize();
    writer.close();

    runSearch();
    }

    private void createAndIndex(IndexWriter writer,
    String title, String author, LocalDate date, String result)
    throws IOException {
    Document document = createDocument(title,
    author, result, date);
    writer.addDocument(document);
    }

    private void runSearch() throws IOException,
    ParseException {
    LocalDate date = new LocalDate();

    Directory dir = FSDirectory.open(new File(INDEX_DIR_PREFIX));
    IndexSearcher is = new IndexSearcher(dir, true);
    QueryParser parser = new QueryParser(Version.LUCENE_32, "hello",
    new StandardAnalyzer(Version.LUCENE_32));

    String date1 = FMT2.print(date.plusDays(5));

    String queryStr = "date:[" + date1 + " TO * ]";
    log.info("query=" + queryStr);
    Query query = parser.parse(queryStr);

    MyCollector collector = new MyCollector(is);
    is.search(query, collector);

    is.close();
    }

    private static class MyCollector extends Collector {
    private IndexSearcher is;

    public MyCollector(IndexSearcher is) {
    this.is = is;
    }

    @Override
    public void setScorer(Scorer scorer) throws IOException {
    log.info("scorer=" + scorer);
    }

    @Override
    public void collect(int doc) throws IOException {
    Document doc2 = this.is.doc(doc);
    log.info("collect=" + doc + " entry=" + doc2.get("url"));
    }

    @Override
    public void setNextReader(IndexReader reader, int docBase)
    throws IOException {
    log.info("reader = " + reader + " docBase=" + docBase);
    }

    @Override
    public boolean acceptsDocsOutOfOrder() {
    return false;
    }
    }

    private Document createDocument(String title, String author,
    String url, LocalDate dateWritten) {

    String dateTxt = FMT2.print(dateWritten);

    Document document = new Document();
    document.add(new Field("author", author, Field.Store.NO,
    Field.Index.NOT_ANALYZED));
    document.add(new Field("title", title, Field.Store.NO,
    Field.Index.NOT_ANALYZED));
    document.add(new Field("date", dateTxt, Field.Store.NO,
    Field.Index.NOT_ANALYZED));

    log.info("created doc=" + dateTxt + " entry+" + url);
    document.add(new Field("url", url, Field.Store.YES, Field.Index.NO));
    return document;
    }
    }

    -----Original Message-----
    From: Michael Sokolov
    Sent: Sunday, June 19, 2011 4:25 PM
    To: java-user@lucene.apache.org
    Cc: Hiller, Dean x66079
    Subject: Re: ranged query didn't work, got exception...

    I think you need

    field:[20020101 TO *]

    although the "*" option isn't available in some versions (pre 3.1?) and
    you just have to supply a big value:

    field:[20020101 TO 99999999]

    -Mike
    On 6/19/2011 6:18 PM, Hiller, Dean x66079 wrote:
    "here you can simply go for field:[20020101 TO ] and leave the end
    blank. If you want to do fast numeric searches you should use
    NumericRangeQuery instead."

    I tried that and got an exception...

    I need to do something like String queryStr = "author:dean and date:[" + date1 + " TO ]"; so can I do a combination of NumericRangeQuery and text query and is there any example on this?

    org.apache.lucene.queryParser.ParseException: Cannot parse 'author:dean and date:[20110622 TO ]': Encountered " "]" "] "" at line 1, column 34.
    Was expecting one of:
    <RANGEIN_QUOTED> ...
    <RANGEIN_GOOP> ...

    at org.apache.lucene.queryParser.QueryParser.parse(QueryParser.java:211)
    at com.broadridge.papr1.test.other.PlayWithLuceneTest.runSearch(PlayWithLuceneTest.java:82)
    at com.broadridge.papr1.test.other.PlayWithLuceneTest.playWithLucene(PlayWithLuceneTest.java:59)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
    at org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4ClassRunner.java:79)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:71)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:49)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
    at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:49)
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
    Caused by: org.apache.lucene.queryParser.ParseException: Encountered " "]" "] "" at line 1, column 34.
    Was expecting one of:
    <RANGEIN_QUOTED> ...
    <RANGEIN_GOOP> ...

    at org.apache.lucene.queryParser.QueryParser.generateParseException(QueryParser.java:1818)
    at org.apache.lucene.queryParser.QueryParser.jj_consume_token(QueryParser.java:1700)
    at org.apache.lucene.queryParser.QueryParser.Term(QueryParser.java:1455)
    at org.apache.lucene.queryParser.QueryParser.Clause(QueryParser.java:1309)
    at org.apache.lucene.queryParser.QueryParser.Query(QueryParser.java:1266)
    at org.apache.lucene.queryParser.QueryParser.TopLevelQuery(QueryParser.java:1226)
    at org.apache.lucene.queryParser.QueryParser.parse(QueryParser.java:206)
    ... 25 more


    -----Original Message-----
    From: Simon Willnauer
    Sent: Sunday, June 19, 2011 11:48 AM
    To: java-user@lucene.apache.org
    Subject: Re: how to do simple search paging results of 100 each? and query syntax question

    On Sun, Jun 19, 2011 at 7:29 PM, Hiller, Dean x66079
    wrote:
    On the link
    http://lucene.apache.org/java/3_0_3/queryparsersyntax.html#Range%20Searches


    There is ranged searched, how do I specify everything above a date from date 20020101 to end of time?


    Next, I am temporarily using lucene in a noSQL solution(to switch to Solr later after prototype) and

    So I am just indexing basic columns..no need for "top search results", etc.



    When I look at the IndexSearcher and it's list of methods I am not sure how I can grab the first 100

    Results, then the second 100 results(that is if I need them), then the third 100 results (again if needed)
    so what you do here is basically requesting as many documents as you
    need lets say 100, then you display it. Once you need the next hundred
    you search again requesting 200 results and once the search returns
    simply discard the first 100
    use this as the basic method if you simply use a query without filters
    or anything.

    public TopDocs search(Query query, int n)

    I see a TopScoreDocCollector.create method but the IndexSearcher.search(Query, Collector) method states only to call that method if you need ALL the results. I definitely don't need all but need to page through the

    Results and typically exit out around the third page. This is not a web app, so ideally I want a reference held into the indexed tree so it can keep giving me the next 100 results.
    in lucene you must search again to the the next 100 but in general the
    search should be very fast.

    lemme know if you have more quesitons.

    simon

    Thanks,

    Dean

    This message and any attachments are intended only for the use of the addressee and
    may contain information that is privileged and confidential. If the reader of the
    message is not the intended recipient or an authorized representative of the
    intended recipient, you are hereby notified that any dissemination of this
    communication is strictly prohibited. If you have received this communication in
    error, please notify us immediately by e-mail and delete the message and any
    attachments from your system.
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org


    This message and any attachments are intended only for the use of the addressee and
    may contain information that is privileged and confidential. If the reader of the
    message is not the intended recipient or an authorized representative of the
    intended recipient, you are hereby notified that any dissemination of this
    communication is strictly prohibited. If you have received this communication in
    error, please notify us immediately by e-mail and delete the message and any
    attachments from your system.

    This message and any attachments are intended only for the use of the addressee and
    may contain information that is privileged and confidential. If the reader of the
    message is not the intended recipient or an authorized representative of the
    intended recipient, you are hereby notified that any dissemination of this
    communication is strictly prohibited. If you have received this communication in
    error, please notify us immediately by e-mail and delete the message and any
    attachments from your system.
  • Hiller, Dean x66079 at Jun 20, 2011 at 12:12 am
    Oddly, enough, this seems to work and I get one result calling Collector.collect(int docIt)...(I found out AND has to be caps)...

    author:dean AND date:20110623

    but this does not seem to work...

    author:dean AND date:[ 20110623 TO * ]

    You would think this second query would return the single result that was returned in the first query, wouldn't it??? I mean it is asking for the same date inclusive and more...why does Collector.collect never get called like nothing matches....stepping through the Lucene code, Scorer is null for some reason and it skips collecting everything because of that. Any ideas?

    Thanks,
    Dean

    -----Original Message-----
    From: Hiller, Dean x66079
    Sent: Sunday, June 19, 2011 4:37 PM
    To: java-user@lucene.apache.org
    Subject: RE: ranged query didn't work, got exception...

    Sweeeet, * worked, but SHUCKS.....query completely returns no results. I don't get it...it gets into the lucene code and since there is no "Scorer" returned, it seems to skip over any results that would be valid :(....any ideas what is wrong with this program....

    MyCollector.collect(int docId) never gets called :(.

    package com.broadridge.papr1.test.other;

    import java.io.File;
    import java.io.IOException;

    import org.apache.lucene.analysis.standard.StandardAnalyzer;
    import org.apache.lucene.document.Document;
    import org.apache.lucene.document.Field;
    import org.apache.lucene.index.IndexReader;
    import org.apache.lucene.index.IndexWriter;
    import org.apache.lucene.index.IndexWriterConfig;
    import org.apache.lucene.queryParser.ParseException;
    import org.apache.lucene.queryParser.QueryParser;
    import org.apache.lucene.search.Collector;
    import org.apache.lucene.search.IndexSearcher;
    import org.apache.lucene.search.Query;
    import org.apache.lucene.search.Scorer;
    import org.apache.lucene.store.Directory;
    import org.apache.lucene.store.FSDirectory;
    import org.apache.lucene.util.Version;
    import org.joda.time.LocalDate;
    import org.joda.time.format.DateTimeFormat;
    import org.joda.time.format.DateTimeFormatter;
    import org.junit.Test;
    import org.slf4j.Logger;
    import org.slf4j.LoggerFactory;

    public class PlayWithLuceneTest {

    private final static Logger log = LoggerFactory
    .getLogger(PlayWithLuceneTest.class);

    private static final String INDEX_DIR_PREFIX = "output/lucene/lucene-index";
    private static final DateTimeFormatter FMT2 = DateTimeFormat
    .forPattern("yyyyMMdd");

    @Test
    public void playWithLucene() throws Exception {
    IndexWriterConfig conf = new IndexWriterConfig(Version.LUCENE_32,
    new StandardAnalyzer(Version.LUCENE_32));

    Directory dir = FSDirectory.open(new File(INDEX_DIR_PREFIX));
    IndexWriter writer = new IndexWriter(dir, conf);

    LocalDate date = new LocalDate();

    createAndIndex(writer, "title1", "dean", date, "entry1");
    createAndIndex(writer, "title1", "dean2", date.plusDays(8), "entry2");
    createAndIndex(writer, "title1", "dean", date.plusDays(4), "entry3");
    createAndIndex(writer, "title1", "dean2", date.plusDays(5), "entry4");
    createAndIndex(writer, "title1", "dean2", date.plusDays(2), "entry5");
    createAndIndex(writer, "title1", "dean2", date.plusDays(6), "entry6");
    createAndIndex(writer, "title1", "dean", date.plusDays(9), "entry7");
    createAndIndex(writer, "title1", "dean2", date.plusDays(5), "entry8");

    writer.optimize();
    writer.close();

    runSearch();
    }

    private void createAndIndex(IndexWriter writer,
    String title, String author, LocalDate date, String result)
    throws IOException {
    Document document = createDocument(title,
    author, result, date);
    writer.addDocument(document);
    }

    private void runSearch() throws IOException,
    ParseException {
    LocalDate date = new LocalDate();

    Directory dir = FSDirectory.open(new File(INDEX_DIR_PREFIX));
    IndexSearcher is = new IndexSearcher(dir, true);
    QueryParser parser = new QueryParser(Version.LUCENE_32, "hello",
    new StandardAnalyzer(Version.LUCENE_32));

    String date1 = FMT2.print(date.plusDays(5));

    String queryStr = "date:[" + date1 + " TO * ]";
    log.info("query=" + queryStr);
    Query query = parser.parse(queryStr);

    MyCollector collector = new MyCollector(is);
    is.search(query, collector);

    is.close();
    }

    private static class MyCollector extends Collector {
    private IndexSearcher is;

    public MyCollector(IndexSearcher is) {
    this.is = is;
    }

    @Override
    public void setScorer(Scorer scorer) throws IOException {
    log.info("scorer=" + scorer);
    }

    @Override
    public void collect(int doc) throws IOException {
    Document doc2 = this.is.doc(doc);
    log.info("collect=" + doc + " entry=" + doc2.get("url"));
    }

    @Override
    public void setNextReader(IndexReader reader, int docBase)
    throws IOException {
    log.info("reader = " + reader + " docBase=" + docBase);
    }

    @Override
    public boolean acceptsDocsOutOfOrder() {
    return false;
    }
    }

    private Document createDocument(String title, String author,
    String url, LocalDate dateWritten) {

    String dateTxt = FMT2.print(dateWritten);

    Document document = new Document();
    document.add(new Field("author", author, Field.Store.NO,
    Field.Index.NOT_ANALYZED));
    document.add(new Field("title", title, Field.Store.NO,
    Field.Index.NOT_ANALYZED));
    document.add(new Field("date", dateTxt, Field.Store.NO,
    Field.Index.NOT_ANALYZED));

    log.info("created doc=" + dateTxt + " entry+" + url);
    document.add(new Field("url", url, Field.Store.YES, Field.Index.NO));
    return document;
    }
    }

    -----Original Message-----
    From: Michael Sokolov
    Sent: Sunday, June 19, 2011 4:25 PM
    To: java-user@lucene.apache.org
    Cc: Hiller, Dean x66079
    Subject: Re: ranged query didn't work, got exception...

    I think you need

    field:[20020101 TO *]

    although the "*" option isn't available in some versions (pre 3.1?) and
    you just have to supply a big value:

    field:[20020101 TO 99999999]

    -Mike
    On 6/19/2011 6:18 PM, Hiller, Dean x66079 wrote:
    "here you can simply go for field:[20020101 TO ] and leave the end
    blank. If you want to do fast numeric searches you should use
    NumericRangeQuery instead."

    I tried that and got an exception...

    I need to do something like String queryStr = "author:dean and date:[" + date1 + " TO ]"; so can I do a combination of NumericRangeQuery and text query and is there any example on this?

    org.apache.lucene.queryParser.ParseException: Cannot parse 'author:dean and date:[20110622 TO ]': Encountered " "]" "] "" at line 1, column 34.
    Was expecting one of:
    <RANGEIN_QUOTED> ...
    <RANGEIN_GOOP> ...

    at org.apache.lucene.queryParser.QueryParser.parse(QueryParser.java:211)
    at com.broadridge.papr1.test.other.PlayWithLuceneTest.runSearch(PlayWithLuceneTest.java:82)
    at com.broadridge.papr1.test.other.PlayWithLuceneTest.playWithLucene(PlayWithLuceneTest.java:59)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
    at org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4ClassRunner.java:79)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:71)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:49)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
    at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:49)
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
    Caused by: org.apache.lucene.queryParser.ParseException: Encountered " "]" "] "" at line 1, column 34.
    Was expecting one of:
    <RANGEIN_QUOTED> ...
    <RANGEIN_GOOP> ...

    at org.apache.lucene.queryParser.QueryParser.generateParseException(QueryParser.java:1818)
    at org.apache.lucene.queryParser.QueryParser.jj_consume_token(QueryParser.java:1700)
    at org.apache.lucene.queryParser.QueryParser.Term(QueryParser.java:1455)
    at org.apache.lucene.queryParser.QueryParser.Clause(QueryParser.java:1309)
    at org.apache.lucene.queryParser.QueryParser.Query(QueryParser.java:1266)
    at org.apache.lucene.queryParser.QueryParser.TopLevelQuery(QueryParser.java:1226)
    at org.apache.lucene.queryParser.QueryParser.parse(QueryParser.java:206)
    ... 25 more


    -----Original Message-----
    From: Simon Willnauer
    Sent: Sunday, June 19, 2011 11:48 AM
    To: java-user@lucene.apache.org
    Subject: Re: how to do simple search paging results of 100 each? and query syntax question

    On Sun, Jun 19, 2011 at 7:29 PM, Hiller, Dean x66079
    wrote:
    On the link
    http://lucene.apache.org/java/3_0_3/queryparsersyntax.html#Range%20Searches


    There is ranged searched, how do I specify everything above a date from date 20020101 to end of time?


    Next, I am temporarily using lucene in a noSQL solution(to switch to Solr later after prototype) and

    So I am just indexing basic columns..no need for "top search results", etc.



    When I look at the IndexSearcher and it's list of methods I am not sure how I can grab the first 100

    Results, then the second 100 results(that is if I need them), then the third 100 results (again if needed)
    so what you do here is basically requesting as many documents as you
    need lets say 100, then you display it. Once you need the next hundred
    you search again requesting 200 results and once the search returns
    simply discard the first 100
    use this as the basic method if you simply use a query without filters
    or anything.

    public TopDocs search(Query query, int n)

    I see a TopScoreDocCollector.create method but the IndexSearcher.search(Query, Collector) method states only to call that method if you need ALL the results. I definitely don't need all but need to page through the

    Results and typically exit out around the third page. This is not a web app, so ideally I want a reference held into the indexed tree so it can keep giving me the next 100 results.
    in lucene you must search again to the the next 100 but in general the
    search should be very fast.

    lemme know if you have more quesitons.

    simon

    Thanks,

    Dean

    This message and any attachments are intended only for the use of the addressee and
    may contain information that is privileged and confidential. If the reader of the
    message is not the intended recipient or an authorized representative of the
    intended recipient, you are hereby notified that any dissemination of this
    communication is strictly prohibited. If you have received this communication in
    error, please notify us immediately by e-mail and delete the message and any
    attachments from your system.
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org


    This message and any attachments are intended only for the use of the addressee and
    may contain information that is privileged and confidential. If the reader of the
    message is not the intended recipient or an authorized representative of the
    intended recipient, you are hereby notified that any dissemination of this
    communication is strictly prohibited. If you have received this communication in
    error, please notify us immediately by e-mail and delete the message and any
    attachments from your system.

    This message and any attachments are intended only for the use of the addressee and
    may contain information that is privileged and confidential. If the reader of the
    message is not the intended recipient or an authorized representative of the
    intended recipient, you are hereby notified that any dissemination of this
    communication is strictly prohibited. If you have received this communication in
    error, please notify us immediately by e-mail and delete the message and any
    attachments from your system.
  • Michael Sokolov at Jun 20, 2011 at 1:00 am

    On 6/19/2011 8:11 PM, Hiller, Dean x66079 wrote:
    Oddly, enough, this seems to work and I get one result calling Collector.collect(int docIt)...(I found out AND has to be caps)...

    author:dean AND date:20110623
    but this does not seem to work...

    author:dean AND date:[ 20110623 TO * ]
    I'm not sure, but I'd try

    author:dean AND date:[ 20110623 TO 99999999]
    You would think this second query would return the single result that was returned in the first query, wouldn't it??? I mean it is asking for the same date inclusive and more...why does Collector.collect never get called like nothing matches....stepping through the Lucene code, Scorer is null for some reason and it skips collecting everything because of that. Any ideas?

    Thanks,
    Dean

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Hiller, Dean x66079 at Jun 20, 2011 at 1:08 am
    I could kick myself in the head for that one....duh...that worked! Thanks!!! I should have tried that earlier from your earlier email but for some reason I was stuck on trying to get * working.
    Thanks,
    Dean

    -----Original Message-----
    From: Michael Sokolov
    Sent: Sunday, June 19, 2011 7:00 PM
    To: java-user@lucene.apache.org
    Subject: Re: ranged query didn't work
    On 6/19/2011 8:11 PM, Hiller, Dean x66079 wrote:
    Oddly, enough, this seems to work and I get one result calling Collector.collect(int docIt)...(I found out AND has to be caps)...

    author:dean AND date:20110623
    but this does not seem to work...

    author:dean AND date:[ 20110623 TO * ]
    I'm not sure, but I'd try

    author:dean AND date:[ 20110623 TO 99999999]
    You would think this second query would return the single result that was returned in the first query, wouldn't it??? I mean it is asking for the same date inclusive and more...why does Collector.collect never get called like nothing matches....stepping through the Lucene code, Scorer is null for some reason and it skips collecting everything because of that. Any ideas?

    Thanks,
    Dean

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org


    This message and any attachments are intended only for the use of the addressee and
    may contain information that is privileged and confidential. If the reader of the
    message is not the intended recipient or an authorized representative of the
    intended recipient, you are hereby notified that any dissemination of this
    communication is strictly prohibited. If you have received this communication in
    error, please notify us immediately by e-mail and delete the message and any
    attachments from your system.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupjava-user @
categorieslucene
postedJun 19, '11 at 5:30p
activeJul 14, '11 at 1:49p
posts12
users4
websitelucene.apache.org

People

Translate

site design / logo © 2021 Grokbase