Grokbase Groups Lucene dev April 2011
FAQ
Hello-

I think it is worth discussing what we want to do with the lucene
spatial contrib.

If you have followed the spatial development, it started with a large
contribution and has never had much love or attention. Grant did some
great work to get point search working in solr, but much of the lucene
spatial contrib does not work as expected, and it is not clear anyone
intends to improve/fix it. (geometry and tier). I feel partly
responsible for pushing the creation of this contrib, but never really
using it -- the problems it solves do not apply to the work I have so
I have not been able to give it much attention.

Recently I have been working on what I hope a high level lucene
spatial API could look like. This work has taken place at:
https://lucene-spatial-playground.googlecode.com/svn/trunk/

The key idea is that there are many 'strategies' for spatial indexing
-- you may work with simple xy coordinates, or you may need arbitrary
geometry in a crazy projection. I hope a common API can be used for
variety of approaches. The core api works directly with lucene and
has nice bindings to solr. The other key concern is a good testing
framework that works across all strategies.

When I started this 'playground', I hoped to push for a spatial module
within the Apache repo. I'm no longer sure that is the best path
forward, and want to get other peoples opinion.

My two concerns about a spatial module within lucene are community and
3rd party spatial tools

1. Community -- for the most part, developers involved in lucene are
concerned with text search; spatial search is a nice-to-have feature,
but not something that gets serious attention. I believe (perhaps
naively) that with some clever indexing, lucene/solr could be a
serious alternative to PostGIS. Our development environment should
attract contribution from spatial folks.

2. 3rd party tools -- In general people working on complex geographic
problems use JTS and other LGPL tools. There is some great work
happening at Apache SIS now, but it is a long way from being a viable
ASL alternative. Within ASF, it is legally ok to have build
dependencies on LGPL. The Lucene contrib bdb even includes a
dependency on a GPL(ish)! However, these dependencies are not
recommended and only happen if the community (and PMC) think the
tradeoff is worthwhile. Given the primary concerns of the lucene
community, I totally understand why a build dependency on LGPL may not
be acceptable.

As designed the proposed spatial API does not require JTS. All
geometry has a 'simple' implementation what works for points and
boxes. I even refactored the JTS dependencies to different packages
that could be hosted elsewhere. This works, but it is far from ideal
because it makes testing the JTS implementations much more difficult
-- given that the community of developers working on the code is
likely to use the complex implementations, this separation is
unacceptable (for me anyway). If I am going to spend the time to make
good tests, i want to make sure the classes I use are a 1st class
citizens. Likewise, if the real work goes into testing the external
packages, it stinks if it does not apply to the simple implementations
/ base framework. I don't think the spatial developer community
should be split across two code bases and build systems.

In the days of sub-projects, I would have proposed that option, but
now I see two options:

A. Work on spatial lucene outside of apache -- perhaps osgeo or even
just github. (would need a different name)
B. Allow JTS compile-time dependency in lucene, and move spatial
contrib to a real module

I think option A is better long term, but I feel like the kid saying
"if I can't have my way I'll take my ball (code) and go home" -- i
don't want this to sound like an ultimatum, but an honest discussion
about what has the best chance of fostering a thriving development
community.

If we do elect for option A, I would also suggest we delete the
spatial contrib (in 4.0) and have solr depend on the external .jar --
this way lucene users would have what they need directly with the
external .jar, and solr users would get lots of fancy new stuff
off-the-shelf.

Thoughts? Ideas? Concerns?

thanks
ryan

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

Search Discussions

  • Mattmann, Chris A (388J) at Apr 3, 2011 at 8:07 pm
    Hi Ryan,
    On Apr 3, 2011, at 12:50 PM, Ryan McKinley wrote:

    2. 3rd party tools -- In general people working on complex geographic
    problems use JTS and other LGPL tools. There is some great work
    happening at Apache SIS now, but it is a long way from being a viable
    ASL alternative.
    Thanks for actually recognizing the work going on in Apache SIS.

    So far, we've got a 0.1-incubating release that includes:

    1. a working QuadTree implementation (which I've heard you say you think would be a good idea for Lucene/Solr). We think it's a good idea in general.
    2. a method of loading/populating the QuadTree with GeoRSS.
    3. a web service (REST-ful, but could use improvement) to query the QuadTree using a bounding box or point/radius and get back XML.
    4. a demo (part of the web service WAR) that uses GMaps to show it off.

    There are a number of interesting ideas folks have had in the SIS community (including OGC-like layer services that connect to OODT, HBase, etc.) and implementation platforms for services (JAX-RS from CXF, etc) that I think will help make the project awesome. But, like all open source, there is tons of work to do.

    I can say is that we'd welcome you (and anyone else interested) working on geospatial stuff at Apache in SIS, if you're interested. Patches welcome. Contributions welcome. Collaborations welcome.
    From the original SIS proposal, accepted into the Incubator in February 2010 [1]:
    "We propose to construct Apache SIS, an ASL 2.0 licensed toolkit that spatial information system builders or users can leverage to support the aforementioned activities, alleviating much of the software and potentially legal difficulties in implementing SIS/GIS systems. This project will look to expand on those concepts and serve as a place to store reference implementations of spatial algorithms, utilities, services, etc. as well as serve as a sandbox to explore new ideas. Further, the goal is to have Apache SIS grow into a thriving Apache top-level community, where a host of SIS/GIS related software (OGC datastores, REST-ful interfaces, data standards, etc.) can grow from and thrive under the Apache umbrella."

    One thing to note: we're not trying to build our system so that it depends on any underlying storage/search/indexing/etc platform (e.g., Lucene or Solr). We're trying to build it so that that part is swappable out. That doesn't preclude us from being interested in having such a plugin-though.

    Cheers,
    Chris

    [1] http://wiki.apache.org/incubator/SpatialProposal


    Within ASF, it is legally ok to have build
    dependencies on LGPL. The Lucene contrib bdb even includes a
    dependency on a GPL(ish)! However, these dependencies are not
    recommended and only happen if the community (and PMC) think the
    tradeoff is worthwhile. Given the primary concerns of the lucene
    community, I totally understand why a build dependency on LGPL may not
    be acceptable.

    As designed the proposed spatial API does not require JTS. All
    geometry has a 'simple' implementation what works for points and
    boxes. I even refactored the JTS dependencies to different packages
    that could be hosted elsewhere. This works, but it is far from ideal
    because it makes testing the JTS implementations much more difficult
    -- given that the community of developers working on the code is
    likely to use the complex implementations, this separation is
    unacceptable (for me anyway). If I am going to spend the time to make
    good tests, i want to make sure the classes I use are a 1st class
    citizens. Likewise, if the real work goes into testing the external
    packages, it stinks if it does not apply to the simple implementations
    / base framework. I don't think the spatial developer community
    should be split across two code bases and build systems.

    In the days of sub-projects, I would have proposed that option, but
    now I see two options:

    A. Work on spatial lucene outside of apache -- perhaps osgeo or even
    just github. (would need a different name)
    B. Allow JTS compile-time dependency in lucene, and move spatial
    contrib to a real module

    I think option A is better long term, but I feel like the kid saying
    "if I can't have my way I'll take my ball (code) and go home" -- i
    don't want this to sound like an ultimatum, but an honest discussion
    about what has the best chance of fostering a thriving development
    community.

    If we do elect for option A, I would also suggest we delete the
    spatial contrib (in 4.0) and have solr depend on the external .jar --
    this way lucene users would have what they need directly with the
    external .jar, and solr users would get lots of fancy new stuff
    off-the-shelf.

    Thoughts? Ideas? Concerns?

    thanks
    ryan

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: dev-help@lucene.apache.org

    ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    Chris Mattmann, Ph.D.
    Senior Computer Scientist
    NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
    Office: 171-266B, Mailstop: 171-246
    Email: chris.a.mattmann@nasa.gov
    WWW: http://sunset.usc.edu/~mattmann/
    ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    Adjunct Assistant Professor, Computer Science Department
    University of Southern California, Los Angeles, CA 90089 USA
    ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: dev-help@lucene.apache.org
  • Smiley, David W. at Apr 4, 2011 at 9:28 pm

    On Apr 3, 2011, at 3:50 PM, Ryan McKinley wrote:

    In the days of sub-projects, I would have proposed that option, but
    now I see two options:

    A. Work on spatial lucene outside of apache -- perhaps osgeo or even
    just github. (would need a different name)
    B. Allow JTS compile-time dependency in lucene, and move spatial
    contrib to a real module
    It took some convincing by Ryan but I've come around to "A" -- a separate project that can be included just as say Carrot2 or Tika is today. Lucene/Solr can stick with the basics of geospatial (index points, filter point-distance or box, distance sort) that satisfy the majority of user needs. Anything else can be tossed, allowing Lucene/Solr to be more focused on its core capabilities. For more performant and rich geospatial features, users can get this proposed external project that is designed to easily work with Lucene/Solr. It would be good for this project to be featured prominently somehow; the wiki pages should be sufficient.

    ~ David Smiley
    Author: http://www.packtpub.com/solr-1-4-enterprise-search-server/





    ---------------------------------------------------------------------
    To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: dev-help@lucene.apache.org
  • Grant Ingersoll at Apr 5, 2011 at 1:35 pm

    On Apr 3, 2011, at 3:50 PM, Ryan McKinley wrote:


    In the days of sub-projects, I would have proposed that option, but
    now I see two options:

    A. Work on spatial lucene outside of apache -- perhaps osgeo or even
    just github. (would need a different name)
    B. Allow JTS compile-time dependency in lucene, and move spatial
    contrib to a real module
    What about C: Spatial in Lucene/Solr w/ JTS component in Apache Extras as a drop in jar that implements the appropriate bindings? I personally have an interest in good spatial in L/S including more than just point search. This solution need not require any automated downloads/compiles, etc. and it can be noted that the support of that particular jar is handled on Apache Extras (or Githup or wherever)

    Alternatively, has anyone simply asked JTS if they would dual license or something like that? I'm happy to do that, but let's coordinate so that we aren't all bombarding the guy.
    I think option A is better long term, but I feel like the kid saying
    "if I can't have my way I'll take my ball (code) and go home" -- i
    don't want this to sound like an ultimatum, but an honest discussion
    about what has the best chance of fostering a thriving development
    community.
    I personally don't. I think we have enough committers who are active on spatial that we can sustain it once we have a good foundation in place (which we are close to) and we also have several contributors who have been active on spatial and are likely good candidates for being committers to work on it.
  • Ryan McKinley at Apr 5, 2011 at 2:30 pm

    On Tue, Apr 5, 2011 at 9:34 AM, Grant Ingersoll wrote:
    On Apr 3, 2011, at 3:50 PM, Ryan McKinley wrote:

    In the days of sub-projects, I would have proposed that option, but
    now I see two options:

    A.  Work on spatial lucene outside of apache -- perhaps osgeo or even
    just github. (would need a different name)
    B.  Allow JTS compile-time dependency in lucene, and move spatial
    contrib to a real module


    What about C:  Spatial in Lucene/Solr w/ JTS component in Apache Extras as a
    drop in jar that implements the appropriate bindings?  I personally have an
    interest in good spatial in L/S including more than just point search.  This
    solution need not require any automated downloads/compiles, etc. and it can
    be noted that the support of that particular jar is handled on Apache Extras
    (or Githup or wherever)
    If the suggestion is splitting the build so that the JTS shapes and
    math are not tested with the core, then i am -1

    The spatial build and test system should be designed to best test
    spatial -- without jumping through serious hoops, splitting the
    compile/test setup only hurts the project. (unless you don't really
    care about the more complex stuff....)

    I also want to make sure that the concerns of more complex features
    are a top priority of the whole framework -- developing/patching
    across two repos stinks.

    Alternatively, has anyone simply asked JTS if they would dual license or
    something like that?  I'm happy to do that, but let's coordinate so that we
    aren't all bombarding the guy.
    You can always ask... but I think it is too late for a license change.
    This has been around to 10 years and the original copyright owner
    appears to be a defunct company.

    http://www.vividsolutions.com/jts/jtshome.htm (note
    http://tsusiatsoftware.net/jts/main.html

    Also check:
    http://tsusiatsoftware.net/jts/jts-history.html

    I think option A is better long term, but I feel like the kid saying
    "if I can't have my way I'll take my ball (code) and go home"  -- i
    don't want this to sound like an ultimatum, but an honest discussion
    about what has the best chance of fostering a thriving development
    community.

    I personally don't.  I think we have enough committers who are active on
    spatial that we can sustain it once we have a good foundation in place
    (which we are close to) and we also have several contributors who have been
    active on spatial and are likely good candidates for being committers to
    work on it.
    Right now the lucene umbrella is pretty big -- the dev concerns of
    spatial are, and *should* be, of little concern to the core lucene
    development. I think spatial dev should live in a place where the
    spatial concerns are top priority.

    Just because it uses lucene does not mean it needs to be in the lucene umbrella.

    Perhaps moving the whole thing to apache-extras is OK, but that is not
    a visible place to attract participation etc. osgeo seems like a
    natural home since it will attract more spatial developers.

    Other then status quo... why should this live in the lucene repo?

    ryan

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: dev-help@lucene.apache.org
  • Smiley, David W. at Apr 5, 2011 at 3:11 pm
    Alternatively, has anyone simply asked JTS if they would dual license or something like that? I'm happy to do that, but let's coordinate so that we aren't all bombarding the guy.
    I did already. Martin didn't respond but someone else did and the outlook is highly unlikely (see Ryan's points about its history). If I get news otherwise then I will respond ASAP but consider it a long shot.

    Grant, since you too have an interest in spatial, you too could be a developer on lucene-spatial-playground (I look forward to a better name). Just because there are folks interested in spatial involved with Lucene/Solr does not mean that the module needs to actually be in the Lucene/Solr's codebase itself. I think geospatial is both fairly specialized and also only desired by a fraction of Lucene/Solr users, particularly beyond the basic essentials satisfied with a pair of trie doubles (LatLonPoint) and a distance functionquery. If you consider that, then why would it be in Lucene/Solr?

    ~ David
    ________________________________________
    From: Grant Ingersoll [gsingers@apache.org]
    Sent: Tuesday, April 05, 2011 9:34 AM
    To: dev@lucene.apache.org
    Subject: Re: Lucene Spatial Future

    On Apr 3, 2011, at 3:50 PM, Ryan McKinley wrote:


    In the days of sub-projects, I would have proposed that option, but
    now I see two options:

    A. Work on spatial lucene outside of apache -- perhaps osgeo or even
    just github. (would need a different name)
    B. Allow JTS compile-time dependency in lucene, and move spatial
    contrib to a real module


    What about C: Spatial in Lucene/Solr w/ JTS component in Apache Extras as a drop in jar that implements the appropriate bindings? I personally have an interest in good spatial in L/S including more than just point search. This solution need not require any automated downloads/compiles, etc. and it can be noted that the support of that particular jar is handled on Apache Extras (or Githup or wherever)

    Alternatively, has anyone simply asked JTS if they would dual license or something like that? I'm happy to do that, but let's coordinate so that we aren't all bombarding the guy.

    I think option A is better long term, but I feel like the kid saying
    "if I can't have my way I'll take my ball (code) and go home" -- i
    don't want this to sound like an ultimatum, but an honest discussion
    about what has the best chance of fostering a thriving development
    community.

    I personally don't. I think we have enough committers who are active on spatial that we can sustain it once we have a good foundation in place (which we are close to) and we also have several contributors who have been active on spatial and are likely good candidates for being committers to work on it.



    ---------------------------------------------------------------------
    To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: dev-help@lucene.apache.org
  • Michael McCandless at Apr 5, 2011 at 3:23 pm
    Forgive my ignorance, but: are there any technical reasons for spatial
    work to be "in core"?

    Or can all the spatial algos be "safely" (ie, won't lose much
    performance, if any) built "on top" of Lucene's (Solr's) APIs? Do we
    need to open up new APIs (in which case being part of one code base,
    released together, is a strong advantage)?

    For example, incremental field updates is a long needed and often
    requested feature in Lucene, but this really couldn't be
    easily/efficiently build "on top" -- it requires access to lots of
    inside details that are tracked during indexing.

    Mike

    http://blog.mikemccandless.com
    On Tue, Apr 5, 2011 at 11:10 AM, Smiley, David W. wrote:
    Alternatively, has anyone simply asked JTS if they would dual license or something like that?  I'm happy to do that, but let's coordinate so that we aren't all bombarding the guy.
    I did already.  Martin didn't respond but someone else did and the outlook is highly unlikely (see Ryan's points about its history).  If I get news otherwise then I will respond ASAP but consider it a long shot.

    Grant, since you too have an interest in spatial, you too could be a developer on lucene-spatial-playground (I look forward to a better name).  Just because there are folks interested in spatial involved with Lucene/Solr does not mean that the module needs to actually be in the Lucene/Solr's codebase itself.  I think geospatial is both fairly specialized and also only desired by a fraction of Lucene/Solr users, particularly beyond the basic essentials satisfied with a pair of trie doubles (LatLonPoint) and a distance functionquery.  If you consider that, then why would it be in Lucene/Solr?

    ~ David
    ________________________________________
    From: Grant Ingersoll [gsingers@apache.org]
    Sent: Tuesday, April 05, 2011 9:34 AM
    To: dev@lucene.apache.org
    Subject: Re: Lucene Spatial Future

    On Apr 3, 2011, at 3:50 PM, Ryan McKinley wrote:


    In the days of sub-projects, I would have proposed that option, but
    now I see two options:

    A.  Work on spatial lucene outside of apache -- perhaps osgeo or even
    just github. (would need a different name)
    B.  Allow JTS compile-time dependency in lucene, and move spatial
    contrib to a real module


    What about C:  Spatial in Lucene/Solr w/ JTS component in Apache Extras as a drop in jar that implements the appropriate bindings?  I personally have an interest in good spatial in L/S including more than just point search.  This solution need not require any automated downloads/compiles, etc. and it can be noted that the support of that particular jar is handled on Apache Extras (or Githup or wherever)

    Alternatively, has anyone simply asked JTS if they would dual license or something like that?  I'm happy to do that, but let's coordinate so that we aren't all bombarding the guy.

    I think option A is better long term, but I feel like the kid saying
    "if I can't have my way I'll take my ball (code) and go home"  -- i
    don't want this to sound like an ultimatum, but an honest discussion
    about what has the best chance of fostering a thriving development
    community.

    I personally don't.  I think we have enough committers who are active on spatial that we can sustain it once we have a good foundation in place (which we are close to) and we also have several contributors who have been active on spatial and are likely good candidates for being committers to work on it.



    ---------------------------------------------------------------------
    To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: dev-help@lucene.apache.org
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: dev-help@lucene.apache.org
  • Smiley, David W. at Apr 5, 2011 at 3:36 pm
    Mike: Lucene core supports what is required for efficient spatial algorithms to use it, without requiring modification to Lucene. So the answer to your question is "no".

    ~ David
    ________________________________________
    From: Michael McCandless [lucene@mikemccandless.com]
    Sent: Tuesday, April 05, 2011 11:23 AM
    To: dev@lucene.apache.org
    Cc: Smiley, David W.
    Subject: Re: Lucene Spatial Future

    Forgive my ignorance, but: are there any technical reasons for spatial
    work to be "in core"?

    Or can all the spatial algos be "safely" (ie, won't lose much
    performance, if any) built "on top" of Lucene's (Solr's) APIs? Do we
    need to open up new APIs (in which case being part of one code base,
    released together, is a strong advantage)?

    For example, incremental field updates is a long needed and often
    requested feature in Lucene, but this really couldn't be
    easily/efficiently build "on top" -- it requires access to lots of
    inside details that are tracked during indexing.

    Mike

    http://blog.mikemccandless.com
    On Tue, Apr 5, 2011 at 11:10 AM, Smiley, David W. wrote:
    Alternatively, has anyone simply asked JTS if they would dual license or something like that? I'm happy to do that, but let's coordinate so that we aren't all bombarding the guy.
    I did already. Martin didn't respond but someone else did and the outlook is highly unlikely (see Ryan's points about its history). If I get news otherwise then I will respond ASAP but consider it a long shot.

    Grant, since you too have an interest in spatial, you too could be a developer on lucene-spatial-playground (I look forward to a better name). Just because there are folks interested in spatial involved with Lucene/Solr does not mean that the module needs to actually be in the Lucene/Solr's codebase itself. I think geospatial is both fairly specialized and also only desired by a fraction of Lucene/Solr users, particularly beyond the basic essentials satisfied with a pair of trie doubles (LatLonPoint) and a distance functionquery. If you consider that, then why would it be in Lucene/Solr?

    ~ David
    ________________________________________
    From: Grant Ingersoll [gsingers@apache.org]
    Sent: Tuesday, April 05, 2011 9:34 AM
    To: dev@lucene.apache.org
    Subject: Re: Lucene Spatial Future

    On Apr 3, 2011, at 3:50 PM, Ryan McKinley wrote:


    In the days of sub-projects, I would have proposed that option, but
    now I see two options:

    A. Work on spatial lucene outside of apache -- perhaps osgeo or even
    just github. (would need a different name)
    B. Allow JTS compile-time dependency in lucene, and move spatial
    contrib to a real module


    What about C: Spatial in Lucene/Solr w/ JTS component in Apache Extras as a drop in jar that implements the appropriate bindings? I personally have an interest in good spatial in L/S including more than just point search. This solution need not require any automated downloads/compiles, etc. and it can be noted that the support of that particular jar is handled on Apache Extras (or Githup or wherever)

    Alternatively, has anyone simply asked JTS if they would dual license or something like that? I'm happy to do that, but let's coordinate so that we aren't all bombarding the guy.

    I think option A is better long term, but I feel like the kid saying
    "if I can't have my way I'll take my ball (code) and go home" -- i
    don't want this to sound like an ultimatum, but an honest discussion
    about what has the best chance of fostering a thriving development
    community.

    I personally don't. I think we have enough committers who are active on spatial that we can sustain it once we have a good foundation in place (which we are close to) and we also have several contributors who have been active on spatial and are likely good candidates for being committers to work on it.



    ---------------------------------------------------------------------
    To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: dev-help@lucene.apache.org
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: dev-help@lucene.apache.org
  • Ryan McKinley at Apr 5, 2011 at 3:39 pm

    On Tue, Apr 5, 2011 at 11:23 AM, Michael McCandless wrote:
    Forgive my ignorance, but: are there any technical reasons for spatial
    work to be "in core"?
    There is no reason it needs to be in core.

    As is, it is a maven build that depends on -SNAPSHOT and everything works great.
    Or can all the spatial algos be "safely" (ie, won't lose much
    performance, if any) built "on top" of Lucene's (Solr's) APIs?  Do we
    need to open up new APIs (in which case being part of one code base,
    released together, is a strong advantage)?
    Not that i can think of -- it is right now using features that will
    get better when new things are added to lucene (for example the
    docvalues branch) but i think that is true of any project built on top
    of lucene.

    My push to have a Numeric field that both solr and lucene use was
    motivated by this project. Again, not a unique problem to spatial.

    Also, i recently changed some package protected constants in solr to
    protected -- so that an external library could make complex solr
    FieldTypes.

    For example, incremental field updates is a long needed and often
    requested feature in Lucene, but this really couldn't be
    easily/efficiently build "on top" -- it requires access to lots of
    inside details that are tracked during indexing.
    Nothing like that in spatial contrib. The closest I can think of (and
    this is a long shot) I can see writing a spatial codec that writes
    values to an rtree. But even this should be able to work within
    published (or SNAPSHOT) apis.

    More importantly, the people who would work on the core lucene
    problems vs the people who would work on the majority of spatial
    problems are pretty different.

    It would be nice if spatial contributions could not possibly break the
    lucene build.

    ryan

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: dev-help@lucene.apache.org
  • Ryan McKinley at Apr 5, 2011 at 4:04 pm
    Grant, since you too have an interest in spatial, you too could be a developer on lucene-spatial-playground (I look forward to a better name).  Just because there are folks interested in spatial involved with Lucene/Solr does not mean that the module needs to actually be in the Lucene/Solr's codebase itself.  I think geospatial is both fairly specialized and also only desired by a fraction of Lucene/Solr users, particularly beyond the basic essentials satisfied with a pair of trie doubles (LatLonPoint) and a distance functionquery.  If you consider that, then why would it be in Lucene/Solr?
    my aspirations may be high, but I think spatial+lucene/solr can be a
    platform to rival PostGIS -- or at the very least good option when you
    need good text+spatial search. (PostGIS and oracle spatial are the
    only options now)

    Some key features that would get lots of attention from geo devlopers are:
    * native CS-W support from solr. This is super important for GIS
    catalogs. Actually now required by law in the EU!
    * native GeoRSS and opensearch geo from solr
    * lucene/solr as a geotools data source -- this opens up a huge range
    of off-the-shelf processing

    The people who care about these problems and the build system and
    dependencies to support them are pretty distinct from lucene core.

    ryan

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: dev-help@lucene.apache.org
  • Grant Ingersoll at Apr 5, 2011 at 6:47 pm

    On Apr 3, 2011, at 3:50 PM, Ryan McKinley wrote:
    If we do elect for option A, I would also suggest we delete the
    spatial contrib (in 4.0) and have solr depend on the external .jar --
    this way lucene users would have what they need directly with the
    external .jar, and solr users would get lots of fancy new stuff
    off-the-shelf.

    At the end of the day, Solr only uses a few methods in the Lucene contrib: the geohash stuff and a few other distance utils (some static methods that I moved from Solr down to Lucene). That's about it. Everything else is just FieldTypes and function queries (the tier stuff is broken b/c of the sinusoidal projection impl. and I didn't see a need for the geometry stuff). If we move function queries to modules, those utils could just be hidden in there (or they could just be put in Solr for that matter if function queries stay where they are) and there would be no need for any external dependency. Then, there is no spatial package at all and anyone who wishes to work on Spatial can go do it in the proposed external project if they need anything beyond point search.

    That being said, a separate package/project is what LocalLucene/LocalSolr was. You are of course free to do as you wish, but to me things that are first order supported by the community seem to stick better than those that don't and have the benefit of our extensive testing framework as well as resources, especially something as popular as spatial. I think, though, at the end of the day, you are just going to see, once again, a forking as people will likely be confused about where to contribute. Some will continue to contribute to Lucene/Solr b/c that is what they are working on (b/c they started w/ point search) and others will contribute to the external project. Nothing wrong with that, of course, people can do as they wish, it's just why I would prefer a single solution as part of our modules. If SIS or something else w/ a better license was as mature as JTS, we probably wouldn't even be having the discussion.

    On a different level, for me personally, and I really stress this just my personal choice, I don't have much interest in contributing to non-ASF (or similar foundation based open source projects) because they don't offer the same legal framework, branding, resources and other opportunities that the ASF does.

    -Grant



    ---------------------------------------------------------------------
    To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: dev-help@lucene.apache.org
  • William Bell at Apr 5, 2011 at 6:50 pm
    Let's just do it in Solr directly. I like the jar idea for JTS.

    We just need more committers to contribute and support the people
    doing the work.

    Bill

    On Tue, Apr 5, 2011 at 2:46 PM, Grant Ingersoll wrote:
    On Apr 3, 2011, at 3:50 PM, Ryan McKinley wrote:
    If we do elect for option A, I would also suggest we delete the
    spatial contrib (in 4.0) and have solr depend on the external .jar --
    this way lucene users would have what they need directly with the
    external .jar, and solr users would get lots of fancy new stuff
    off-the-shelf.

    At the end of the day, Solr only uses a few methods in the Lucene contrib:  the geohash stuff and a few other distance utils (some static methods that I moved from Solr down to Lucene).  That's about it.   Everything else is just FieldTypes and function queries (the tier stuff is broken b/c of the sinusoidal projection impl. and I didn't see a need for the geometry stuff).  If we move function queries to modules, those utils could just be hidden in there (or they could just be put in Solr for that matter if function queries stay where they are) and there would be no need for any external dependency.  Then, there is no spatial package at all and anyone who wishes to work on Spatial can go do it in the proposed external project if they need anything beyond point search.

    That being said, a separate package/project is what LocalLucene/LocalSolr was.  You are of course free to do as you wish, but to me things that are first order supported by the community seem to stick better than those that don't and have the benefit of our extensive testing framework as well as resources, especially something as popular as spatial.   I think, though, at the end of the day, you are just going to see, once again, a forking as people will likely be confused about where to contribute.  Some will continue to contribute to Lucene/Solr b/c that is what they are working on (b/c they started w/ point search) and others will contribute to the external project.  Nothing wrong with that, of course, people can do as they wish, it's just why I would prefer a single solution as part of our modules.  If SIS or something else w/ a better license was as mature as JTS, we probably wouldn't even be having the discussion.

    On a different level, for me personally, and I really stress this just my personal choice, I don't have much interest in contributing to non-ASF (or similar foundation based open source projects) because they don't offer the same legal framework, branding, resources and other opportunities that the ASF does.

    -Grant



    ---------------------------------------------------------------------
    To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: dev-help@lucene.apache.org
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: dev-help@lucene.apache.org
  • Ryan McKinley at Apr 5, 2011 at 8:09 pm

    On Tue, Apr 5, 2011 at 2:46 PM, Grant Ingersoll wrote:
    On Apr 3, 2011, at 3:50 PM, Ryan McKinley wrote:
    If we do elect for option A, I would also suggest we delete the
    spatial contrib (in 4.0) and have solr depend on the external .jar --
    this way lucene users would have what they need directly with the
    external .jar, and solr users would get lots of fancy new stuff
    off-the-shelf.

    At the end of the day, Solr only uses a few methods in the Lucene contrib:  the geohash stuff and a few other distance utils (some static methods that I moved from Solr down to Lucene).  That's about it.   Everything else is just FieldTypes and function queries (the tier stuff is broken b/c of the sinusoidal projection impl. and I didn't see a need for the geometry stuff).  If we move function queries to modules, those utils could just be hidden in there (or they could just be put in Solr for that matter if function queries stay where they are) and there would be no need for any external dependency.  Then, there is no spatial package at all and anyone who wishes to work on Spatial can go do it in the proposed external project if they need anything beyond point search.
    The spatial API in google code takes a pretty different approach to
    spatial search in general. It is organized into three key packages:
    1. core stuff, no lucene dependencies. Most of the math is here
    2. lucene stuff
    3. solr interface -- configure and manage the lucene implementations

    It does not rely on function queries (it will use ValueSorceQuery when
    necessary). The solr integration is handled via
    FieldType.getFieldQuery()

    This way we can send complex queries in the regular 'q' param or an
    'fq' (or even a function query) -- syntax looks like:

    q=geo:"Operation( shape )"

    Operation is one of:
    BBoxIntersects
    BBoxWithin
    Contains
    Intersects
    IsEqualTo
    IsDisjointTo
    Overlaps
    SimilarTo

    And the same is either a point, bbox, point+radius, or complex polygon
    q=geofield:"Intersects( -10 20 -10 20 )"
    q=geofield:"IsWithin( PointDistance( x y distance )"
    q=geofield:"Intersects( POLYGON ((30 10, ...)) )"

    Different fields can be indexed with different strategies, but the
    interface stays the same

    Nothing wrong with that, of course, people can do as they wish, it's just why I would prefer a single solution as part of our modules.
    Hypothetically, if a stable Apache licensed spatial module were
    released by a reputable 3rd party foundation (osgeo.org) are you
    against including it in solr as a .jar? It could get tested directly
    in solr framework the same we do with carrot2 and tika.

    Assuming the legal stuff is up-to-snuff and the build is solid, I'm
    confused why spatial support for lucene needs to be in lucene modules?


    If SIS or something else w/ a better license was as mature as JTS, we probably wouldn't even be having the discussion.
    There is more to it then that -- the JTS license issue just slams the
    issue to the forefront so we can not ignore it.

    For me the bigger issue is that the development choices for a serious
    spatial module are not the same as for core lucene development. Also
    the people are potentially quite different (with overlap of course)

    On a different level, for me personally, and I really stress this just my personal choice, I don't have much interest in contributing to non-ASF (or similar foundation based open source projects) because they don't offer the same legal framework, branding, resources and other opportunities that the ASF does.
    This is why I suggest osgeo.org -- they are a foundation similar to
    ASF. They are the home to most of the key opensource geographic
    projects, including: PostGIS, GDAL, Geotools, Openlayers, and
    MapServer. Unlike the ASF, there is not a single license model across
    all projects, but they make sure work is licensed properly and have
    most of the same procedures as ASF (incubation, infrastructure,
    mentors, etc)

    I could suggest a new ASF project, but there seems like too much
    overlap with SIS and very different philosophy on 3rd party libraries.
    In the end, osgeo.com seems like a more natural home and has better
    branding for spatial related work anyway.

    ryan

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: dev-help@lucene.apache.org
  • Grant Ingersoll at Apr 6, 2011 at 1:30 pm

    On Apr 5, 2011, at 4:08 PM, Ryan McKinley wrote:
    On Tue, Apr 5, 2011 at 2:46 PM, Grant Ingersoll wrote:
    On Apr 3, 2011, at 3:50 PM, Ryan McKinley wrote:
    If we do elect for option A, I would also suggest we delete the
    spatial contrib (in 4.0) and have solr depend on the external .jar --
    this way lucene users would have what they need directly with the
    external .jar, and solr users would get lots of fancy new stuff
    off-the-shelf.

    At the end of the day, Solr only uses a few methods in the Lucene contrib: the geohash stuff and a few other distance utils (some static methods that I moved from Solr down to Lucene). That's about it. Everything else is just FieldTypes and function queries (the tier stuff is broken b/c of the sinusoidal projection impl. and I didn't see a need for the geometry stuff). If we move function queries to modules, those utils could just be hidden in there (or they could just be put in Solr for that matter if function queries stay where they are) and there would be no need for any external dependency. Then, there is no spatial package at all and anyone who wishes to work on Spatial can go do it in the proposed external project if they need anything beyond point search.
    The spatial API in google code takes a pretty different approach to
    spatial search in general. It is organized into three key packages:
    1. core stuff, no lucene dependencies. Most of the math is here
    Aren't you just replicating what SIS is doing for this piece? If you don't have a JTS requirement, that means you are going to need equivalent math, right? Isn't that what SIS is about?
    2. lucene stuff
    3. solr interface -- configure and manage the lucene implementations

    It does not rely on function queries (it will use ValueSorceQuery when
    necessary). The solr integration is handled via
    FieldType.getFieldQuery()

    This way we can send complex queries in the regular 'q' param or an
    'fq' (or even a function query) -- syntax looks like:

    q=geo:"Operation( shape )"

    Operation is one of:
    BBoxIntersects
    BBoxWithin
    Contains
    Intersects
    IsEqualTo
    IsDisjointTo
    Overlaps
    SimilarTo

    And the same is either a point, bbox, point+radius, or complex polygon
    q=geofield:"Intersects( -10 20 -10 20 )"
    q=geofield:"IsWithin( PointDistance( x y distance )"
    q=geofield:"Intersects( POLYGON ((30 10, ...)) )"

    Different fields can be indexed with different strategies, but the
    interface stays the same
    Cool.
    Nothing wrong with that, of course, people can do as they wish, it's just why I would prefer a single solution as part of our modules.
    Hypothetically, if a stable Apache licensed spatial module were
    released by a reputable 3rd party foundation (osgeo.org) are you
    against including it in solr as a .jar? It could get tested directly
    in solr framework the same we do with carrot2 and tika.

    Assuming the legal stuff is up-to-snuff and the build is solid, I'm
    confused why spatial support for lucene needs to be in lucene modules?
    It doesn't, we just already have point search working and released. Which means it would have to go through deprecation, etc. Obviously, it can be done.

    If SIS or something else w/ a better license was as mature as JTS, we probably wouldn't even be having the discussion.
    There is more to it then that -- the JTS license issue just slams the
    issue to the forefront so we can not ignore it.

    For me the bigger issue is that the development choices for a serious
    spatial module are not the same as for core lucene development.
    That's true of all of our modules. That's why they are modules and not core.
    This is why I suggest osgeo.org -- they are a foundation similar to
    ASF. They are the home to most of the key opensource geographic
    projects, including: PostGIS, GDAL, Geotools, Openlayers, and
    MapServer. Unlike the ASF, there is not a single license model across
    all projects, but they make sure work is licensed properly and have
    most of the same procedures as ASF (incubation, infrastructure,
    mentors, etc)

    I could suggest a new ASF project, but there seems like too much
    overlap with SIS and very different philosophy on 3rd party libraries.
    In the end, osgeo.com seems like a more natural home and has better
    branding for spatial related work anyway.

    By all means go for it. I don't see any reason not too. I guess in the end, I'm not sure what you are asking us to do. Do you want Lucene/Solr to remove all of our spatial support in favor of incorporating this new project or do you just want those who are interested in spatial to join the new project and it can be seen as an add on?
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: dev-help@lucene.apache.org
  • Yonik Seeley at Apr 6, 2011 at 2:17 pm

    On Wed, Apr 6, 2011 at 9:30 AM, Grant Ingersoll wrote:
    By all means go for it.  I don't see any reason not too.  I guess in the end, I'm not sure what you are asking us to do.  Do you want Lucene/Solr to remove all of our spatial support in favor of incorporating this new project or do you just want those who are interested in spatial to join the new project and it can be seen as an add on?

    Let's not confuse the issue... what is being discussed really has no
    impact on the basic spatial search that was added to Solr. As you
    said yourself, the Solr geo stuff uses very little of the spatial
    contrib stuff.

    This is about building and maintaining a spatial module, and the best
    place to do it (which I'll leave up to those doing the work... I'm
    pretty happy with basic point, radius, bounding-box).

    -Yonik
    http://www.lucenerevolution.org -- Lucene/Solr User Conference, May
    25-26, San Francisco

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: dev-help@lucene.apache.org
  • Ryan McKinley at Apr 6, 2011 at 2:46 pm

    The spatial API in google code takes a pretty different approach to
    spatial search in general.  It is organized into three key packages:
    1. core stuff, no lucene dependencies.  Most of the math is here
    Aren't you just replicating what SIS is doing for this piece?  If you don't have a JTS requirement, that means you are going to need equivalent math, right?  Isn't that what SIS is about?
    This package defines the general interfaces and concepts used in the
    project. Things like SpatialOperations, Shape, PrefixGrid and
    DistanceCalculator -- these can then be backed by simple math, JTS, or
    eventually maybe SIS.

    The other key stuff in this package is the client side objects that to
    build spatial queries. Essentially everything that could be bundled
    with solrj.

    I could suggest a new ASF project, but there seems like too much
    overlap with SIS and very different philosophy on 3rd party libraries.
    In the end, osgeo.com seems like a more natural home and has better
    branding for spatial related work anyway.

    By all means go for it.  I don't see any reason not too.  I guess in the end, I'm not sure what you are asking us to do.  Do you want Lucene/Solr to remove all of our spatial support in favor of incorporating this new project or do you just want those who are interested in spatial to join the new project and it can be seen as an add on?
    I'm trying to have an open discussion about what makes sense for
    spatial development. I don't *want* to start a new project... but I
    think we need a dev/test environment that can support the whole range
    of spatial needs -- without reinventing many wheels, this includes
    JTS.

    Lucene currently has LGPL compile dependencies, but they are on the
    way out, and (unless I'm missing something) i don't think folks are
    open to adding a JTS build/test dependency -- Maybe I should call a
    vote on the JTS issue, though i suspect most binding votes are -0 or
    -1. I *totally* understand if other people don't want JTS in the
    build system -- it is not a core concern to most people involved.

    If the lucene build/test environment does not support spatial
    development, this leads me to think about other places to host the
    project... wherever it makes the most sense. I would prefer staying
    within lucene because it is easiest for me.

    I don't want this to be competition or duplicate effort. I hope it
    lets us clean up the broken stuff from lucene and overtime deprecate
    the parts that are better supported elsewhere.

    I want the best spatial support available in solr out-of-the-box. If
    this project is eventually built and maintained outside of lucene, i
    would like the .jar distributed in solr.


    ryan

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: dev-help@lucene.apache.org
  • Grant Ingersoll at Apr 6, 2011 at 3:39 pm

    On Apr 6, 2011, at 10:45 AM, Ryan McKinley wrote:
    I'm trying to have an open discussion about what makes sense for
    spatial development. I don't *want* to start a new project... but I
    think we need a dev/test environment that can support the whole range
    of spatial needs -- without reinventing many wheels, this includes
    JTS.

    Lucene currently has LGPL compile dependencies, but they are on the
    way out, and (unless I'm missing something) i don't think folks are
    open to adding a JTS build/test dependency -- Maybe I should call a
    vote on the JTS issue, though i suspect most binding votes are -0 or
    -1. I *totally* understand if other people don't want JTS in the
    build system -- it is not a core concern to most people involved.
    Until there is a specific patch that brings in and shows how JTS would be incorporated (via reflection and as a totally optional piece, presumably, per the ASF LGPL guidelines), there really isn't anything to vote on.

    I don't want this to be competition or duplicate effort. I hope it
    lets us clean up the broken stuff from lucene and overtime deprecate
    the parts that are better supported elsewhere.
    I totally agree. I hope I wasn't framing it that way. I'm just trying to understand what's being proposed. I can see advantages to both.
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: dev-help@lucene.apache.org
  • Smiley, David W. at Apr 6, 2011 at 4:06 pm

    On Apr 6, 2011, at 11:38 AM, Grant Ingersoll wrote:

    Until there is a specific patch that brings in and shows how JTS would be incorporated (via reflection and as a totally optional piece, presumably, per the ASF LGPL guidelines), there really isn't anything to vote on.
    I think what is being asked to vote on is deprecation/removal of Lucene's spatial contrib module with its replacement being an externally hosted ASL-licened module expressly designed to work with Lucene/Solr 4.0 and beyond (temporarily known as lucene-spatial-playground). What would stay is the _basic_ spatial support that got into Lucene/Solr 3.1. Furthermore, no future spatial work would be accepted on Lucene/Solr aside from support of the basic capability.

    This module isn't quite ready so perhaps the vote should wait till it is.

    ~ David Smiley
    Author: http://www.packtpub.com/solr-1-4-enterprise-search-server/





    ---------------------------------------------------------------------
    To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: dev-help@lucene.apache.org
  • Grant Ingersoll at Apr 6, 2011 at 6:11 pm

    On Apr 6, 2011, at 12:06 PM, Smiley, David W. wrote:
    On Apr 6, 2011, at 11:38 AM, Grant Ingersoll wrote:

    Until there is a specific patch that brings in and shows how JTS would be incorporated (via reflection and as a totally optional piece, presumably, per the ASF LGPL guidelines), there really isn't anything to vote on.
    I think what is being asked to vote on is deprecation/removal of Lucene's spatial contrib module

    Just FYI, It is already deprecated in 3.x and slated for removal in 4.0. Someone just needs to axe the appropriate bits (and either move what's needed to Solr or to modules)
    with its replacement being an externally hosted ASL-licened module expressly designed to work with Lucene/Solr 4.0 and beyond (temporarily known as lucene-spatial-playground). What would stay is the _basic_ spatial support that got into Lucene/Solr 3.1. Furthermore, no future spatial work would be accepted on Lucene/Solr aside from support of the basic capability.
    That is the piece I was wondering about and why I said yesterday it isn't likely to work, as it will just fork. How do you tell people not to put in patches to L/S, especially when part of it is native and part of it isn't?
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: dev-help@lucene.apache.org
  • Smiley, David W. at Apr 6, 2011 at 6:22 pm

    On Apr 6, 2011, at 2:08 PM, Grant Ingersoll wrote:
    with its replacement being an externally hosted ASL-licened module expressly designed to work with Lucene/Solr 4.0 and beyond (temporarily known as lucene-spatial-playground). What would stay is the _basic_ spatial support that got into Lucene/Solr 3.1. Furthermore, no future spatial work would be accepted on Lucene/Solr aside from support of the basic capability.
    That is the piece I was wondering about and why I said yesterday it isn't likely to work, as it will just fork. How do you tell people not to put in patches to L/S, especially when part of it is native and part of it isn't?
    I think risk of this is mitigated if the proposed external module is highly visible in L/S -- in other words, it's downloaded and packaged up as part of the distribution -- a jar, sitting along side the other contrib module jars (no JTS of course!). Users would be referred to this module for non-basic spatial via the wiki and community in general. Of course I would prominently mention this module in the 2nd edition of my book ;-) which is well underway.

    ~ David Smiley
    Author: http://www.packtpub.com/solr-1-4-enterprise-search-server/





    ---------------------------------------------------------------------
    To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: dev-help@lucene.apache.org
  • Yonik Seeley at Apr 6, 2011 at 6:32 pm

    On Wed, Apr 6, 2011 at 2:08 PM, Grant Ingersoll wrote:
    On Apr 6, 2011, at 12:06 PM, Smiley, David W. wrote:
    with its replacement being an externally hosted ASL-licened module expressly designed to work with Lucene/Solr 4.0 and beyond (temporarily known as lucene-spatial-playground).  What would stay is the _basic_ spatial support that got into Lucene/Solr 3.1. Furthermore, no future spatial work would be accepted on Lucene/Solr aside from support of the basic capability.
    That is the piece I was wondering about and why I said yesterday it isn't likely to work, as it will just fork.  How do you tell people not to put in patches to L/S, especially when part of it is native and part of it isn't?
    Right - there's no need to try and make promises about the future. It
    seems unrelated to the questions at hand here.

    -Yonik
    http://www.lucenerevolution.org -- Lucene/Solr User Conference, May
    25-26, San Francisco

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: dev-help@lucene.apache.org
  • Ryan McKinley at Apr 6, 2011 at 7:36 pm

    Right - there's no need to try and make promises about the future.  It
    seems unrelated to the questions at hand here.
    To be clear... I don't see any of this as promises -- obviously
    nothing happens until there is somethign concrete to evaluate.

    The point of this thread (for me anyway) is to raise my concerns, see
    what people are thinking, and be transparent about my choices.

    This discussion has made me feel like the right choice (for me) is to
    pursue spatial development somewhere else -- likely osgeo -- and down
    the road figure out how that could/should fit with solr.

    ryan

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: dev-help@lucene.apache.org
  • Ryan McKinley at Apr 6, 2011 at 4:39 pm

    -1.  I *totally* understand if other people don't want JTS in the
    build system -- it is not a core concern to most people involved.
    Until there is a specific patch that brings in and shows how JTS would be incorporated (via reflection and as a totally optional piece, presumably, per the ASF LGPL guidelines), there really isn't anything to vote on.
    fair point -- the optional logistics are working in a maven build.
    I'm reluctant to convert to the ant build system if there is already
    strong opposition to the idea. If folks are OK with the idea, I will
    happily make concrete patch/branch that we could vote on.

    so maybe i'm just looking for a POLL not a vote -- find out if this is
    a non-starter or not (i am under the impression that it might be)

    FYI, the optional support is now handled by a static
    SpatialContextProvider' that you can ask for a SpatialContext. By
    default it makes a SimpleSpatialContext -- if you set some system
    properties, it uses reflection to load a different instance.
    Eventually, this should be replaced with the standard java service
    loader stuff (i think)

    ryan

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: dev-help@lucene.apache.org
  • Chris Male at Apr 6, 2011 at 1:53 am
    Hi,
    On Wed, Apr 6, 2011 at 6:46 AM, Grant Ingersoll wrote:

    On Apr 3, 2011, at 3:50 PM, Ryan McKinley wrote:
    If we do elect for option A, I would also suggest we delete the
    spatial contrib (in 4.0) and have solr depend on the external .jar --
    this way lucene users would have what they need directly with the
    external .jar, and solr users would get lots of fancy new stuff
    off-the-shelf.

    At the end of the day, Solr only uses a few methods in the Lucene contrib:
    the geohash stuff and a few other distance utils (some static methods that
    I moved from Solr down to Lucene). That's about it. Everything else is
    just FieldTypes and function queries (the tier stuff is broken b/c of the
    sinusoidal projection impl. and I didn't see a need for the geometry stuff).
    If we move function queries to modules, those utils could just be hidden in
    there (or they could just be put in Solr for that matter if function queries
    stay where they are) and there would be no need for any external dependency.
    Then, there is no spatial package at all and anyone who wishes to work on
    Spatial can go do it in the proposed external project if they need anything
    beyond point search.

    That being said, a separate package/project is what LocalLucene/LocalSolr
    was. You are of course free to do as you wish, but to me things that are
    first order supported by the community seem to stick better than those that
    don't and have the benefit of our extensive testing framework as well as
    resources, especially something as popular as spatial. I think, though, at
    the end of the day, you are just going to see, once again, a forking as
    people will likely be confused about where to contribute. Some will
    continue to contribute to Lucene/Solr b/c that is what they are working on
    (b/c they started w/ point search) and others will contribute to the
    external project. Nothing wrong with that, of course, people can do as they
    wish, it's just why I would prefer a single solution as part of our modules.
    If SIS or something else w/ a better license was as mature as JTS, we
    probably wouldn't even be having the discussion.

    On a different level, for me personally, and I really stress this just my
    personal choice, I don't have much interest in contributing to non-ASF (or
    similar foundation based open source projects) because they don't offer the
    same legal framework, branding, resources and other opportunities that the
    ASF does.
    After some consideration, I agree with a lot of what you say here. Although
    it should be pointed out that bringing LocalLucene into Lucene did not help
    its functionality. Instead we are faced with deprecating/deleting it and
    starting from scratch basically. But that is also something we have to
    prevent from happing again, wherever the code is.

    Given that JTS, like it or not, is at the core of 'beyond point' spatial
    search at the moment, what are our options? You mentioned having the JTS
    components in Apache-extras, isn't that sort of splitting things up too? I
    am pretty keen to see this work stay within the ASF, but I'm struggling to
    see how we can do that at the moment.

    -Grant



    ---------------------------------------------------------------------
    To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: dev-help@lucene.apache.org

    --
    Chris Male | Software Developer | JTeam BV.| www.jteam.nl

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupdev @
categorieslucene
postedApr 3, '11 at 7:50p
activeApr 6, '11 at 7:36p
posts24
users9
websitelucene.apache.org

People

Translate

site design / logo © 2021 Grokbase