At a high level, I think the idea is fine (and I've seen a number of
people that wanted it).
The question is more around one of implementation... would it make a
mess of things or not.
The answer to that I think is probably mostly related to issues around
how zookeeper is currently handled.
I don't see any issues with other things like spinning up a core when
a request comes in for it.

On Tue, Aug 13, 2013 at 4:26 PM, Erick Erickson wrote:
There was a question on the user's list today about making lazily-loaded
(aka transient) cores work with SolrCloud where I basically punted and said
"not designed with that in mind". I've kind of avoided thinking about this
as the use-case; the transient code wasn't written with SolrCloud in mind.

But what is the general reaction to that pairing? Mostly I'm looking for
feedback at the level of "no way that could work without invasive changes to
SolrCloud, don't even go there" or "sure, just allow ZK to get a list of all
cores and it'll be fine, the user is responsible for the quirks though".
Some questions that come to my mind:
Is a core that's not loaded be considered "live" by ZK? Would simply
returning a list of all cores (both loaded and not loaded) be sufficient for
ZK? (this list is already available so the admin UI can list all cores).
Does SolrCloud distributed update processing go through (or could be made
to go through) the path that autoloads a core?
Ditto for querying. I suspect the answer to both is that it'll "just
Would the idea of waiting for all the cores to load on all the nodes for
an update be totally unacceptable? We already have the distributed deadlock
potential, this seems to make that more likely by lengthening out the time
the semaphore in question is held.
Would re-synching/leader election be an absolute nightmare? I can imagine
that if all the cores for a particular shard weren't loaded at startup,
there'd be a terrible time waiting for leader election for instance.
Stuff I haven't thought of
Mostly I'm trying to get a "sense of the community" here about whether
supporting transient cores in SolrCloud mode would be something that would
be easy/do-able/really_hard/totally_unacceptable.

To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

Search Discussions

Discussion Posts


Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 2 of 5 | next ›
Discussion Overview
groupdev @
postedAug 13, '13 at 8:27p
activeAug 14, '13 at 2:42a



site design / logo © 2021 Grokbase