FAQ
Hi list,

we have a slightly different use case of Optimistic Concurrency (as
explained in http://yonik.com/solr/optimistic-concurrency/).

Our current stack uses AWS DynamoDB to guarantee consistency of the
documents, as explained bellow:

Fase 1: an external application puts documents into an AWS SQS queue (Q1)
Fase 2: a job that we wrote reads these documents, does some processing and
writes the result to a AWS DynamoDB table AND to another AWS SQS queue (Q2)
Fase 3: another job reads Q2 and then writes the document to Solr

Since SQS doesn't guarantee message order, the Fase 2's job does a
"Conditional write" to DynamoDB, so that the document will only be written
to the table should the document's timestamp be newer than the version that
is already there.

We want to eliminate this conditional write to DynamoDB using Solr own
capabilities. Since we need to know the current _version_ of Solr document,
we would need to make a GET request (to fetch the current _version_) before
issuing the PUT request. Did we get how the way Solr's Optimistic
Concurrency system works right?

Our ideia would be to change the _version_'s check be less restrictive and
allow for it to be greater or equal to the current value (as opposed to the
equal check that is currently implemented). This change could be done here:

https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/update/processor/DistributedUpdateProcessor.java#L1038-L1047

- if ( versionOnUpdate == foundVersion || (versionOnUpdate < 0
&& foundVersion < 0) || (versionOnUpdate==1 && foundVersion > 0) ) {
+ if ( checkVersion(versionOnUpdate, foundVersion) ) {
+ protected boolean checkVersion(long versionOnUpdate, long foundVersion)
{
+ return versionOnUpdate == foundVersion || (versionOnUpdate < 0 &&
foundVersion < 0) || (versionOnUpdate==1 && foundVersion > 0);
+ }
+

Any thoughts?

Best regards,
[image: Visite o nosso site] <http://www.vivareal.com.br/>
     Daniel Mezzatto
Desenvolvedor de Software
+55 (11) 3150 4646 | Skype:(daniel.mezzatto)
daniel.mezzatto@vivareal.com
Rua Bela Cintra, 539 - Consolação - 01415-001 - São Paulo -SP

Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupsolr-user @
categorieslucene
postedJun 11, '16 at 10:45p
activeJun 11, '16 at 10:45p
posts1
users1
websitelucene.apache.org...

1 user in discussion

daniel Mezzatto Rodrigues: 1 post

People

Translate

site design / logo © 2019 Grokbase