FAQ
Hello.

for my Delta-Import, i get the Id's which are should be updatet from an
extra table in my database.

... when dih finished the delta-import it's necessary, that the table with
the ID's is to delete.

can i put a sql query in the DIH for that issue ? this code should only be
send to the database when import was succesfully ...

any suggestions ??

thxxx
--
View this message in context: http://lucene.472066.n3.nabble.com/DIH-behavior-after-a-import-Log-delete-table-tp823232p823232.html
Sent from the Solr - User mailing list archive at Nabble.com.

Search Discussions

  • Ahmet Arslan at May 17, 2010 at 10:21 am

    for my Delta-Import, i get the Id's which are should be
    updatet from an
    extra table in my database.

    ... when dih finished the delta-import it's necessary, that
    the table with
    the ID's is to delete.

    can i put a sql query in the DIH for that issue ?
    deletedPkQuery (sql query) is used in delta-import to delete documents from solr index. Is this what you mean?
  • Stockii at May 17, 2010 at 10:24 am
    hm .. no =(

    i want to delete from a mysql database, not from my solr-index
    --
    View this message in context: http://lucene.472066.n3.nabble.com/DIH-behavior-after-a-import-Log-delete-table-tp823232p823264.html
    Sent from the Solr - User mailing list archive at Nabble.com.
  • Stockii at May 17, 2010 at 10:54 am
    hm i think i can use "deletedPkQuery" but it dont works for me, but maybe you
    can help me. here is my conifg.

    <entity name="item" pk="id" transformer="script:BoostDoc"
    query="select i.id, i.shop_id, i.is_active, i.shop ...

    deltaImportQuery="select i.id, i.s ...WHERE ... AND
    i.id='${dataimporter.delta.update_id}

    deltaQuery= "SELECT update_id FROM solr_imports"

    deletedPkQuery ="DELETE FROM solr_imports WHERE
    solr_imports.update_id='${dataimporter.item.update_id}' ">


    so, i only want to delete these ID'S which are updatet. this is my
    exception:
    SCHWERWIEGEND: Delta Import Failed
    org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to
    execute query: DELETE FROM solr_imports WHERE solr_imports.update_id=''
    Processing Document # 1


    so the deletedPkQuery get no ID's =(



    --
    View this message in context: http://lucene.472066.n3.nabble.com/DIH-behavior-after-a-import-Log-delete-table-tp823232p823298.html
    Sent from the Solr - User mailing list archive at Nabble.com.
  • Ahmet Arslan at May 17, 2010 at 11:57 am

    hm i think i can use "deletedPkQuery" but it dont works for
    me, but maybe you
    can help me. here is my conifg.

    <entity name="item" pk="id"
    transformer="script:BoostDoc"
    query="select i.id, i.shop_id, i.is_active, i.shop
    ...

    deltaImportQuery="select i.id, i.s ...WHERE
    ...  AND
    i.id='${dataimporter.delta.update_id}

    deltaQuery=    "SELECT update_id FROM
    solr_imports"

    deletedPkQuery ="DELETE FROM solr_imports
    WHERE
    solr_imports.update_id='${dataimporter.item.update_id}'
    ">


    so, i only want to delete these ID'S which are updatet.
    this is my
    exception:
    SCHWERWIEGEND: Delta Import Failed
    org.apache.solr.handler.dataimport.DataImportHandlerException:
    Unable to
    execute query: DELETE FROM solr_imports WHERE
    solr_imports.update_id=''
    Processing Document # 1


    so the deletedPkQuery  get no ID's =(
    I am not sure what will happen with this kind of deletedPkQuery. Probably you won't be able to use ${dataimporter.delta.update_id} variable. But i am also curios what will happen. can you try this:

    deletedPkQuery ="DELETE FROM solr_imports WHERE
    solr_imports.update_id='${dataimporter.delta.update_id}'

    Since your deltaQuery does not contain WHERE clause, why not delete (with another program or script) solr_imports table after delta-import?
  • Stockii at May 17, 2010 at 12:05 pm
    thats what i try ! :D

    i dont want to do this with another script, because i never know when a
    delta-import is finished, and when he is completed, i dont know with which
    result. complete, fail, ?!?!?

    so i thought dih can delete the updated ID's in my database =(

    i try also to empty the table like this: TRUNCTATE TABLE solr_imports

    this works, but i get new exception...
    --
    View this message in context: http://lucene.472066.n3.nabble.com/DIH-behavior-after-a-import-Log-delete-table-tp823232p823413.html
    Sent from the Solr - User mailing list archive at Nabble.com.
  • Ahmet Arslan at May 17, 2010 at 5:00 pm

    thats what i try ! :D

    i dont want to do this with another script, because i never
    know when a
    delta-import is finished, and when he is completed, i dont
    know with which
    result. complete, fail, ?!?!?
    If you are updating your index *only* with DIH, after every full/delta import commit and optimize occurs by default. And you can auto-run your special code after every optimize/commit.

    http://wiki.apache.org/solr/SolrConfigXml#A.22Update.22_Related_Event_Listeners

    solrconfig.xml:

    <listener event="postCommit" class="solr.RunExecutableListener">
    <str name="exe">solr/bin/test</str>
    <str name="dir">.</str>
    <bool name="wait">true</bool>
    </listener>

    test file:

    #!/bin/bash
    java -jar /home/search/junk.jar
  • Stockii at May 17, 2010 at 7:38 pm
    oh, nice.

    so i can me make a jar-file with the query i need and in solrconfig.xml i
    need to define this..
    --
    View this message in context: http://lucene.472066.n3.nabble.com/DIH-behavior-after-a-import-Log-delete-table-tp823232p824484.html
    Sent from the Solr - User mailing list archive at Nabble.com.
  • Stockii at May 18, 2010 at 8:32 am
    how can i say that solr should start the jar after every Delta-Import NOT
    after every Full-Import ?????
    --
    View this message in context: http://lucene.472066.n3.nabble.com/DIH-behavior-after-a-import-Log-delete-table-tp823232p825717.html
    Sent from the Solr - User mailing list archive at Nabble.com.
  • Ahmet Arslan at May 18, 2010 at 1:49 pm

    how can i say that solr should start the jar after every
    Delta-Import NOT
    after every Full-Import ?????
    You cannot distinguish between delta or full. So you need to do it in your jar program. In your java program you need to send GET method to url
    http://localhost:8080/solr/dataimport

    if result string/xml contains contains '<str name="status">idle</str>' and 'Delta Dump started' then you can truncate your table.

    if result string contains contains '<str name="status">idle</str>' and 'Full Dump Started' then do nothing.
  • Stockii at May 19, 2010 at 2:30 pm
    hey, thx

    i did all what you say.

    createn an Jar-file. this jar file delete my table.

    but SOLR absolute dont want to start this JAR. i put a run.bat file into my
    folder where is my jar saved. this batch-file runs and delete the table, but
    when solr start this batch-file. it doesnt work. i dont know why. !?!?!?
    i test the batch-file in different wayy and it should be work... help ^^

    windows xp for test ;-)
    --
    View this message in context: http://lucene.472066.n3.nabble.com/DIH-behavior-after-a-import-Log-delete-table-tp823232p829230.html
    Sent from the Solr - User mailing list archive at Nabble.com.
  • Ahmet Arslan at May 19, 2010 at 3:47 pm

    createn an Jar-file. this jar file delete my table.

    but SOLR absolute dont want to start this JAR. i put a
    run.bat file into my
    folder where is my jar saved. this batch-file runs and
    delete the table, but
    when solr start this batch-file. it doesnt work. i dont
    know why. !?!?!?
    i test the batch-file in different wayy and it should be
    work... help ^^

    windows xp for test ;-)
    I don't know why but, it seems that we need to set dir other than '.'
    Anyway I got it working in Windows in two ways:

    1-)
    <updateHandler class="solr.DirectUpdateHandler2">
    <listener event="postCommit" class="solr.RunExecutableListener">
    <str name="exe">java</str>
    <str name="dir">solr/bin</str>
    <arr name="args"> <str>-jar</str> <str>junk.jar</str> </arr>
    <bool name="wait">true</bool>
    </listener>
    </updateHandler>

    2-) Giving full paths:

    <updateHandler class="solr.DirectUpdateHandler2">
    <listener event="postCommit" class="solr.RunExecutableListener">
    <str name="exe">C:\test.bat</str>
    <str name="dir">C:\</str>
    <bool name="wait">true</bool>
    </listener>
    </updateHandler>

    It should work this time on windows.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupsolr-user @
categorieslucene
postedMay 17, '10 at 10:05a
activeMay 19, '10 at 3:47p
posts12
users2
websitelucene.apache.org...

2 users in discussion

Stockii: 7 posts Ahmet Arslan: 5 posts

People

Translate

site design / logo © 2021 Grokbase