FAQ
hello,

     I am trying to make a scenario execute using Camel Spring DSL .

     My idea is to processing some files in a numerical seq order from a
folder , after processing a file I need to check for the next file
sequence which need to be processed from DB.
     The logic of get the next sequence is okay but i am not sure how do i
pass this information to another route to process that single file only . I
though about using sequencer etc. but it does not seem fitting it. The only
option i see is using File Filter .
      But how do I pass this information to a another route . For example I
have to test below approach(Not sure if its right way to go).


    I tried the UserCSVTODB route just to test , the filter worked fine but
the other files also got moved to the complete folder. I expected that only
the matched file in filter should be moved and not others. Is there any way
I can make this happen , so in the next run it picks
   the next sequence file . (probably I need to loop back which is another
question )


<route shutdownRunningTask="CompleteCurrentTaskOnly" id="1">
     <from uri="direct:start"/>
    <setHeader headerName="filename">
     <simple>data.4.csv</simple> (this will be replaced which a processor
or bean call)
   </setHeader>
       <to uri="UserCSVToDB"/>
   </route>


<route shutdownRunningTask="CompleteCurrentTaskOnly" id="UserCSVToDB">
     <from
uri="file:///C:\FS\processing?readLock=true&amp;charset=utf-8&amp;move=C:\FS\complete"/>
    <filter>
        <simple>${in.header.CamelFileName} == 'data.4.csv'</simple>
        <to uri="mock:result"/>
    </filter>
  </route>



regards,
Felix

Search Discussions

  • Claus Ibsen at Jan 28, 2016 at 7:59 am
    You can use a singleton java bean to store the state of that sequence
    and then in the file route use that in a file filter to only pickup
    the correct file.

    You can implement a custom file filter, using the filter option, where
    you can then access that singleton bean to get that sequence number.
    The other route can then also call that singleton bean where it can
    update the sequence.

    On Wed, Jan 27, 2016 at 10:34 PM, Felix Thomas wrote:
    hello,

    I am trying to make a scenario execute using Camel Spring DSL .

    My idea is to processing some files in a numerical seq order from a
    folder , after processing a file I need to check for the next file
    sequence which need to be processed from DB.
    The logic of get the next sequence is okay but i am not sure how do i
    pass this information to another route to process that single file only . I
    though about using sequencer etc. but it does not seem fitting it. The only
    option i see is using File Filter .
    But how do I pass this information to a another route . For example I
    have to test below approach(Not sure if its right way to go).


    I tried the UserCSVTODB route just to test , the filter worked fine but
    the other files also got moved to the complete folder. I expected that only
    the matched file in filter should be moved and not others. Is there any way
    I can make this happen , so in the next run it picks
    the next sequence file . (probably I need to loop back which is another
    question )


    <route shutdownRunningTask="CompleteCurrentTaskOnly" id="1">
    <from uri="direct:start"/>
    <setHeader headerName="filename">
    <simple>data.4.csv</simple> (this will be replaced which a processor
    or bean call)
    </setHeader>
    <to uri="UserCSVToDB"/>
    </route>


    <route shutdownRunningTask="CompleteCurrentTaskOnly" id="UserCSVToDB">
    <from
    uri="file:///C:\FS\processing?readLock=true&amp;charset=utf-8&amp;move=C:\FS\complete"/>
    <filter>
    <simple>${in.header.CamelFileName} == 'data.4.csv'</simple>
    <to uri="mock:result"/>
    </filter>
    </route>



    regards,
    Felix


    --
    Claus Ibsen
    -----------------
    http://davsclaus.com @davsclaus
    Camel in Action 2: https://www.manning.com/ibsen2
  • Fxthomas at Jan 28, 2016 at 8:13 am
    thanks, will try that



    --
    View this message in context: http://camel.465427.n5.nabble.com/Camel-file-component-URI-dynamic-options-Filter-query-tp5776861p5776884.html
    Sent from the Camel - Users mailing list archive at Nabble.com.
  • Fxthomas at Jan 29, 2016 at 4:41 pm
    hello claus,

       How do I get the singleton bean nside my FileFilter java code. I tried
    using the @BeanInject annotation, but it looks like it creates different
    bean object and not using the one I have defined in the XML.

    regards,
    Felix T





    --
    View this message in context: http://camel.465427.n5.nabble.com/Camel-file-component-URI-dynamic-options-Filter-query-tp5776861p5776996.html
    Sent from the Camel - Users mailing list archive at Nabble.com.
  • Claus Ibsen at Jan 29, 2016 at 4:45 pm
    You can just refer to it in xml too, to setup the IoC in XML

    <bean id="foo">

    <bean id="myFilter" ..>
        <property name="foo" ref="foo"/>
    </bean>


    On Fri, Jan 29, 2016 at 5:40 PM, fxthomas wrote:
    hello claus,

    How do I get the singleton bean nside my FileFilter java code. I tried
    using the @BeanInject annotation, but it looks like it creates different
    bean object and not using the one I have defined in the XML.

    regards,
    Felix T





    --
    View this message in context: http://camel.465427.n5.nabble.com/Camel-file-component-URI-dynamic-options-Filter-query-tp5776861p5776996.html
    Sent from the Camel - Users mailing list archive at Nabble.com.


    --
    Claus Ibsen
    -----------------
    http://davsclaus.com @davsclaus
    Camel in Action 2: https://www.manning.com/ibsen2
  • Fxthomas at Feb 1, 2016 at 2:24 pm
    hello,

             I have tried with the below route & filter configuration , its
    works fine once , the filter is never called again after the first run .
    Do i need to call the filter explicitly to start the poll again? Since in
    my Filter I am getting the sequence from the Db always in the accept method.

    <route shutdownRunningTask="CompleteCurrentTaskOnly" id="UserCSVToDB" >
           <from
    uri="file:///C:\FS\processing\?delay=3600000&amp;readLock=true&amp;move=C:\FS\complete&amp;filter=#filterDS&amp;charset=utf-8"/>
             <setHeader headerName="sourcename">
              <simple>DS</simple>
                   </setHeader>
                    <setHeader headerName="fileseq">
              <method ref="DatabaseBean"
    method="getSequence(${header.sourcename})" />
                   </setHeader>

           <split streaming="true">
             <tokenize token="\n"></tokenize>
             <unmarshal>
               <csv/>
             </unmarshal>
             <log message="The message contains ${body}"/>
           </split>
           <to
    uri="bean:DatabaseBean?method=updateSequence(${header.sourcename},${header.fileseq})"/>
         </route>





    --
    View this message in context: http://camel.465427.n5.nabble.com/Camel-file-component-URI-dynamic-options-Filter-query-tp5776861p5777100.html
    Sent from the Camel - Users mailing list archive at Nabble.com.
  • Fxthomas at Feb 1, 2016 at 5:08 pm
    I tried with a new approach it works atleast for basic condition . But the
    problem is if there is any file left in the folder which did not match in
    the Run , the route keeps on polling and the filter is called everytime .

    1) Is there any way in which the route can be told to stop pooling after
    subsequent polls which did not result any file processing.

    2) There could be a gap in the sequence, i.e. after 2 , 4 can be the next
    file how can i come to know that the route has finished processing one
    cylce, so that I can increment again in filter and the route will work
    again.

    3) Incase there is a file les than sequence or any junk file , it should be
    moved to different folder. Since I am using a Filter , not sure if that can
    be done . I can configure a different route to counter this.



      <route shutdownRunningTask="CompleteCurrentTaskOnly" id="RouteFilter" >
          <from uri="timer://foo?period=2h"/>
          <to uri="controlbus:route?routeId=UserCSVToDB&amp;action=start" />
        </route>


         <route shutdownRunningTask="CompleteCurrentTaskOnly" id="UserCSVToDB"
    autoStartup="false">
           <from
    uri="file:///C:\FS\processing\?readLock=true&amp;move=C:\FS\complete&amp;filter=#filterDS&amp;charset=utf-8"/>

             <setHeader headerName="sourcename">
              <simple>DS</simple>
                   </setHeader>
                    <setHeader headerName="fileseq">
              <method ref="DatabaseBean"
    method="getSequence(${header.sourcename})" />
                   </setHeader>

           <split streaming="true">
             <tokenize token="\n"></tokenize>
             <unmarshal>
               <csv/>
             </unmarshal>
             <log message="The message contains ${body}"/>
           </split>
           <to
    uri="bean:DatabaseBean?method=updateSequence(${header.sourcename},${header.fileseq})"/>
         </route>



    --
    View this message in context: http://camel.465427.n5.nabble.com/Camel-file-component-URI-dynamic-options-Filter-query-tp5776861p5777107.html
    Sent from the Camel - Users mailing list archive at Nabble.com.
  • Claus Ibsen at Feb 2, 2016 at 8:42 am

    On Mon, Feb 1, 2016 at 6:08 PM, fxthomas wrote:
    I tried with a new approach it works atleast for basic condition . But the
    problem is if there is any file left in the folder which did not match in
    the Run , the route keeps on polling and the filter is called everytime .

    1) Is there any way in which the route can be told to stop pooling after
    subsequent polls which did not result any file processing.
    You can use a route policy, with sendEmptyMessageWhenIdle=true, and
    then detect if its an empty file and stop the route.
    And mind you would need to filter the empty file in your route so you
    do not process the empty file.

    Also you can use the backoff to not poll so often when there is no
    files. So it wont poll so often, though its still running.

    2) There could be a gap in the sequence, i.e. after 2 , 4 can be the next
    file how can i come to know that the route has finished processing one
    cylce, so that I can increment again in filter and the route will work
    again.
    You can add a callback to the exchange which gets called when its done
    / or use event notifier etc.



    3) Incase there is a file les than sequence or any junk file , it should be
    moved to different folder. Since I am using a Filter , not sure if that can
    be done . I can configure a different route to counter this.



    <route shutdownRunningTask="CompleteCurrentTaskOnly" id="RouteFilter" >
    <from uri="timer://foo?period=2h"/>
    <to uri="controlbus:route?routeId=UserCSVToDB&amp;action=start" />
    </route>


    <route shutdownRunningTask="CompleteCurrentTaskOnly" id="UserCSVToDB"
    autoStartup="false">
    <from
    uri="file:///C:\FS\processing\?readLock=true&amp;move=C:\FS\complete&amp;filter=#filterDS&amp;charset=utf-8"/>

    <setHeader headerName="sourcename">
    <simple>DS</simple>
    </setHeader>
    <setHeader headerName="fileseq">
    <method ref="DatabaseBean"
    method="getSequence(${header.sourcename})" />
    </setHeader>

    <split streaming="true">
    <tokenize token="\n"></tokenize>
    <unmarshal>
    <csv/>
    </unmarshal>
    <log message="The message contains ${body}"/>
    </split>
    <to
    uri="bean:DatabaseBean?method=updateSequence(${header.sourcename},${header.fileseq})"/>
    </route>



    --
    View this message in context: http://camel.465427.n5.nabble.com/Camel-file-component-URI-dynamic-options-Filter-query-tp5776861p5777107.html
    Sent from the Camel - Users mailing list archive at Nabble.com.


    --
    Claus Ibsen
    -----------------
    http://davsclaus.com @davsclaus
    Camel in Action 2: https://www.manning.com/ibsen2
  • Fxthomas at Feb 2, 2016 at 9:28 am
    hello Claus,

    Thanks for the Tip.

    I had tried for the point 2) using a eventnotifer class, but the results are
    quiet confusing or maybe I am handling some other event . But i see the
    Exchange completed is fired for each message sent. So ite becomes diffcult
    to know which is the exact exchaneg complete event. I will try with a
    callback and see if that works fine.

    Below is MY event notifier code.

    @Override
         public void notify(EventObject event) throws Exception {

      if (event instanceof ExchangeCompletedEvent) {
          ExchangeCompletedEvent exchangeCompletedEvent =
    (ExchangeCompletedEvent) event;
          Exchange exchange = exchangeCompletedEvent.getExchange();
          String FileName =
    (String)exchange.getProperty(Exchange.FILE_NAME_CONSUMED);
          System.out.println("EXCHANGE COMPLETED "+FileName);

      }else if(event instanceof ExchangeCreatedEvent){
          ExchangeCreatedEvent exchangeCreatedEvent = (ExchangeCreatedEvent)
    event;
          Exchange exchange = exchangeCreatedEvent.getExchange();
          String FileName = (String)
    exchange.getProperty(Exchange.FILE_NAME_CONSUMED);
          System.out.println("EVENTTTTTTTTTTTT "+FileName);


      }else{
          System.out.println("EVENT NOt KNOWN "+event.getClass().getName());
      }

         }


         protected void doStart() throws Exception {
      // filter out unwanted events
      setIgnoreCamelContextEvents(true);
      setIgnoreServiceEvents(true);
      setIgnoreRouteEvents(false);
      setIgnoreExchangeCreatedEvent(false);
      setIgnoreExchangeCompletedEvent(false);
      setIgnoreExchangeFailedEvents(false);
      setIgnoreExchangeRedeliveryEvents(true);
      setIgnoreExchangeSentEvents(false);
         }



    The Results On Console.


    EVENTTTTTTTTTTTT null
    1 CSV Line - printed
    EXCHANGE COMPLETED null

    EVENTTTTTTTTTTTT null
    2 second CSV line - printed
    EXCHANGE COMPLETED null

    EVENT NOt KNOWN org.apache.camel.management.event.ExchangeSendingEvent
    Updating sequence 0
    EVENT NOt KNOWN org.apache.camel.management.event.ExchangeSentEvent
    EXCHANGE COMPLETED null



    --
    View this message in context: http://camel.465427.n5.nabble.com/Camel-file-component-URI-dynamic-options-Filter-query-tp5776861p5777145.html
    Sent from the Camel - Users mailing list archive at Nabble.com.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupusers @
categoriescamel
postedJan 27, '16 at 9:34p
activeFeb 2, '16 at 9:28a
posts9
users2
websitecamel.apache.org

2 users in discussion

Fxthomas: 6 posts Claus Ibsen: 3 posts

People

Translate

site design / logo © 2021 Grokbase