FAQ
Hey guys,

I'm trying to write a Camel route that will only kick off once two files are
present on the server, but I can't seem to figure out an easy way to do
this. I was hoping there'd be a Simple command for this or some feature in
the File component, but I haven't found anything that looks promising.

I was thinking I could start my route with a file endpoint, then do a poll
enrich on the second file and kick out an error if the second file doesn't
exist, but this seems a little kludgy. Is there a better way to accomplish
this?

Thanks.



--
View this message in context: http://camel.465427.n5.nabble.com/File-Exists-tp5746608.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Search Discussions

  • Richard Kettelerij at Jan 30, 2014 at 7:52 am
    Hi Jakob,

    Perhaps you can do a poll using an generic filename filter that matches
    both files and combine this with an aggregator (
    http://camel.apache.org/aggregator2.html)?
    So you have the aggregator wait until both files are polled, this can
    probably be done by using a completionPredicate on the aggregator.

    Kind regards,
    Richard
    On Thu, Jan 30, 2014 at 4:49 AM, JKemp wrote:

    Hey guys,

    I'm trying to write a Camel route that will only kick off once two files
    are
    present on the server, but I can't seem to figure out an easy way to do
    this. I was hoping there'd be a Simple command for this or some feature in
    the File component, but I haven't found anything that looks promising.

    I was thinking I could start my route with a file endpoint, then do a poll
    enrich on the second file and kick out an error if the second file doesn't
    exist, but this seems a little kludgy. Is there a better way to accomplish
    this?

    Thanks.



    --
    View this message in context:
    http://camel.465427.n5.nabble.com/File-Exists-tp5746608.html
    Sent from the Camel - Users mailing list archive at Nabble.com.
  • Claus Ibsen at Jan 30, 2014 at 10:15 am

    On Thu, Jan 30, 2014 at 4:49 AM, JKemp wrote:
    Hey guys,

    I'm trying to write a Camel route that will only kick off once two files are
    present on the server, but I can't seem to figure out an easy way to do
    this. I was hoping there'd be a Simple command for this or some feature in
    the File component, but I haven't found anything that looks promising.
    Yeah write a custom filter, and then only return true when both files are there.

    I was thinking I could start my route with a file endpoint, then do a poll
    enrich on the second file and kick out an error if the second file doesn't
    exist, but this seems a little kludgy. Is there a better way to accomplish
    this?

    Thanks.



    --
    View this message in context: http://camel.465427.n5.nabble.com/File-Exists-tp5746608.html
    Sent from the Camel - Users mailing list archive at Nabble.com.


    --
    Claus Ibsen
    -----------------
    Red Hat, Inc.
    Email: cibsen@redhat.com
    Twitter: davsclaus
    Blog: http://davsclaus.com
    Author of Camel in Action: http://www.manning.com/ibsen
    Make your Camel applications look hawt, try: http://hawt.io
  • Ravindra Godbole at Jan 30, 2014 at 10:35 am
    If custom filter is written then it will return true when both files are present and then first file will be picked up but what about the second file.

    During processing of second file, custom filter will return false ?

    Need to check how it behaves ?



    -----Original Message-----
    From: Claus Ibsen
    Sent: Thursday, January 30, 2014 3:44 PM
    To: users@camel.apache.org
    Subject: Re: File Exists
    On Thu, Jan 30, 2014 at 4:49 AM, JKemp wrote:
    Hey guys,

    I'm trying to write a Camel route that will only kick off once two files are
    present on the server, but I can't seem to figure out an easy way to do
    this. I was hoping there'd be a Simple command for this or some feature in
    the File component, but I haven't found anything that looks promising.
    Yeah write a custom filter, and then only return true when both files are there.

    I was thinking I could start my route with a file endpoint, then do a poll
    enrich on the second file and kick out an error if the second file doesn't
    exist, but this seems a little kludgy. Is there a better way to accomplish
    this?

    Thanks.



    --
    View this message in context: http://camel.465427.n5.nabble.com/File-Exists-tp5746608.html
    Sent from the Camel - Users mailing list archive at Nabble.com.


    --
    Claus Ibsen
    -----------------
    Red Hat, Inc.
    Email: cibsen@redhat.com
    Twitter: davsclaus
    Blog: http://davsclaus.com
    Author of Camel in Action: http://www.manning.com/ibsen
    Make your Camel applications look hawt, try: http://hawt.io
    This e-mail and any files transmitted with it are for the sole use of the intended recipient(s) and may contain confidential and privileged information. If you are not the intended recipient(s), please reply to the sender and destroy all copies of the original message. Any unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this email, and/or any action taken in reliance on the contents of this e-mail is strictly prohibited and may be unlawful.
  • Claus Ibsen at Jan 30, 2014 at 10:38 am

    On Thu, Jan 30, 2014 at 11:35 AM, wrote:
    If custom filter is written then it will return true when both files are present and then first file will be picked up but what about the second file.

    During processing of second file, custom filter will return false ?

    Need to check how it behaves ?
    Camel filters first the files, before it processes the batch of files
    that was accepted.


    -----Original Message-----
    From: Claus Ibsen
    Sent: Thursday, January 30, 2014 3:44 PM
    To: users@camel.apache.org
    Subject: Re: File Exists
    On Thu, Jan 30, 2014 at 4:49 AM, JKemp wrote:
    Hey guys,

    I'm trying to write a Camel route that will only kick off once two files are
    present on the server, but I can't seem to figure out an easy way to do
    this. I was hoping there'd be a Simple command for this or some feature in
    the File component, but I haven't found anything that looks promising.
    Yeah write a custom filter, and then only return true when both files are there.

    I was thinking I could start my route with a file endpoint, then do a poll
    enrich on the second file and kick out an error if the second file doesn't
    exist, but this seems a little kludgy. Is there a better way to accomplish
    this?

    Thanks.



    --
    View this message in context: http://camel.465427.n5.nabble.com/File-Exists-tp5746608.html
    Sent from the Camel - Users mailing list archive at Nabble.com.


    --
    Claus Ibsen
    -----------------
    Red Hat, Inc.
    Email: cibsen@redhat.com
    Twitter: davsclaus
    Blog: http://davsclaus.com
    Author of Camel in Action: http://www.manning.com/ibsen
    Make your Camel applications look hawt, try: http://hawt.io
    This e-mail and any files transmitted with it are for the sole use of the intended recipient(s) and may contain confidential and privileged information. If you are not the intended recipient(s), please reply to the sender and destroy all copies of the original message. Any unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this email, and/or any action taken in reliance on the contents of this e-mail is strictly prohibited and may be unlawful.


    --
    Claus Ibsen
    -----------------
    Red Hat, Inc.
    Email: cibsen@redhat.com
    Twitter: davsclaus
    Blog: http://davsclaus.com
    Author of Camel in Action: http://www.manning.com/ibsen
    Make your Camel applications look hawt, try: http://hawt.io
  • Ravindra Godbole at Feb 1, 2014 at 2:54 am
    As Claus has pointed out in the earlier thread, depending on the configuration, listing of files is passed on to the filter for processing. Any new file will be picked up in the next iteration.

    -Ravi
    ________________________________________
    From: JKemp [kemp.jacob@gmail.com]
    Sent: Saturday, February 01, 2014 1:17 AM
    To: users@camel.apache.org
    Subject: Re: File Exists

    Ah, whoops, my fault. I was looking at the File documentation and not at
    File2.

    I think I have a working solution, but I'm wondering if I'm going about this
    in the best way.

    I set up my route with the filter:

             <bean id="bothFilesReceivedFilter"
    class="com.mycompany.BothFilesReceivedFilter" />

             <camelContext xmlns="http://camel.apache.org/schema/spring">
                     <route id="report-file-consolidation">
                             <from
    uri="file:///data/input?delete=true&amp;include=.*.txt&amp;maxMessagesPerPoll=10&amp;filter=#bothFilesReceivedFilter"
    />
                             <split streaming="true">
                                     <tokenize includeTokens="true" token="\n" group="5000" />
                                     <convertBodyTo type="String" charset="UTF-8" />
                                     <setHeader headerName="CamelFileName">
                                             <constant>output-file.txt</constant>
                                     </setHeader>
                     <to uri="log:lineparser?showBody=true&amp;showHeaders=true"
    />
                                     <to uri="file:///data/output?fileExist=Append" />
                             </split>
                     </route>
             </camelContext>

    And I set my filter up like this:

    public class BothFilesReceivedFilter<T> implements GenericFileFilter<T> {
             Logger log = Logger.getLogger();

             HashSet<String> namesReceived = new HashSet<String>();

             public boolean accept(GenericFile<T> file) {
                     log.debug("Received file named: " + file.getFileName());

                     namesReceived.add(file.getFileName());

                     if (file.getFileName().equals("foo.txt")){
                             return namesReceived.contains("bar.txt");
                     }
                     else if (file.getFileName().equals("bar.txt")){
                             return namesReceived.contains("foo.txt");
                     }
                     else {
                             return false;
                     }
             }
    }

    And this seems to work. Is this sort of what you guys meant? The only
    thing I don't really like about this solution is that the file filtering
    still seem to be getting evaluated separately from each other. So with the
    above solution, after both files have been received, if we receive another
    input file with the same name, it gets processed immediately even when there
    isn't the companion file in the input directory as well, because the HashSet
    still shows that both files have been received. But if I clear out the Set
    once both files have been received, I think it will ignore the second file.

    Since I was using maxMessagesPerPoll, I thought this would work as a batch
    consumer, but I don't see the CamelBatchSize header populated when I print
    the headers. Is there any way to make the filtering inspect all the files
    in the batch at once?

    Am I off track here?

    Thanks again.



    --
    View this message in context: http://camel.465427.n5.nabble.com/File-Exists-tp5746608p5746675.html
    Sent from the Camel - Users mailing list archive at Nabble.com.
    This e-mail and any files transmitted with it are for the sole use of the intended recipient(s) and may contain confidential and privileged information. If you are not the intended recipient(s), please reply to the sender and destroy all copies of the original message. Any unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this email, and/or any action taken in reliance on the contents of this e-mail is strictly prohibited and may be unlawful.
  • JKemp at Feb 7, 2014 at 7:41 am
    I feel like I'm still missing something here. I don't see anything in the
    GenericFile that is passed in that represents a listing of the file names
    that the batch polling picked up.
    http://camel.apache.org/maven/camel-2.12.0/camel-core/apidocs/org/apache/camel/component/file/GenericFile.html
    <http://camel.apache.org/maven/camel-2.12.0/camel-core/apidocs/org/apache/camel/component/file/GenericFile.html>

    Can you please clarify how the listing is passed to the filter?

    Thank you.



    --
    View this message in context: http://camel.465427.n5.nabble.com/File-Exists-tp5746608p5746916.html
    Sent from the Camel - Users mailing list archive at Nabble.com.
  • JKemp at Feb 14, 2014 at 2:24 am
    Hey guys,

    Does anyone have any further ideas here? I don't see any obvious way to get
    the listing of file names. Or should I be looking at another way to solve
    this problem?

    Thanks.



    --
    View this message in context: http://camel.465427.n5.nabble.com/File-Exists-tp5746608p5747277.html
    Sent from the Camel - Users mailing list archive at Nabble.com.
  • Claus Ibsen at Feb 14, 2014 at 7:20 am

    On Thu, Feb 13, 2014 at 9:23 PM, JKemp wrote:
    Hey guys,

    Does anyone have any further ideas here? I don't see any obvious way to get
    the listing of file names. Or should I be looking at another way to solve
    this problem?
    You can use the java.io file api to check if a file exists and use
    that to check for that 2nd file.

    Thanks.



    --
    View this message in context: http://camel.465427.n5.nabble.com/File-Exists-tp5746608p5747277.html
    Sent from the Camel - Users mailing list archive at Nabble.com.


    --
    Claus Ibsen
    -----------------
    Red Hat, Inc.
    Email: cibsen@redhat.com
    Twitter: davsclaus
    Blog: http://davsclaus.com
    Author of Camel in Action: http://www.manning.com/ibsen
    Make your Camel applications look hawt, try: http://hawt.io

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupusers @
categoriescamel
postedJan 30, '14 at 4:10a
activeFeb 14, '14 at 7:20a
posts9
users4
websitecamel.apache.org

People

Translate

site design / logo © 2021 Grokbase