FAQ
Hi

I've got a use case where i need to enrich a message by polling a FTP server
so as to get files like "request_1.xml", "request_2.xml", etc.
This FTP server acts as a resource provider, and i want to be able to get
the appropriate file every time a message is handled (the request number is
held in a header of current exchange).

I'm doing the polling in a processor, as pollEnrich can't use headers from
current exchange.

So, i've configured the URI of the FTP server with idempotent=false (which
theoritically allows me to poll the same file multiple times).

But it appears that the name of the file is held in the in-memory
inProgressRepository of the endpoint and is never removed.
So the second time I try to get a request.xml file, it doesn't work :
[TRACE] remote.FtpConsumer Skipping as file is already in progress:
request_1.xml

A workaround is to remove the file name from the inProgressRepository by
myself after getting the file and closing the consumer.

Is there a way to automatically clean the inProgressRepository after file is
handled ? Am I missing something ?

Thanks.

--
View this message in context: http://camel.465427.n5.nabble.com/inProgressRepository-not-cleaned-in-GenericFileEndpoint-tp4722713p4722713.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Search Discussions

  • Claus Ibsen at Aug 22, 2011 at 10:12 am

    On Mon, Aug 22, 2011 at 11:22 AM, aurelienmasse wrote:
    Hi

    I've got a use case where i need to enrich a message by polling a FTP server
    so as to get files like "request_1.xml", "request_2.xml", etc.
    This FTP server acts as a resource provider, and i want to be able to get
    the appropriate file every time a message is handled (the request number is
    held in a header of current exchange).

    I'm doing the polling in a processor, as pollEnrich can't use headers from
    current exchange.

    So, i've configured the URI of the FTP server with idempotent=false (which
    theoritically allows me to poll the same file multiple times).
    What version of Camel are you using?
    And have you tried upgrading to the latest release and see if it now works?

    But it appears that the name of the file is held in the in-memory
    inProgressRepository of the endpoint and is never removed.
    So the second time I try to get a request.xml file, it doesn't work :
    [TRACE] remote.FtpConsumer Skipping as file is already in progress:
    request_1.xml

    A workaround is to remove the file name from the inProgressRepository by
    myself after getting the file and closing the consumer.

    Is there a way to automatically clean the inProgressRepository after file is
    handled ? Am I missing something ?

    Thanks.

    --
    View this message in context: http://camel.465427.n5.nabble.com/inProgressRepository-not-cleaned-in-GenericFileEndpoint-tp4722713p4722713.html
    Sent from the Camel - Users mailing list archive at Nabble.com.


    --
    Claus Ibsen
    -----------------
    FuseSource
    Email: cibsen@fusesource.com
    Web: http://fusesource.com
    Twitter: davsclaus, fusenews
    Blog: http://davsclaus.blogspot.com/
    Author of Camel in Action: http://www.manning.com/ibsen/
  • Aurelienmasse at Aug 22, 2011 at 10:21 am
    I'm using the released version 2.8.0


    Below is a code snippet of what i'm currently doing :

    String uri = buildFullUri(baseUrl, fileName);

    GenericFileEndpoint endpoint = (GenericFileEndpoint)
    exchange.getContext().getEndpoint(uri);

    PollingConsumer consumer = endpoint.createPollingConsumer();

    consumer.start();
    Exchange result = consumer.receive(DEFAULT_TIMEOUT);
    consumer.stop();
    if (result == null) {
    throw new IllegalStateException("Requested resource [" + fileName + "]
    has not been found.");
    }
    GenericFile body = result.getIn().getBody(GenericFile.class);

    endpoint.getInProgressRepository().remove(body.getAbsoluteFilePath());



    --
    View this message in context: http://camel.465427.n5.nabble.com/inProgressRepository-not-cleaned-in-GenericFileEndpoint-tp4722713p4722880.html
    Sent from the Camel - Users mailing list archive at Nabble.com.
  • Claus Ibsen at Aug 22, 2011 at 10:38 am
    Ah read the javadoc of the consumer template, the doneUoW
    http://camel.apache.org/maven/current/camel-core/apidocs/org/apache/camel/ConsumerTemplate.html

    We should actually add same details to the PollingConsumer API javadoc.

    Since you return an Exchange you need to manually done its UoW when
    you are done using it.


    On Mon, Aug 22, 2011 at 12:20 PM, aurelienmasse
    wrote:
    I'm using the released version 2.8.0


    Below is a code snippet of what i'm currently doing :

    String uri = buildFullUri(baseUrl, fileName);

    GenericFileEndpoint endpoint = (GenericFileEndpoint)
    exchange.getContext().getEndpoint(uri);

    PollingConsumer consumer = endpoint.createPollingConsumer();

    consumer.start();
    Exchange result = consumer.receive(DEFAULT_TIMEOUT);
    consumer.stop();
    if (result == null) {
    throw new IllegalStateException("Requested resource [" + fileName + "]
    has not been found.");
    }
    GenericFile body = result.getIn().getBody(GenericFile.class);

    endpoint.getInProgressRepository().remove(body.getAbsoluteFilePath());



    --
    View this message in context: http://camel.465427.n5.nabble.com/inProgressRepository-not-cleaned-in-GenericFileEndpoint-tp4722713p4722880.html
    Sent from the Camel - Users mailing list archive at Nabble.com.


    --
    Claus Ibsen
    -----------------
    FuseSource
    Email: cibsen@fusesource.com
    Web: http://fusesource.com
    Twitter: davsclaus, fusenews
    Blog: http://davsclaus.blogspot.com/
    Author of Camel in Action: http://www.manning.com/ibsen/
  • Aurelienmasse at Aug 22, 2011 at 12:03 pm
    Hello Claus !

    thanks for the quick answer.

    I've changed the code to use a consumerTemplate instead of a
    pollingConsumer, so as to use the doneUoW method. So the changed code is :

    String uri = buildFullUri(baseUrl, fileName);

    ConsumerTemplate template = exchange.getContext().createConsumerTemplate();
    Exchange result = template.receive(uri, DEFAULT_TIMEOUT);

    GenericFile body = result.getIn().getBody(GenericFile.class);
    template.doneUoW(exchange);
    template.stop();

    Unfortunately, it didn't change the issue, on the second time I poll the
    endpoint, I still got :
    [TRACE] remote.FtpConsumer Skipping as file is already in progress:
    request_1.xml




    --
    View this message in context: http://camel.465427.n5.nabble.com/inProgressRepository-not-cleaned-in-GenericFileEndpoint-tp4722713p4723141.html
    Sent from the Camel - Users mailing list archive at Nabble.com.
  • Claus Ibsen at Aug 22, 2011 at 12:06 pm

    On Mon, Aug 22, 2011 at 2:03 PM, aurelienmasse wrote:
    Hello Claus !

    thanks for the quick answer.

    I've changed the code to use a consumerTemplate instead of a
    pollingConsumer, so as to use the doneUoW method. So the changed code is :

    String uri = buildFullUri(baseUrl, fileName);

    ConsumerTemplate template = exchange.getContext().createConsumerTemplate();
    Exchange result = template.receive(uri, DEFAULT_TIMEOUT);

    GenericFile body = result.getIn().getBody(GenericFile.class);
    template.doneUoW(exchange);
    template.stop();

    Unfortunately, it didn't change the issue, on the second time I poll the
    endpoint, I still got :
    [TRACE] remote.FtpConsumer Skipping as file is already in progress:
    request_1.xml

    What version of Camel are you using?

    --
    View this message in context: http://camel.465427.n5.nabble.com/inProgressRepository-not-cleaned-in-GenericFileEndpoint-tp4722713p4723141.html
    Sent from the Camel - Users mailing list archive at Nabble.com.


    --
    Claus Ibsen
    -----------------
    FuseSource
    Email: cibsen@fusesource.com
    Web: http://fusesource.com
    Twitter: davsclaus, fusenews
    Blog: http://davsclaus.blogspot.com/
    Author of Camel in Action: http://www.manning.com/ibsen/
  • Aurelienmasse at Aug 22, 2011 at 12:08 pm
    I'm using 2.8.0

    --
    View this message in context: http://camel.465427.n5.nabble.com/inProgressRepository-not-cleaned-in-GenericFileEndpoint-tp4722713p4723151.html
    Sent from the Camel - Users mailing list archive at Nabble.com.
  • Claus Ibsen at Aug 23, 2011 at 7:00 am

    On Mon, Aug 22, 2011 at 2:07 PM, aurelienmasse wrote:
    I'm using 2.8.0
    I created an unit test on trunk, and it works fine there
    http://svn.apache.org/viewvc?rev=1160565&view=rev

    --
    View this message in context: http://camel.465427.n5.nabble.com/inProgressRepository-not-cleaned-in-GenericFileEndpoint-tp4722713p4723151.html
    Sent from the Camel - Users mailing list archive at Nabble.com.


    --
    Claus Ibsen
    -----------------
    FuseSource
    Email: cibsen@fusesource.com
    Web: http://fusesource.com
    Twitter: davsclaus, fusenews
    Blog: http://davsclaus.blogspot.com/
    Author of Camel in Action: http://www.manning.com/ibsen/
  • Aurelienmasse at Aug 23, 2011 at 7:51 am
    Hello Claus

    Ok i've found the problem, it appeared to be a simple mistake on my code.
    It works perfectly now using the ConsumerTemplate.

    Thanks a lot for your help and for the time you spent.



    --
    View this message in context: http://camel.465427.n5.nabble.com/inProgressRepository-not-cleaned-in-GenericFileEndpoint-tp4722713p4725854.html
    Sent from the Camel - Users mailing list archive at Nabble.com.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupusers @
categoriescamel
postedAug 22, '11 at 9:22a
activeAug 23, '11 at 7:51a
posts9
users2
websitecamel.apache.org

2 users in discussion

Aurelienmasse: 5 posts Claus Ibsen: 4 posts

People

Translate

site design / logo © 2022 Grokbase