FAQ
Short version:

Using [% c.req.uri_with({ page => pager.next_page }) %] is fine for a simple
single-field search (where the form uses GET instead of POST)... but how do
we PAGE through (and/or cache) a multi-field form search that uses POST?


Long version:

This is probably already a posted recipe somewhere, but I haven't had much
luck finding answers via googling combinations of 'perl catalyst post cache
pager' ... so pointers are welcome:


# Form has lots of fields, we'll just nab a sample handful from the POST:

my @terms = map{s/\s+//; $_} split /,/, $form->field('keywords');
# I know, but it's just an example, this isn't a robust search-field parser
:)

my %search = (
asof => {
# date field "asof" must be within date range
'>=' => $form->field('start_date'),
'<=' => $form->field('end_date'),
},
terms => [
# field "terms" can contain any of the keywords
map { +{ -like => '%' . $_ . '%' } } @terms
],
);

my $page = $c->req->param('page');
$page = 1 if ! defined( $page ) || ! $page || $page =~ /\D/;
my $result = $c->model('Package')->search( \%search, {page=>$page} );

$c->stash->{results} = $result;
$c->stash->{pager} = $result->pager;


Then, in the template:


<a href="[% c.req.uri_with({page => pager.prev_page}) %]">Prev</a>
<a href="[% c.req.uri_with({page => pager.next_page}) %]">Next</a>


That works well for simple GET forms where the ?field=val syntax is used in
the URI.

What's the approach for paging the (cached?) query results from a
complex-field POSTed search form?

I'm imagining a two-table DB solution where we cache the found row-id's in
table `cached_search_row` and link that to `cached_search`, then have the
cached_search.id mentioned in the URI. I'm hoping there's a better way
someone has already conjured up that doesn't have all the drawbacks of this
approach that we haven't even thought of...

Thanks in advance!


--
will trillich
"It's only by saying 'no' that you can concentrate on the things that are
really important." -- Steve Jobs

--
will trillich
"It's only by saying 'no' that you can concentrate on the things that are
really important." -- Steve Jobs
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.scsys.co.uk/pipermail/catalyst/attachments/20100602/ecc0907c/attachment.htm

Search Discussions

  • Stuart Watt at Jun 2, 2010 at 6:11 pm
    We do a redirect code 303 from the POST request handler to a GET handler
    and page through it instead. This appears to be the HTTP 1.1 norm, and
    anything else is risky, as POST is expected to change stuff in the
    database. Paging through a POST result will (might) cause multiple
    changes. This way caching is supposed to be handled right.

    --S

    Stuart Watt
    ARM Product Developer
    Information Balance
    On 6/2/2010 1:56 PM, will@serensoft.com wrote:
    Short version:

    Using [% c.req.uri_with({ page => pager.next_page }) %] is fine for a
    simple single-field search (where the form uses GET instead of
    POST)... but how do we PAGE through (and/or cache) a multi-field form
    search that uses POST?

    _______________________________________________
    List: Catalyst@lists.scsys.co.uk
    Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
    Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
    Dev site: http://dev.catalyst.perl.org/
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: http://lists.scsys.co.uk/pipermail/catalyst/attachments/20100602/043c81a6/attachment.htm
  • Stuart Watt at Jun 2, 2010 at 6:23 pm
    Actually, I'll elaborate our more detailed solution.

    1. We don't actually use POST requests for search, but our GET requests
    have many fields and strange Dojo magic
    2. We serialize the query with its many fields, using a bit of
    compression on the URI query string, and base 64 encoding, into a
    relatively opaque and relatively short token
    3. This string is used by our search request handler, which unpacks the
    string and allows a pageable search by merging in a few additional
    fields (_page and _page_size) which are not serialized (that was
    underscore magic). Because the serialized/compressed search query token
    is opaque and you can't have two searches with the same query token, we
    use this as a cache key extensively for performance.

    We find this works well. Also, since we have a model object for the
    search (with serialize/deserialize) we can create views on it which
    allows us to generate nice textual descriptions of the search -- very
    handy for user feedback. And our users like to keep a history using
    these descriptions, so they can go back and look/run previous searches.

    The only problem would appear to be when the URLs become excessive for a
    GET request. When this happens, the POST can handle the form, generate
    the serialized/compressed search query bundle, and then hand off to the
    GET request with that instead.

    One caveat we hit was Microsoft's IIS rejected path elements in URLs
    which were more than 240 characters (bytes?) even the URL was sound. It
    probably thought they might be files. So we did pass stuff as query
    elements, as this seems to be more viable on Microsoft servers anyway.

    --S

    Stuart Watt
    ARM Product Developer
    Information Balance
    On 6/2/2010 1:56 PM, will@serensoft.com wrote:
    Short version:

    Using [% c.req.uri_with({ page => pager.next_page }) %] is fine for a
    simple single-field search (where the form uses GET instead of
    POST)... but how do we PAGE through (and/or cache) a multi-field form
    search that uses POST?

    _______________________________________________
    List: Catalyst@lists.scsys.co.uk
    Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
    Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
    Dev site: http://dev.catalyst.perl.org/
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: http://lists.scsys.co.uk/pipermail/catalyst/attachments/20100602/81dccca0/attachment.htm
  • Will at Jun 9, 2010 at 3:56 pm
    Stuart -- This sounds very intriguing.

    Can you share some code or pseudocode? That would be very instructive.

    Thanks!

    On Wed, Jun 2, 2010 at 1:23 PM, Stuart Watt wrote:

    Actually, I'll elaborate our more detailed solution.

    1. We don't actually use POST requests for search, but our GET requests
    have many fields and strange Dojo magic
    2. We serialize the query with its many fields, using a bit of compression
    on the URI query string, and base 64 encoding, into a relatively opaque and
    relatively short token
    3. This string is used by our search request handler, which unpacks the
    string and allows a pageable search by merging in a few additional fields
    (_page and _page_size) which are not serialized (that was underscore magic).
    Because the serialized/compressed search query token is opaque and you can't
    have two searches with the same query token, we use this as a cache key
    extensively for performance.

    We find this works well. Also, since we have a model object for the search
    (with serialize/deserialize) we can create views on it which allows us to
    generate nice textual descriptions of the search -- very handy for user
    feedback. And our users like to keep a history using these descriptions, so
    they can go back and look/run previous searches.

    The only problem would appear to be when the URLs become excessive for a
    GET request. When this happens, the POST can handle the form, generate the
    serialized/compressed search query bundle, and then hand off to the GET
    request with that instead.

    One caveat we hit was Microsoft's IIS rejected path elements in URLs which
    were more than 240 characters (bytes?) even the URL was sound. It probably
    thought they might be files. So we did pass stuff as query elements, as this
    seems to be more viable on Microsoft servers anyway.

    --S

    Stuart Watt
    ARM Product Developer
    Information Balance

    On 6/2/2010 1:56 PM, will@serensoft.com wrote:

    Short version:

    Using [% c.req.uri_with({ page => pager.next_page }) %] is fine for a
    simple single-field search (where the form uses GET instead of POST)... but
    how do we PAGE through (and/or cache) a multi-field form search that uses
    POST?

    _______________________________________________
    List: Catalyst@lists.scsys.co.uk
    Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
    Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
    Dev site: http://dev.catalyst.perl.org/


    _______________________________________________
    List: Catalyst@lists.scsys.co.uk
    Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
    Searchable archive:
    http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
    Dev site: http://dev.catalyst.perl.org/

    --
    will trillich
    "I think it would be worse to expect nothing than to be disappointed." --
    Anne (with an 'e') Shirley
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: http://lists.scsys.co.uk/pipermail/catalyst/attachments/20100609/faf0c3d7/attachment.htm
  • Oleg Pronin at Jun 3, 2010 at 12:51 pm
    <form id="myform">
    <input type="hidden" name="..." value="..."/>
    ... any number of params
    <input type="hidden" name="page" value="[%page%]"/>
    ...

    <a href="#" onclick="$('#myform input[name=page]').val(2);
    $('#myform').submit()">
    Page 2
    </a>

    </form>

    2010/6/2 will@serensoft.com <will@serensoft.com>:
    Short version:
    Using [% c.req.uri_with({ page => pager.next_page }) %] is fine for a simple
    single-field search (where the form uses GET instead of POST)... but how do
    we PAGE through (and/or cache) a multi-field form search that uses POST?

    Long version:
    This is probably already a posted recipe somewhere, but I haven't had much
    luck finding answers via googling?combinations?of 'perl catalyst post cache
    pager' ... so pointers are welcome:

    # Form has lots of fields, we'll just nab a sample handful from the POST:
    my @terms = map{s/\s+//; $_} split /,/, $form->field('keywords');
    # I know, but it's just an example, this isn't a robust search-field parser
    :)
    my %search = (
    ?? ?asof => {
    ?? ? ? ?# date field "asof" must be within date range
    ?? ? ? ?'>=' =>?$form->field('start_date'),
    ?? ? ? ?'<=' => $form->field('end_date'),
    ?? ?},
    ?? ?terms => [
    ?? ? ? ?# field "terms" can contain any of the keywords
    ?? ? ? ?map { +{ -like => '%' . $_ . '%' } } @terms
    ?? ?],
    );
    my $page = $c->req->param('page');
    $page = 1 if ! defined( $page ) || ! $page || $page =~ /\D/;
    my $result = $c->model('Package')->search( \%search, {page=>$page} );
    $c->stash->{results} = $result;
    $c->stash->{pager} = $result->pager;

    Then, in the template:

    <a href="[% c.req.uri_with({page => pager.prev_page}) %]">Prev</a>
    <a href="[% c.req.uri_with({page => pager.next_page}) %]">Next</a>

    That works well for simple GET forms where the ?field=val syntax is used in
    the URI.
    What's the approach for paging the (cached?) query results from a
    complex-field POSTed search form?
    I'm imagining a two-table DB solution where we cache the found row-id's in
    table `cached_search_row` and link that to `cached_search`, then have the
    cached_search.id mentioned in the URI. I'm hoping there's a better way
    someone has already conjured up that doesn't have all the drawbacks of this
    approach that we haven't even thought of...
    Thanks in advance!

    --
    will trillich
    "It's only by saying 'no' that you can concentrate on the things that are
    really important." -- Steve Jobs
    --
    will trillich
    "It's only by saying 'no' that you can concentrate on the things that are
    really important." -- Steve Jobs

    _______________________________________________
    List: Catalyst@lists.scsys.co.uk
    Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
    Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
    Dev site: http://dev.catalyst.perl.org/
  • Oleg Pronin at Jun 3, 2010 at 12:52 pm
    i forgot

    <form method="POST">

    2010/6/3 Oleg Pronin <syber.rus@gmail.com>:
    <form id="myform">
    ?<input type="hidden" name="..." value="..."/>
    ?... any number of params
    ?<input type="hidden" name="page" value="[%page%]"/>
    ?...

    ?<a href="#" onclick="$('#myform input[name=page]').val(2);
    $('#myform').submit()">
    ? ?Page 2
    ?</a>

    </form>

    2010/6/2 will@serensoft.com <will@serensoft.com>:
    Short version:
    Using [% c.req.uri_with({ page => pager.next_page }) %] is fine for a simple
    single-field search (where the form uses GET instead of POST)... but how do
    we PAGE through (and/or cache) a multi-field form search that uses POST?

    Long version:
    This is probably already a posted recipe somewhere, but I haven't had much
    luck finding answers via googling?combinations?of 'perl catalyst post cache
    pager' ... so pointers are welcome:

    # Form has lots of fields, we'll just nab a sample handful from the POST:
    my @terms = map{s/\s+//; $_} split /,/, $form->field('keywords');
    # I know, but it's just an example, this isn't a robust search-field parser
    :)
    my %search = (
    ?? ?asof => {
    ?? ? ? ?# date field "asof" must be within date range
    ?? ? ? ?'>=' =>?$form->field('start_date'),
    ?? ? ? ?'<=' => $form->field('end_date'),
    ?? ?},
    ?? ?terms => [
    ?? ? ? ?# field "terms" can contain any of the keywords
    ?? ? ? ?map { +{ -like => '%' . $_ . '%' } } @terms
    ?? ?],
    );
    my $page = $c->req->param('page');
    $page = 1 if ! defined( $page ) || ! $page || $page =~ /\D/;
    my $result = $c->model('Package')->search( \%search, {page=>$page} );
    $c->stash->{results} = $result;
    $c->stash->{pager} = $result->pager;

    Then, in the template:

    <a href="[% c.req.uri_with({page => pager.prev_page}) %]">Prev</a>
    <a href="[% c.req.uri_with({page => pager.next_page}) %]">Next</a>

    That works well for simple GET forms where the ?field=val syntax is used in
    the URI.
    What's the approach for paging the (cached?) query results from a
    complex-field POSTed search form?
    I'm imagining a two-table DB solution where we cache the found row-id's in
    table `cached_search_row` and link that to `cached_search`, then have the
    cached_search.id mentioned in the URI. I'm hoping there's a better way
    someone has already conjured up that doesn't have all the drawbacks of this
    approach that we haven't even thought of...
    Thanks in advance!

    --
    will trillich
    "It's only by saying 'no' that you can concentrate on the things that are
    really important." -- Steve Jobs
    --
    will trillich
    "It's only by saying 'no' that you can concentrate on the things that are
    really important." -- Steve Jobs

    _______________________________________________
    List: Catalyst@lists.scsys.co.uk
    Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
    Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
    Dev site: http://dev.catalyst.perl.org/
  • Bill Crawford at Jun 4, 2010 at 3:58 pm

    On 2 June 2010 18:56, will@serensoft.com wrote:
    Short version:
    Using [% c.req.uri_with({ page => pager.next_page }) %] is fine for a simple
    single-field search (where the form uses GET instead of POST)... but how do
    we PAGE through (and/or cache) a multi-field form search that uses POST?
    What version of Catalyst::Runtime adds uri_with? Not found here using
    Ubuntu Lucid Loser (5.80022) ...
  • Will at Jun 4, 2010 at 4:26 pm
    Bill -- We're running Catalyst::Runtime 5.80022 but uri_with comes via
    Catalyst::Request which doesn't appear to have a $VERSION specified. This is
    on Debian 5.0.4 (stable) but we've had to use CPAN to get more modern
    versions of many of the Debian libraries to get around the "uses NEXT which
    is deprecated" warnings.

    Oleg -- that seems delightfully sneaky for a GET form, definitely worth
    playing with. But how does that work the the $c->stash->{pager} paradigm to
    skip to "last page" for example?

    Stuart -- when you say you serialize the search form, are you using
    Javascript to compress/encapsulate things before they go to the server for
    unwrapping? And if you can elaborate more on the mechanics of 303-forwarding
    from a POST handler to a GET handler (or share a link or two) I'd be most
    appreciative.


    On Fri, Jun 4, 2010 at 10:58 AM, Bill Crawford
    wrote:
    On 2 June 2010 18:56, will@serensoft.com wrote:
    Short version:
    Using [% c.req.uri_with({ page => pager.next_page }) %] is fine for a simple
    single-field search (where the form uses GET instead of POST)... but how do
    we PAGE through (and/or cache) a multi-field form search that uses POST?
    What version of Catalyst::Runtime adds uri_with? Not found here using
    Ubuntu Lucid Loser (5.80022) ...

    _______________________________________________
    List: Catalyst@lists.scsys.co.uk
    Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
    Searchable archive:
    http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
    Dev site: http://dev.catalyst.perl.org/


    --
    will trillich
    "It's only by saying 'no' that you can concentrate on the things that are
    really important." -- Steve Jobs
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: http://lists.scsys.co.uk/pipermail/catalyst/attachments/20100604/0198b34d/attachment.htm
  • Bill Crawford at Jun 4, 2010 at 6:14 pm

    On 4 June 2010 17:26, will@serensoft.com wrote:
    Bill -- We're running Catalyst::Runtime 5.80022 but uri_with comes via
    Catalyst::Request which doesn't appear to have a $VERSION specified. This is
    on Debian 5.0.4 (stable) but we've had to use CPAN to get more modern
    versions of many of the Debian?libraries?to get around the "uses NEXT which
    is deprecated" warnings.
    It's OK, I <span style="font-size: xx-small"> typo'd a line to say
    "$c-req->uri_with(...)" and it gave me the helpfullest warning
    :)</span>

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcatalyst @
categoriescatalyst, perl
postedJun 2, '10 at 5:56p
activeJun 9, '10 at 3:56p
posts9
users4
websitecatalystframework.org
irc#catalyst

People

Translate

site design / logo © 2022 Grokbase