[Catalyst] create search engine friendly uri from string

$str =~ s{\W}{}g;
$str = lc($str);
$str =~ s{\s+}{-}g;

You might also want to add these two to the end:

=~ s/--+/-/g;
=~ s/-$//g;

Scott McWhirter wrote:

My not very elegant, but working solution:

my $str = " Beta Launch Invites: Kwyno Brings The Web Into Your IM
And (Soon) SMS Inboxes ";
$str =~ s/^\s+|\s+$//g;
$str =~ s/\W/ /g;
$str =~ s/\s{1,}/ /g;
$str =~ s/\s/-/g;
$str = lc($str);

print "$str\n";

You are so great. Thanks guys!

What do you think? should I put those in a Template Plugin module and
push it to the CPAN?

Any name suggestions?

cheers

I've been meaning to, for use with DBIx::Class::Tokenize ... but
better to have something like Text::Filter::URI and then a Template
Toolkit Filter that simply uses that underlying module.

Just my $0.02,

-J

I agree. I initially wrote Template::Plugin::$do_something_or_other then
soon realised that I didn't only want to use it in TT.

I recently factored out the useful stuff into its own module and made
the plugin not much more than a TT wrapper around it.

Just a caveat that should be in the doc you end up with if not
possible in the code. URIs are supposed to be unique to a resource.
Titles of articles and pages are not (necessarily). The end user/dev
will have to check for duplicates against the data and append a -II
or -2, and so on.

-Ashley

I'll go with this approach:
I use my primary key (happens to be a uuid string) and append the string
from the title.
I read just the uuid string and dismiss everything else.
This way I don't have to worry about uniqueness and have seo friendly urls
anyway.

mo

A wee bit shorter:

my $str = " Beta Launch Invites: Kwyno Brings The Web Into Your IM
And (Soon) SMS Inboxes ";
$str =~ s/^\s+|\s+$//g;
$str =~ s/\W+/-/g;
$str = lc($str);

print "$str\n";

Before putting that into a module though you might want to think about
what should happen to characters outside the [a-z0-9] range as \W will
match differently based on locale. I'm not sure what the recommended
behaviour is for such cases.

That's what I'm thinking about right now. I couldn't find a reference which
says that \W matches differently based on locale.

Ptyhon can convert an utf8 string to an ascii string and replaces
characters like "?" with the most equivalent character "a". Is there such
a thing for perl?

cheers,

moritz

From perlre

A "\w" matches a single alphanumeric character (an alphabetic
character, or a decimal digit) or "_", not a whole word. Use
"\w+" to
match a string of Perl-identifier characters (which isn't the
same as
matching an English word). If "use locale" is in effect, the
list of
alphabetic characters generated by "\w" is taken from the
current
locale.
There's a host of modules on CPAN that do things like that, but I
don't know if one is accepted as the better way to go. The problem is
that if you want to cover all your bases it can become a rather
extensive problem. For instance you might want to convert "?" to "e",
but do you want to map "??" to "beijing"?

The simple solution is probably to have one option that encodes to IRI
friendly, and another to URI friendly, and let people who want
something more complicated roll up their own. See http://annevankesteren.nl/2004/08/uri-design
for some thoughts related to this, or http://www.w3.org/International/iri-edit/draft-duerst-iri-bis.html
.

But that doesn't address the locale issue. For that be sure to toss in
a no locale (which is lexical) or to define your own character classes
instead of \w, \s, and friends.

From: <onken@houseofdesign.de>
Maybe Text::Unidecode could be helpful...

Octavian

or Search::Tools::Transliterate

From: "Peter Karman" <peter@peknet.com>
I have tried:

use strict;
use Search::Tools::Transliterate;
use utf8;

my $tr = Search::Tools::Transliterate->new;
$tr->ebit(0);
print $tr->convert("??????????????");

#It prints:
ast??AST??????

I want to print ai instead of ?? and AI instead of ??. Am I using $tr->ebit
correctly?

The latest 4 chars are 4 new UTF-8 chars in romanian language (U+0218,
U+0219, U+021A, U+021B). Can they be transliterated?
They are ???? but with a comma below, and not with a sedila. Can they be
displayed as sStT?

Thanks.

Octavian

Did you have those problems with Text::Unidecode, too?

From: "Moritz Onken" <onken@houseofdesign.de>
No, I don't, but as I explained in a previous message, Text::Unidecode has
some other problems but they both are good for what they do.

Octavian

no. you must set ebit in new(), not after instantiation. I've added a
note to the docs to emphasize that.

my $tr = Search::Tools::Transliterate->new( ebit => 0 );
sure. Just add them via the map() method. I believe that's documented
with an example, but here's another:

---------------------------
#!/usr/bin/perl
use strict;
use Search::Tools::Transliterate;
use utf8;

binmode STDERR, ':utf8';

my $string = "??????????";

# new romanian utf8 chars
$string .= "\x{0218}";
$string .= "\x{0219}";
$string .= "\x{021A}";
$string .= "\x{021B}";

my $tr = Search::Tools::Transliterate->new(ebit=>0);
$tr->map->{"\x{0218}"} = 's';
$tr->map->{"\x{0219}"} = 'S';
$tr->map->{"\x{021A}"} = 't';
$tr->map->{"\x{021B}"} = 'T';

print STDERR $string . "\n";
print STDERR $tr->convert($string) . "\n";

------------------

I added the above code as part of a new test and just uploaded 0.19 to
cpan.

If you have suggestions for permanent additions/changes to the character
mapping file, please open a RT ticket and I'll see that they get
reviewed for a future release.

Thanks for the feedback.

From: "Peter Karman" <peter@peknet.com>
Just as a feedback, here is a short comparison I've made between these 2
modules:

Text::Unidecode is 5 or 6 times faster than S::T::T.

I haven't tested what S::T::T does internally, but Text::Unidecode uses many
other perl modules which are loaded dynamicly, and the current ActiveState
PDK can't load them automaticly, so it is harder to use Text::Unidecode.

Because it is able to use the map hash, S::T::T is more flexible than
Text::Unidecode.

I found that Text::Unidecode gives "Bei Jing" for the string
"\x{5317}\x{4EB0}\n" while S::T::T just gives 2 spaces.

And I've tried to transliterate those new 4 romanian chars using these 2
modules:

use Text::Unidecode;
print unidecode("\x{0218}\x{0219}\x{021A}\x{021B}");
#It printed: SsTt

use Search::Tools::Transliterate;

my $tr = Search::Tools::Transliterate->new(ebit => 0);

open(OUT, ">:utf8", "test.txt");
print OUT $tr->convert("\x{0218}\x{0219}\x{021A}\x{021B}");

It printed: ????
Well, without using the map hash, this doesn't print the "correct" string,
but it is interesting because it prints the corresponding characters which
are used now instead of those new characters with a comma instead of a
sedila below them.

HTH.

Octavian

Interesting. I had never compared times since STT was always "fast enough" for
my purposes.

I just refactored convert() in STT and uploaded 0.20 to pause. My benchmark
script shows the new version is 244% faster.

At $work we use Text::Unaccent for this -
http://search.cpan.org/~ldachary/Text-Unaccent-1.08/Unaccent.pm

Unless I've missed your point, but I hadn't seen it mentioned in this
thread so far

From: "Oliver Charles" <oliver.g.charles@googlemail.com>
Yes it is also helpful because it allows to specify the encoding of the
source string. It prints the correct transliteration for those 4 chars in
romanian language. I've seen that it doesn't print "Bei Jing" for those 2
chinese chars however.

Thanks for telling about this module.

Octavian

Here's some code I use for renaming strange files, there might
be something useful there (or not, given I'm decades behind on
this list)

use strict;
use Convert::Translit;

my $trans = Convert::Translit->new('Latin1' => 'ascii');

sub clean {
local($_) = shift;

$_ = $trans->transliterate($_);
$_=lc;
s/&/ and /g;
s/\+/ plus /g;
s/'//g;
y/()~/---/s;
# . || - || _ for the rest
s/(?:(-)|(\.)|[\W_])+/$+||'_'/ge;
s/^[._-]//;
s/[._-]$//;
return $_;
}

Hi:

I'm building an App but I want to have my database in another IP. Is
there a way to enter the parameters to do this instead of the normal
connection that appears in the manual?

This is my model (a very standar one):

package App::Model::ModelDB;

use strict;
use base 'Catalyst::Model::DBIC::Schema';

__PACKAGE__->config(
schema_class => 'App::Schema::ModelDB',
connect_info => [
'dbi:mysql:model',
'model',
'model',
{
mysql_enable_utf8 => 0,
on_connect_do => [
"SET NAMES 'utf8'",
"SET CHARACTER SET 'utf8'",
],
}
],
);

How can I change it to connect to a DB in another IP?

TIA,

Giancarlo

This is a mysql question. DBI passes the connection line is mysql. See
here for an example of what you can write in it, but I cannot confirm, not
being a mysl user:

http://www.apachefriends.org/f/viewtopic.php?f&t'481

Look for any examples of mysql connection in DBI manuals.

--Gene

You probably want to do this in the config, something like this:

If the config is yaml

Model::ModelDB:
connect_info:
- 'dbi:mysql:dbname=my_db_name;host=my_host_or_ip'
- 'my_dbuser'
- 'my_dbpass'
- AutoCommit: 1
mysql_auto_reconnect: 0


Or if the config is conf

<Model::ModelDB>
connect_info dbi:mysql:dbname=my_db_name;host=my_host_or_ip
connect_info my_dbuser
connect_info my_dbpass
<connect_info>
AutoCommit 1
mysql_auto_reconnect 0
</connect_info>
</Model::ModelDB>



Thanks,
John

I recommend that you also install the MySQL client software (not
necessarily the whole database engine, but the client component only) on
your computer and then first try to connect to the MySQL server you are
interested in using the MySQL client. That lets you test your
connectivity before you code that into the Catalyst configuration. It is
also great for testing out your SQL statements if you are shaky on SQL
like me.

I also wonder if Catalyst can make use of a .my.cnf file if there is
one. Why is this important? Because you can keep your connection
password private.

Bob Cochran

Good afternoon,

On 19/01/09 at 8:11 PM -0500, Robert L Cochran
wrote:
You can pass a path to a "user defaults" file as part of the DBI
connect string. I don't recall the syntax though.


Charlie

This is how I usually do it. The __path_to()__ in configs gets
expanded properly.

Model::DBIC:
schema_class: MyApp::Schema
connect_info:
- dbi:mysql:myappdb;mysql_read_default_file=__path_to
(etc/.mysql.cnf)__
- ~
- ~
- RaiseError: 1
PrintError: 0
AutoCommit: 1
ChopBlanks: 1

-Ashley

Thank you all for your answers:
Actually, what I did is to follow the basic instructions from mysql
forum but what I was missing is to set the user with full access from
any host, which I created later from phpmyadmin.

Regards,

Giancarlo

full access? any host? this makes me nervous (for you and your data)

lol I know but I'm doing this inside a private lan where we are building
this app with my coleagues :p
of course later on we will have specific accounts with limited access

Your colleagues can be your biggest security problem. It just takes an
argument...a jealousy...someone who feels belittled...someone who just
wants to take advantage of lax data security. It is real good practice
to use least-privilege access to data from the very beginning.

Bob

Exactly, also how many of those "we will of course do X later..." TODO's end
up hanging out there forever.