[Perl-beginners] Longest prefix match

Don't reinvent the wheel (unless it is for fun or educational purposes):
http://search.cpan.org/~dcantrell/Number-Phone-1.58/lib/Number/Phone/Country.pm

first build a hash which record all countries and their codes.
my %countries = (852 => 'Hong Kong', 853 => 'Macau' ...);

then use regex or other ways (I prefer substr) to get the prefix.
my $prefix = substr($string,1,3);

last get the country name via the prefix code.
my $country = $countries{$prefix};

2007/8/27, Mihir Kamdar <[email protected]>:

sorry, I forgot to send my mail to the list:


Jeff Pang schrieb:
I think the problem is that some countrycodes are only 2 digits some are 3
( or more ? )
So you dont know how long your prefix is.


A further suggestion that extends jeffs idea could be following:
you have a longest key in your hash

lets say it stored in $prefix_len, so you could do (pseudocode):
my $prefix = substr($string,1, $prefex_len);
if($prefix eq somekey_of_your_hash){
yay :-)
}else{
try it again with substr($string,1, $prefix_len - 1)
}

HTH
Regards
Martin

The sample code below on executing gives result as:-
Macau
Hong Kong

But there are 0 keys in the hash. I expected the result to be
Hong Kong
Macau

Why is the hash getting empty here?


#!/usr/bin/perl

my %countries = ('+852' => 'Hong Kong', '+853' => 'Macau' );

my @string = {'+8521235567','+8531764458'} ;


while (my($prefix,$country) = each %countries)

{

$prefix = substr($string,1,3);

print "there are " . (keys %hash) . " keys in hash\n";

print $country."\n" ;

}



Thanks,
Mihir

The codes have some problems.I'd modify them to,

use strict;
use warnings;

my %countries = ('+852' => 'Hong Kong', '+853' => 'Macau' );
my @string = ('+8521235567','+8531764458');

for (@string) {
my $prefix = substr($_,0,4);
print $countries{$prefix},"\n";
}

__DATA__
Hong Kong
Macau

But as Chas and Martin said,you'd better use that CPAN module for doing this.
Because countries's tel prefixes length are not the same.

thanks,

But i have a list of 65 countries, and I only want to implement for them.
Also, as you said, definitely for some, prefix length is only 1 digit, while
for others, it is 2 or 3 digits. So, I was thinking to build a hash and then
implement this.
Also, my aim is not actually to get the country name.
On the basis of the prefix, I have the corresponding callrate/sec. Foe ex,

Country_Name Country_Code Rate/sec
Argentina 54 RM0.99
So, if the longest prefix match of the 6th field(called_number) of my CSV
file matches with country code of Argentina, then read the 13th
field(Duration), divide it by 60 and multiply it by the rate/sec. of
Argentina and write the result into the 14th field.

please advice on how something like this can be achieved??

Thanks,
Mihir

Hi,

do you understand my pseudo code, i think it should work for your hash?

Regards
Martin

Hi Martin,

Yes, I guess I understood yr code...if I am correct, i just need to know the
longest country code digits and store them in $prefix_len, and then compare
it with the string...if it doesnt match...decrement length by 1 and
compare...yeah this shud work...
but in my previous thread, i have mentioned by actual requirement....can u
just have a look and comment on that..

Thanks,
Mihir

i never worked with CSV so far but

http://search.cpan.org/~danboo/Tie-Handle-CSV-0.09/lib/Tie/Handle/CSV.pm

sounds quite good. (hope it can handle if you're writing something to the
hash too. I didn't read the code... if not there are a lot of CSV modules
at cpan. )

<snip>
Country_Name Country_Code Rate/sec
Argentina 54 RM0.99
</snip>

if you use that Module you could access your CountryCoude with
$csv->{Country-Code}

use my algorithm and simply do
# then read the 13th
# field(Duration), divide it by 60 and multiply it by the rate/sec. of
# Argentina and write the result into the 14th field.

my $tmp = $csv->{Duration} / 60 * $csv->{Rate/sec};
$csv->{what_ever_the_name_of_your_14th_field_is} = $tmp;

On 8/27/07, Mihir Kamdar wrote:
snip
I can't recommend reinventing the wheel, I would use the module to
lookup the country name and the use hash where the country name is the
key and the rate is the value to get the value, but if you are dead
set on not using the module your best bet is something like this:

#!/usr/bin/perl

use strict;
use warnings;

my %prefix_to_rate = (
12 => 0.30,
1234 => 0.35,
134 => 0.50,
44 => 0.70
);

#if there are more than about 50 country codes
#get rid of the sort and implement min and max
#in Perl, otherwise sort is actually faster
my ($shortest, $longest) =
(sort { $a <=> $b } map {length} keys %prefix_to_rate)[0, -1];

while (<DATA>) {
my ($num, $min) = split;
my $rate;
for my $len (reverse $shortest .. $longest) {
my $key = substr $num, 1, $len;
last if $rate = $prefix_to_rate{$key};
}
if (defined $rate) {
print "the rate for $num is $rate: ", $rate * $min, "\n";
} else {
print "$num does not have a rate\n";
}
}

__DATA__
+12555 10
+44555 20
+12345 30
+55555 55

I can't recommend reinventing the wheel, I would use the module to
Hi,

I tried to implement the way Chas suggested, but am getting a lot of errors.
Plz check my mistakes:-

#!/usr/bin/perl

use strict;
use warnings;

my $file_path =
'/home/user71/RangerDatasource/Customization/TelekomMalaysia/Scripts/Tests/cprogs/files/ratetest';
my %prefix_to_rate = (
'+12' =>0.30,
'+1234' =>0.35,
'+134' =>0.50,
'+44' =>0.70
);
my %times ;

my ($shortest, $longest) =
(sort { $a <=> $b } map {length} keys %prefix_to_rate)[0,
-1];

my $continue = 1;

$SIG{INT} = $SIG{TERM} = sub { $continue = 0 };

while ($continue) {
opendir my $dh, $file_path or die $!;
while (my $file = readdir $dh) {
my $fname = "$file_path/$file" ;
next unless -f $fname;
unless (exists $times{$file}){
my $line;
open (my $IN_FILE,"<","$file_path/$file") or die
$!." file not found" ;


while ($line=readline($IN_FILE))
{
my @cdr=split (/,/, $line) ;
($cdr[3],my $min) = split;
my $rate;
for my $len (reverse $shortest .. $longest) {
my $key = substr $cdr[3], 1, $len;
last if $rate = $prefix_to_rate{$key};
}

if (defined $rate) {
print "the rate for $cdr[3] is $rate: ", $rate * $min, "\n";
}
else
{
print "$cdr[3] does not have a rate\n";
}
}
}
}
closedir $dh ;
}


Thanks,
Mihir

Please ignore the previous reply of mine. I have made changes and now the
code is not giving any errors, but is not giving me the desired result. Plz
check:-


#!/usr/bin/perl

use strict;
use warnings;

my $file_path =
'/home/user71/RangerDatasource/Customization/TelekomMalaysia/Scripts/Tests/cprogs/files/ratetest';
my $write_path =
'/home/user71/RangerDatasource/Customization/TelekomMalaysia/Scripts/Tests/cprogs/files/rateop'
;
my %prefix_to_rate = (
'+12' =>0.30,
'+1234' =>0.35,
'+134' =>0.50,
'+44' =>0.70
);
my %times ;

my ($shortest, $longest) =
(sort { $a <=> $b } map {length} keys %prefix_to_rate)[0,
-1];

my $continue = 1;

$SIG{INT} = $SIG{TERM} = sub { $continue = 0 };

while ($continue) {
opendir my $dh, $file_path or die $!;
while (my $file = readdir $dh) {
my $fname = "$file_path/$file" ;
next unless -f $fname;
unless (exists $times{$file}){
my $line;
open (my $IN_FILE,"<","$file_path/$file") or die
$!." file not found" ;


while ($line=readline($IN_FILE))
{
my @cdr=split (/,/, $line) ;
#($cdr[3],my $min) = split;
my $rate;
for my $len (reverse $shortest .. $longest) {
my $key = substr $cdr[3], 1, $len;
last if $rate = $prefix_to_rate{$key};
}
open (my $OUT_FILE,">","$write_path/$file.out") or die $!;

if (defined $rate) {
print $OUT_FILE "the rate for $cdr[3] is $rate: ",
($cdr[6]/20)*$rate , "\n";
}
else
{
print $OUT_FILE "$cdr[3] does not have a rate\n";
}
}
close $IN_FILE ;
}
}

closedir $dh ;
}


Thanks,
Mihir

On 8/27/07, Mihir Kamdar wrote:
snip
snip
snip

Your substr should use 0 instead of 1. If you look back at my example
you can see that the string the corresponds to your $cdr[3] had + at
the start, but my keys did not. This is because I was using an offset
of 1 which is the second character (i.e. the character after the +).

Hi,

Still same result...

Look at the following:-

if (defined $rate) {
print $OUT_FILE "found rate","\n" ;
print $OUT_FILE "the rate for $cdr[3] is $rate: ",
($cdr[6]/20)*$rate , "\n";
}
else
{
print $OUT_FILE "$cdr[3] does not have a
rate\n";
}

Even if my $cdr[3] field in few of the records matches with the keys in the
hash, it is still not going to the if part at all.
It just takes one of the not matching records and printing the else part in
the $OUT_FILE.

OUT_FILE content:-
+60199479787 does not have a rate

Any idea?

Thanks,
Mihir

On 8/27/07, Mihir Kamdar wrote:
snip
snip

Not a one. Can you provide some test data?

Sure Chas...

Following is some sample records:-

2006/09/01 00:19:30,999,+60132868382,+126312437,,1,253,2006/09/01
00:15:17,350370010515510,515111061171520,00043,3,1,0.0
,7,1,1,1,,0,8,+60132868382,Normal,,TELMT,Alcatel,,,
2006/09/01 00:21:13,999,+60132860550,+1235111519,,1,111,2006/09/01
00:19:22,350370010466140,515111060160867,00038,3,1,0.0
,7,1,1,1,,0,8,+60132860550,Normal,,TELMT,Alcatel,,,
2006/09/01 00:28:34,999,+60132868277,+60135768776,,1,142,2006/09/01
00:26:12,350370010512060,515111061171688,00035,0,1,0.0
,7,1,1,1,,0,262144,+60132868277,Normal,,TELMT,Alcatel,,,
2006/09/01 01:42:40,999,+60132860290,+60199479787,,1,4162,2006/09/01
00:33:18,350370010519310,515111060160355,00038,0,1,0.0
,7,1,1,1,,0,262144,+60132860290,Normal,,TELMT,Alcatel,,,
2006/09/01 00:41:02,999,+60132866293,+4429253850,,1,6,2006/09/01
00:40:56,350370010523460,515111061170603,00037,3,1,0.0
,7,1,1,1,,0,8,+60132866293,Normal,,TELMT,Alcatel,,,
2006/09/01 00:57:39,999,+60132866401,+0168445479,,1,774,2006/09/01
00:44:45,350370010522420,515111061170711,00037,3,1,0.0
,7,1,1,1,,0,8,+60132866401,Normal,,TELMT,Alcatel,,,
2006/09/01 00:49:01,999,+60132866293,+0129253850,,1,223,2006/09/01
00:45:18,350370010523460,515111061170603,00037,3,1,0.0
,7,1,1,1,,0,8,+60132866293,Normal,,TELMT,Alcatel,,,
2006/09/01 01:22:44,999,+60132860290,+60199479787,,1,1461,2006/09/01
00:58:23,350370010519310,515111060160355,00038,0,1,0.0
,7,1,1,1,,0,262144,+60132860290,Normal,,TELMT,Alcatel,,,

Here, I need to read the 4th field and compare with the hash....

Plz tell me if you want anything more

snip

Seems to be working for me (with substr $cdr[3], 0, $len;). I get the output

the rate for +126312437 is 0.3: 3.795
the rate for +1235111519 is 0.3: 1.665
+60135768776 does not have a rate
+60199479787 does not have a rate
the rate for +4429253850 is 0.7: 0.21
+0168445479 does not have a rate
+0129253850 does not have a rate
+60199479787 does not have a rate

This makes sense since the only valid keys are +44, +1234, +12, and
+134. There are no matches that start with +6 or +0 so they do not
get a rate. You will need to add keys to %prefix_to_rate if you wish
to be able to handle them.

Hi Chas,

Look at the following code. It does not change the $cdr[13] value after
processing as is desired.

#!/usr/bin/perl

use strict;
use warnings;

my $file_path =
'/home/user71/RangerDatasource/Customization/TelekomMalaysia/Scripts/Tests/cprogs/files/ratetest';
my $write_path =
'/home/user71/RangerDatasource/Customization/TelekomMalaysia/Scripts/Tests/cprogs/files/rateop'
;
my %prefix_to_rate = (
'+1' =>0.20,
'+65' =>0.30,
'+673'=>0.45,
'+86' =>0.45,
'+852'=>0.45,
'+853'=>0.45,
'+66' =>0.45,
'+212'=>1.50,
'+234'=>1.50,
'+47' =>1.50,
'+48' =>1.50,
'+7' =>1.50,
'+27' =>1.50,
'+46' =>1.50,
'+380'=>1.50,
'+93' =>1.50,
'+62' =>0.22
);
my %times ;
my %hash ;
my ($shortest, $longest) =
(sort { $a <=> $b } map {length} keys %prefix_to_rate)[0,
-1];

my $continue = 1;

$SIG{INT} = $SIG{TERM} = sub { $continue = 0 };

while ($continue) {
opendir my $dh, $file_path or die $!;
while (my $file = readdir $dh) {
my $fname = "$file_path/$file" ;
next unless -f $fname;
unless (exists $times{$file}){
my $line;
open (my $IN_FILE,"<","$file_path/$file") or die
$!." file not found" ;


while ($line=readline($IN_FILE))
{
my @cdr=split (/,/, $line) ;
my $rate;
if($cdr[22] eq "Budget-IDD" )
{
for my $len (reverse $shortest .. $longest)
{
my $key = substr $cdr[3],0,$len;
last if $rate = $prefix_to_rate{$key};
if (defined $rate) {
$cdr[13] = ($cdr[6]/20)*$rate ;
}
}
}
$line = join(",",@cdr);
$hash{@cdr[2,3,6,7]}=$line;
}
close $IN_FILE ;

open (my $OUT_FILE,">>","$write_path/$file.out") or
die $!;

while (my($key, $value) = each %hash)
{
print $OUT_FILE $value;
}
close $OUT_FILE ;
}
}
closedir $dh ;
}


Following is the sample input data:-

2007/08/18 23:09:51,999,+60139169925,+6281917457237,,1,17,2007/08/18
23:09:34,357953001453930,502191402472982,30421,3,4117773,5.5
,2,1,1,1,,0,8,+60139169925,Budget-IDD,,9KTTB,Alcatel,5419496593,1
2007/08/18 23:10:03,999,+60199514513,+6681805297189,,1,5,2007/08/18
23:09:58,353377004203470,502191200047108,33761,3,8893845,5.5
,2,1,1,1,,0,8,+60199514513,Budget-IDD,,9KTTB,Alcatel,5419496808,1
2007/08/18 23:10:09,999,+60139484541,+62081933039638,,1,5,2007/08/18
23:10:04,359772006475961,502191402477236,36433,3,4357348,5.5
,2,1,1,1,,0,8,+60139484541,Budget-IDD,,9KTTB,Alcatel,5419496906,1
2007/08/18 23:10:19,999,+60139433881,+4781917453095,,1,559,2007/08/18
23:01:00,354388004016271,502191501305123,33802,3,10638343,55,2,1,1,1,,0,8,+60139433881,Budget-IDD,,9KTTB,Alcatel,5419497064,1
2007/08/18 23:10:27,999,+60148482721,+6285937021269,,1,86,2007/08/18
23:09:01,354813018414143,502191201919440,36643,3,4891400,11,2,1,1,1,,0,8,+60148482721,Budget-IDD,,9KTTB,Alcatel,5419497197,1



Thanks,
Mihir

On 8/27/07, Mihir Kamdar wrote:
snip
snip

The last will execute if $rate is set, so the "if (defined" will never
be true (unless rate is 0, which I would assume is an error). You
either need to move the "if (defined" outside of the for loop or you
need to change the "last if" to

if (exists $prefix_to_rate{$key}) {
$cdr[13] = $cdr[6]/20*$prefix_to_rate{$key};
last;
}

Also, you need to work on your indenting. It is all over the place
(which is probably why your if statement is in the wrong place). You
might consider installing perltidy and running it on your code.

Hi Chas,

I modified it like below:-

while ($line=readline($IN_FILE))
{
my @cdr=split (/,/, $line) ;
my $rate;
if($cdr[22] eq "Budget-IDD" )
{
for my $len (reverse $shortest .. $longest)
{
my $key = substr $cdr[3],0,$len;
last if $rate = $prefix_to_rate{$key};
}
if (defined $rate)
{
$cdr[13] = ($cdr[6]/20)*$rate ;
}
}
$line = join(",",@cdr);
$hash{@cdr[2,3,6,7]}=$line;
}
close $IN_FILE ;


Still not getting the result?

Thanks..

#!/usr/bin/perl

use strict;
use warnings;

Off hand I would say the problem is in your hash key. The following
outputs data as I would expect. Let me know where/if it differs from
what you want.

my %prefix_to_rate = (
'+1' => 0.20,
'+65' => 0.30,
'+673' => 0.45,
'+86' => 0.45,
'+852' => 0.45,
'+853' => 0.45,
'+66' => 0.45,
'+212' => 1.50,
'+234' => 1.50,
'+47' => 1.50,
'+48' => 1.50,
'+7' => 1.50,
'+27' => 1.50,
'+46' => 1.50,
'+380' => 1.50,
'+93' => 1.50,
'+62' => 0.22
);

my ( $shortest, $longest ) =
( sort { $a <=> $b } map { length } keys %prefix_to_rate )[ 0, -1 ];

my %hash;
while ( my $line = <> ) {
my @cdr = split( /,/, $line );
my $rate;
if ( $cdr[22] eq "Budget-IDD" ) {
for my $len ( reverse $shortest .. $longest ) {
my $key = substr $cdr[3], 0, $len;
last if $rate = $prefix_to_rate{$key};
}
if ( defined $rate ) {
$cdr[13] = ( $cdr[6] / 20 ) * $rate;
}
}
$line = join ',', @cdr;
$hash{join ',', @cdr[2,3,6,7]} = $line;
}

print $hash{$_} for sort keys %hash;

Hi Chas,

It works...but the problem I have is that some of the records are totally
lost. And also the output file keeps on writing multiple times and grows
into a huge size. almost 10 times, it keeps on growing until I kill the
script.

I checked few of the records which have calculated properly and giving me
the desired result, while the others are totally lost in the output. Check
my code after closing the file. Is it the culprit?

close $IN_FILE ;

open (my $OUT_FILE,">>","$write_path/$file.out") or
die $!;

while (my($key, $value) = each %hash)
{
print $OUT_FILE $value;
}
close $OUT_FILE ;
}
}
closedir $dh ;
}


Thanks..

Hi,

if you open your outputfile like
open(FH, ">>", $file)

you will append all the stuff to the file. Probably you want to
replace your file if you run your script again.

open(FH, ">", $file)

HTH, Martin


On Mon, 27 Aug 2007 22:53:16 +0530
"Mihir Kamdar" wrote:

It looks like you aren't moving or deleting the files you have already
processed, so you are processing them over and over again. You should
consider renaming the files or creating a list of already processed
files (remember to store the list to a file and to reread the list on
start up).

perldoc perlrequick
perldoc perlretut
perldoc perlre

Good luck!