Grokbase › Groups › Perl › beginners › February 2007
FAQ
Hi,

I have to combine several Unix password files and remove any duplicate
accounts - putting this into LDAP.

I have the following code that will remove any duplicate whole lines but
I need to remove lines only if the first field of the password file is a
duplicate.

---------------------
use strict;
use warnings;

my $passwdfile = "pwcomb";
my %seen = ();
{
local @ARGV = ($passwdfile);
local $^I = '.bac';
while(<>){
$seen{$_}++;
next if $seen{$_} > 1;
print;
}
}
print "finished";
---------------------

Could someone help with a solution to this?

Thanks,
Greg.

Search Discussions

  • Jeff Pang at Feb 28, 2007 at 10:09 am ⇧

    -----Original Message-----
    From: "Keenan, Greg John (Greg)** CTR **" <gjkeenan@alcatel-lucent.com>
    Sent: Feb 28, 2007 12:39 PM
    To: beginners@perl.org
    Subject: remove line if field one is duplicate

    Hi,

    I have to combine several Unix password files and remove any duplicate
    accounts - putting this into LDAP.

    I have the following code that will remove any duplicate whole lines but
    I need to remove lines only if the first field of the password file is a
    duplicate.

    Hello,

    Just give a simple test and I think this may work:

    $ perl -Mstrict -e '
    my %hash; open HD,"/etc/passwd" or die $!;
    my @uniq_users = map { $_->[1] } grep { ++$hash{$_->[0]} < 2 } map { [(split/:/)[0],$_] } <HD>;
    print @uniq_users '

    --
    Jeff Pang
    EMAIL: pangj<at>earthlink.net AIM: jeffpang
  • Chas Owens at Feb 28, 2007 at 3:52 pm ⇧

    On 2/27/07, Keenan, Greg John (Greg)** CTR ** wrote:
    Hi,

    I have to combine several Unix password files and remove any duplicate
    accounts - putting this into LDAP.

    I have the following code that will remove any duplicate whole lines but
    I need to remove lines only if the first field of the password file is a
    duplicate.
    This should work

    perl -ne 'print unless $h{(split/:/)[0]}++'
  • Oryann9 at Mar 1, 2007 at 8:14 pm ⇧

    --- Chas Owens wrote:

    On 2/27/07, Keenan, Greg John (Greg)** CTR **
    wrote:
    Hi,

    I have to combine several Unix password files and
    remove any duplicate
    accounts - putting this into LDAP.

    I have the following code that will remove any
    duplicate whole lines but
    I need to remove lines only if the first field of
    the password file is a
    duplicate.
    This should work

    perl -ne 'print unless $h{(split/:/)[0]}++'

    This is creating a anonymous hash, correct?



    ____________________________________________________________________________________
    We won't tell. Get more on shows you hate to love
    (and love to hate): Yahoo! TV's Guilty Pleasures list.
    http://tv.yahoo.com/collections/265
  • John W. Krahn at Mar 1, 2007 at 10:00 pm ⇧

    oryann9 wrote:
    --- Chas Owens wrote:
    This should work

    perl -ne 'print unless $h{(split/:/)[0]}++'
    This is creating a anonymous hash, correct?
    No, incorrect. The hash is named %h.


    John
    --
    Perl isn't a toolbox, but a small machine shop where you can special-order
    certain sorts of tools at low cost and in short order. -- Larry Wall
  • Chas Owens at Mar 2, 2007 at 10:03 pm ⇧
    On 3/1/07, oryann9 wrote:
    snip
    perl -ne 'print unless $h{(split/:/)[0]}++'
    This is creating a anonymous hash, correct?
    snip

    No, the hash is named %h. The proper way to read $h{(split/:/)[0]}++
    is "Increment the value whose key in %h is the first element of $_
    when split on ':'." The following code does the same thing, but
    requires a lot more typing:

    #!/usr/bin/perl

    use strict;
    use warnings;

    my %seen;

    while (<>) {
    my $key = (split /:/)[0];
    if (exists $seen{$key}) {
    $seen{$key}++;
    } else {
    $seen{$key} = 1;
    print;
    }
    }
  • Rob Dixon at Feb 28, 2007 at 10:07 pm ⇧

    Keenan, Greg John (Greg)** CTR ** wrote:
    Hi,

    I have to combine several Unix password files and remove any duplicate
    accounts - putting this into LDAP.

    I have the following code that will remove any duplicate whole lines but
    I need to remove lines only if the first field of the password file is a
    duplicate.

    ---------------------
    use strict;
    use warnings;

    my $passwdfile = "pwcomb";
    my %seen = ();
    {
    local @ARGV = ($passwdfile);
    local $^I = '.bac';
    while(<>){
    $seen{$_}++;
    next if $seen{$_} > 1;
    print;
    }
    }
    print "finished";
    ---------------------

    Could someone help with a solution to this?
    Hi Greg

    This will take all the characters from the beginning of each line up to the
    first colon and put it into $field1. The line will be printed only if the value
    for $field1 hasn't been seen before.

    HTH,

    Rob



    use strict;
    use warnings;

    my $passwdfile = "pwcomb";
    my %seen;

    local @ARGV = ($passwdfile);
    local $^I = '.bac';

    while (<>) {

    my ($field1) = /([^:]+)/;
    print unless $seen{$field1};
    }

    print "finished\n";
  • Oryann9 at Mar 1, 2007 at 9:02 pm ⇧

    --- Chris Charley wrote:

    This is creating a anonymous hash, correct?
    This is saying
    print unless the first field has been seen before

    which is what he wants his code to do.

    $h is a good short notation - but using %seen
    instead of %h may make it clearer for you.

    perl -ne 'print unless $seen{(split/:/)[0]}++'

    Chris
    I understand what its doing, but just wanted make sure
    if its creating a anonymous hash b/c of the { }. So is
    it b/c u never said yes or no?

    thank you



    ____________________________________________________________________________________
    TV dinner still cooling?
    Check out "Tonight's Picks" on Yahoo! TV.
    http://tv.yahoo.com/

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupbeginners @
categoriesperl
postedFeb 28, '07 at 4:39a
activeMar 2, '07 at 10:03p
posts8
users6
websiteperl.org

6 users in discussion

Oryann9: 2 posts Chas Owens: 2 posts Keenan, Greg John \(Greg\)** CTR **: 1 post Rob Dixon: 1 post Jeff Pang: 1 post John W. Krahn: 1 post

Content

People

Support

Translate

site design / logo © 2022 Grokbase