FAQ
Hi Gurus,



I want to remove duplicate lines across files.



Below is the detailed problem,





I have file1 file2,file3,file4,file5,



The lines in file1 are there in file2,file3,file4,file5 I want to remove
all the lines which are there in file1 from file2,file3,file4,file5.



Can anybody please tell me to solve the above problem?









Thanks in Advance,

Siva

Search Discussions

  • John W. Krahn at Jan 3, 2008 at 10:34 am

    Siva Prasad wrote:

    Hi Gurus, Hello,
    I want to remove duplicate lines across files.

    Below is the detailed problem,


    I have file1 file2,file3,file4,file5,

    The lines in file1 are there in file2,file3,file4,file5 I want to remove
    all the lines which are there in file1 from file2,file3,file4,file5.

    Can anybody please tell me to solve the above problem?
    UNTESTED:

    #!/usr/bin/perl
    use warnings;
    use strict;

    my $file = 'file1';

    open my $fh, '<', $file or die "Cannot open '$file' $!";

    my %lines;
    @lines{ <$fh> } = ();

    close $fh;

    { local ( $^I, @ARGV ) = qw/ .bak file2 file3 file4 file5 /;

    while ( <> ) {
    next if exists $lines{ $_ };
    print;
    }
    }

    __END__



    John
    --
    Perl isn't a toolbox, but a small machine shop where you
    can special-order certain sorts of tools at low cost and
    in short order. -- Larry Wall
  • Siva Prasad at Jan 4, 2008 at 12:41 pm
    Hi Gurus,



    Sorry I did not give detailed description of the problem, Here it goes





    I have a array of file names (file1, file2, file3, file4).



    Each the duplicates should be removed in the below fashion



    The lines in file1 should be removed from file2,fiile3,file4,If they
    exists.

    The lines in file2 should be removed from file3,file4 if They exists,

    The lines in file3 should be removed from file4 if they exists.



    Is there any way apart from forloop



    Thanks in Advance

    Siva









    -----Original Message-----
    From: John W. Krahn
    Sent: Thursday, January 03, 2008 4:04 PM
    To: Perl Beginners
    Subject: Re: removing duplicate lines across files.



    Siva Prasad wrote:

    >
    Hi Gurus,


    Hello,


    I want to remove duplicate lines across files. >
    Below is the detailed problem,
    >

    >
    I have file1 file2,file3,file4,file5, >
    The lines in file1 are there in file2,file3,file4,file5 I want to remove
    all the lines which are there in file1 from file2,file3,file4,file5. >
    Can anybody please tell me to solve the above problem?


    UNTESTED:



    #!/usr/bin/perl

    use warnings;

    use strict;



    my $file = 'file1';



    open my $fh, '<', $file or die "Cannot open '$file' $!";



    my %lines;

    @lines{ <$fh> } = ();



    close $fh;



    { local ( $^I, @ARGV ) = qw/ .bak file2 file3 file4 file5 /;



    while ( <> ) {

    next if exists $lines{ $_ };

    print;

    }

    }



    __END__







    John

    --

    Perl isn't a toolbox, but a small machine shop where you

    can special-order certain sorts of tools at low cost and

    in short order. -- Larry Wall



    --

    To unsubscribe, e-mail: beginners-unsubscribe@perl.org

    For additional commands, e-mail: beginners-help@perl.org

    http://learn.perl.org/
  • Rob Dixon at Jan 3, 2008 at 10:46 am

    Siva Prasad wrote:

    I want to remove duplicate lines across files.

    Below is the detailed problem,



    I have file1 file2,file3,file4,file5,

    The lines in file1 are there in file2,file3,file4,file5 I want to
    remove all the lines which are there in file1 from file2,file3,file4,file5.

    Can anybody please tell me to solve the above problem?
    You have been asked already not to makes posts in blue lettering. Please
    have the courtesy to post in a fashion appropriate for the list.

    read the faq article

    perldoc -q duplicate elements

    which describes how to do this with an array of data. Hopefully you can
    make the step to doing the same thing with disk files.

    HTH,

    Rob

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupbeginners @
categoriesperl
postedJan 3, '08 at 10:14a
activeJan 4, '08 at 12:41p
posts4
users3
websiteperl.org

People

Translate

site design / logo © 2022 Grokbase