[Perl-beginners] Re: Deleting Duplicate Lines one-liner

Here's a one-liner:

perl -nle 'print if !$seen{$_}++'

and here's another:

perl -pe '$_ x= !$seen{$_}++' (attributed to some of Larry's genius)

and another, for use in a program

$seen{$_} ||= print OUT while <IN>;

Have fun. :)

LOL!!! Twistedly brilliant!

Ok, lemme see if I can parse this.....

perl -pe says print each line after the -e code has been executed.

"$_ x= !$seen{$_}++" says:

$_ x=
assign to $_ itself, repeated a number of times

!$seen{$_}
evaluate $seen{$_} in a negated boolean context for the number

postfix ++ increments after returning the previous value.

so "$_ x= !$seen{$_}++" says

1) check $seen{$_} (have we seen this before)
2) then increment it (we've seen it now!)
3) negate the boolean value of $seen{$_} from before the increment
4) assign $_ to itself a number of times equal to the boolean return
(i.e., 0 or 1)

Then -p prints $_, which is either what it was before, or ''
(repetition zero times!)

Is that right?

LOL!!! That's truly, beautifully, perversely thick! >:O]

__________________________________________________

On Jul 30, Paul said:
Yes. I attribute that to Larry Wall, because he wrote something like

perl -pe '$_ x= /pattern/'

to make a poor man's grep. I found that beautiful.

I'd have to say I agree with you.
His is less twisted, more subtle;
I think both deserve to be framed. =o)

__________________________________________________

It is beautiful, but I fear it could scare a beginner away. I'd rather
such brilliance were directed to the Fun With Perl list than exposed to
people many of whom are no doubt still wondering whether Perl is a
write-only language.

Which might be a cue for Tim Maher to pipe up and talk about his "minimal
Perl" dialect, if he's here...

Yikes! This is what I was talking about. Amazing.

Let me take a crack at the first one -- should be entertaining for everyone
;)

From: Jeff 'japhy/Marillion' Pinyan
The dash n (-n) puts the command 'print if !$seen{$_}++' in a while (<>) {
... } loop. So we get:

while (<>) {
print if !$seen{$_}++
}

$seen{$_}

Tries to lookup the line in the hash of lines we've already seen.

$seen{$_}++

This is a complete guess, I can't seem to find anything like this in the
'Programming Perl' book.
It seems that if you say:

$seen{$_} = 1;

it causes the key to be added to the hash with the value 1, which is true in
boolean context.
So, if the key (line) wasn't previously seen, line"

$seen{$_}

might return a 0 or "false" to indicate it wasn't found. Then the line

$seen{$_}++

might take that 0/"false", increment it by one turning it to 1/"true"
causing the key/$_/"line" to be added with a value of 1/"true". If the
$_/"line" were already seen, it would have been added initially with a value
1/"true"; the ++ in this situation would just increment the value to
2,3,4...n, all of which are "true" values.

!$seen{$_}

Might negate the 1/"true" return of looking up a key that previously existed
in the hash, causing the

print

statement to execute, which is just short for

print STDOUT $_;

So how close am I and where can I read about this?

This would bypass the need for the print statement, but I'm not sure how the
'$_ x= ' in the statement works.
This is tons of fun! Dying to know the answer!

Thanks,
David

Harrumph, I wasn't caught up with the earlier part of the thread, so I
apologize for picking on Jeff. Besides, it's appropriately marked [OT].

The whole thread is still scary for a beginner, though. Let's all focus on
being clear rather than clever. There's a list called FWP (see
lists.perl.org) for the latter.

It's just a postfix increment on the value in %seen as looked up using
$_ as the key. You've got the idea.
lol -- you kind of have to look up the pieces. ;o]
Or maybe there's something in the FAQ's?
This is the one that tickled me most.
I parsed it in another post. ;o]
left-hand argument if the current value of the left-hand argument is
boolean false. "$a ||= 1" assigns 1 to $a if $a was '',0, or undef.
Otherwise it leaves it alone.

Since print returns a bool, and while <IN> assigns to $_ (which is
print()'s default), it prints the line just read and assigns 1 to
$seen{$_} if $seen{$_} had no value, but if $seen{$_} already has a
value, it just returns that in a void context -- a no-op. =o)
Then it reads the next line of <IN>.

I love Perl. =o)

Of course, writing readable code is always a good idea, but explaining
*tight* code is a great learning experience! lol!

__________________________________________________

Truly astounding.


From: Paul
So the command not only weeds out the duplicate lines, it counts as well!
That's great! I hate to even think of how many lines in java it would take
to do the same thing. It takes on line just to create the hash.
I am a very accomplished java programmer and many aspects of programming are
just old news. I really have zero need to learn another language in my
professional career. But since I've ran across Perl, I've found so many new
ways to do things that were just old hat -- I just have to learn it. Perl
takes something I've been doing for years and makes it new again.

David

On Mon, Jul 30, 2001 at 02:16:42PM -0600, Canavan, John wrote:
: Hopefully, at least some beginners would find this playful and entertaining
: enough that they'd be willing to work a little harder to understand the
: code. When I'm learning a new language, I enjoy reading playful but hard
: code some of the time.

Yes, I imagine it would be a fun thing for beginners to see. Some
might try hard to understand them. A teaser would be fine but, the
entirety of a one-liner thread most likley doesn't belong on the
beginners lists[*]. Not to mention, Fun With Perl is so low traffic
that if they are interested they can surely sign up.

* If someone is asking a question about "How to X in a one-liner",
it's probably ok here, to a point. That point gets pushed further
and further away the more usefull and real (for some value of real)
the one-liner is.

Casey West

Well, it does, though this code doesn't take advantage of that.
The boolean NOT ( the ! ) always returns 1 if the $seen{$_} is false,
or '' if it's true -- and any value other than 0,'', or undef is true.
lol -- true. But remember one thing for which Perl is famous: the word
is "cryptocontext". Expressions do/return different things in different
contexts. Important to understand, and handy to use. Java expressions
don't change behavior or value if you change their context. Perl has
tools many other languages don't have, but they can bite you. =o)
Well, it *does* always look good on a resumé to have a few extra hats
available. =o)
again.

Yeah. >;O}

Gearhead Heaven! lol!

__________________________________________________

1) Is that a "let's drop this thread" request?
While *I* am having a ball with it, I'd rather not stress the
Nu-B's...

2) Since we mention it (and it realy does sound like fun to me), where
exactly does one subscripbe to this "Fun With Perl" list? {:^)

__________________________________________________

On Mon, Jul 30, 2001 at 02:19:00PM -0700, Paul wrote:
:
: --- Casey West wrote:
: > Yes, I imagine it would be a fun thing for beginners to see. Some
: > might try hard to understand them. A teaser would be fine but, the
: > entirety of a one-liner thread most likley doesn't belong on the
: > beginners lists[*]. Not to mention, Fun With Perl is so low traffic
: > that if they are interested they can surely sign up.
:
: 1) Is that a "let's drop this thread" request?
: While *I* am having a ball with it, I'd rather not stress the
: Nu-B's...

I suppose it is, yes.

: 2) Since we mention it (and it realy does sound like fun to me), where
: exactly does one subscripbe to this "Fun With Perl" list? {:^)

Subscribe at fwp-subscribe@perl.org

Casey West

: : --- Casey West wrote:
: : > Yes, I imagine it would be a fun thing for beginners to see. Some
: : > might try hard to understand them. A teaser would be fine but, the
: : > entirety of a one-liner thread most likley doesn't belong on the
: : > beginners lists[*]. Not to mention, Fun With Perl is so low traffic
: : > that if they are interested they can surely sign up.
: :
: : 1) Is that a "let's drop this thread" request?
: : While *I* am having a ball with it, I'd rather not stress the
: : Nu-B's...
:
: I suppose it is, yes.

I should clarify. If this thread can continue with cool examples
*and* good explinations, there is no reason why it should be removed.
People will learn something from it. If those requirements can't be
met then, it probably won't be a very usefull thread. On FWP you
don't have to explain. Pick your poison. :-)

Casey West