Tying sub arguments is rare?

Ken Fox wrote
This has also concerned me. You seem to be making tie operate on
a reference to a reference, rather than just a reference.

Does this mean that code like this will break?

my @a = (1..5);
my $b = \($a[2]);
tie $a[2], 'MyWatcher';
$$b = 4; # should be seen by watcher


Mike Guy

Yes. Or, more precisely, on an lvalue reference, a la
\$a = \$b;
which we needed to make work anyway.
Yes.

Doggone it, if I have to change this I'm going to be ticked.

Just reorder line 2 to be after line 3. What's the big deal?

Can anyone think of an example that is less contrived and actually hard
to fix?

Right. I should've made that clearer, thanks.

Is having it behave differently from perl5 a big deal?

That's the key question, isn't it? I hardly think tying variables to
which references already exist is a common practice. And we already
have disavowed any claims of 100% Perl 5 compatibility.

Then, unless it has changed, I guess you have not used Tk recently
as IIRC there are some modules there which call tie via a reference
that was passed to the sub, so it is quite possible that other
references exist.

I think your assumption that this is not practice is not used a lot
is a badly informed one.

But then Tk will need a lot of work if XS is not going to be compatible
as it stresses XS now :-)
Maybe, but the number is getting smaller all the time.

Tk! Aaugh! The people in eyeball hats! I'm ready for my close-up,
Mr. Wall....
I see a thin thread of hope. Are the ties via references user-visible?

I am not sure what you mean, but I have seen tie used as

sub abc {

# ... code ...
tie $_[1], 'Package';
# ... code ...
}

and also

sub def {
my $ref = shift;

tie $$ref, 'Package';

# ... code ...
}

It is the latter of the two that I seem to remember being used
with some Tk modules. Normally to detect when the variable
was changed.

I mean, can Tk be changed to work without C<tie $$ref>, or is it part
of Tk's public (documented) interface?

From what I can remember, and I have not used it for a while, but it
was part of the documented API for some widgets. Usually composite
widgets that people have implemented on top of the base Tk stuff.

As I say it has been a while since I have used it so I may be
mis-remembering and/or things could have changed. But this is not
the only place that I have seen tie $$ref used.

the documented interface is

$obj->configure(-variable => \$foo)

where configure is an XS defined method. Given the reference the XS code
must be able to arrange for C code to be called on either FETCH or STORE.
It is currently done with 'U' magic.

Other than wholesale changes XS side (which are inevitable) I don't see
a problem.

Well, that about wraps it up. Tie by reference must work. Oh well.

Can you be more specific? Is the tieing to detect when a variable
changes?
Please inform us.

Well, XSUBs are going to have to undergo some serious translation
anyway in order to work in Topaz. So it wouldn't be too hard to turn
them into the equivalent of (\$) prototypes (i.e. lvalue refs).
It's a potential problem for both. You've seen the XSUB problem, but
there's also the implicit aliasing of @_ elements:

tie $_[0], 'SomePackage'; # urque

Doggone it, that's really unfortunate. If that has to keep working, I
may have to revisit large parts of the design. Grrr.
I think we have to continue allowing simple value modification. Even
transmogrification is no issue, because it's simply an implementation
technique limited to a of the type tree (PoolScalars), and thus
irrelevant to classes that fall outside that subtree.

On the other hand, yying is a very thorny issue. We have to be able
to tie _any_ variable, even one of a type that isn't known when Perl
is compiled. Combining that with aliased @_ ... brrr.

So the following works:

my $x = 1;
foo($x);

sub foo {
$_[0] = "blah";
}

$x starts out life as a ThinIntegerScalar and then transmogrifies
into a ThinStringScalar -- which is no problem because that's supported
for all objects derived from PoolScalar. Cool. One thing that occurs
to me though is that this is a "yank-the-rug-from-under-the-compiler"
sort of thing so some compiler optimizations (like caching vtbl
pointers) might break things.

The problem case is when a PoolScalar must transmogrify to a non-Pool
Scalar. (But Topaz doesn't have any non-PoolScalars yet... ;) One
possible solution is to have transmogrify() generate a forwarding
object if the new object is larger than the old one. (Don't lisp
systems with copying garbage collectors call these broken hearts?)

A forwarding system might be useful for moving things like FatScalar
out of the PoolScalar hierarchy too. A forwarding object should only
be a few bytes larger than the xguts pointer in FatScalar (and the
forwarding object will eventually be GC'd).

BTW, why are ThinScalars so big? ThinIntegerScalar is 32 bytes on
my machine (Sun Ultra-60 w/ gcc 2.95). Is this common? FatScalar by
comparison is only 24 bytes.

- Ken

Point taken. However, given the possibility of delete/new during any
subroutine call, I doubt that such an optimization would comply with
the ANSI standard.
The hard case is when a Scalar of undetermined type -- say, a
KenFoxScalar -- is tied. Since I have no knowledge of the size or
lifetime or other characteristics of KenFoxScalars, I can't assume
that transmogrification will work. Thus my intention to restrict
the semantics of tie.

However, if by Rule #1 we have to support tying via reference, there
is still a way out. It's ugly, but here goes:

We would have to impose two additional requirements on all tyable
Values. First, they would have to know how to displace themselves to
new storage and replace themselves back into the original storage.
Second, they would have to be at least as large as a TiedScalar (or
TiedArray or ...), so as to leave room for one to be contructed.

Those two requirements together would permit transmogrification of
any Value into a Tied{Scalar,Array,whatever}. Problem solved, but
only at great cost to my self-respect. (Get me the soap, quick.)
It's because ThinIntegerScalar is using a 'long long' currently.
That'll change in the next snapshot.

We haven't seen TiedScalar yet, but if you think you can make the
core work, then extension authors will probably just copy whatever
you come up with. We've been able to learn magic after all, so how
hard can transmogrification be? ;)

Is tying the only case where transmogrification hurts? This bit of
code seems painful too:

my $x = 1;
$x = foo();
sub foo { my $y : KenFoxScalar = 2; $y }

BTW, I've never really been sure if extension writers are going to
be creating new scalars or new Guts. Doesn't all of this trouble go
away if extension writers only create new Guts? Then everything
works like a Thin or Fat scalar -- just like Perl 5, but with the
ability to stash small values right in the SV.

- Ken

Tying is the only thing that hurts, yes -- it's the only feature that
puts a whole new personality into the body of an existing value.

(However, it's not the transmogrification that causes the pain, but
the _inability_ to transmogrify. (Perl 5's equivalent is a simple
change of the SvTYPE() and/or addition of a MAGIC, which is relatively
easy because everything above a certain SvTYPE has a common layout for
*SvANY(). No such easy way out for us.))
Oh, that's easy. Scalars are allowed to be tricky, but as long as
they're actually scalars they have to know how to do value assignment.
(Unless they want to break that on purpose, of course.) So the author
of KenFoxScalar provides an appropriate KenFoxScalar::copy_to(Scalar *),
or else inherits the default one which does the obvious (:-)) things
with final(), is(), str(), num(), etc.

(I think the current snapshot says assign_to(), BTW.)
New Scalars, I should think. FatScalar has a Guts pointer, true, but
all other Guts objects are members, not pointers. (See FatArray,
FatHash, etc.) So polymorphism doesn't have a chance to get started.

Rationale: Any given class's Guts are private. On the other hand, the
Value::Guts class will probably be documented as a convenient way to
create Guts classes with all the standard behavior(s).
Yes, but then we're back to Perl 5's requirement of an extra level of
indirection on EVERY SINGLE DATUM. I'm sick of that.

"You can solve any programming problem with an extra level of
indirection, except the problem of too many levels of indirection."
-- Tom C.

Custom representations aren't very useful if they get "sliced" whenever
they get copied. Perl-wrapped constructors become impossible too. The
copy_to() method has to be able to do the moral equivalent of sv_upgrade()
which puts us back into the transmogrification quandry because (Scalar *)
isn't enough.
Huh? The core can still do ThinScalars -- just make extension writers
do FatScalar-like scalars. I have no objection to my customizations
taking an indirection penalty in exchange for simple transmogrification.

- Ken

Sure they are. Think of $$. After C<$a = $$>, you don't expect $a to
be a magical pid variable, just a copy of the current pid.

On the other hand, in C<$a = $b>, $b's copy_to() is allowed to do a
dynamic_cast<>() on $a, and do something special if it's a known type.
That allows assignments to be fatter than usual when appropriate.
There's were reasons Larry made all objects references. I'll bet this
is one of them.
In that case, FatScalar et al become mere forwarders. That would
impose an extra vtable dispatch on *every* operation that currently
requires only one.
But not only your customizations would pay that penalty. FatScalar,
FatArray, FatHash, etc. would also pay it. That would add up.

I never worry about giving away a Factor of Two in a program.
The problem is, neither do nine of my friends.
-- Stu Feldman (progenitor of Make)

That's because $$ IMHO isn't a representation -- it's merely magical.
If I write a ScalarWithUnit representation, I don't want the unit sliced
off whenever an assignment happens. I think of representation as being
equivalent to Perl 5's IV, NV, PV, etc. Those things don't disappear in
an assignment -- why should a custom representation disappear?

BTW, this brings up another facet of representation -- overloading.
Perl 5's IV, NV, PV, etc. reps are overloaded. Custom reps should allow
overloading by default too.
Which is fine if custom scalars can transmogrify the target.
Now I'm confused about whether you believe objects and representations
should be the same thing. IMHO, a custom rep should return ref() false.
I said FatScalar-*like* not FatScalar derivatives. Build a virtual class
CustomScalar as peers of Thin and Fat scalars. Require all extension writers
to subclass from Thin, Fat or Custom *only*. Thin and Fat can be extensively
optimized. Custom is only there to make hard things possible.

- Ken

You're talking about the copying semantics of overloading, while I'm
talking about the copying semantics of tying.

I intend to support overloading with an entirely separate mechanism
from what we've been discussing so far, because overloaded assignment
is just the tip of the iceberg---everything else has to be overloaded,
too.

(I sometimes confuse the two sometimes because $$ is going to be a
specialized representation, just one that doesn't mind being sliced.)
But they do! For example: An SVt_PVMG that happens to hold an undef,
when assigned to a target SV, will not force the target to upgrade to
PVMG. It will simply turn off the OK flags of the target.
Ah, now I see! I must ponder this idea. It has great potential.

... It's even possible that this could be the core construct of
support for overloading.