FAQ
Hi,

As a followup to the discussion in January, I'd like post a revised patch to
this list that implements closures and anonymous functions in PHP.

INTRODUCTION
------------

Closures and lambda functions can make programming much easier in
several ways:

1. Lambda functions allow the quick definition of throw-away functions
that are not used elsewhere. Imaging for example a piece of code that
needs to call preg_replace_callback(). Currently, there are three
possibilities to acchieve this:

a. Define the callback function elsewhere. This distributes code that
belongs together throughout the file and decreases readability.

b. Define the callback function in-place (but with a name). In
that case
one has to use function_exists() to make sure the function is only
defined once. Example code:

<?php
function replace_spaces ($text) {
if (!function_exists ('replace_spaces_helper')) {
function replace_spaces_helper ($matches) {
return str_replace ($matches[1], ' ', '&nbsp;').' ';
}
}
return preg_replace_callback ('/( +) /',
'replace_spaces_helper',
$text);
}
?>

Here, the additional if() around the function definition makes the
source code difficult to read.

c. Use the present create_function() in order to create a function at
runtime. This approach has several disadvantages: First of all,
syntax
highlighting does not work because a string is passed to the
function.
It also compiles the function at run time and not at compile
time so
opcode caches can't cache the function.

2. Closures provide a very useful tool in order to make lambda
functions even
more useful. Just imagine you want to replace 'hello' through
'goodbye' in
all elements of an array. PHP provides the array_map() function which
accepts a callback. If you don't wan't to hard-code 'hello' and
'goodbye'
into your sourcecode, you have only four choices:

a. Use create_function(). But then you may only pass literal values
(strings, integers, floats) into the function, objects at best as
clones (if var_export() allows for it) and resources not at
all. And
you have to worry about escaping everything correctly.
Especially when
handling user input this can lead to all sorts of security issues.

b. Write a function that uses global variables. This is ugly,
non-reentrant and bad style.

c. Create an entire class, instantiate it and pass the member function
as a callback. This is perhaps the cleanest solution for this
problem
with current PHP but just think about it: Creating an entire
class for
this extremely simple purpose and nothing else seems overkill.

d. Don't use array_map() but simply do it manually (foreach). In this
simple case it may not be that much of an issue (because one simply
wants to iterate over an array) but there are cases where doing
something manually that a function with a callback as parameter
does
for you is quite tedious.

[Yes, I know that str_replace also accepts arrays as a third
parameter so
this example may be a bit useless. But imagine you want to do a more
complex operation than simple search and replace.]

PROPOSED PATCH
--------------

I now propose a patch that implements compile-time lambda functions and
closures for PHP while keeping the patch as simple as possible. The patch is
based on a previous patch on mine which was based on ideas discussed here
end of December / start of January.

Userland perspective
--------------------

1. The patch adds the following syntax as a valid expression:

function & (parameters) { body }

(The & is optional and indicates - just as with normal functions - that the
anonymous function returns a reference instead of a value)

Example usage:

$lambda = function () { echo "Hello World!\n"; };

The variable $lambda then contains a callable resource that may be called
through different means:

$lambda ();
call_user_func ($lambda);
call_user_func_array ($lambda, array ());

This allows for simple lambda functions, for example:

function replace_spaces ($text) {
$replacement = function ($matches) {
return str_replace ($matches[1], ' ', '&nbsp;').' ';
};
return preg_replace_callback ('/( +) /', $replacement, $text);
}

2. The patch implements closures by defining an additional keyword 'lexical'
that allows an lambda function (and *only* an lambda function) to import
a variable from the "parent scope" to the lambda function scope. Example:

function replace_in_array ($search, $replacement, $array) {
$map = function ($text) {
lexical $search, $replacement;
if (strpos ($text, $search) > 50) {
return str_replace ($search, $replacement, $text);
} else {
return $text;
}
};
return array_map ($map, array);
}

The variables $search and $replacement are variables in the scope of the
function replace_in_array() and the lexical keyword imports these variables
into the scope of the closure. The variables are imported as a reference,
so any change in the closure will result in a change in the variable of the
function itself.

3. If a closure is defined inside an object, the closure has full access
to the current object through $this (without the need to use 'lexical' to
import it seperately) and all private and protected methods of that class.
This also applies to nested closures. Essentially, closures inside
methods are
added as public methods to the class that contains the original method.

4. Closures may live longer as the methods that declared them. It is
perfectly
possible to have something like this:

function getAdder($x) {
return function ($y) {
lexical $x;
return $x + $y;
};
}

Zend internal perspective
-------------------------

The patch basically changes the following in the Zend engine:

When the compiler reaches a lambda function, it creates a unique name
for that
function ("\0__compiled_lambda_FILENAME_N" where FILENAME is the name of the
file currently processed and N is a per-file counter). The use of the
filename
in the function name ensures compability with opcode caches. The lambda
function is then immediately added to the function table (either the global
function table or that of the current class if declared inside a class
method).
Instead of a normal ZEND_DECLARE_FUNCTION opcode the new
ZEND_DECLARE_LAMBDA_FUNC is used as an opcode at this point. The op_array
of the new function is initialized with is_lambda = 1 and is_closure = 0.

When parsing a 'lexical' declaration inside an anonymous function the parser
saves the name of the variable that is to be imported in an array stored
as a member of the op_array structure (lexical_names).

The opcode handler for ZEND_DECLARE_LAMBDA_FUNC does the following: First of
all it creates a new op_array and copies the entire memory structure of the
lambda function into it (the opcodes themselves are not copied since they
are only referenced in the op_array structure). Then it sets is_closure = 1
on the new op_array, and for each lexical variable name that the compiler
added to the original op_array it creates a reference to that variable from
the current scope into a HashTable member in the new op_array. It also saves
the current object pointer ($this) as a member of the op_array in order to
allow for the closure to access $this. Finally it registers the new op_array
as a resource and returns that resource.

The opcode handler of the 'lexical' construct simply fetches the variable
from that HashTable and imports it into local scope of the inner function
(just like with 'global' only with a different hash table).

Some hooks were added that allow the 'lambda function' resource to be
called.
Also, there are several checks in place that make sure the lambda function
is not called directly, i.e. if someone explicitely tries to use the
internal
function name instead of using the resource return value of the declaration.

The patch
---------

The patch is available here:
<http://www.christian-seiler.de/temp/closures-php-5.3-2008-06-16-1.diff>

Please note that I did NOT include the contents of zend_language_scanner.c
in the patch since that can easily be regenerated and just takes up enormous
amounts of space.

The patch itself applies against the 5.3 branch of PHP.

If I understand the discussion regarding PHP6 on this list correctly, some
people are currently undergoing the task of removing the unicode_semantics
switch and if (UG(unicode)). As soon as this task is finished I will also
provide a patch for CVS HEAD (it doesn't make much sense adopting the patch
now and then having to change it again completely afterwards).

BC BREAKS
---------

* Introduction of a new keyword 'lexical'. Since it is very improbable
that
someone should use it as a function, method, class or property name, I
think this is an acceptable break.

Other that that, I can find no BC breaks of my patch.

CAVEATS / POSSIBLE WTFS
-----------------------

* On writing $func = function () { }; there is a semicolon necessary.
If left
out it will produce a compile error. Since any attempt to remove that
necessity would unecessarily bloat the grammar, I suggest we simply keep
it the way it is. Also, Lukas Kahwe Smith pointed out that a single
trailing semicolon after a closing brace already exists: do { }
while ();

* The fact that 'lexical' creates references may cause certain WTFs:

for ($i = 0; $i < 10; $i++) {
$arr[$i] = function () { lexical $i; return $i; };
}

This will not work as expected since $i is a reference and thus all
created closures would reference the same variable. In order to get this
right one has to do:

for ($i = 0; $i < 10; $i++) {
$loopIndex = $i;
$arr[$i] = function () { lexical $loopIndex; return $loopIndex; };
unset ($loopIndex);
}

This can be a WTF for people that don't expect lexical to create an
actual reference, especially since other languages such as JavaScript
don't do it. On the other hand, global and static both DO create
references so that behaviour is consistent with current PHP.

But complex constructions such as this will probably not be used by
beginners so maintaining a good documentation should solve this.

* The fact that 'lexical' is needed at all may cause WTFs. Other languages
such as JavaScript implicitely have the entire scope visible to child
functions. But since PHP does the same thing with global variables, I
find a keyword like 'lexical' much more consistent than importing the
entire scope (and always importing the entire scope costs unnecessary
performance).

FINAL THOUGHTS
--------------

My now proposed patch addresses the two main problems of my previous patch:
Support for closures in objects (with access to $this) and opcode caches. My
patch applies against PHP_5_3 and does not break any tests. It adds a
valuable
new language feature which I'd like to see in PHP.

Regards,
Christian

Search Discussions

  • Christian Seiler at Jun 16, 2008 at 6:57 pm
    Hi,

    Lukas Kahwe Smith asked me to put my proposal into the PHP wiki, which I
    have done:

    http://wiki.php.net/rfc/closures

    I also have incorporated the last comment by troels knak-nielsen about
    JS behaving the same way as my patch and thus not being that much of a
    WTF at all (I somehow had a different behaviour for JS in mind, I
    probably got confused with yet another language).

    Anyway, feel free to comment.

    Regards,
    Christian
  • Larry Garfield at Jun 17, 2008 at 2:24 am

    On Monday 16 June 2008, Christian Seiler wrote:
    Hi,

    Lukas Kahwe Smith asked me to put my proposal into the PHP wiki, which I
    have done:

    http://wiki.php.net/rfc/closures

    I also have incorporated the last comment by troels knak-nielsen about
    JS behaving the same way as my patch and thus not being that much of a
    WTF at all (I somehow had a different behaviour for JS in mind, I
    probably got confused with yet another language).

    Anyway, feel free to comment.
    Should comments from user-space folk be posted here or as comments at the
    bottom of the wiki page?

    --
    Larry Garfield AIM: LOLG42
    larry@garfieldtech.com ICQ: 6817012

    "If nature has made any one thing less susceptible than all others of
    exclusive property, it is the action of the thinking power called an idea,
    which an individual may exclusively possess as long as he keeps it to
    himself; but the moment it is divulged, it forces itself into the possession
    of every one, and the receiver cannot dispossess himself of it." -- Thomas
    Jefferson
  • Philip Olson at Jun 17, 2008 at 2:53 am

    .
    Anyway, feel free to comment.
    Should comments from user-space folk be posted here or as comments
    at the
    bottom of the wiki page?
    We're all users... so here.

    Regards,
    Philip
  • Larry Garfield at Jun 17, 2008 at 3:57 am
    Thoughts from a user-land denizen:

    - I conceptually really like. lambdas and closures are a feature I've been
    jealous of Javascript for having since I learned how Javascript "really"
    worked. :-)

    - I recall earlier discussion pondering if we should be using the
    keyword "function" to describe this new construct. It is a function, but not
    as we know it. (Thank you, Mr. Spock.) Should we consider using the
    keyword "lambda" instead, or do we think re-using "function" is acceptable?
    (I'm cool either way; I just wanted to make sure it was a conscious
    decision.)

    - I don't think the lexical keyword is a wtf at all, especially given that it
    works essentially like global. It simplifies both the internal code and the
    user-space code, because it becomes more self-documenting. The semi-colon is
    a slight wtf, but since it is the same as Javascript I think most people will
    get used to it. I have no objection to it.

    - I am a little confused about the OOP interaction. How does a function
    become a public method of the class?

    class Example {
    private $a = 2;

    function myMethod($b) {
    $lambda = function() {
    lexical $b;
    return $this->a * $b; // This part I get
    };
    return $lambda;
    }
    }

    $e = new Example();
    $lambda = $e->myMethod();
    $e->$lambda(5);

    That doesn't seem right at all, but that's how I interpret "Essentially,
    closures inside methods are added as public methods to the class that
    contains the original method." Can you give an example of what that actually
    means?

    - Related to that, would it then be possible to add methods to a class at
    runtime using lambda functions as the added methods? If so, how? If not, is
    that something that could reasonably be added here without hosing performance
    (or at least doing so less than stacking __call() and call_user_func_array()
    does)?

    I guess I just need some code examples to see how this behaves in relation to
    objects, because I'm not really following it as is.

    - My minuscule knowledge of the engine internals looks at the description
    there and doesn't see anything to be scared about, but I am far from an
    authority in that subject. It does seem to side-step the "can't compile at
    compile time" problem that was raised back in December, which is good.

    Overall, big +1 pending clarification of how lambdas and classes interact,
    which I still don't grok.
    On Monday 16 June 2008, Christian Seiler wrote:
    Hi,

    Lukas Kahwe Smith asked me to put my proposal into the PHP wiki, which I
    have done:

    http://wiki.php.net/rfc/closures

    I also have incorporated the last comment by troels knak-nielsen about
    JS behaving the same way as my patch and thus not being that much of a
    WTF at all (I somehow had a different behaviour for JS in mind, I
    probably got confused with yet another language).

    Anyway, feel free to comment.

    Regards,
    Christian

    --
    Larry Garfield AIM: LOLG42
    larry@garfieldtech.com ICQ: 6817012

    "If nature has made any one thing less susceptible than all others of
    exclusive property, it is the action of the thinking power called an idea,
    which an individual may exclusively possess as long as he keeps it to
    himself; but the moment it is divulged, it forces itself into the possession
    of every one, and the receiver cannot dispossess himself of it." -- Thomas
    Jefferson
  • Alexey Zakhlestin at Jun 17, 2008 at 4:59 am

    On 6/17/08, Larry Garfield wrote:
    - I am a little confused about the OOP interaction. How does a function
    become a public method of the class?

    class Example {
    private $a = 2;

    function myMethod($b) {
    $lambda = function() {
    lexical $b;
    return $this->a * $b; // This part I get
    };
    return $lambda;
    }
    }

    $e = new Example();
    $lambda = $e->myMethod();
    $e->$lambda(5);

    That doesn't seem right at all, but that's how I interpret "Essentially,

    closures inside methods are added as public methods to the class that

    contains the original method." Can you give an example of what that actually
    means?

    As far as I understand, it means following:

    class Example
    {
    private $a = 1;

    private function b()
    {
    return 2;
    }

    public function getLambda($param)
    {
    $lambda = function($lparam) {
    lexical $param;

    return $this->a + $this->b() + $param + $lparam;
    }

    return $lambda;
    }
    }

    $obj = new Example();
    $lambda = $obj->getLambda(3);

    $result = $lambda(4); // 1+2+3+4 => 10
  • Nathan Nobbe at Jun 17, 2008 at 5:57 am

    On Mon, Jun 16, 2008 at 9:57 PM, Larry Garfield wrote:

    Thoughts from a user-land denizen:

    - Related to that, would it then be possible to add methods to a class at
    runtime using lambda functions as the added methods? If so, how? If not,
    is
    that something that could reasonably be added here without hosing
    performance
    (or at least doing so less than stacking __call() and
    call_user_func_array()
    does)?

    im curious about this as well, and i would welcome such functionality.
    adding methods to class instances from within the definition of the class
    seems intuitive, the $this keyword should be available, but when adding
    methods via a variable which holds a handle to a class instance, well, i
    wonder, would the $this keyword be available within those anonymous function
    definitions?

    class Dynamic {
    private $someVar = 5;
    /// adding a function to instances from within the class
    public function addMethodAtRuntime() {
    $this->dynamicFunc1 = function() {
    return $this->someVar; // expected to work
    }
    }
    }

    /// adding a function to an instance externally
    $dynamic = new Dynamic();
    $anotherVar = 6;
    $dynamic->dynamicFunc2 = function() {
    lexical $anotherVar; // expected to work
    return $this->someVar + $anotherVar; // would this work ?
    }

    /// invoking dynamically added methods
    /// (anticipated behavior given above definitions)
    $dynamic->addMethodAtRuntime();
    echo $dynamic->dynamicFunc1(); // 5
    echo $dynamic->dynamicFunc2(); // 11

    -nathan
  • Christian Seiler at Jun 17, 2008 at 7:45 am
    Hi!
    class Dynamic {
    private $someVar = 5;
    /// adding a function to instances from within the class
    public function addMethodAtRuntime() {
    $this->dynamicFunc1 = function() {
    return $this->someVar; // expected to work
    }
    }
    }

    /// invoking dynamically added methods
    /// (anticipated behavior given above definitions)
    $dynamic->addMethodAtRuntime();
    echo $dynamic->dynamicFunc1(); // 5
    This will not work - for the same reason as this does not work:

    class SpecialChars {
    public $process = 'htmlspecialchars';
    }

    $sc = new SpecialChars ();
    var_dump ($sc->process ('<>')); // call to undefined ...

    On the other hand, the following will work:

    $sc = new SpecialChars ();
    $processor = $sc->process;
    var_dump ($processor ('<>')); // string(8) "&lt;&gt;"

    The same with closures:

    echo $dynamic->dynamicFunc1(); // call to undefined ...
    $func = $dynamic->dynamicFunc1;
    echo $func (); // 5
    echo call_user_func ($dynamic->dynamicFunc1); // 5

    As I sead in my other mail: I don't see closures as a method for somehow
    making it possible to extend classes dynamically - if you want that, use
    runkit.

    Regards,
    Christian
  • Christian Seiler at Jun 17, 2008 at 7:40 am
    Hi,
    - I am a little confused about the OOP interaction. How does a function
    become a public method of the class?
    To clarify: the "public method" ist just the internal representation of
    the lambda function and has *nothing* to do with the semantics of
    calling the lambda itself. The "method" only means that the lambda
    function defined inside another method can access the class members and
    "public" only means that the lambda function can still be called from
    outside the class.
    class Example {
    private $a = 2;

    function myMethod($b) {
    $lambda = function() {
    lexical $b;
    return $this->a * $b; // This part I get
    };
    return $lambda;
    }
    }

    $e = new Example();
    $lambda = $e->myMethod();
    $e->$lambda(5);
    No, that's not what my patch does. My patch does:

    class Example {
    private $a = 2;

    public function myMethod ($b) {
    return function () {
    lexical $b;
    return $this->a * $b;
    };
    }
    }

    $e = new Example ();
    $lambda = $e->myMethod (4);
    var_dump ($lambda ()); // int(8)
    $lambda2 = $e->myMethod (6);
    var_dump ($lambda2 ()); // int(12)

    So esentially, it does not matter whether you define a lambda function
    inside a method or a function (or in global scope, for that matter), you
    always use it the same way. The in-class-method lambda function only has
    the additional advantage of being able to access the private and
    protected class members since *internally* it is treated like a public
    class method.
    - Related to that, would it then be possible to add methods to a class at
    runtime using lambda functions as the added methods? No.
    If not, is that something that could reasonably be added here without
    hosing performance (or at least doing so less than stacking __call() and
    call_user_func_array() does)?
    If you want to add methods dynamically to classes, why not use the
    runkit extension? I really don't see a point in making lambda functions
    and closures something they are not.

    Regards,
    Christiaan
  • Larry Garfield at Jun 18, 2008 at 1:59 am

    On Tuesday 17 June 2008, Christian Seiler wrote:
    Hi,
    - I am a little confused about the OOP interaction. How does a function
    become a public method of the class?
    To clarify: the "public method" ist just the internal representation of
    the lambda function and has *nothing* to do with the semantics of
    calling the lambda itself. The "method" only means that the lambda
    function defined inside another method can access the class members and
    "public" only means that the lambda function can still be called from
    outside the class.
    If one knew how to access it, which it seems is not possible/feasible for
    user-space code.
    class Example {
    private $a = 2;

    function myMethod($b) {
    $lambda = function() {
    lexical $b;
    return $this->a * $b; // This part I get
    };
    return $lambda;
    }
    }

    $e = new Example();
    $lambda = $e->myMethod();
    $e->$lambda(5);
    No, that's not what my patch does. My patch does:

    class Example {
    private $a = 2;

    public function myMethod ($b) {
    return function () {
    lexical $b;
    return $this->a * $b;
    };
    }
    }

    $e = new Example ();
    $lambda = $e->myMethod (4);
    var_dump ($lambda ()); // int(8)
    $lambda2 = $e->myMethod (6);
    var_dump ($lambda2 ()); // int(12)

    So esentially, it does not matter whether you define a lambda function
    inside a method or a function (or in global scope, for that matter), you
    always use it the same way. The in-class-method lambda function only has
    the additional advantage of being able to access the private and
    protected class members since *internally* it is treated like a public
    class method.
    I see. It would be great if you could update the RFC with this information so
    that it's clearer.
    If you want to add methods dynamically to classes, why not use the
    runkit extension? I really don't see a point in making lambda functions
    and closures something they are not.
    I was asking if they could be used for that, not to make them into a different
    animal. As for using runkit, do I really need to answer that? :-)

    Two other questions that just occurred to me:

    1) What is the interaction with namespaces, if any? Are lambdas as
    implemented here ignorant of namespace, or do they take the namespace where
    they are lexically defined?

    2) What happens with the following code?

    class Foo {
    private $a;
    }

    $f = new Foo();

    $b = 5;

    $f->myfunc = function($c) {
    lexical $b;
    print $a; // This generates an error, no?
    print $b; // This prints 5, right?
    print $c; // Should print whatever $c is.
    }

    $f->myfunc(3);

    Or is the above a parse error entirely?

    --
    Larry Garfield AIM: LOLG42
    larry@garfieldtech.com ICQ: 6817012

    "If nature has made any one thing less susceptible than all others of
    exclusive property, it is the action of the thinking power called an idea,
    which an individual may exclusively possess as long as he keeps it to
    himself; but the moment it is divulged, it forces itself into the possession
    of every one, and the receiver cannot dispossess himself of it." -- Thomas
    Jefferson
  • Christian Seiler at Jun 18, 2008 at 11:45 am
    Hi!
    - I am a little confused about the OOP interaction. How does a function
    become a public method of the class?
    To clarify: the "public method" ist just the internal representation of
    the lambda function and has *nothing* to do with the semantics of
    calling the lambda itself. The "method" only means that the lambda
    function defined inside another method can access the class members and
    "public" only means that the lambda function can still be called from
    outside the class.
    If one knew how to access it, which it seems is not possible/feasible for
    user-space code.
    No, that's not what I meant. The engine uses the following internal trick:

    a) Upon copmilation, my patch simply adds the lambdas as normal
    functions to the function table with an automatically generated
    unique (!) name. If it happens to be defined within a class method,
    the function will be added as a public final method to that class.

    b) That added function is not directly callable due to checks of a flag
    in the internal structure of that function.

    c) At the place of the function definition the compiler leaves an
    opcode "grab function $generatedname and make a closure out of it".
    This opcode then looks up the generated lambda function, copies the
    function structure, saves the bound variables in that structure and
    returns the copied structure as a resource.

    d) Normally, when a function is called, the name is looked up in the
    function table. The function structure that is retrieved from there
    is then used to execute the function. Since a lambda resource is
    already a function structure, there is no necessity to look up
    anything in the function table but the function structure can be
    directly passed on to the executor.

    Please note step d): The closure functionality only changes the *lookup*
    of the function - so instead of getting the function structure from a
    hash table lookup I get the function structure by retrieving it from the
    resource. But *after* the lookup of a class method there are checks for
    the access protection of that method. So these access protection checks
    also apply to closures that were called. If a lambda function was not
    declared public, it could not be used outside of the class it was
    defined in. Perhaps this makes it clearer?
    I see. It would be great if you could update the RFC with this information so
    that it's clearer.
    Done: http://wiki.php.net/rfc/closures
    Two other questions that just occurred to me:

    1) What is the interaction with namespaces, if any? Are lambdas as
    implemented here ignorant of namespace, or do they take the namespace where
    they are lexically defined?
    My patch itself is namespace-ignorant, but the net result is not:

    a) The generated internal function names do not contain the current
    namespace name, but since namespace names in function names are only
    used for lookup if you want to call the function. And calling
    lambdas by name (!) directly doesn't work anyway (is not supposed
    to work) so this poses no problem.

    b) The code *inside* the closure is namespace-aware because the
    information of which namespace is used is added at compile time.
    Either the name lookup is done entirely at compile time or the
    current compiler namespace is automatically added to all runtime
    lookup calls (this is already the case with current code). So
    the information which namespace a function resides in is currently
    *irrelevant* at runtime when calling other functions.

    For (b) let me make two examples:

    Suppose you have the following code:

    namespace Foo;

    function baz () {
    return "Hello World!\n";
    }

    function bar () {
    return function () {
    echo baz ();
    };
    }

    and in another file:

    $lambda = Foo::bar ();
    $lambda ();

    This will - as expected - print "Hello World!\n".

    The reason is that the compiler upon arriving at the baz() function call
    inside the closure already looks up the function in the function table
    directly (it knows the current namespace) - and simply creates a series
    of opcodes that will call the function with the name "Foo::baz" (the
    lookup is already done at compile time).

    Consider this other code:

    foo-bar.php:

    namespace Foo;

    function bar () {
    return function () {
    echo baz ();
    };
    }

    foo-baz.php:

    namespace Foo;

    function baz () {
    return "Hello World!\n";
    }

    baz.php:

    function baz () {
    return "PHP!\n";
    }

    test1.php:

    require 'foo-bar.php';
    require 'foo-baz.php';

    $lambda = Foo::bar ();
    $lambda ();

    test2.php:

    require 'foo-bar.php';
    require 'baz.php';

    $lambda = Foo::bar ();
    $lambda ();

    Running test1.php yields "Hello World!" whereas running test2.php yields
    "PHP!". Why is this? Because when the compiler reaches the baz ()
    function call in the closure, it cannot find the function so it cannot
    determine whether it's a function in global or in namespace scope. So it
    will simply add a series of opcodes that say "try Foo::bar and if that
    does not exist try bar". Here again, Foo:: is added by the compiler to
    the opcode because the compiler has the necessary information which
    namespace the call is currently in.

    The runtime execution engine NEVER looks at the function as to which
    namespace the function belongs to, it ONLY looks at the function calling
    opcodes that the compiler already correctly (!) generates.

    Please note that I have only described what the PHP engine does with
    namespaces until now ANYWAY, the closures DO NOT CHANGE ANYTHING related
    to this. That's why my patch doesn't have to care about namespaces at
    all but the net-result closures will.
    2) What happens with the following code?

    class Foo {
    private $a;
    }

    $f = new Foo();

    $b = 5;

    $f->myfunc = function($c) {
    lexical $b;
    print $a; // This generates an error, no?
    print $b; // This prints 5, right?
    print $c; // Should print whatever $c is.
    }

    $f->myfunc(3);

    Or is the above a parse error entirely?
    Well, first of all, it's a parse error because you forgot the ; after
    the } of the closure. ;-)

    Then: Closures only have access to the scope in which they are defined
    in. Whether you assign that closure to a class property or not does not
    change the semantics of the closure itself. The closure definition is
    simply the following part: function ($c) { ... }. You then use the part
    to store the closure as a class property. The closure you define is
    entirely oblivious of the class since it is not defined inside a class
    method. Consider this code:

    function foo () {
    echo $a;
    }

    $f->myfunc = 'foo';

    Well, of course, first of all $a will not work anyway because $a has
    never been the object member access method of PHP. But even if you use
    $this->a, that won't work with foo either. For the exact same reasons it
    will not work with closures.

    Your comments inside the closure are correct though: $a gives an error
    (notice: undefined variable), $b gives 5 (unless $b is modified later
    on) and $c gives the current value of the parameter $c.

    But: As I already said last time, $f->myfunc(3) won't work. That will
    give "no such method". For the same reason as $f->myfunc = 'foo';
    $f->myfunc(); will not work - because -> can only be used to call
    METHODS, not dynamic functions stored in properties.

    Regards,
    Christian
  • Edward Z. Yang at Jun 17, 2008 at 11:48 am

    Larry Garfield wrote:
    $e = new Example();
    $lambda = $e->myMethod();
    $e->$lambda(5);

    That doesn't seem right at all, but that's how I interpret "Essentially,
    closures inside methods are added as public methods to the class that
    contains the original method." Can you give an example of what that actually
    means?
    At first blush, this sets off warning bells in my head, I suppose
    because my notion of a lambda says that the lambda should not carry any
    baggage about the context it was created in.

    However, with further thought, I believe that binding the lambda's
    lexical scope to the place it was defined is:

    * Conducive to good coding (you will always be able to look outside the
    lambda to find out where the lexical variables are coming form)
    * Adds functionality, since anything you want to pass to the function
    via the callee's context can be passed via a parameter

    What would be neat, however, is the ability to rebind the lambda to
    another context. Also, I don't know how other languages do it (Python?
    Lisp?).

    --
    Edward Z. Yang GnuPG: 0x869C48DA
    HTML Purifier <http://htmlpurifier.org> Anti-XSS Filter
    [[ 3FA8 E9A9 7385 B691 A6FC B3CB A933 BE7D 869C 48DA ]]
  • Marcus Boerger at Jun 17, 2008 at 10:03 am
    Hello Christian,

    very nice work. I think we should really add this to 5.3. The only thing I
    don't like is the function naming (“\0__compiled_lambda_FILENAME_N”). Can
    we drop the \0? For methods inside classes, do we have to provide real
    private methods or do we support visibility fully? Or did you use the \0
    prefix to prevent direct invocations? If so, it doesn't help, as the user
    can simply create the function call with the \0.

    I think the best option would be to force lambda functions being public
    always. The last question is about changing visibility and overriding
    functions, do we want that, or should we mark lamdas as final? Note that
    preceeding the function names with a \0 does not help. In fact it might
    confuse reflection. Actually how does it integrate with reflection?

    Comments about the implementation:

    - ZendEngine2/zend_compile.c:

    Why provide forward declaration for free_filename(), simply putting the
    new function above the first user avoids it and does the same with
    increased maintainability.

    s/static void add_lexical_var (/static void add_lexical_var(/

    Other than that it all looks fine.


    marcus

    Monday, June 16, 2008, 7:39:19 PM, you wrote:
    Hi,
    As a followup to the discussion in January, I'd like post a revised patch to
    this list that implements closures and anonymous functions in PHP.
    INTRODUCTION
    ------------
    Closures and lambda functions can make programming much easier in
    several ways:
    1. Lambda functions allow the quick definition of throw-away functions
    that are not used elsewhere. Imaging for example a piece of code that
    needs to call preg_replace_callback(). Currently, there are three
    possibilities to acchieve this:
    a. Define the callback function elsewhere. This distributes code that
    belongs together throughout the file and decreases readability.
    b. Define the callback function in-place (but with a name). In
    that case
    one has to use function_exists() to make sure the function is only
    defined once. Example code:
    <?php
    function replace_spaces ($text) {
    if (!function_exists ('replace_spaces_helper')) {
    function replace_spaces_helper ($matches) {
    return str_replace ($matches[1], ' ', '&nbsp;').' ';
    }
    }
    return preg_replace_callback ('/( +) /',
    'replace_spaces_helper',
    $text);
    }
    ?>
    Here, the additional if() around the function definition makes the
    source code difficult to read.
    c. Use the present create_function() in order to create a function at
    runtime. This approach has several disadvantages: First of all,
    syntax
    highlighting does not work because a string is passed to the
    function.
    It also compiles the function at run time and not at compile
    time so
    opcode caches can't cache the function.
    2. Closures provide a very useful tool in order to make lambda
    functions even
    more useful. Just imagine you want to replace 'hello' through
    'goodbye' in
    all elements of an array. PHP provides the array_map() function which
    accepts a callback. If you don't wan't to hard-code 'hello' and
    'goodbye'
    into your sourcecode, you have only four choices:
    a. Use create_function(). But then you may only pass literal values
    (strings, integers, floats) into the function, objects at best as
    clones (if var_export() allows for it) and resources not at
    all. And
    you have to worry about escaping everything correctly.
    Especially when
    handling user input this can lead to all sorts of security issues.
    b. Write a function that uses global variables. This is ugly,
    non-reentrant and bad style.
    c. Create an entire class, instantiate it and pass the member function
    as a callback. This is perhaps the cleanest solution for this
    problem
    with current PHP but just think about it: Creating an entire
    class for
    this extremely simple purpose and nothing else seems overkill.
    d. Don't use array_map() but simply do it manually (foreach). In this
    simple case it may not be that much of an issue (because one simply
    wants to iterate over an array) but there are cases where doing
    something manually that a function with a callback as parameter
    does
    for you is quite tedious.
    [Yes, I know that str_replace also accepts arrays as a third
    parameter so
    this example may be a bit useless. But imagine you want to do a more
    complex operation than simple search and replace.]
    PROPOSED PATCH
    --------------
    I now propose a patch that implements compile-time lambda functions and
    closures for PHP while keeping the patch as simple as possible. The patch is
    based on a previous patch on mine which was based on ideas discussed here
    end of December / start of January.
    Userland perspective
    --------------------
    1. The patch adds the following syntax as a valid expression:
    function & (parameters) { body }
    (The & is optional and indicates - just as with normal functions - that the
    anonymous function returns a reference instead of a value)
    Example usage:
    $lambda = function () { echo "Hello World!\n"; };
    The variable $lambda then contains a callable resource that may be called
    through different means:
    $lambda ();
    call_user_func ($lambda);
    call_user_func_array ($lambda, array ());
    This allows for simple lambda functions, for example:
    function replace_spaces ($text) {
    $replacement = function ($matches) {
    return str_replace ($matches[1], ' ', '&nbsp;').' ';
    };
    return preg_replace_callback ('/( +) /', $replacement, $text);
    }
    2. The patch implements closures by defining an additional keyword 'lexical'
    that allows an lambda function (and *only* an lambda function) to import
    a variable from the "parent scope" to the lambda function scope. Example:
    function replace_in_array ($search, $replacement, $array) {
    $map = function ($text) {
    lexical $search, $replacement;
    if (strpos ($text, $search) > 50) {
    return str_replace ($search, $replacement, $text);
    } else {
    return $text;
    }
    };
    return array_map ($map, array);
    }
    The variables $search and $replacement are variables in the scope of the
    function replace_in_array() and the lexical keyword imports these variables
    into the scope of the closure. The variables are imported as a reference,
    so any change in the closure will result in a change in the variable of the
    function itself.
    3. If a closure is defined inside an object, the closure has full access
    to the current object through $this (without the need to use 'lexical' to
    import it seperately) and all private and protected methods of that class.
    This also applies to nested closures. Essentially, closures inside
    methods are
    added as public methods to the class that contains the original method.
    4. Closures may live longer as the methods that declared them. It is
    perfectly
    possible to have something like this:
    function getAdder($x) {
    return function ($y) {
    lexical $x;
    return $x + $y;
    };
    }
    Zend internal perspective
    -------------------------
    The patch basically changes the following in the Zend engine:
    When the compiler reaches a lambda function, it creates a unique name
    for that
    function ("\0__compiled_lambda_FILENAME_N" where FILENAME is the name of the
    file currently processed and N is a per-file counter). The use of the
    filename
    in the function name ensures compability with opcode caches. The lambda
    function is then immediately added to the function table (either the global
    function table or that of the current class if declared inside a class
    method).
    Instead of a normal ZEND_DECLARE_FUNCTION opcode the new
    ZEND_DECLARE_LAMBDA_FUNC is used as an opcode at this point. The op_array
    of the new function is initialized with is_lambda = 1 and is_closure = 0.
    When parsing a 'lexical' declaration inside an anonymous function the parser
    saves the name of the variable that is to be imported in an array stored
    as a member of the op_array structure (lexical_names).
    The opcode handler for ZEND_DECLARE_LAMBDA_FUNC does the following: First of
    all it creates a new op_array and copies the entire memory structure of the
    lambda function into it (the opcodes themselves are not copied since they
    are only referenced in the op_array structure). Then it sets is_closure = 1
    on the new op_array, and for each lexical variable name that the compiler
    added to the original op_array it creates a reference to that variable from
    the current scope into a HashTable member in the new op_array. It also saves
    the current object pointer ($this) as a member of the op_array in order to
    allow for the closure to access $this. Finally it registers the new op_array
    as a resource and returns that resource.
    The opcode handler of the 'lexical' construct simply fetches the variable
    from that HashTable and imports it into local scope of the inner function
    (just like with 'global' only with a different hash table).
    Some hooks were added that allow the 'lambda function' resource to be
    called.
    Also, there are several checks in place that make sure the lambda function
    is not called directly, i.e. if someone explicitely tries to use the
    internal
    function name instead of using the resource return value of the declaration.
    The patch
    ---------
    The patch is available here:
    <http://www.christian-seiler.de/temp/closures-php-5.3-2008-06-16-1.diff>
    Please note that I did NOT include the contents of zend_language_scanner.c
    in the patch since that can easily be regenerated and just takes up enormous
    amounts of space.
    The patch itself applies against the 5.3 branch of PHP.
    If I understand the discussion regarding PHP6 on this list correctly, some
    people are currently undergoing the task of removing the unicode_semantics
    switch and if (UG(unicode)). As soon as this task is finished I will also
    provide a patch for CVS HEAD (it doesn't make much sense adopting the patch
    now and then having to change it again completely afterwards).
    BC BREAKS
    ---------
    * Introduction of a new keyword 'lexical'. Since it is very improbable
    that
    someone should use it as a function, method, class or property name, I
    think this is an acceptable break.
    Other that that, I can find no BC breaks of my patch.
    CAVEATS / POSSIBLE WTFS
    -----------------------
    * On writing $func = function () { }; there is a semicolon necessary.
    If left
    out it will produce a compile error. Since any attempt to remove that
    necessity would unecessarily bloat the grammar, I suggest we simply keep
    it the way it is. Also, Lukas Kahwe Smith pointed out that a single
    trailing semicolon after a closing brace already exists: do { }
    while ();
    * The fact that 'lexical' creates references may cause certain WTFs:
    for ($i = 0; $i < 10; $i++) {
    $arr[$i] = function () { lexical $i; return $i; };
    }
    This will not work as expected since $i is a reference and thus all
    created closures would reference the same variable. In order to get this
    right one has to do:
    for ($i = 0; $i < 10; $i++) {
    $loopIndex = $i;
    $arr[$i] = function () { lexical $loopIndex; return $loopIndex; };
    unset ($loopIndex);
    }
    This can be a WTF for people that don't expect lexical to create an
    actual reference, especially since other languages such as JavaScript
    don't do it. On the other hand, global and static both DO create
    references so that behaviour is consistent with current PHP.
    But complex constructions such as this will probably not be used by
    beginners so maintaining a good documentation should solve this.
    * The fact that 'lexical' is needed at all may cause WTFs. Other languages
    such as JavaScript implicitely have the entire scope visible to child
    functions. But since PHP does the same thing with global variables, I
    find a keyword like 'lexical' much more consistent than importing the
    entire scope (and always importing the entire scope costs unnecessary
    performance).
    FINAL THOUGHTS
    --------------
    My now proposed patch addresses the two main problems of my previous patch:
    Support for closures in objects (with access to $this) and opcode caches. My
    patch applies against PHP_5_3 and does not break any tests. It adds a
    valuable
    new language feature which I'd like to see in PHP.
    Regards,
    Christian



    Best regards,
    Marcus
  • Christian Seiler at Jun 17, 2008 at 10:20 am
    Hi Marcus,
    very nice work. Thanks!
    The only thing I don't like is the function naming
    (“\0__compiled_lambda_FILENAME_N”). Can we drop the \0?
    I used \0 because it is already used in two other places:

    1) create_function (run-time lambda functions) uses \0__lambda_N
    2) build_runtime_defined_function_key uses \0 to start function names.

    I can drop it if you like; personally, I don't care for either solution
    - it's an internal name that *may* leak to userspace in some
    circumstances but is never really useful for userspace anyway..

    A minor side-note here: I oriented myself at
    build_runtime_defined_function_key at the time of writing but I have
    noticed a slight discrepancy between function names generated by
    build_runtime_defined_function_key and the normal function names: When
    stored in the corresponding function_table hash, for runtime defined
    function keys it is opline->op1.u.constant.value.str.len, whereas for
    normal function names it is »Z_STRLEN_P(...) + 1« and thus including the
    *trailing* (not preceding!) \0 in the hash key for normal function names
    but not including it for runtime defined function keys. Any idea why
    that is the case? [For the record: I'm refering to the code that is
    already used in PHP, not to my patch!]
    Or did you use the \0 prefix to prevent direct invocations?
    No, direct invocations are prevented by the is_lambda == 1 &&
    is_closure == 0 check.
    I think the best option would be to force lambda functions being public
    always.
    They are. If you look at my modified version of
    zend_do_begin_function_declaration, you will see that:

    if (is_lambda && CG(active_class_entry)) {
    is_method = 1;
    fn_flags = ZEND_ACC_PUBLIC;
    if (CG(active_op_array)->fn_flags & ZEND_ACC_STATIC) {
    fn_flags |= ZEND_ACC_STATIC;
    }
    } else if (is_lambda) {
    fn_flags = 0;
    }

    The only attribute that is "inherited" from the parent function is
    whether that function is static or not.
    The last question is about changing visibility and overriding
    functions, do we want that, or should we mark lamdas as final?
    Internal representations of lambda fuctions should never be overridden,
    so yes, ZEND_ACC_FINAL would probably be a good idea. Overriding them
    won't work anyway since the new opcode that "instantiates" a closure
    will always use the class in which the closure was defined to look it up.

    I'll add that.
    Actually how does it integrate with reflection?
    Good question, I will investigate that and come back to you.
    Comments about the implementation:

    - ZendEngine2/zend_compile.c:

    Why provide forward declaration for free_filename(), simply putting the
    new function above the first user avoids it and does the same with
    increased maintainability.

    s/static void add_lexical_var (/static void add_lexical_var(/
    Ok, I will fix that.

    Regards,
    Christian
  • Christian Seiler at Jun 17, 2008 at 12:24 pm
    Hi Marcus,

    I now have revised my patch to include your suggestions:

    http://www.christian-seiler.de/temp/closures-php-5.3-2008-06-17-2.diff

    The changes to the previous version:

    - \0 at the start of the compiled lambda function name is dropped.
    - lambdas which are class members are now marked as final
    - the generated name of the lambda is now also stored within the
    op_array (before op_array->function_name was simply "lambda")
    - your suggestions for code cleanups in zend_compile.c
    Actually how does it integrate with reflection?
    Consider the following class:

    class Example {
    private $x = 0;

    public function getIncer () {
    return function () {
    $this->x++;
    };
    }

    public function show () {
    $this->reallyShow ();
    }

    protected function reallyShow () {
    echo "{$this->x}\n";
    }
    }

    Running

    Reflection::export(new ReflectionClass('Example'));

    will yield (among other things):

    - Methods [4] {
    Method [ <user> public method getIncer ] {
    @@ /home/christian/dev/php5.3/c-tests/reflection.php 6 - 10
    }

    Method [ <user> final public method
    __compiled_lambda_/home/christian/dev/php5.3/c-tests/reflection.php_0 ] {
    @@ /home/christian/dev/php5.3/c-tests/reflection.php 7 - 9
    }

    Method [ <user> public method show ] {
    @@ /home/christian/dev/php5.3/c-tests/reflection.php 12 - 14
    }

    Method [ <user> protected method reallyShow ] {
    @@ /home/christian/dev/php5.3/c-tests/reflection.php 16 - 18
    }
    }

    So lambda functions appear simply as additional methods in the class,
    with their generated name.

    Of course, the ReflectionMethod / ReflectionFunction classes could be
    extended so that it provides and additional method isLambda() in order
    to determine whether a function actually is a lambda function. But I'd
    rather do that in a separate step and a separate patch.

    Regards,
    Christian
  • Chris Stockton at Jun 17, 2008 at 4:25 pm
    Hello,

    Great patch and a much needed feature. One thing I do not agree with is the
    point in the lexical key word, seems it should be natural to inherit the
    outer scope. I guess the choice of adding lexical and going slightly against
    the grain of typical closure implementations like scheme or ecmascript is
    that is not really consistent with php so i can understand disagreement and
    your note you made on performance. Seems like the right choice to force
    manual inheritance of outer scope. But great work on this, hope it gets
    added and none of the core developers say it is not the php way or is only
    useful in brainless languages.

    -Chris
  • Marcus Boerger at Jun 17, 2008 at 7:10 pm
    Hello Christian, Johannes,

    Tuesday, June 17, 2008, 2:24:01 PM, you wrote:
    Hi Marcus,
    I now have revised my patch to include your suggestions:
    http://www.christian-seiler.de/temp/closures-php-5.3-2008-06-17-2.diff
    The changes to the previous version:
    - \0 at the start of the compiled lambda function name is dropped.
    - lambdas which are class members are now marked as final
    - the generated name of the lambda is now also stored within the
    op_array (before op_array->function_name was simply "lambda")
    - your suggestions for code cleanups in zend_compile.c
    Actually how does it integrate with reflection?
    Consider the following class:
    class Example {
    private $x = 0;
    public function getIncer () {
    return function () {
    $this->x++;
    };
    }
    public function show () {
    $this->reallyShow ();
    }
    protected function reallyShow () {
    echo "{$this->x}\n";
    }
    }
    Running
    Reflection::export(new ReflectionClass('Example'));
    will yield (among other things):
    - Methods [4] {
    Method [ <user> public method getIncer ] {
    @@ /home/christian/dev/php5.3/c-tests/reflection.php 6 - 10
    }
    Method [ <user> final public method
    __compiled_lambda_/home/christian/dev/php5.3/c-tests/reflection.php_0 ] {
    @@ /home/christian/dev/php5.3/c-tests/reflection.php 7 - 9
    }
    Method [ <user> public method show ] {
    @@ /home/christian/dev/php5.3/c-tests/reflection.php 12 - 14
    }
    Method [ <user> protected method reallyShow ] {
    @@ /home/christian/dev/php5.3/c-tests/reflection.php 16 - 18
    }
    }
    So lambda functions appear simply as additional methods in the class,
    with their generated name.
    Of course, the ReflectionMethod / ReflectionFunction classes could be
    extended so that it provides and additional method isLambda() in order
    to determine whether a function actually is a lambda function. But I'd
    rather do that in a separate step and a separate patch.
    Yep, Reflection details can and should be addressed separately.

    I think the next step is confirming with the rest of the core team that we
    all ewant this. IMO it has often enough been requested and your patch
    solves every tiny piece that was missing so far. And if people agree then
    unfortunately you need to provide a patch for HEAD first. Actually you
    could do so just now.

    Johannes, what's your take on this one for 5.3?


    Best regards,
    Marcus
  • Stanislav Malyshev at Jun 17, 2008 at 7:20 pm
    Hi!
    Johannes, what's your take on this one for 5.3?
    I'm not Johannes and I didn't review the proposal in detail yet, but I
    think we have enough for 5.3 right now. I'd think we better concentrate
    on tying the loose ends and rolling beta out and then moving towards the
    release than adding more and more features and never releasing it. 5.3
    is not the final release until the end of times, there will be 5.4 etc.
    and 6, so there will be ample opportunity to add stuff. And 5.3 has
    enough stuff to be released, there's no rush to add more new things,
    especially radically new ones. My opinion is that we better take some
    time with it and not tie it to 5.3.
    --
    Stanislav Malyshev, Zend Software Architect
    stas@zend.com http://www.zend.com/
    (408)253-8829 MSN: stas@zend.com
  • Steph Fox at Jun 17, 2008 at 7:23 pm

    I'm not Johannes and I didn't review the proposal in detail yet, but I
    think we have enough for 5.3 right now. I'd think we better concentrate
    on tying the loose ends and rolling beta out and then moving towards the
    release than adding more and more features and never releasing it. 5.3
    is not the final release until the end of times, there will be 5.4 etc.
    and 6, so there will be ample opportunity to add stuff. And 5.3 has
    enough stuff to be released, there's no rush to add more new things,
    especially radically new ones. My opinion is that we better take some
    time with it and not tie it to 5.3.
    Amen to that.

    - Steph
    --
    Stanislav Malyshev, Zend Software Architect
    stas@zend.com http://www.zend.com/
    (408)253-8829 MSN: stas@zend.com

    --
    PHP Internals - PHP Runtime Development Mailing List
    To unsubscribe, visit: http://www.php.net/unsub.php
  • Alexey Zakhlestin at Jun 17, 2008 at 7:35 pm

    On 6/17/08, Stanislav Malyshev wrote:

    I'm not Johannes and I didn't review the proposal in detail yet, but I
    think we have enough for 5.3 right now. I'd think we better concentrate on
    tying the loose ends and rolling beta out and then moving towards the
    release than adding more and more features and never releasing it. 5.3 is
    not the final release until the end of times, there will be 5.4 etc. and 6,
    so there will be ample opportunity to add stuff. And 5.3 has enough stuff to
    be released, there's no rush to add more new things, especially radically
    new ones. My opinion is that we better take some time with it and not tie it
    to 5.3.
    although I really want to have this functionality right now, I agree with Stas.
    5.3 has a lot of new things already. this should go to HEAD, and,
    hopefully, it will attract more people to actually try php-6
  • Marcus Boerger at Jun 17, 2008 at 7:51 pm
    Hello Stanislav,

    nicely put but not in agreement with the PHP world. First we cannot add
    a new feature like this in a mini release as it comes with an API change.
    And second PHP is not anywhere close so we'd have to do it in a PHP 5.4
    and personally I would like to avoid it.

    marcus

    Tuesday, June 17, 2008, 9:19:56 PM, you wrote:
    Hi!
    Johannes, what's your take on this one for 5.3?
    I'm not Johannes and I didn't review the proposal in detail yet, but I
    think we have enough for 5.3 right now. I'd think we better concentrate
    on tying the loose ends and rolling beta out and then moving towards the
    release than adding more and more features and never releasing it. 5.3
    is not the final release until the end of times, there will be 5.4 etc.
    and 6, so there will be ample opportunity to add stuff. And 5.3 has
    enough stuff to be released, there's no rush to add more new things,
    especially radically new ones. My opinion is that we better take some
    time with it and not tie it to 5.3.
    --
    Stanislav Malyshev, Zend Software Architect
    stas@zend.com http://www.zend.com/
    (408)253-8829 MSN: stas@zend.com



    Best regards,
    Marcus
  • Stanislav Malyshev at Jun 17, 2008 at 8:13 pm
    Hi!
    nicely put but not in agreement with the PHP world. First we cannot add
    a new feature like this in a mini release as it comes with an API change.
    And second PHP is not anywhere close so we'd have to do it in a PHP 5.4
    and personally I would like to avoid it.
    You meant "PHP 6 not anywhere close"? well, if we keep adding completely
    new stuff to 5.3 then 5.3 would not be anywhere close so what we earn by
    that? I understand that we all want new and cool stuff in PHP, but
    endlessly delaying 5.3 is not the answer.
    And this feature is a big thing, it's not adding yet another module or
    function - it should be thoroughly reviewed and seen how it affect all
    other things. It's probably very cool addition, but yet more reason to
    consider all the side effects and surprises on the way.
    Pushing it in now would mean we'd either have to delay 5.3 yet another
    half-year or we'd release buggy 5.3 and then we'd be bound by API
    compatibility and couldn't fix things that we might want to fix. I think
    as much as we want to add yet another cool feature - we have to release
    versions for the users to actually be able to enjoy features we already
    added, from time to time. And time for 5.3 is very much due, and we
    still have a bunch of work to do on it.
    --
    Stanislav Malyshev, Zend Software Architect
    stas@zend.com http://www.zend.com/
    (408)253-8829 MSN: stas@zend.com
  • Andrei Zmievski at Jun 19, 2008 at 10:09 pm
    Yes, I would rather put it in 5.4 (or whatever the next version) is and
    make sure that along with lambdas/closures we have a way of referring to
    functions/methods as first-class objects.

    -Andrei

    Marcus Boerger wrote:
    Hello Stanislav,

    nicely put but not in agreement with the PHP world. First we cannot add
    a new feature like this in a mini release as it comes with an API change.
    And second PHP is not anywhere close so we'd have to do it in a PHP 5.4
    and personally I would like to avoid it.

    marcus

    Tuesday, June 17, 2008, 9:19:56 PM, you wrote:
    Hi!
    Johannes, what's your take on this one for 5.3?
    I'm not Johannes and I didn't review the proposal in detail yet, but I
    think we have enough for 5.3 right now. I'd think we better concentrate
    on tying the loose ends and rolling beta out and then moving towards the
    release than adding more and more features and never releasing it. 5.3
    is not the final release until the end of times, there will be 5.4 etc.
    and 6, so there will be ample opportunity to add stuff. And 5.3 has
    enough stuff to be released, there's no rush to add more new things,
    especially radically new ones. My opinion is that we better take some
    time with it and not tie it to 5.3.
    --
    Stanislav Malyshev, Zend Software Architect
    stas@zend.com http://www.zend.com/
    (408)253-8829 MSN: stas@zend.com



    Best regards,
    Marcus
  • Stanislav Malyshev at Jun 20, 2008 at 5:23 am
    Hi!
    Yes, I would rather put it in 5.4 (or whatever the next version) is and
    make sure that along with lambdas/closures we have a way of referring to
    functions/methods as first-class objects.
    Maybe we could make some object handler so that $object($foo) would work
    and treat object as "functional object" called on $foo and then have
    both reflection and closure object implement it?
    --
    Stanislav Malyshev, Zend Software Architect
    stas@zend.com http://www.zend.com/
    (408)253-8829 MSN: stas@zend.com
  • Lars Strojny at Jun 17, 2008 at 7:52 pm
    Hi Markus, hi Stas,

    Am Dienstag, den 17.06.2008, 12:19 -0700 schrieb Stanislav Malyshev:
    [...]
    I'm not Johannes and I didn't review the proposal in detail yet, but I
    think we have enough for 5.3 right now. I'd think we better concentrate
    on tying the loose ends and rolling beta out and then moving towards the
    release than adding more and more features and never releasing it. 5.3
    is not the final release until the end of times, there will be 5.4 etc.
    and 6, so there will be ample opportunity to add stuff. And 5.3 has
    enough stuff to be released, there's no rush to add more new things,
    especially radically new ones. My opinion is that we better take some
    time with it and not tie it to 5.3.
    I would like to see 5.3 released first before we add really cool
    features like this. I'm really +1 for closures but I please for 5.4 and
    6.

    cu, Lars
  • Christian Seiler at Jun 17, 2008 at 9:04 pm
    Hi!
    I'm not Johannes and I didn't review the proposal in detail yet, but I
    think we have enough for 5.3 right now. I'd think we better concentrate
    on tying the loose ends and rolling beta out and then moving towards the
    release than adding more and more features and never releasing it. 5.3
    is not the final release until the end of times, there will be 5.4 etc.
    and 6, so there will be ample opportunity to add stuff. And 5.3 has
    enough stuff to be released, there's no rush to add more new things,
    especially radically new ones. My opinion is that we better take some
    time with it and not tie it to 5.3.
    I would like to see 5.3 released first before we add really cool
    features like this. I'm really +1 for closures but I please for 5.4 and
    6.
    If I may add my own personal (and biased ;-)) opinion (which may not
    count much but I'd like to present the arguments): I'd like to see it in
    PHP 5.3. Mainly because of two reasons:

    First: My patch is quite non-intrusive, it only adds things in a few
    places (new opcode, a few checks). If you only look at the non-generated
    files (i.e. excluding files generated by re2c or zend_vm_gen.php), the
    patch is actually not even that long:
    http://www.christian-seiler.de/temp/closures-php-5.3-2008-06-17-3-redux.diff
    Except for the introduction of a 'lexical' keyword I carefully designed
    the patch not to have *any* impact *at all* on PHPs other behaviour. I'd
    be genuinely surprised if any code breaks with my patch. I also don't
    see how this would delay 5.3 - of course things have to be tested but at
    least as far as I can tell the major showstoppers currently are class
    inheritance rules and namespaces which still cause quite a few headaches
    of their own.

    Second: If closures are not supported in PHP 5.3, even with the release
    of PHP 6 backwards compability will be a hindrance in using them. Since
    PHP 6 will have Unicode support and thus quite a few semantic changes,
    this will of course not matter much for the actual PHP applications
    since these will have to change anyway. But think of class libraries:
    There are many things that can be implemented in class libraries where
    unicode support doesn't matter at all - such as for example advanced
    date and time calculations (beyond timelib), financial calculations etc.
    Such libraries will probably want to maintain compability with PHP 5.3
    as long as possible. But these libraries may profit from closures.

    If you still decide to include closures only from post PHP 5.3, I
    suggest to at least declare 'lexical' a reserved keyword in PHP 5.3.

    Just my 2 cents...

    As to the patch for HEAD: I thought it best to wait for
    unicode.semantics to go away along with the if (UG(unicode)) checks
    before implementing it (everything else would be a waste of time - since
    if I'm not mistaken someone is actually currently removing those). If I
    really am mistaken in my interpretation of the discussions here on this
    topic and they are not going away (at least not in the short term), I
    can of course provide one now (meaning the next few days).

    Regards,
    Christian
  • Stanislav Malyshev at Jun 18, 2008 at 3:05 pm
    Hi!
    First: My patch is quite non-intrusive, it only adds things in a few
    places (new opcode, a few checks). If you only look at the non-generated
    I think it falls into "famous last words" category. While I did not have
    time yet to look into the patch in the detail, I have hard time to
    believe patch creating wholly new concept in PHP, new opcodes, etc.
    would have zero impact. You have to consider at least the following:
    tests, documentation, how lexical interacts with other references
    (global? static? just variable passed by-ref?), how closure interacts
    with various reflection capabilities, how it works with bytecode caches,
    what happens with lifetimes of the variables saved in closures -
    especially implicit ones like $this, etc., etc. I know these questions
    can be answered, and maybe even easily answered, but I think they have
    to be answered without pressure of 5.3 release and commitment to the
    fixed API hanging over us.

    I understand your urge to have it inside ASAP - if you didn't want it,
    you'd not gone through this effort to create it :) However, I still
    think we better not make 5.3 dependent on yet another new feature.
    As for adoption - I think it would take a long time for off-the-shelf
    libraries and mainstream users to use this anyway, and for the hackers
    among us it will be available in development version pretty soon after
    5.3. I think if we would decide that every new feature anybody can think
    about should enter into 5.3 because it will be harder to adopt it
    otherwise, we'd never release 5.3 at all - look at the RFCs, we have a
    bunch of ideas already, and I'm sure there will be more. We need to
    release some time - what happened to that "release often" thing?

    Please do not consider this to be opinion about (or against) the patch -
    I think the idea is good and from preliminary glance the implementation
    is very nice too, but IMHO we just can not have everything in one release.
    --
    Stanislav Malyshev, Zend Software Architect
    stas@zend.com http://www.zend.com/
    (408)253-8829 MSN: stas@zend.com
  • Marcus Boerger at Jun 20, 2008 at 8:42 am
    Hello Stanislav,

    Wednesday, June 18, 2008, 5:00:09 PM, you wrote:
    Hi!
    First: My patch is quite non-intrusive, it only adds things in a few
    places (new opcode, a few checks). If you only look at the non-generated
    I think it falls into "famous last words" category. While I did not have
    time yet to look into the patch in the detail, I have hard time to
    You have time to answer every little mail. It would only be fair if you
    showed respect by at least looking into patches that people provide
    because they tried to address long outstanding issues and even address
    every little comment we all including you made in the past. Given the wiki
    he even clearly showed that he understands what he is doing and that he
    did care about a hell of detail.
    believe patch creating wholly new concept in PHP, new opcodes, etc.
    would have zero impact. You have to consider at least the following:
    tests, documentation, how lexical interacts with other references
    (global? static? just variable passed by-ref?), how closure interacts
    with various reflection capabilities, how it works with bytecode caches,
    what happens with lifetimes of the variables saved in closures -
    especially implicit ones like $this, etc., etc. I know these questions
    can be answered, and maybe even easily answered, but I think they have
    to be answered without pressure of 5.3 release and commitment to the
    fixed API hanging over us.
    I understand your urge to have it inside ASAP - if you didn't want it,
    you'd not gone through this effort to create it :) However, I still
    think we better not make 5.3 dependent on yet another new feature.
    As for adoption - I think it would take a long time for off-the-shelf
    libraries and mainstream users to use this anyway, and for the hackers
    among us it will be available in development version pretty soon after
    5.3. I think if we would decide that every new feature anybody can think
    about should enter into 5.3 because it will be harder to adopt it
    otherwise, we'd never release 5.3 at all - look at the RFCs, we have a
    bunch of ideas already, and I'm sure there will be more. We need to
    release some time - what happened to that "release often" thing?
    Please do not consider this to be opinion about (or against) the patch -
    I think the idea is good and from preliminary glance the implementation
    is very nice too, but IMHO we just can not have everything in one release.
    --
    Stanislav Malyshev, Zend Software Architect
    stas@zend.com http://www.zend.com/
    (408)253-8829 MSN: stas@zend.com



    Best regards,
    Marcus
  • Christopher Jones at Jun 17, 2008 at 11:23 pm

    Christian Seiler wrote:
    As a followup to the discussion in January, I'd like post a revised
    patch to this list that implements closures and anonymous functions
    in PHP.
    Did I miss seeing its phpt tests?
    A really thorough test suite might the case for inclusion in PHP 5.3.

    Chris

    --
    Christopher Jones, Oracle
    Email: christopher.jones@oracle.com Tel: +1 650 506 8630
    Blog: http://blogs.oracle.com/opal/ Free PHP Book: http://tinyurl.com/f8jad
  • Andi Gutmans at Jun 18, 2008 at 6:02 am
    Hi Christian,

    This is a very nice piece of work. Definitely addresses a lot of the issues we have raised in the past.
    I would like to see such a solution make its way into PHP (see below re: timing).

    There are some things I'd like to consider:
    1) I am not sure that the current semantics of the "lexical" keyword is great in all cases. Is the reason why you don't allow by-value binding so that we don't have to manage more than one lambda instance per declaration?
    2) [minor curiosity - do we want to consider reusing "parent" instead of "lexical"? I guess that could be confusing but it's not the first time we reuse a keyword when it's clear that the usage is in two different places (this is minor and I don't mind much either way although lexical doesn't mean too much to me).]
    3) I am concerned about binding to classes. First of all we need to look into more detail what the implications are for bytecode caches when changing class entries at run-time. We may want to also consider an option where the lambda binds to the object and only has public access although I realize that may be considered by some as too limiting. We'll review these two things in the coming days.

    Re: timing, I think the biggest issue we have right now with PHP 5.3 is that we are not making a clear cut on features. There's always pressure on release managers to include more (I went through the same with 5.0) but at some point you just have to stop at some place or things will never go out as there are always good ideas flowing in. Unfortunately with 5.3 that cut isn't happening and it seems to drag out longer than needed. I prefer having this discussion in the context of a hard date for a beta release after which we'll be especially strict with accepting new features. Each new feature will drag out the beta/RC cycle as they need enough time for testing/feedback/tweaks.

    Andi
    -----Original Message-----
    From: Christian Seiler
    Sent: Monday, June 16, 2008 10:39 AM
    To: php-dev List
    Subject: [PHP-DEV] [PATCH] [RFC] Closures and lambda functions in PHP

    Hi,

    As a followup to the discussion in January, I'd like post a revised patch to
    this list that implements closures and anonymous functions in PHP.

    INTRODUCTION
    ------------

    Closures and lambda functions can make programming much easier in
    several ways:

    1. Lambda functions allow the quick definition of throw-away functions
    that are not used elsewhere. Imaging for example a piece of code that
    needs to call preg_replace_callback(). Currently, there are three
    possibilities to acchieve this:

    a. Define the callback function elsewhere. This distributes code that
    belongs together throughout the file and decreases readability.

    b. Define the callback function in-place (but with a name). In
    that case
    one has to use function_exists() to make sure the function is only
    defined once. Example code:

    <?php
    function replace_spaces ($text) {
    if (!function_exists ('replace_spaces_helper')) {
    function replace_spaces_helper ($matches) {
    return str_replace ($matches[1], ' ', '&nbsp;').' ';
    }
    }
    return preg_replace_callback ('/( +) /',
    'replace_spaces_helper',
    $text);
    }
    ?>

    Here, the additional if() around the function definition makes the
    source code difficult to read.

    c. Use the present create_function() in order to create a function at
    runtime. This approach has several disadvantages: First of all,
    syntax
    highlighting does not work because a string is passed to the
    function.
    It also compiles the function at run time and not at compile
    time so
    opcode caches can't cache the function.

    2. Closures provide a very useful tool in order to make lambda
    functions even
    more useful. Just imagine you want to replace 'hello' through
    'goodbye' in
    all elements of an array. PHP provides the array_map() function which
    accepts a callback. If you don't wan't to hard-code 'hello' and
    'goodbye'
    into your sourcecode, you have only four choices:

    a. Use create_function(). But then you may only pass literal values
    (strings, integers, floats) into the function, objects at best as
    clones (if var_export() allows for it) and resources not at
    all. And
    you have to worry about escaping everything correctly.
    Especially when
    handling user input this can lead to all sorts of security issues.

    b. Write a function that uses global variables. This is ugly,
    non-reentrant and bad style.

    c. Create an entire class, instantiate it and pass the member function
    as a callback. This is perhaps the cleanest solution for this
    problem
    with current PHP but just think about it: Creating an entire
    class for
    this extremely simple purpose and nothing else seems overkill.

    d. Don't use array_map() but simply do it manually (foreach). In this
    simple case it may not be that much of an issue (because one simply
    wants to iterate over an array) but there are cases where doing
    something manually that a function with a callback as parameter
    does
    for you is quite tedious.

    [Yes, I know that str_replace also accepts arrays as a third
    parameter so
    this example may be a bit useless. But imagine you want to do a more
    complex operation than simple search and replace.]

    PROPOSED PATCH
    --------------

    I now propose a patch that implements compile-time lambda functions and
    closures for PHP while keeping the patch as simple as possible. The patch is
    based on a previous patch on mine which was based on ideas discussed here
    end of December / start of January.

    Userland perspective
    --------------------

    1. The patch adds the following syntax as a valid expression:

    function & (parameters) { body }

    (The & is optional and indicates - just as with normal functions - that the
    anonymous function returns a reference instead of a value)

    Example usage:

    $lambda = function () { echo "Hello World!\n"; };

    The variable $lambda then contains a callable resource that may be called
    through different means:

    $lambda ();
    call_user_func ($lambda);
    call_user_func_array ($lambda, array ());

    This allows for simple lambda functions, for example:

    function replace_spaces ($text) {
    $replacement = function ($matches) {
    return str_replace ($matches[1], ' ', '&nbsp;').' ';
    };
    return preg_replace_callback ('/( +) /', $replacement, $text);
    }

    2. The patch implements closures by defining an additional keyword 'lexical'
    that allows an lambda function (and *only* an lambda function) to import
    a variable from the "parent scope" to the lambda function scope. Example:

    function replace_in_array ($search, $replacement, $array) {
    $map = function ($text) {
    lexical $search, $replacement;
    if (strpos ($text, $search) > 50) {
    return str_replace ($search, $replacement, $text);
    } else {
    return $text;
    }
    };
    return array_map ($map, array);
    }

    The variables $search and $replacement are variables in the scope of the
    function replace_in_array() and the lexical keyword imports these variables
    into the scope of the closure. The variables are imported as a reference,
    so any change in the closure will result in a change in the variable of the
    function itself.

    3. If a closure is defined inside an object, the closure has full access
    to the current object through $this (without the need to use 'lexical' to
    import it seperately) and all private and protected methods of that class.
    This also applies to nested closures. Essentially, closures inside
    methods are
    added as public methods to the class that contains the original method.

    4. Closures may live longer as the methods that declared them. It is
    perfectly
    possible to have something like this:

    function getAdder($x) {
    return function ($y) {
    lexical $x;
    return $x + $y;
    };
    }

    Zend internal perspective
    -------------------------

    The patch basically changes the following in the Zend engine:

    When the compiler reaches a lambda function, it creates a unique name
    for that
    function ("\0__compiled_lambda_FILENAME_N" where FILENAME is the name of the
    file currently processed and N is a per-file counter). The use of the
    filename
    in the function name ensures compability with opcode caches. The lambda
    function is then immediately added to the function table (either the global
    function table or that of the current class if declared inside a class
    method).
    Instead of a normal ZEND_DECLARE_FUNCTION opcode the new
    ZEND_DECLARE_LAMBDA_FUNC is used as an opcode at this point. The op_array
    of the new function is initialized with is_lambda = 1 and is_closure = 0.

    When parsing a 'lexical' declaration inside an anonymous function the parser
    saves the name of the variable that is to be imported in an array stored
    as a member of the op_array structure (lexical_names).

    The opcode handler for ZEND_DECLARE_LAMBDA_FUNC does the following: First of
    all it creates a new op_array and copies the entire memory structure of the
    lambda function into it (the opcodes themselves are not copied since they
    are only referenced in the op_array structure). Then it sets is_closure = 1
    on the new op_array, and for each lexical variable name that the compiler
    added to the original op_array it creates a reference to that variable from
    the current scope into a HashTable member in the new op_array. It also saves
    the current object pointer ($this) as a member of the op_array in order to
    allow for the closure to access $this. Finally it registers the new op_array
    as a resource and returns that resource.

    The opcode handler of the 'lexical' construct simply fetches the variable
    from that HashTable and imports it into local scope of the inner function
    (just like with 'global' only with a different hash table).

    Some hooks were added that allow the 'lambda function' resource to be
    called.
    Also, there are several checks in place that make sure the lambda function
    is not called directly, i.e. if someone explicitely tries to use the
    internal
    function name instead of using the resource return value of the declaration.

    The patch
    ---------

    The patch is available here:
    <http://www.christian-seiler.de/temp/closures-php-5.3-2008-06-16-1.diff>

    Please note that I did NOT include the contents of zend_language_scanner.c
    in the patch since that can easily be regenerated and just takes up enormous
    amounts of space.

    The patch itself applies against the 5.3 branch of PHP.

    If I understand the discussion regarding PHP6 on this list correctly, some
    people are currently undergoing the task of removing the unicode_semantics
    switch and if (UG(unicode)). As soon as this task is finished I will also
    provide a patch for CVS HEAD (it doesn't make much sense adopting the patch
    now and then having to change it again completely afterwards).

    BC BREAKS
    ---------

    * Introduction of a new keyword 'lexical'. Since it is very improbable
    that
    someone should use it as a function, method, class or property name, I
    think this is an acceptable break.

    Other that that, I can find no BC breaks of my patch.

    CAVEATS / POSSIBLE WTFS
    -----------------------

    * On writing $func = function () { }; there is a semicolon necessary.
    If left
    out it will produce a compile error. Since any attempt to remove that
    necessity would unecessarily bloat the grammar, I suggest we simply keep
    it the way it is. Also, Lukas Kahwe Smith pointed out that a single
    trailing semicolon after a closing brace already exists: do { }
    while ();

    * The fact that 'lexical' creates references may cause certain WTFs:

    for ($i = 0; $i < 10; $i++) {
    $arr[$i] = function () { lexical $i; return $i; };
    }

    This will not work as expected since $i is a reference and thus all
    created closures would reference the same variable. In order to get this
    right one has to do:

    for ($i = 0; $i < 10; $i++) {
    $loopIndex = $i;
    $arr[$i] = function () { lexical $loopIndex; return $loopIndex; };
    unset ($loopIndex);
    }

    This can be a WTF for people that don't expect lexical to create an
    actual reference, especially since other languages such as JavaScript
    don't do it. On the other hand, global and static both DO create
    references so that behaviour is consistent with current PHP.

    But complex constructions such as this will probably not be used by
    beginners so maintaining a good documentation should solve this.

    * The fact that 'lexical' is needed at all may cause WTFs. Other languages
    such as JavaScript implicitely have the entire scope visible to child
    functions. But since PHP does the same thing with global variables, I
    find a keyword like 'lexical' much more consistent than importing the
    entire scope (and always importing the entire scope costs unnecessary
    performance).

    FINAL THOUGHTS
    --------------

    My now proposed patch addresses the two main problems of my previous patch:
    Support for closures in objects (with access to $this) and opcode caches. My
    patch applies against PHP_5_3 and does not break any tests. It adds a
    valuable
    new language feature which I'd like to see in PHP.

    Regards,
    Christian

    --
    PHP Internals - PHP Runtime Development Mailing List
    To unsubscribe, visit: http://www.php.net/unsub.php
  • Alexey Zakhlestin at Jun 18, 2008 at 6:36 am

    On 6/18/08, Andi Gutmans wrote:
    1) I am not sure that the current semantics of the "lexical" keyword is great in all cases. Is the reason why you don't allow by-value binding so that we don't have to manage more than one lambda instance per declaration?
    by-reference binding is much closer to other languages symantics. I
    guess, that was the main reason Christian chose it.
    "by-value" may still exist, if people find, that they need it, but
    only in addition, please.

    lambda has to reflect changing state of context, to be truly useful
  • Gwynne Raskind at Jun 18, 2008 at 8:02 am

    On Jun 18, 2008, at 2:36 AM, Alexey Zakhlestin wrote:
    1) I am not sure that the current semantics of the "lexical"
    keyword is great in all cases. Is the reason why you don't allow by-
    value binding so that we don't have to manage more than one lambda
    instance per declaration?
    by-reference binding is much closer to other languages symantics. I
    guess, that was the main reason Christian chose it.
    "by-value" may still exist, if people find, that they need it, but
    only in addition, please.

    lambda has to reflect changing state of context, to be truly useful
    In Lua, the language in which I've seen the most of closures and
    lambda, lexical scoping is handled this way:

    someVariable1 = "asdf";
    someVariable2 = "jkl;";
    SomeFunction = function()
    local someVariable2 = "1234";

    print someVariable1.." "..someVariable2.."\n";
    end
    print gettype(SomeFunction).."\n";
    SomeFunction();
    someVariable1 = "qwer";
    someVariable2 "0987";
    SomeFunction();

    The resulting output of this code fragment would be:
    function
    asdf 1234
    qwer 1234

    The Lua interpreter handles this by resolving variable references as
    they're made; "someVariable1" is looked up in the closure's scope and
    not found, so the interpreter steps out one scope and looks for it
    there, repeat as necessary. Once found outside the closure's scope,
    something similar to the proposed "lexical" keyword happens. Closures
    and lexical variables can be nested this way, to the point where a
    single variable in a sixth-level closure could still have been
    originally found in the global scope.

    I'm not sure this would work for PHP, I'm curious what others think.

    Of course, that fragment does a very poor job of showing off the
    extreme flexibility of Lua with regards to functions and scoping, but
    hopefully it illustrates the concept.

    -- Gwynne, Daughter of the Code
    "This whole world is an asylum for the incurable."
  • Richard Quadling at Jun 18, 2008 at 10:09 am

    2008/6/18 Gwynne Raskind <gwynne@wanderingknights.org>:
    On Jun 18, 2008, at 2:36 AM, Alexey Zakhlestin wrote:

    1) I am not sure that the current semantics of the "lexical" keyword is
    great in all cases. Is the reason why you don't allow by-value binding so
    that we don't have to manage more than one lambda instance per declaration?
    by-reference binding is much closer to other languages symantics. I
    guess, that was the main reason Christian chose it.
    "by-value" may still exist, if people find, that they need it, but
    only in addition, please.

    lambda has to reflect changing state of context, to be truly useful
    In Lua, the language in which I've seen the most of closures and lambda,
    lexical scoping is handled this way:

    someVariable1 = "asdf";
    someVariable2 = "jkl;";
    SomeFunction = function()
    local someVariable2 = "1234";

    print someVariable1.." "..someVariable2.."\n";
    end
    print gettype(SomeFunction).."\n";
    SomeFunction();
    someVariable1 = "qwer";
    someVariable2 "0987";
    SomeFunction();

    The resulting output of this code fragment would be:
    function
    asdf 1234
    qwer 1234

    The Lua interpreter handles this by resolving variable references as
    they're made; "someVariable1" is looked up in the closure's scope and not
    found, so the interpreter steps out one scope and looks for it there, repeat
    as necessary. Once found outside the closure's scope, something similar to
    the proposed "lexical" keyword happens. Closures and lexical variables can
    be nested this way, to the point where a single variable in a sixth-level
    closure could still have been originally found in the global scope.

    I'm not sure this would work for PHP, I'm curious what others think.

    Of course, that fragment does a very poor job of showing off the extreme
    flexibility of Lua with regards to functions and scoping, but hopefully it
    illustrates the concept.

    -- Gwynne, Daughter of the Code
    "This whole world is an asylum for the incurable."



    --
    PHP Internals - PHP Runtime Development Mailing List
    To unsubscribe, visit: http://www.php.net/unsub.php
    Is "nested scope" just the same as "namespace" in this regard?



    --
    -----
    Richard Quadling
    Zend Certified Engineer : http://zend.com/zce.php?c=ZEND002498&r=213474731
    "Standing on the shoulders of some very clever giants!"
  • Stanislav Malyshev at Jun 18, 2008 at 3:05 pm
    Hi!
    The Lua interpreter handles this by resolving variable references as
    they're made; "someVariable1" is looked up in the closure's scope and
    not found, so the interpreter steps out one scope and looks for it
    You may get into a problem here - creator's scope may not exist when you
    execute the closure, and using caller's scope would be very unexpected -
    usually closures are intended to capture part of creating environment,
    not calling environment. It would also impose serious penalty if you
    just use undefined variable - you'd have to go through whole stack up to
    the top.
    there, repeat as necessary. Once found outside the closure's scope,
    something similar to the proposed "lexical" keyword happens. Closures
    lexical in the proposal binds to creator's scope, not caller's scope, as
    I understood. Anyway, binding to caller's immediate scope doesn't seem
    that useful since you could just pass it as a parameter when calling.
    --
    Stanislav Malyshev, Zend Software Architect
    stas@zend.com http://www.zend.com/
    (408)253-8829 MSN: stas@zend.com
  • Gwynne Raskind at Jun 18, 2008 at 4:32 pm

    On Jun 18, 2008, at 11:01 AM, Stanislav Malyshev wrote:
    The Lua interpreter handles this by resolving variable references
    as they're made; "someVariable1" is looked up in the closure's
    scope and not found, so the interpreter steps out one scope and
    looks for it
    You may get into a problem here - creator's scope may not exist when
    you execute the closure, and using caller's scope would be very
    unexpected - usually closures are intended to capture part of
    creating environment, not calling environment. It would also impose
    serious penalty if you just use undefined variable - you'd have to
    go through whole stack up to the top.
    This lookup happens at the time the closure is first declared, and the
    value is stored for later use by the closure; the calling scope
    doesn't need to exist anymore. The problem with going to the top of
    the stack is an issue, though; the Lua interpreter's idea of "scope"
    is rather different from PHPs, and it's not nearly the same penalty
    there.
    >


    -- Gwynne, Daughter of the Code
    "This whole world is an asylum for the incurable."
  • Stanislav Malyshev at Jun 18, 2008 at 4:38 pm
    Hi!
    This lookup happens at the time the closure is first declared, and the
    value is stored for later use by the closure; the calling scope doesn't
    This would work for $var, but what about $$var and various other ways of
    indirect variable access?
    --
    Stanislav Malyshev, Zend Software Architect
    stas@zend.com http://www.zend.com/
    (408)253-8829 MSN: stas@zend.com
  • Christian Seiler at Jun 18, 2008 at 4:38 pm
    Hi,

    [I'm going to collect here a bit:]

    Stanislav Malyshev wrote:
    lexical in the proposal binds to creator's scope, not caller's scope, as
    I understood. Anyway, binding to caller's immediate scope doesn't seem
    that useful since you could just pass it as a parameter when calling.
    Correct and I completely agree.

    Chris Stockton wrote:
    I am curious if is_callable will be able to detect these?
    Yes, as is call_user_func able to call them. (But as I was verifying
    that I saw that there was a tiny bug in the code that makes sure the
    internal names are not used directly, I will fix that.)

    Richard Quadling wrote:
    [JS Example]
    I'm not sure I would say that there is a reference used there.

    I'm no expert, but x, f() and n are all working like normal variables with x
    having to look outside of f().

    Unless I'm getting confused with "pass by reference".
    Yes, shure, ok, it's not a reference in the classical sense BUT the
    effect is the same: A change INSIDE the closure changes the variable
    outside. The only useful way of doing that in PHP without rewriting the
    complete engine is using references - and such things are *already* done
    via references - namely global variables: If you import a variable via
    $global, in reality you are creating a reference to the actual global
    variable - global $foo is actually more or less the same as $foo =&
    $_GLOBALS['foo'].

    Regards,
    Christian
  • Christian Seiler at Jun 18, 2008 at 12:18 pm
    Hi!
    1) I am not sure that the current semantics of the "lexical" keyword
    is great in all cases. Is the reason why you don't allow by-value
    binding so that we don't have to manage more than one lambda instance
    per declaration?
    First of all: global and static are also used to create references to
    other variables (OK, with static they are not visible to the outside,
    but nevertheless...) and second because other languages do the same. As
    someone corrected me a while ago, even JS uses references, test the
    following JavaScript code:

    function foo () {
    var x = 5;
    var f = function (n) {
    x += n;
    };
    alert (x);
    f (2);
    alert (x);
    }

    foo ();

    That will yield first 5 and then 7.
    2) [minor curiosity - do we want to consider reusing "parent" instead
    of "lexical"? I guess that could be confusing but it's not the first
    time we reuse a keyword when it's clear that the usage is in two
    different places (this is minor and I don't mind much either way
    although lexical doesn't mean too much to me).]
    Consider this code:

    class A {
    public function printSomething ($var) {
    echo "$var\n";
    }
    }

    class B extends A {
    public function printSomething ($var) {
    $printer = function () {
    parent $var;
    parent::printSomething ('I print: ' . $var);
    };
    $printer ();
    }
    }

    Yeah, of course, my example is extremely stupid since it could be done
    entirely without closures but I really dread the perspective of having
    to explain someone the difference between those two lines...
    3) I am concerned about binding to classes. First of all we need to
    look into more detail what the implications are for bytecode caches
    when changing class entries at run-time.
    Well, that's the thing: My patch does NOT change classes at runtime, so
    that is totally a non-issue. :-)

    When creating a lambda function inside a class method, it adds a new
    class method for the lambda function at compile time (!). This
    compile-time added method has a dynamic name (__compiled_lambda_F_N
    where F is the filename and N is a per-file counteŕ). To an opcode cache
    processing this class this added method will appear no different than a
    normal class method - it can be cached just the same.

    Now, upon execution of the code containing the closure, the new opcode
    just copies the zend_function structure into a copy, registers that copy
    as a resource and returns that resource. As soon as the resource is
    garbage collected (or explicitly unset), the op_array copy is destroyed.
    No modification of the actual class is done at all - the cache remains
    happy.

    Just for clarity I have posted a sample output of PHP with my Patch and
    VLD active (<153> is the new ZEND_DECLARE_LAMBDA_FUNC opcode that VLD
    does not yet know about):

    http://www.christian-seiler.de/temp/php-closure-opcodes.txt

    Perhaps this helps to understand better how my patch works?
    We may want to also consider an option where the lambda binds to the
    object and only has public access although I realize that may be
    considered by some as too limiting. We'll review these two things in
    the coming days.
    What do you mean with "binds to the object"?

    But if you only want to grant access to public object members: If I
    declare a closure inside a class method, from a programmers point of
    view I am still within that class - why shouldn't I be able to access
    all class properties there? I would find such a limitation quite odd
    (and technically unecessary).

    Regards,
    Christian
  • Chris Stockton at Jun 18, 2008 at 3:05 pm
    Hello,

    I am curious if is_callable will be able to detect these? Or do we need a
    is_lamba, or is_function or something. You may have mentioned it but reading
    through I did not notice. I am only curious how to know when someone passed
    me one of these. Maybe a type hint would be nice too but that is a different
    conversation I guess.

    -Chris
  • Andi Gutmans at Jun 18, 2008 at 6:13 pm
    Hi Christian,

    Thanks for the clarifications. This helped a lot and makes me feel very confident about this implementation. I think this is a very strong proposal.

    A few additional things I thought about while taking a closer look:
    - You mention "global" and "static" as examples of how we do things today. They are actually not good examples because the binding by reference which they do has been a real pain over the years. This is why we introduced the $GLOBALS[] array so that you could also assign by reference ($GLOBALS["foo"] =& $var). Now that I think of this example I'd actually prefer to see $LEXICALS[] or something similar to access variables then go with the broken global/static behavior. This will bite us and people will complain... In general, I always recommend to people to keep away from "global" and go with "$GLOBALS[]".
    - Minor implementation suggestion: I am not sure we need those flags for closures and have those if() statements before function calls. We took the same approach with other obfuscated functions/methods/variables. If the developer *really* wants to cheat the engine and assemble an obfuscated name then he can. It's like doing the following in C: ((fun(*)()) 0x454544)(). I say, be my guest. It just simplifies implementation a bit. No biggy but consistent with the rest of PHP.
    - Please check eval(). I assume it will bind to global scope but let's just make sure what happens esp. when it's called from within a method...
    - In PHP 5, object storage is resources done right. I don't think we should be using the resource infrastructure for this implementation and would prefer to use the object one. It's better. I suggest to take a look at it.

    Will also look into byte code cache implementation issues incl. performance pieces but it looks like there shouldn't be any show stoppers here but I want to verify.

    Thanks again for your hard work!

    Andi
  • Stanislav Malyshev at Jun 18, 2008 at 6:29 pm
    Hi!
    by reference ($GLOBALS["foo"] =& $var). Now that I think of this
    example I'd actually prefer to see $LEXICALS[] or something similar
    The problem here might be that if we do something like $LEX[$foo] in
    runtime, we don't know which parts of parent's scope we need to
    preserve. While I like the syntax, it may not work this way.

    Which brings me to another point - how bad would it be if closure's
    lifetime would be limited to parent's lifetime? Of course, this would
    limit some tricks, but would allow some other - like this direct access
    to parent's scope.
    - Minor implementation suggestion: I am not sure we need those flags
    for closures and have those if() statements before function calls. We
    In any case I think we don't need to waste 2 bytes (or more with
    alignment) on something that's essentially 2 bits. I know it's
    nitpicking, but every little bit helps :) Of course, if we drop the
    flags the point is moot.
    --
    Stanislav Malyshev, Zend Software Architect
    stas@zend.com http://www.zend.com/
    (408)253-8829 MSN: stas@zend.com
  • Alexey Zakhlestin at Jun 18, 2008 at 7:59 pm

    On 6/18/08, Stanislav Malyshev wrote:

    Which brings me to another point - how bad would it be if closure's
    lifetime would be limited to parent's lifetime? Of course, this would limit
    some tricks, but would allow some other - like this direct access to
    parent's scope.
    that would be seriously bad, because it will eliminate possibility of
    lambda-generating functions
  • Christian Seiler at Jun 18, 2008 at 8:15 pm
    Hi Andi, Hi Stanislav,
    - You mention "global" and "static" as examples of how we do things
    today. They are actually not good examples because the binding by
    reference which they do has been a real pain over the years. This
    is why we introduced the $GLOBALS[] array so that you could also
    assign by reference ($GLOBALS["foo"] =& $var). Now that I think of
    this example I'd actually prefer to see $LEXICALS[] or something
    similar to access variables then go with the broken global/static
    behavior. This will bite us and people will complain... In general,
    I always recommend to people to keep away from "global" and go with
    "$GLOBALS[]".
    The problem here might be that if we do something like $LEX[$foo] in
    runtime, we don't know which parts of parent's scope we need to
    preserve. While I like the syntax, it may not work this way.
    Yes, that's the point. 'lexical $foo' does two things (instead of global
    simply doing one thing):

    a) At compile time (!) remember the name of the variable specified
    in an internal list assigned to the function. Example:

    function () {
    lexical $data, $i;
    }

    This will cause op_array->lexical_names to be the list "data", "i".

    b) At run time, the lexical keyword creates a reference to the
    lexical variables that are stored in op_array->lexical_variables
    (just as global does with global scope)

    The op_array->lexical_variables itself is filled in the new opcode which
    is executed upon *assignment* (read: creation) of the closure. It's
    essentially a for loop that goes through op_array->lexical_names and
    adds a reference from the current symbol table to the
    op_array->lexical_variables table.

    So, to make an example (with line numbers for reference):

    1 $data = "foo";
    2 $i = 4;
    3 $func = function () {
    4 lexical $data, $i;
    5 return array ($data, $i);
    6 };
    7
    8 $func ();

    Step 1 (Line 4 at compile time): op_array->lexical_names is set to
    "data", "i".

    Step 2 (Line 3 at run time): The ZEND_DECLARE_LAMBDA_FUNC opcode is
    executed, it creates a copy of the op_array to store in the return
    value, in the copy it initializes the hash table
    op_array->lexical_variables and then creates two new variables in
    op_array->lexical_variables which are references to the current scope
    varialbes $data and $i:

    +---------------+ +-------------------------+
    lex_variables | | EG(active_symbol_table) |
    +---------------+ ref +-------------------------+
    data ------|------------------|-> data |
    i ------|------------------|-> i |
    func |
    ... |
    +---------------+ +-------------------------+

    Step 3 (Line 8 at run time): The closure is executed.

    Step 4 (Line 4 at run time): The lexical keyword retrieves the $data and
    $i variables from op_array->lexical_variables and adds a reference to
    them:

    +-------------------------+ +---------------+ +-------------+
    EG(active_symbol_table) | | lex_variables | | parent s.t. |
    +-------------------------+ +---------------+ +-------------+
    data --------|------|-> data ---|------|-> data |
    i --------|------|-> i ---|------|-> i |
    func |
    ... |
    +-------------------------+ +---------------+ +-------------+

    Btw: The grammar for lexical_variable contains only T_VARIABLE (and not
    ${...} etc.) on purpose - to be sure the name can be extracted at
    compile time.

    (Just as a clarification how the patch internally works.)

    Frankly, I don't really see a problem with using references. It fits
    into what's already there in PHP and it assures that closures have the
    necessary properties to make them useful.
    Which brings me to another point - how bad would it be if closure's
    lifetime would be limited to parent's lifetime? Of course, this would
    limit some tricks, but would allow some other - like this direct
    access to parent's scope.
    That "trick" would actually completely destroy the concept of closures:
    The idea behind closures is that the lexical scope during *creation* of
    the closure is saved. If you say "I want direct access via $LEXICALS"
    then the lexical scope during the *execution* of the closure will be
    used (yeah sure, the scope will be the scope during the creation of the
    closure but the *state* of that scope will be the scope during execution
    and not creation - "unbinding" variables after defining the closure (and
    therefore for example loops) will not be possible at that point).

    Furthermore, the idea that the closure lives longer than the scope in
    which it was declared is one of the other most basic ideas behind
    closures. Also, consider this code:

    function foo () {
    $lambda = function () { echo "Hello World!\n"; };
    $lambda ();
    return $lambda;
    }

    $lambda = foo ();
    $lambda (); # What should happen here?

    That would be a *major* WTF in my eyes... You return something that was
    perfectly valid inside the function WITHOUT CHANGE TO IT and just
    because you leave the function it becomes invalid?

    Personally, I don't like the idea of dumping two essential concepts of
    closures just because using variable references may seem a bit of a pain
    in some way.
    - Minor implementation suggestion: I am not sure we need those
    flags for closures and have those if() statements before function
    calls. We took the same approach with other obfuscated
    functions/methods/variables. If the developer *really* wants to
    cheat the engine and assemble an obfuscated name then he can. It's
    like doing the following in C: ((fun(*)()) 0x454544)(). I say, be
    my guest. It just simplifies implementation a bit. No biggy but
    consistent with the rest of PHP.
    Personally, I do like to catch all possible errors, even if they don't
    matter that much. But if you think this is superflous, I can remove it.

    Granted, if called directly without being a closure, all lexical
    variables will be NULL, so it doesn't really represent a problem.
    - Please check eval(). I assume it will bind to global scope but
    let's just make sure what happens esp. when it's called from within
    a method...
    Hmm, closures inside eval() will bind variables to the scope in which
    eval() was called. But closures defined inside eval will NOT be class
    methods, even if eval() is called within a class.

    But I do find that behaviour consistent with what PHP currently does
    with normal functions and variables: If eval()'d or include()'d inside a
    function, variables will the "global scope" of eval() or the included
    file will actually be the local function scope whereas defined functions
    inside will automatically become global functions.

    Of course, this behaviour should be documented but I don't see a reason
    to try and change it.
    - In PHP 5, object storage is resources done right. I don't think
    we should be using the resource infrastructure for this
    implementation and would prefer to use the object one. It's better.
    I suggest to take a look at it.
    Hmm, seems like a good idea. If nobody objects in the next few days,
    I'll rewrite my patch to use objects instead of resources. What class
    name do you suggest?

    Regards,
    Christian

    PS: Somebody made me aware of a segfault in my code when destroying the
    closure variable while still inside the closure. I'll fix that.
  • Andi Gutmans at Jun 19, 2008 at 6:47 am

    See below:

    -----Original Message-----
    From: Christian Seiler
    Sent: Wednesday, June 18, 2008 1:14 PM
    To: php-dev List
    Subject: Re: [PHP-DEV] [PATCH] [RFC] Closures and lambda functions in PHP

    Frankly, I don't really see a problem with using references. It fits
    into what's already there in PHP and it assures that closures have the
    necessary properties to make them useful.
    I think you are right that there isn't really a good alternative as the "parent" scope does not necessarily exist anymore. Your solution is likely the best.
    - Please check eval(). I assume it will bind to global scope but
    let's just make sure what happens esp. when it's called from within
    a method...
    Hmm, closures inside eval() will bind variables to the scope in which
    eval() was called. But closures defined inside eval will NOT be class
    methods, even if eval() is called within a class.

    But I do find that behaviour consistent with what PHP currently does
    with normal functions and variables: If eval()'d or include()'d inside a
    function, variables will the "global scope" of eval() or the included
    file will actually be the local function scope whereas defined functions
    inside will automatically become global functions.

    Of course, this behaviour should be documented but I don't see a reason
    to try and change it.
    I agree. It behaves as I would expect I just wanted to make sure you verify that because I didn't have the opportunity to do so. You'd actually have to work very hard for it not to behave in that way :) Let's just make sure we have unit tests for both cases just so we have a good regression on this one.
    - In PHP 5, object storage is resources done right. I don't think
    we should be using the resource infrastructure for this
    implementation and would prefer to use the object one. It's better.
    I suggest to take a look at it.
    Hmm, seems like a good idea. If nobody objects in the next few days,
    I'll rewrite my patch to use objects instead of resources. What class
    name do you suggest?
    Great. I think Closure is probably a good name.
    [Btw, if we want to get fancy we could even have a __toString() method on those which would print out information about the Closure. But this is not a must, just something which eventually could be nice for debugging purposes...]
    PS: Somebody made me aware of a segfault in my code when destroying the
    closure variable while still inside the closure. I'll fix that.
    :)

    Thanks,
    Andi
  • Troels knak-nielsen at Jun 19, 2008 at 8:52 am

    On Thu, Jun 19, 2008 at 8:44 AM, Andi Gutmans wrote:
    - In PHP 5, object storage is resources done right. I don't think
    we should be using the resource infrastructure for this
    implementation and would prefer to use the object one. It's better.
    I suggest to take a look at it.
    Hmm, seems like a good idea. If nobody objects in the next few days,
    I'll rewrite my patch to use objects instead of resources. What class
    name do you suggest?
    Great. I think Closure is probably a good name.
    [Btw, if we want to get fancy we could even have a __toString() method on those which would print out information about the Closure. But this is not a must, just something which eventually could be nice for debugging purposes...]
    Using objects, instead of resources is an excellent idea. Would it be
    possible to introduce a general __invoke (Or whatever name is more
    fitting) magic-method, so that whichever object implements that
    method, is callable with call_user_func (and directly through
    variable-function-syntax). Eg.:
    class Foo {
    function __invoke($thing) {
    echo "Foo: " . $thing;
    }
    }

    $foo = new Foo();
    $foo("bar"); // > echoes "Foo: bar"

    I'm not sure how this would play together with lexical scope?

    --
    troels
  • Marcus Boerger at Jun 20, 2008 at 8:36 am
    Hello Andi,

    Thursday, June 19, 2008, 8:44:07 AM, you wrote:
    See below:
    -----Original Message-----
    From: Christian Seiler
    Sent: Wednesday, June 18, 2008 1:14 PM
    To: php-dev List
    Subject: Re: [PHP-DEV] [PATCH] [RFC] Closures and lambda functions in PHP

    Frankly, I don't really see a problem with using references. It fits
    into what's already there in PHP and it assures that closures have the
    necessary properties to make them useful.
    I think you are right that there isn't really a good alternative as the
    "parent" scope does not necessarily exist anymore. Your solution is likely the best.
    I though we are speaking of PHP here? And all I remeber is that PHP has
    reference counting. So it doesn't matter if we do reference or value
    binding. We simply have to increase the internal reference counter - done.
    - Please check eval(). I assume it will bind to global scope but
    let's just make sure what happens esp. when it's called from within
    a method...
    Hmm, closures inside eval() will bind variables to the scope in which
    eval() was called. But closures defined inside eval will NOT be class
    methods, even if eval() is called within a class.

    But I do find that behaviour consistent with what PHP currently does
    with normal functions and variables: If eval()'d or include()'d inside a
    function, variables will the "global scope" of eval() or the included
    file will actually be the local function scope whereas defined functions
    inside will automatically become global functions.

    Of course, this behaviour should be documented but I don't see a reason
    to try and change it.
    I agree. It behaves as I would expect I just wanted to make sure you
    verify that because I didn't have the opportunity to do so. You'd
    actually have to work very hard for it not to behave in that way :) Let's
    just make sure we have unit tests for both cases just so we have a good regression on this one.
    - In PHP 5, object storage is resources done right. I don't think
    we should be using the resource infrastructure for this
    implementation and would prefer to use the object one. It's better.
    I suggest to take a look at it.
    Hmm, seems like a good idea. If nobody objects in the next few days,
    I'll rewrite my patch to use objects instead of resources. What class
    name do you suggest?
    Great. I think Closure is probably a good name.
    [Btw, if we want to get fancy we could even have a __toString() method
    on those which would print out information about the Closure. But this is
    not a must, just something which eventually could be nice for debugging purposes...]
    PS: Somebody made me aware of a segfault in my code when destroying the
    closure variable while still inside the closure. I'll fix that.
    :)
    Thanks,
    Andi



    Best regards,
    Marcus
  • Kalle Sommer Nielsen at Jun 20, 2008 at 9:51 am
    Hey

    I must say that the lexical keywords makes alot more sense to me which keeps
    the syntax readable without making it too cryptic for the unexperinced or new
    developer to php.

    I think introducing both the lexical keyword and as Andi proposed a
    $LEXICAL as
    a to the global / $GLOBALS.

    Both ways have potential, but I would personal go with the lexical keyword.


    Regrads, Kalle
  • Alexander Wagner at Jun 20, 2008 at 12:00 am
    First, a comment from haskell-land:
    http://www.haskell.org/pipermail/haskell-cafe/2008-June/044533.html
    http://www.haskell.org/pipermail/haskell-cafe/2008-June/thread.html#44379
    On Wednesday 18 June 2008, Christian Seiler wrote:
    Frankly, I don't really see a problem with using references. It fits
    into what's already there in PHP and it assures that closures have the
    necessary properties to make them useful.
    References are necessary, but an easy way to obtain copies of variables from
    the lexical context would be really nice.

    I have been introduced to functional programming through Haskell, where values
    are immutable, so a reference is basically the same as a copy. I like this
    behaviour because it makes closures distinctly non-dangerous by default.

    Getting the same behaviour out of PHP should not be as difficult as this:
    for ($i = 0; $i < 10; $i++) {
    $loopIndex = $i;
    $arr[$i] = function () { lexical $loopIndex; return $loopIndex; };
    unset ($loopIndex);
    }
    This is not only quite a hassle (making beer much cheaper than water, so to
    speak), I also believe it to be error-prone. A lot of programmers are going
    to forget that unset().

    I would prefer something like this:
    for ($i = 0; $i < 10; $i++) {
    $arr[$i] = function () { lexical_copy $i; return $i; };
    }

    An alternative would be to let lexical behavie like function parameters:
    - copies by default
    lexical $x;
    - objects referenced by default
    lexical $obj;
    - other references optional
    lexical &$y;

    Of course this would make lexical behave quite differently from global in this
    regard, decreasing consistency, but f*ck global, nobody should use that
    anyway. Better to have nice lexical closures.

    Gesundheit
    Wag

    --
    Be careful about reading health books. You may die of a misprint.
    - Mark Twain
  • Stanislav Malyshev at Jun 20, 2008 at 12:57 am
    Hi!
    Thanks for the links, very interesting. Even a couple of comments in the
    thread going beyond "PHP sucks" and really discussing the matter. :)
    Best account is this:

    * A closure must only keep alive the varables it references, not the
    whole pad on which they are allocated
    [Check]
    * A closure must be able to call itself recursively (via a
    higher-order function typically)
    [Check, since you can use variable you assigned closure to inside the
    closure, if I understand correctly]
    * Multiple references to the same body of code with different bindings
    must be able to exist at the same time
    [Check]
    * Closures must be nestable.
    [Dunno - does the patch allow nesting and foo(1)(2)?]
    Getting the same behaviour out of PHP should not be as difficult as this:
    Well, I don't see any other way if you use references. Variables _are_
    mutable in PHP. You could, of course, use copies, but then you'd lose
    ability to update. Maybe if we drop "lexical" and use Dmitry's proposal of

    $arr[$i] = function () ($i) { return $i; };

    where ($i) would be copy, (&$i) would be by-ref, then it'd be easier to
    control it. I know function()() is weird, but not everybody likes
    lexical either :) Maybe we can do lexical &$y, but that looks weird too...
    Of course this would make lexical behave quite differently from global in this
    I wouldn't spend too much thought on making lexical work like global.
    global is for different purpose (and with $GLOBALS is obsolete anyway :)
    --
    Stanislav Malyshev, Zend Software Architect
    stas@zend.com http://www.zend.com/
    (408)253-8829 MSN: stas@zend.com
  • Alexander Wagner at Jun 20, 2008 at 1:52 am

    On Friday 20 June 2008, Stanislav Malyshev wrote:
    * A closure must be able to call itself recursively (via a
    higher-order function typically)
    [Check, since you can use variable you assigned closure to inside the
    closure, if I understand correctly]
    This is a matter of implementation rather than design, so it should be
    resolved by testing rather than by reading the spec ;-)
    Well, I don't see any other way if you use references. Variables _are_
    mutable in PHP.
    They are also copied by default (passed by value). So if lexical used copies
    by default (and passed objects by reference), it would be consistent with all
    of php except for global. Let global be the outcast and be consistent with
    exerything else. As long as references are easily available, I think this is
    the much better trade-off. And it makes water slightly cheaper than beer.
    I know function()() is weird
    And would become weirder if foo(1)(2) is implemented. +1 to that by the way,
    allowing dereferencing for methods ( $obj->method1()->method2(); ) but not
    for functions is kinda mean.

    Maybe function( ) [ ] { } instead of function( ) ( ) { }
    That way the different parts actually look different. Also, confusion with
    arrays should be pretty much impossible here, both for the parser and human
    readers.

    I prefer "lexical", though. Functional programming is not the default paradigm
    in PHP, so rather err on the side of explicitness.

    Gesundheit
    Wag

    --
    Remember, growing older is mandatory. Growing up is optional.
  • Stanislav Malyshev at Jun 23, 2008 at 5:56 pm
    Hi!
    Hmm, seems like a good idea. If nobody objects in the next few days,
    I'll rewrite my patch to use objects instead of resources. What class
    name do you suggest?
    While we are at it maybe even having special standard handler
    (__invoke?) that could be also used by objects created by reflection and
    maybe later of some other purposes. I.e. if we do $foo($bar, $baz) and
    $foo is an object and it defines __invoke, then we call it (in which
    case if $foo is Closure it does its thing) otherwise we get an error
    "object $foo is not callable". Of course, this goes also for
    is_callable, etc.
    What do you think?
    --
    Stanislav Malyshev, Zend Software Architect
    stas@zend.com http://www.zend.com/
    (408)253-8829 MSN: stas@zend.com

Related Discussions

People

Translate

site design / logo © 2022 Grokbase