[PHP-INTERNALS] Please let's not bitch about lazy users not learning C to implement THEIR missing feature. (Was Re: [PHP-DEV] 5.4 again)

Hi Richard,

Again what I commented on other thread and again you barely see what I
mentioned, the feature is ALREADY written in C and compatible with
latest PHP trunk.
I'm not bitching against do and don't dos... I'm bitching about
ignored feature that are not even discussed.

I agree with you, it's an Open Source project and any help from any
front is valuable.
But as I pointed out on other thread, only developers with php-src
karma had features evaluated.
If you think I'm wrong, please re-read the entire thread of
Annotations and then get back to me with a single message on thread
where actually had a discussion with some mature content instead of
personal feelings.

Regards,

A lot of people who understand the problem very well weighed in on this
feature. Framework authors and others who have a real use for
annotations, and there was still no clear agreement.

As far as RFCs on the wiki only coming from people with php-src
accounts, that is true, since that is our auth system for the wiki. But
getting an account, even if it is simply to submit an RFC to the wiki is
rather trivial.

Or looking at it another way, if the feature is so popular, getting one
of the 2000 people with an account to submit the RFC shouldn't be a problem.

-Rasmus

Rasmus,

I already wrote an RFC, I already wrote a patch and none from php-src
gave me some valuable feedback.
During private conversations while flaming messages were popping on ML
thread, I updated the code to be more PHP compatible and when I went
to update the RFC on wiki, it became offline.

BTW, if you think Annotations wouldn't be so popular, please tell the
Symfony users (Routing, Validation), Doctrine users (Entire Mapping),
Typo3 users, Zend Framework (XML-RPC), PHPUnit users that this feature
is useless. If this doesn't count 2000 users using the feature, I
think only wordpress users may count this.

Cheers,

Nobody has argued that there isn't a use for annotations. There
obviously is. The argument is whether it needs to be in the core of the
language when it isn't inherently a runtime thing. A single standard for
annotations and non-runtime tools for manipulating that standard is a
viable approach as well. That is what people are doing now, except they
all picked different ways of doing it. By putting it into the core you
are solving that problem since everyone will likely switch to it, but
the argument is that that is not a good enough justification for putting
it into the core of the language.

-Rasmus

Hi Rasmus,

Thanks a lot for the response. This was the first email that I got
that is not rude against my patch.

I have worked on Doctrine annotations support (which is being used by
Symfony and also Typo3), which is a LL(*) parser that processes
docblocks and uses runtime classes to build associated information.
This relies on 2 points to correctly work: a reader and a cache. These
ones are required for 2 reasons:
- PHP doesn't currently support annotations (so
ReflecionClass/ReflectionProperty/ReflectionMethod cannot have a
simplified API to getAnnotations()). The Reader is a composition over
ReflectionAPI.
- Parsing is expensive and I cannot plugin opcache. To fix the
overhead of processing every request, I plugged a cache support.

What I thought it could be changed is:
- Allow PHP to support it natively and also take advantage of opcode cache
- Make API cleaner

That's where the idea came.
I voted for having it native to ZE because a code with and without
comments should behave the same. So this made me to work on something
that could be merged into php-src.

If possible, could you look at the patch and give me high level ideas
of what could be changed?

Thanks

Hi!
If the patch is the same RFC that is at
https://wiki.php.net/rfc/annotations, the same problems that were voiced
a number of times on the list stay:

- it is overly complex (see class User example - it's really a piece of
code, I think it should be in the code)
- it introduces method call syntax not compatible with the rest of PHP
- it introduces object instantiation syntax not compatible with the rest
of PHP

These issues were mentioned before - were they fixed?
The RFC also does not clarify where the code contained in annotations is
run and how it would play with bytecode caches.

please read the other thread, I brought up the original thread (the more php
friendly version), as it was mentioned, the rfc didn't up to date AFAIK.

Tyrael

Hi Stas,

Comments inline.
Currently it's not.
When I was going to update it, wiki went offline.
That's just an example. I'll simplify it.
The main point there was to illustrate the ability to support nested
annotations. Exmaple using @:

@A(@B)
This is already fixed.
@A(@B) would require class A like this:

class A {
public function __construct(B $b) { ... }
}

class B {}
Sorry, this was one of the things that I was going to update. I'll
include it too.

Guilherme,

As per many of the conversations on annotations one of that hardest parts of
it is that there are generally 3 conversations going on about it when this
starts to be discussed. It seems many threads are hi-jacked and I can
understand why.

I would like to state that annotations in the core may be a good idea;
however, we have PECL and PECL seems like the perfect place for this. Quite
a few extensions have started off in PECL and made their way to the core and
several have been moved from the core to PECL. To me annotations support
seems like it would be perfect in PECL. Dedicated machines and virtual
machines for end users are higher than ever and they seem to continue to
grow. This should easily allow folks to put in the PECL module as needed.

I would argue that the introduction of this into the core is adding more
feature bloat into the language that is not quite needed at this point.
There are obviously several improvements that this would allow to be
improved and consistent: WSDL / JSON-RPC / XML-RPC / Doctrine / etc.

However, I feel extensions like Mongo / Memcached / Gearman have more to add
to the PHP core than adding annotations and they live in PECL. Just
speaking from the practical point of view. It is great that certain
projects that have utilized annotations and created a layer in user land to
make annotations a nice thing to utilize. But to continue to argue this
point; it just does not seem logical other than the fact that a few projects
want to promote annotations should cause it to drop into the core.

I for one would like to see this go to PECL and see the up-take then
identify if it is needed for the next PHP release after 5.4. It seems a bit
early and like it would be crammed into the PHP core without enough
discussion. There are obviously many thoughts on this and it will take time
to sort out and implementation details then would be further down the trail
after some consensus is happening within this feature.

Regards,

Mike

Hi,

Annotations as proposed in the RFC can not (or hardly) be develop as an
extension (and so can not go into PECL). The proposed feature require
modifications directly into the Zend Engine like for the inclusion of a new
syntax which imply modification of the parser.

Regards,
Pierrick

Annotations cannot be considered bloat because are being used increasingly
everywhere that is a clear indication that they are required as part of the
PHP core as much as many of the Spl classes. It should be clear by now that
the PHP community really do want annotations. At this stage, if someone has
done the work to make this happen, the discussion really should be more
about polishing that contribution and making sure it provides a robust
solution to this feature than trying to postpone or find reasons to put this
off.

Regards,

Drak

*IS* it clear by now that the majority of users want this? And the argument that
'You don't have to use it' does not wash either since once it has been pushed
in, some of the libraries we are using are going to start requiring it simply
because those developers do like the idea, but it does not necessarily mean that
THE CURRENT PROPOSAL is the right way of doing it? What I seem to have lost is
the evidence that annotations are going in because we HAVE all agreed to them?
If they CAN'T be implemented as a PECL extension, then they need a very good
implementation covering all aspects even before it is decided that they should
be pushed in? Much like Traits ...

The current standards such as phpdoc are falling well behind because nobody has
the time to implement some of the ALREADY created new features. It is this that
is more annoying given the investment in time just to try and keep up, And
creating tools intended for users who don't like the existing tools is further
watering down the project :( The existing tools had been working well, but
nowadays things are simply becoming a mess ...

I agree.

Why not fixing the several hundreds of bugs in PHP before just even thinking about adding new features ???
I much respect people using my programs thus for my projects I just forbid myself to release any new
version which is not bug free as far as I know.

It's getting harder and harder to make my php programs work because of memory leaks/corruptions :-(

YMMV

I especially oppose the complexity of the current proposal. One of the
reasons I prefer PHPDoc to the proposed Annotations is because they're
a simple key=>value syntax.

I'm already doing my coding in PHP - why do I have to code in a new
sub-language when all I want is a litte bit of meta-data?

My main question is: Why do we need more than key=>value? When you say
that "everyone" supports annotations (if that is true), are you sure
they actually want more than key=>value?

Discussion of this does not seem to appear in your "Why do we need
Class Metadata?" section.

that would be the same argument that we don't need objects because we have
arrays, and if you only need something to store your structures, then both
can be used for that.

nobody is forcing you to use annotations, it won't replace the docblocks.

at least Doctrine, Symfony and FLOW3 does.
Sebastian expressed that he is fine with the current Docblock support for
PHPUnit.
the FLOW3 used to use single key values in the past, but I'm not familiar
with the current situation.

for actual use-cases you can check
http://blog.seric.at/2011/05/03/annotations-with-symfony2/
or
http://www.doctrine-project.org/docs/orm/2.0/en/reference/annotations-reference.html#annref-column


I also think that it would be a good idea to link or describe annotations in
general, because it seems that nobody bothers to read that up without
joining the conversation...

Tyrael

From the user-end perspective, what I don't understand is this:

What is the goal of having Annotations embedded in PHP? To nail down a
common syntax? To provide an interface for meta-information on a class?

Why can't this be PHP code? Why should I have to learn a whole new kind
of syntax? We already have a common syntax (PHP interface) for this as
well as an interface (static Class-functions/Object-methods).

To explain what I mean, I'll use the example provided in the RFC. Could
anyone please explain the advantages of having "passive" annotations
over "active" PHP Code.

<Entity("users")>
class User
{
<Column("integer")>
<Id>
<GeneratedValue("AUTO")>
protected $id;

// ...

<ManyToMany("Phonenumber")>
protected $Phonenumbers;
}

*** Example ***

class User implements EntityAnnotation {
protected $id;
protected $Phonenumbers;

public function getEntityAnnotation(){
return new User_EntityAnnotation();
}
}

class User_EntityAnnotation {
public function getEntityName(){
return 'users';
}

public function getColumnInfo($property){
switch( $property ) {
case 'id': return array(
'column'=>'integer',
'isPrimary'=>true,
'autoIncrement'=>true
);

case 'Phonenumbers': return array(
'manytomany'=>'Phonenumber'
);
}
}
}

***************

I think the main reasons are standardization of the syntax and
performance of the parsing. At the moment everyone has to cache the
stuff because hitting the tokenizer every time is quite expensive.

Honestly if there was a way to parse them into the docblocks, I think
it would be better because it would allow us to keep BC with older PHP
versions, but since docblocks contain documentation as well, I think
you can't really parse it all without creating a mess.
Annotations are not code, but more like metadata to tell the
surrounding libraries how to work best with a piece of code.
I think your example shows very well why annotations are good, it's
much more concise.

Cheers

If implemented within PHP the existing opcode-caches could work wonders
on this.
But my example is much more verbose (which is good in my view), uses
common syntax and is very flexible! It gives you the whole power of PHP
instead of a semi-language (no offense).

By concise, you mean shorter, easier to read?

I have a really hard time looking at:

<Entity("users")>
class User {
<Column("integer")>
<Id>
<GeneratedValue("AUTO")>
<ManyToMany("Phonenumber")>
protected $Phonenumbers;
}

as a PHP user and instantly understanding what this does. It looks
completely foreign from a PHP syntax point of view.

Whereas:

class User_EntityAnnotation {
public function getEntityName() {
return 'users';
}

public function getColumnInfo($property) {
switch($property) {
case 'id': return array(
'column'=>'integer',
'isPrimary'=>true,
'autoIncrement'=>true
);

case 'Phonenumbers': return array(
'manytomany'=>'Phonenumber'
);
}
}
}

is way more verbose, but I can instantly grok what is going on without
learning a new syntax, without waiting for my opcode cache to support
it, and without waiting for my editor to understand the new syntax.

Don't underestimate the difficulty in getting opcode cache support for
something like this. If you think the engine code is complex, try
digging into the opcode cache code. It will be much much harder to write
the opcode cache support than it was to write the engine code for it.

I also have a personal problem with code that needs to introspect on
every web request in order to run. But that is likely because I am old
and gray and used to stare sceptically at the assembly output of the
first C compilers to see if I could come up with an alternative that
would take fewer cycles.

-Rasmus

The editor argument is out of place
do you really think that the engine should we built around what IDE
supports?

At least the much quoted user-base would welcome syntax-support for this
feature, wouldn't you agree? If support is already there, that's a big plus!

And given the amount of time it is taking to get things like GIT and HG even
vaguely functional in Eclipse and other IDE's who is expected to implement even
the existing new features. Even the classname stuff still needs someone to
support it in the IDE's :(

As an interesting aside, HG is based on python2 and now that python3 is out
questions are being asked about supporting that, but the core development team
do not have the time since the two will not be backwards compatible. So it's not
just PHP that is having difficulties bringing the user base along with these new
developments. I'm only just getting a handle on python2 so that I can integrate
my hg repo's with the rest of my PHP based code management ....

IDEs are part of the PHP ecosystem, just as much as frameworks, op
code caches, documentation, bug reports, maintenance issues and even
current technology trends. The benefits of a new feature must be
calculated taking into account its impact on the whole ecosystem.

Chris

creating an official EBNF would solve this problem, among others as well.
http://marc.info/?l=php-internals&m=129387252319019

patches welcome ;)

Tyrael

A formal syntax description might help with highligting, not with all
assisting features an IDE provides (good and useful autocomplete, having
some visualisation of the code structure ("class tree" incl. annotation)
and all these)

The main point in regards to IDEs and such thing imo is that it shows
how long it takes for the eco system to pick a change up. The result is
that it takes "ages" till a new standard can be relied upon. This is
worse for a new syntax as there is no way to have code compatible with
5.4 and 5.3 then. There focussing around the existing way (doc block,
more efficient doc block parsing, ...) helps.

Then again "proper" annotations might solve tomorrows problem.

johannes, who does not argue about annotations themselves

And many of us do not recognise some of the 'complaints' simply because our own
ecosystems have already solved the problem, but it's these 'new developments'
that are threatening to destroy a perfectly stable system.

sorry my FUD counter just overflowed with your last comment.

Tyrael

Hi all,

Based on an extensive chat with Matthew, I think we reached some consensus.
I'll write another RFC related to Annotations in docblocks, then we
can chat until reach some standardization and availability.

I'll keep the old one for history purposes. It seems that none from
core php devs accepted it, so I'll move it to rejected.
As I told you previously, all I wanted was some good feedback to give
a north and that's what I had.

As soon as I finish the new RFC, I'll open another thread here for
fine-grain the support and discuss architecture.
I'll keep Stas comments in mind when creating it, so it would help in
discussions. It seems we still have 2 weeks to discuss the new idea
and less than 2 months to get it ready if everyone agreed.

Regards,

One suggestion. Be very careful about anything that requires changes in
the opcode caches out there. Such changes will be very slow in coming,
if at all.

-Rasmus

It's unrelated to this thread but, what is the status of merging APC
into the PHP core? I remember this being discussed at some point.

Drak

Next major release. We don't have enough manpower to do it for 5.4.
Right now it is only Gopal that is working on that code. Once we merge
it is going to take people quite a while to get their heads around it.

-Rasmus

Please first take a look at the current RFC regarding parsing
docblocks: https://wiki.php.net/rfc/docblockparser . Even if you want
to put up a competing RFC, at least you can use it as a point of
reference.

Chad

Additionally I would recommend reading the JavaDoc and phpDocumentor
rules concerning DocBlocks.
As far as I can see the RFC mentioned above is incomplete with respect
to the current syntax guidelines for DocBlocks as mentioned in the
documents above; which is used for quite a bit of projects out there.

I'll keep my eyes open for the new RFC to see if I can contribute some
of my experiences to the efforts.
I hope that my experiences building the Static Reflection for DocBlocks
in DocBlox can prove useful.

as phpDocumentor is dead and doesn't support "new" things like namespace and
closures, we should take those into account also.

Tyrael

Nothing stops it from working perfectly well on the years of code that it still
supports! Finding people with the time to ADD new features is the problem
especially when they are not currently using those new features themselves.

It is one of the packages that I have managed some patches on myself, but some
of it's esoteric ways of working make contributing a little difficult.

What do you suggest as an alternative - is there any?

DocBlox (http://www.docblox-project.org) is a rising project which
offers support for all these new things and uses less processing time
and memory.
It is goal is to serve as an alternative for phpDocumentor (and an
improvement, it has several features of it's own such as incremental
parsing).

DISCLAIMER: I am the Lead Dev of DocBlox.

(and before you asked: yes I tried patching phpDocumentor but its core
design would not allow enough room to patch the memory and performance
problems, which prevented me from running it at our own project)

I would also suggest DocBlox, pretty alive project, open for feedbacks
(contributed some ideas myself), and Zend Framework started using it, so now
it has a pretty nice userbase.

Tyrael

I'll hit the bitweaver code with it and see what I get ;)
Takes about 45 minutes with phpdoc but I needed 8Gb of memory to get it to run :)

ZF docs were taking between 80 and 110 minutes with phpDocumentor, and
consuming ~2GB of RAM. They now take around 10 minutes and consume less
than 1GB of RAM. :)

phpdox generates documentation for Zend Framework in less than two
minutes using less than 50 megabytes of memory ;-)

I forgot to mention that the above is for a run without an existing
cache. With an existing cache it is 5 seconds and 5 megabytes.

It's good to hear that Arne is making such fine progress; those are nice
stats.
Is this including or excluding transforming the reflector output to a
full documentation site?
(Perhaps best to answer off-group to reduce off-topic 'chatter'?)

Allow me to showcase more detailed stats of my own for completeness
(then I will stop going off-topic unless requested ;)):

A full parse including the generation of a full website takes 9.2
minutes and 44.8M RAM.
An incremental parse (which you call cache?) takes 3.3 minutes and
42.51M RAM.

Next generation documentation tools such as phpdox [1] handle these
newer language features just fine.

Sorry you feel that way, but obviously there are more people with my view that
we simply do not agree on IF annotation should be implemented. I'm a lot more
comfortable with something that works WITH what we already have rather than
going off on yet another tangent. Tidying existing docblock content to an
updated format makes a lot more sense then having to do a wholesale re-write
many thousands of files. Adding another copy of much of the same data in a
complete different format is equally insane?

PDO is another case in point - that is still not accepted and fully functional
as a replacement for the genric drivers ... ADOdb still provides a valid
abstraction layer, and if you must use PDO then it just loads that instead of
the generic one ... and it runs just as fast on either. PDO is functional, but
needs more work, however there are few people that find the need for improving it?

Hi Lester,
Documentation != Annotation.

A documentation is something that is human readable and understandable
bu humans, but not for applications.
An annotation is a behavioral functionality that is between human
readable and machine readable.
It's the starting point for AOP, which allows you to intercept
execution. I won't enter in details because I assume you have enough
experience to know how does it work.
PDO is horrible. You have to implement workarounds for almost every driver
For example Oracle driver... I already reported to Chris tons of
issues (bug ids) that even include patches for each situation and I
saw a *VERY* few bugs fixed.
I know he has priorities and etc, but it seems that you flood your
mouth to talk about PDO, but actually, writing abstraction drivers
around it is very painful.

I feel that enough people have explained why the current docblock standard
should not be replaced by yet another layer, but I have yet to see something
'annotation' related that has not already been used for several years within the
docblock wrapper format. I come from BCB where compiler commands have always
been in comments so it's natural for me anyway.
Something we can agree on! PDO was never the right solution to any problem, and
had the time been spent on bringing the ADOdb accelerator code into PHP properly
I can't help feeling we would have something much better today. I still use
ADOdb and have since day one, and cringe when people ask why it hasn't been
replaced by PDO ... when I can run the same test suite in ADOdb using a native
and the equivalent PDO driver then things may be better, but you still need
ADOdb to handle the SQL abstraction anyway.

AND fit it into the 16k of core store which was all an ICL1900 computer had to
run on ...

Martin Scotta

Annotated code integrates best with library/frameworks without the need to
"extends" or "implements".
Without annotation you will need to extend some class or to implement some
interface. That means more code to write, more chances to shoot you foot.

With annotation your classes are unaware of the other components, which
implies:
* shorter, concise code => less bugs
* no extra dependencies => easy to test

class UserFoo extends LibraryFoo { }
class UserBar implements LibraryBar {
// even worst you will need to write some methods here
}

With annotations classes are "free" to live on you own herarchy

<LibraryFoo> class UserFoo extends UserFooBase { }
<LibraryBar> class UserBar { }

Umm. Is there any way with the Annotation-proposal to validate a class
to have all the necessary annotations to work with a certain framework?
Is there an annotation-Schema or DTD which I can apply against my
classes and validate them? Because that's what you get when using
interfaces. Which means less "shooting in the foot".

As for writing less code...If there is any shared code, we now have
traits!;) hurray!
I can see that extending does not really work when using frameworks. But
when using interfaces and traits in conjunction, this is not necessary
anymore.
I don't get what the problem with writing methods is. They are readable
by any PHP5 developer and you get all kind of support in writing them.
The same holds true for interfaces.

Hi,

You all think that mapping something can always be abstracted into a
few lines like the one you presented.
Well, in certain cases your idea is valid. I'd then point you an
Entity mapping of Doctrine 2 with and without Annotations, so you can
imagine how much it can abstract:

With Doctrine Annotations: http://pastie.org/1885284
With my proposal: http://pastie.org/1885294
Without Annotations: http://pastie.org/1885252

Is that still simple?


Regarding usability, I thought that it is not "it must be used by
10,000k people to be relevant".
I think the best thing that can prove that is "goto". I didn't see a
lot of people requesting for it, but it was still included.
Let me ask you then: if it wasn't widely requested (considering your
argument that only massive requests should be relevant), why was it
included?

Regarding BC, IDEs, etc... Traits also includes new keywords and none
complained about "IDE won't support immediately".
Should I consider then that your arguments that you are strongly
forcing to be valid on this subject should also be strong on traits?
If yes, why traits was included?

Regarding readability, traits and annotations follow a similar path.
Everything that annotations can do, with much more work PHP can do
too. The same applies to traits. So again I ask: if with PHP is
possible to do the same things that traits included, why was it
included?


I'm not putting traits support inclusion on risk. I'm a string +1 to it.
All I want is that you stop giving stupid arguments to be against the
patch instead of giving *real* relevant arguments.


Regards,

On Tue, May 10, 2011 at 11:12 AM, Lars Schultz
wrote: