[Xml-compile] XML::Compile::Schema validation on xs:ID has problem

What is "xs:ID validation"? Is that a special kind of validation?
Patterns are translated into reg-exes, because the XML::LibXML module
does not export the related libxml2 functions (yet). So there could
be something wrong.
But please send me some code which does shows the problem (off-
mailinglist might be better)

If not, is there any existing work-around?
This looks ok. But reading did work, as you reported. Writing didn't.

I'm not very familar about spec. Here is text from *XML Schema 2nd Edtion*,
O'Reilly, ISBN 0-596-00252-1, chapter 16 Datatype Reference Guide, xs:ID
section:

Identifiers defined using this datatype are global to a document and
does such task.
So I guess validators should guarantee the unique restriction of xs:ID.
After all, it is not difficult to implement, right?

I quote the correct behavior is just to show my superise: xs:pattern based
on xs:ID can be checked perfectly, but xs:ID itself is not checked. That's
all.

Other people's input welcome!

* MENG Xin (zinces@gmail.com) [080515 09:54]:
Well, that's second-hand info.
The XML-Schema spec is at http://www.w3.org/TR/xmlschema-2/#ID and
says really nothing useful:
3.3.8 ID
[Definition:] ID represents the ID attribute type from [XML 1.0 (Second
Edition)]. The ?value space? of ID is the set of all strings that
?match? the NCName production in [Namespaces in XML]. The ?lexical
space? of ID is the set of all strings that ?match? the NCName
production in [Namespaces in XML]. The ?base type? of ID is NCName.

The XML 1.0 spec says:

http://www.w3.org/TR/2000/WD-xml-2e-20000814
Validity constraint: ID
Values of type ID must match the Name production. A name must not
appear more than once in an XML document as a value of this type;
i.e., ID values must uniquely identify the elements which bear them.

The RFC definition of "must" is less strong than "shall". The former
means: "should be, but not deadly", where the latter means: error.
I do not known how to interpret the XML use of the word "must".

Anyone else: what is correct behavior? An error or warning on double
IDs, of simply ignore it?
There are quite some different needs from checking seperate values against
some rules (very locally) and document global ID administration. Although
the second is possible (of course).

It sounds you have accepted my "feature request", aha?
Thanks a lot! :)

No, although a feature request is always accepted (preferrably
via rt.cpan.org) I need input from more people.

To quote myself:
So, I try to express here: I do not know how to interpret this.

If I cannot decide based on a good argument, for instance because there
are some formal decissions by the XML design board about it, then I will
not implement it (even when it is easy to do)

Yes, you are right to make the module fully compatible with RFC before
having a clear understanding.

My point is, RFC defines "must" that means "should be, but not deadly". It
could be something like "enhancement". I think adding this feature can be a
great improvement of XML::Compile::Schema module. It enriches capability of
this module and makes it more complete and powerful. Everyone is willing to
have a fully functional module. On the other hand, it is still compatible
with RFC. So I can hardly find any reason not to add this feature.

Anyway, all above are my personal suggestions (I'm trying to give some "good
arguments"), final decision is on you, of course.

What about a set of 'strictness' flags for this "should be, but not
deadly" things? The flags could even be set globally (I don't see a
system accepting different levels of strictness in XML). If the flag is
set, it dies, else, it warns.

daniel

This puts the responsibility of correct behavior with the user. I do not
like that. Most users do not read the specs.

The XML specification says that all messages shall be validated, and if
not correct, the message shall be refused. So, the answer I am looking
for is whether the re-appearance of an ID in a message is a capitical
offence, or should be treated mildly. That will dictate whether it should
be an error or a warning.

There is a lot of "informal specification" with XML Schemas, which you
need to know for good interoperability.

The point is... If the specification is not 'strict' on a definition,
there's no "correct" behavior. There are good practices and bad
practices, but they are all valid (in terms of compliance to spec), a
framework should promote the good practice (make "die" by default), but
should allow the variations that are valid according to the spec...

daniel

Thanks for your input.

Are you sure that you are not confusing the regulations set for RFCs
(from the IETF) "be strict in your implementation, but forgiving in what
you receive", to the regulations for XML (by W3C) which are "be correct
or you'll be refused"?

Yesterday, at the half-annual Dutch Unix User Group conference, I had
two separate discussions with small groups of UNIX/Internet die-hards.
And everyone agreed that the RFCs approach was a very bad idea: bad
clients and servers do often not know that they are bad. Spam filters,
for instance, suffer from the consequences; they have to work with
untrustable meta-data from justified but broken mailers. The XML concept
is much preferred.

XML::Compile is not to be forgiving like RFCs, is not to be sloppy and
DWIMming like Perl (use SOAP::Lite if you want that), but is ment to be
really XML compliant, also in this political aspect. In this particular
case, I am still unsure what "compliant" is, because the usual documents
are unclear about the requirement to enforce uniqueness of IDs within
the document. It is better not to implement a check, than having to
change the interface in a later version.

Yes. You're right. You actually have only two modes (validating and
non-validating processors). If you're "validating", uniqueness of IDs
MUST be checked, but if you're not checking it MUST NOT be checked.

Relevant quotes...

[xml spec sect. 5.1]
Non-validating processors are REQUIRED to check only the document
entity, including the entire internal DTD subset, for well-formedness.
[Definition: While they are not required to check the document for
validity, they are REQUIRED to process all the declarations they read
in the internal DTD subset and in any parameter entity that they read,
up to the first reference to a parameter entity that they do not read;
that is to say, they MUST use the information in those declarations to
normalize attribute values, include the replacement text of internal
entities, and supply default attribute values.] Except when
standalone="yes", they MUST NOT process entity declarations or
attribute-list declarations encountered after a reference to a
parameter entity that is not read, since the entity may have contained
overriding declarations; when standalone="yes", processors MUST process
these declarations.

Note that when processing invalid documents with a non-validating
processor the application may not be presented with consistent
information. For example, several requirements for uniqueness within
the document may not be met, including more than one element with the
same id, duplicate declarations of elements or notations with the same
name, etc. In these cases the behavior of the parser with respect to
reporting such information to the application is undefined.

[xml spec sect. 3.3.1]
Values of type ID MUST match the Name production. A name MUST NOT
appear more than once in an XML document as a value of this type;
i.e., ID values MUST uniquely identify the elements which bear them.
?
[rfc2119]
1. MUST This word, or the terms "REQUIRED" or "SHALL", mean that the
definition is an absolute requirement of the specification.
2. MUST NOT This phrase, or the phrase "SHALL NOT", mean that the
definition is an absolute prohibition of the specification.

daniel

Daniel,

You found what I needed to know - and didn't find (yet) because serious
lack of time. It is clear that the uniqueness test MUST==SHALL be
done when the validate option is on (the default in compile()), and
it will always die in that case.

Thank you very much.

* Daniel Ruoso (daniel@ruoso.com) [080516 10:43]:

Daniel, we are talking about whether to implement this checking rather than
how to implement, I guess you might have misunderstandings. Let me summarize
it.

As defined in spec, xs:ID restriction is used to guarantee the uniqueness of
contents. The question is who guarantee the uniqueness, users or tools?
Mark's opinion is that users who create XML instance files should guarantee
the uniqueness and XML::Compile::Schema just builds XML reader according to
schema and does not check the uniqueness. My opinion is that users have used
xs:ID to define schema and expect tools to check uniqueness, if tools leave
this task to users, why do we need xs:ID restriction?

Mark further argues that there is no deadly definitions on this checking in
spec, and I point out this implementation is a valuable enhancement to his
module. (Frankly speaking, I hope he can release a new version with this
checking implemented.)

That's all.

Mark is observing discussions in this group and waiting more good arguments
to help him make a wise decision. So I'm waiting on this thread, too.

Daniel showed in a follow-up that validating schema handlers shall check
the uniqueless. XML::Compile is validating on the messages by default,
so should check it and die on collissions.

A question which I still have is: define uniqueless. The xs:ID must
be unique. But, you can have extensions to that ID type: are they
seperate ID spaces or the same? And is xml:ID, schema-1999:ID
and schema-2001:ID one and the same space?

<attribute name="first" type="xs:ID" />
<attribute name="second" type="xs:ID" />

<simpleType name="personnel-number">
<extension base="xs:ID" />
</simpleType>
<attribute name="third" type="xs:personnel-number" />

If I read the specs well, they are one. People may get big surprises
with name-clashes between ID types. Or should I create an option?
That would complicate things a lot.

Another thing what worries me, is that I probably need to run-time
check each element whether is has an ID attribute, which can be
costly. I hope XML::LibXML can help me in some way.

The last problem to solve, is how to return the id index (because
if you keep it, you probably should give people access to it as well).
my $r = $reader->($xml);
has no space for this extension... probably something like this:

my %ids;
my $reader = $schemas->compile(READER => $type, ids => \%ids);

while(1)
{ %ids = ();
$reader->($data)
}

But I am not sure yet, because there also is a
$xml->getElementById('myid');

So... still needs some thoughts.

Based on my understanding, xs:ID is global in one XML file. So all elements
constricts by xs:ID share one name space.

I really underestimate the difficulties of checking implementation, it is
not as simple as my thought...