Subject: XML related issues in metalink, was: Last Call: draft-bryan-metalink (The Metalink Download Description Format) to Proposed Standard

From: Julian Reschke <[email-masked]>
Date: Sun, 13 Dec 2009 11:09:29 +0100

The IESG wrote:
> The IESG has received a request from an individual submitter to consider 
> the following document:
> 
> - 'The Metalink Download Description Format '
>    <draft-bryan-metalink-24.txt> as a Proposed Standard
> ...

Hi,

I did a quick check on a few XML related issues, and found:

1) References to W3C specs

- the references currently use many different formats (for 
seriesName/value), it would be great if those were consistent (I 
recommend to use the format used for "REC-xml" for compactness)

- the up-to-date check yielded:

REC-xml-20060816: [FirstEdition] obsoleted by REC-xml-20081126
REC-xml-infoset-20040204: [REC] ok
REC-xml-names-20060816: [FirstEdition] obsoleted by REC-xml-names-20091208
REC-xmlbase-20010627: [FirstEdition] obsoleted by REC-xmlbase-20090128
REC-xmldsig-core-20080610: [REC] ok

(where the out-of-date xml-names reference is excused :-).

2) RNC

- was there an automated check that the collected RNC and the fragments 
are in sync? For "metalinkFile" I see a difference in ordering which may 
indicate that this didn't always happen (the difference appears to be 
irrelevant, but who knows...)

- I found the RNC to miss a few characters (commas, closing braces), 
which indicates it may not have been checked recently. I recommend to do 
that, and also to validate the examples in the spec against the RNC.

3) XML vs whitespace

I'm not sure I understand the whitespace treatment.

One example has:

     <url location="de" priority="1">
        ftp://ftp.example.com/example.ext
     </url>

while the prose says in Section 3: "Note that there MUST NOT be any 
white space in a Date construct or in any IRI."

I personally would prefer that whitespace is NOT ignorable, but in any 
case this should be stated somewhere more clearly.

(Note I didn't review the spec, I just did a few XML related checks)

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Sun, 13 Dec 2009 20:39:37 +0100

Anthony Bryan wrote:
>> while the prose says in Section 3: "Note that there MUST NOT be any white
>> space in a Date construct or in any IRI."
>>
>> I personally would prefer that whitespace is NOT ignorable, but in any case
>> this should be stated somewhere more clearly.
> 
> Very helpful, your sentence was added:
> 
> "All leading and trailing whitespace is part of the element content,
> and MUST be ignored. Consequently, it is disallowed for elements where
> the defined type does not allow whitespace, such as dates, integers,
> or IRIs."

Um, that should be "MUST NOT be ignored", right?

BR, Julian

---

From: "Thomson, Martin" <[email-masked]>
Date: Mon, 14 Dec 2009 07:48:38 +0800

...
> Fixed.

Why is whitespace so important?  The alternative to constraining use as you have done, which requires that you also "fix" all the examples, is to use the type that fits better with user expectations: token.

In the *value space* for the token type, the above example is simply "ftp://ftp.example.com/example.ext", with leading and trailing whitespace stripped.

Then you can remove the following note:
   Note that there MUST NOT be any white space in a Date construct or in
   any IRI.  Some XML-generating implementations erroneously insert
   white space around values by default, and such implementations will
   generate invalid Metalink Documents.

... and with it, avoid all the errors that inevitably occur when you make whitespace significant.  Unless you believe that Metalink documents will never be authored by anything other than software.

In looking into this, I noted this:
   # Unconstrained; it's not entirely clear how IRI fit into
   # xsd:anyURI so let's not try to constrain it here

I wonder why you haven't taken the plunge on xsd:anyURI, even if xsd:anyURI has dubious official status with regards to IRIs.  In practice, IRIs are commonly placed in xsd:anyURI.  The lexical space accommodates them, no implementation I'm aware of prevents use of IRIs.

--Martin

---

From: Julian Reschke <[email-masked]>
Date: Mon, 14 Dec 2009 10:00:24 +0100

> Why is whitespace so important?  The alternative to constraining use as you have done, which requires that you also "fix" all the examples, is to use the type that fits better with user expectation…

IMHO, what's important is to decide, to specify it properly, and to have 
examples consistent with it. And, optimally, test cases.

> ... and with it, avoid all the errors that inevitably occur when you make whitespace significant.  Unless you believe that Metalink documents will never be authored by anything other than software.

Well, by making it non-significant, you'll might get interop problems as 
well.

> I wonder why you haven't taken the plunge on xsd:anyURI, even if xsd:anyURI has dubious official status with regards to IRIs.  In practice, IRIs are commonly placed in xsd:anyURI.  The lexical spac…

I'll assume it's inherited from RFC 4287, and that the Atom WG had good 
reasons not to use xsd:anyURI...

Best regards, Julian

---

From: Anthony Bryan <[email-masked]>
Date: Sun, 13 Dec 2009 14:34:38 -0500

On Sun, Dec 13, 2009 at 5:09 AM, Julian Reschke <[email-masked]> wrote:
> seriesName/value), it would be great if those were consistent (I recommend
> to use the format used for "REC-xml" for compactness)
>

These references have been updated and made more consistent.

> 2) RNC
>
> - was there an automated check that the collected RNC and the fragments are
> in sync? For "metalinkFile" I see a difference in ordering which may
> - I found the RNC to miss a few characters (commas, closing braces), which
> indicates it may not have been checked recently. I recommend to do that, and
> also to validate the examples in the spec against the RNC.

Fixed.

> (Note I didn't review the spec, I just did a few XML related checks)

Thank you, Julian, for these needed checks! Your fresh eyes picked up
things unnoticed. "It takes a whole village to raise a spec" :)

For those interested, the txt, XML, and html updates in revision 25
are at http://metalinks.svn.sourceforge.net/viewvc/metalinks/internetdraft/

-- 
(( Anthony Bryan ... Metalink [ http://www.metalinker.org ]
  )) Easier, More Reliable, Self Healing Downloads

---

From: Anthony Bryan <[email-masked]>
Date: Sun, 13 Dec 2009 14:47:19 -0500

On Sun, Dec 13, 2009 at 2:39 PM, Julian Reschke <[email-masked]> wrote:
>>> case

Fixed.

Thanks for re-re-re-re-checking my errors. (That's a full time job).
You must want this to come out ok. :)

---

From: "Thomson, Martin" <[email-masked]>
Date: Wed, 16 Dec 2009 06:27:28 +0800

Inline...

> -----Original Message-----
> From: Julian Reschke [mailto:[email-masked]]
> Sent: Monday, 14 December 2009 8:00 PM
> To: Thomson, Martin
> Cc: Anthony Bryan; Apps Discuss; [email-masked]
> Subject: Re: XML related issues in metalink, was: Last Call: draft-
> bryan-metalink (The Metalink Download Description Format) to Proposed
> Standard
> 
> Thomson, Martin wrote:
> > Why is whitespace so important?  The alternative to constraining use
> as you have done, which requires that you also "fix" all the examples,
> is to use the type that fits better with user expectations: token.
> 
> IMHO, what's important is to decide, to specify it properly, and to
> have examples consistent with it. And, optimally, test cases.

I tend to favour non-significant whitespace - it fits with common expectations.  After all, the examples seem to assume that.

But in the end, making a decision and sticking to it is the best plan.

> Well, by making it non-significant, you'll might get interop problems
> as well.

It's XML.  Of course there will be whitespace issues.

> > I wonder why you haven't taken the plunge on xsd:anyURI, even if
> xsd:anyURI has dubious official status with regards to IRIs.  In
> practice, IRIs are commonly placed in xsd:anyURI.  The lexical space
> accommodates them, no implementation I'm aware of prevents use of IRIs.
> 

Ah.  Can't argue with inherited wisdom.

---

From: Peter Pöml <[email-masked]>
Date: Tue, 15 Dec 2009 03:02:57 +0100

Hi Martin,

Am 14.12.2009 um 00:48 schrieb Thomson, Martin:

> as you have done, which requires that you also "fix" all the  
> examples, is to use the type that fits better with user  
> expectations: token.
>
> In the *value space* for the token type, the above example is simply  
> "ftp://ftp.example.com/example.ext", with leading and trailing  
> whitespace stripped.
>
> ... and with it, avoid all the errors that inevitably occur when you  
> make whitespace significant.  Unless you believe that Metalink  
> documents will never be authored by anything other than software.
>

Thanks for the interesting suggestion. Thought about it for a while,  
asked some people, but now I think that what matters most is that it's  
clearly specified, as Julian pointed out. I now think it's fine as it  
is.

We still have to fix the examples. I'll do it now in svn.

Peter