Subject: Re: New Last Call: 'Tags for Identifying Languages' to BCP

From: "Peter Constable" <[email-masked]>
Date: Mon, 13 Dec 2004 15:34:12 -0800

Bruce Lilly has posted comments on the IETF list in response to the
last-call announcement for a proposed revision to RFC 3066. His comments
were generally negative, raising a number of concerns. I and others
involved in preparation of the revision have discussed Bruce's concerns
with him, but they were not made available on the IETF list since those
of us other than Bruce were not subscribed to this list. I wish to
briefly summarize the outcome of that discussion for the benefit of
people here.

Some of Bruce's comments were purely editorial (e.g. formatting of
draft); I will not review those.

Bruce's substantive concerns were:

-         Accessibility of source ISO standards was referred to in the
announcment as a major reason for the proposed revision, but
accessibility has not been a problem in his experience.

-         RFC directed users to source ISO standards; the proposed
revision would establish a registry that includes all ISO identifiers
considered valid for use in language tags, but the documentation for
those identifiers in this registry does not include both English and
French language / country names. 

-         The proposed revision makes referene to ISO 8601 time/date
format being used in the registry, which is a complex and
not-readily-available specification.

-         The ABNF used in the proposed draft permits many strings that
do not conform with RFC 3066.

-         The proposed revision imposes no bounds on the length of tags
(same as RFC 3066), and does not require registration of complete tags
(different from RFC 3066).

-         The lack of an "Internationalization considerations" section
as recommended by RFC 2277 (a.k.a. BCP 18).

As a result of Bruce's comments, those of us contributing to the
development of this revision have suggested certain revisions to which
the authors have indicated openness. As I will explain, these revisions
would provide clarification on various matters, but would not constitute
technical changes in the draft.

1. Re accessibility: it was pointed out that the draft itself does not
identify accessibility of source ISO standards as one of the primary
reasons for the revision. There are some minor accessibility concerns
having to do with uncertainty of the on-going availability to the
relevant ISO code tables, and to change histories for each of the
relevant ISO standards. The proposed changes to the language-tag
registry address these concerns, though there were bigger reasons for
the proposed registry changes, particularly having to do with stability.

2. Re the lack of French descriptions in the registry: it was pointed
out that the registry defined by RFC 3066 did not include French
descriptions, and that the revised registry is not intended to replace
the source ISO standards or make them irrelevant. The meaning of IDs
would still be established from the ISO standards from which they were
drawn, and the proposed revision would continue to make reference to
them. As a result of Bruce's comments, it was suggested that wording be
revised in the draft to make this relationship clearer.

3. Re ISO 8601 time/date format: What is used in the registry is dates
expressed in the format "YYYY-MM-DD". It was agreed that it would be
better to identify the format precisely rather than make the generic
reference to ISO 8601.

4. Re the less restrictive ABNF: the one place that had less restrictive
syntax was a production rule that was subject to additional strict
constraints, namely that only certain pre-existing tags registered under
RFC 3066 could fall under that production. A change to the ABNF has been
suggested that would make the ABNF at that point consistent with the
ABNF for RFC 3066. This does not constitute a change having any
technical consequence as there is no resulting change in the set of
valid tags.

5. Re upper bounds on length of tags: It was pointed out that
private-use tags for both RFC 3066 and the proposed revision have no
bounds on their length. The greater concern was for non-private-use
tags. For these, it was pointed out that RFC 3066 also imposes no bounds
on length. Admittedly, though, there is a difference because RFC 3066
requires registration of complete tags, so one can determine at any time
what is the longest valid tag that may be encountered, whereas the
proposed revision requires registration of sub-tags which can then be
combined productively, and one cannot predict with certainty what
combinations may be used. (This, IMO, is the most significant of the
concerns Bruce raised.)

While the proposed revision allows productive combinations of registered
sub-tags, there are some limits on how combinations can be made, as
specified by the ABNF. The ABNF does allow unlimited numbers of certain
elements - specifically three. 

One of these ('extlang') is defined by the ABNF in anticipation of
possible future extension of the language tag specification to
incorporate mechanisms expected in a new part to ISO 639 that is in
preparation, but is not made avaialble for use at this time. 

Another ('variant') requires sub-tags to be registered, and requires
that the registration indicate prefix sub-tags that they are recommended
to be used with. While it may still be technical valid to use a
registered variant in some way other than the recommendatation, that
will be unlikely (just as certain combinations valid under RFC 3066,
such as ja-DE are unlikely). Thus, implementers will have a reasonable
chance of anticipating what combinations will be used. 

The third of these ('extension') is defined as mechanism for extending
language tags for use in future protocols. There is an upper limit of 25
extensions, though this RFC does not define limits on the length of each
extension. There are no extensions defined at this time, and any
extension would require specification in the form of a separate RFC. At
such time as one or more extension RFCs are defined, those
specifications would provide some indication of what limits they do or
don't impose on the length of extensions. In the case of any protocol
that supports this proposed revision to RFC 3066 but does not support
extensions, any extensions that may be included in a language tag are
ignorable.

Apart from extensions, all of the mechanisms introduced in the proposed
revision were in response to the direction users and implementers were
already going with registered tags under RFC 3066. Thus, while the
proposed revision gives greater provision for lengthy tags, this is not
completely unrestrained, and the practical likelihood of encountering
tags of any given length would be no greater under the proposed revision
than it was under RFC 3066.

Even so, verious changes were suggested to highlight issues related to
length, specifically with a view to the possibility that some
applications of RFC 3066 (or this proposed revision) would impose fixed
limits on the length of tags. These suggestions included notes in that
regard in key points within the RFC, but also in sub-tag registrations
and in RFC defining extensions. (For instance, a variant registration
would include not only a recommendation on appropriate prefixes, but
also specific comments on maximal length of tags using the given
variant.) There were no suggestions to impose limits on the length of
tags in the RFC itself (just as RFC 3066 does not impose limits).
Basically, limits on length was seen to be a concern belonging to
particular applications of the language-tag spec and not the spec
itself, but significant additions would be added to the RFC so that
these concerns are highlighted.

6. Re an i18n-considerations section: It was pointed out that language
tags are symbolic identifiers with no culture-specific content; the only
i18n consideration related to the identifiers themselves are charset,
and charset issues are covered in the section on syntax. Bruce was also
concerned about i18n considerations in the registry (see issue #2, above
- lack of French-language descriptions), but it was pointed out that the
content of the registry is not intended as localization data, that there
are well-established precedents for code sets that are not documented in
terms of multilingual content, and therefore that it was not really
necessary to discuss i18n concerns in relation to the registry (no more
than it is necessary to have a section to discuss i18n issues in
relation to the IANA charset registry in RFC 2978).

In conclusion, I think that some of Bruce's concerns were valid, and
suggestions for changes have been presented to the authors accordingly.
I believe all of these changes can be considered to be for clarification
purposes, rather than technical changes. (No changes affecting the set
of valid tags have been made.)

Thanks.

Peter Constable

GIFT | GPTS | MICROSOFT

_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf

---

From: Vernon Schryver <[email-masked]>
Date: Mon, 13 Dec 2004 17:45:51 -0700 (MST)

> From: "Peter Constable" <[email-masked]>
> To: <[email-masked]>
> Cc: [email-masked]

> This is a multi-part message in MIME format.
>
> --===============1521567419==
> Content-class: urn:content-classes:message
> Content-Type: multipart/alternative;
> 	boundary="----_=_NextPart_001_01C4E16C.40BF0707"
>
> This is a multi-part message in MIME format.
>
> ------_=_NextPart_001_01C4E16C.40BF0707
> Content-Type: text/plain;
> 	charset="us-ascii"
> Content-Transfer-Encoding: quoted-printable
>
> =20
> ...

> ...

> ------_=_NextPart_001_01C4E16C.40BF0707
> Content-Type: text/html;
>       charset="us-ascii"
> Content-Transfer-Encoding: quoted-printable
>
> <html>
>
> <head>
> <meta http-equiv=3DContent-Type content=3D"text/html; =
> charset=3Dus-ascii">
> <meta name=3DGenerator content=3D"Microsoft Word 11 (filtered)">
> <style>
> <!--
>  /* Font Definitions */
>  @font-face
>       {font-family:Wingdings;
>       panose-1:5 0 0 0 0 0 0 0 0 0;}
> @font-face
>       {font-family:SimSun;
>       panose-1:2 1 6 0 3 1 1 1 1 1;}
> @font-face

On the contrary, what the authors of a standard intend is not normative.
As much as possible, every standard must say what it means, because
what a standard says *is* its technical content.  For example, I'm
unhappy about an apparent sentiment that would put ABNF on a lower
footing that the English text.  I think I'm like most implementors and
perhaps unlike non-engineers in reversing that precedence.  Whenever
I read an RFC, I rely first and foremost on the ABNF.  I use the English
only for hints, and follow the ABNF instead of the English whenever
there is a conflict.

There are a couple other issues that ought to be addressed.

I think Bruce Lilly started by charging that a potentially disruptive
document had reached last-call without any review by those concerned
with related, affected IETF standards.  That sounds like a process
problem that needs at least 1% as many words as have been spent in
this mailing list in lawyerly talk such as whether "accounts" is more
appropriate than "account."

The other issue is that some of us consider the completely unnecessary
and gratuitous use of duplicate-copy/quoted-printable/HTML email
somewhere among aggressive, offensive, and a security attack.  In
purely text contexts like this mailing list QP/HTML never contributes
to an impression of technical accuracy and relevance of whatever
message it enciphers.  Then there is the use of Microsoft's XML
flavor of HTML mail ...

Vernon Schryver    [email-masked]

---

From: Joe Abley <[email-masked]>
Date: Mon, 13 Dec 2004 20:50:51 -0500

> better to identify the format precisely rather than make the generic 

Why not require dates to be formatted as per RFC 3339?

In general, "YYYY-MM-DD" is ambiguous unless a timezone is specified.

Joe

---

From: John Cowan <[email-masked]>
Date: Tue, 14 Dec 2004 12:22:05 -0500

Vernon Schryver scripsit:

> what a standard says *is* its technical content.

Quite.

> For example, I'm

Then you would be incapable of implementing any programming language compiler,
or an XML parser, for the specs for these things include literally hundreds
of constraints that are specified only in technical English and not in the
BNF.  As far as the BNF is concerned, this is good sound C:

	main(argv, argc) {
		float Argv;
		int* Argc;
		print(32);
		}

-- 
John Cowan  [email-masked]  www.reutershealth.com  www.ccil.org/~cowan
"It's the old, old story.  Droid meets droid.  Droid becomes chameleon. 
Droid loses chameleon, chameleon becomes blob, droid gets blob back
again.  It's a classic tale."  --Kryten, Red Dwarf

---

From: "Addison Phillips [wM]" <[email-masked]>
Date: Tue, 14 Dec 2004 09:08:27 -0800

> only for hints, and follow the ABNF instead of the English whenever

The ABNF is not on a lower footing than the English text. But it is
dependent on the English text in exactly the same way that the ABNF in RFC
3066 was.

I think the suggestion to change the "grandfathered" production is a good
one and will help implementers who start with the ABNF.

I also think, though, that the establishment of a comprehensive (as opposed
to fractional) registry is the real salient point for implementers here. An
implementation of RFC 3066 that follows *only* the ABNF would happily
proliferate garbage tags like "c57-x", not just valid ones. The existence of
a registry in draft-langtags should focus implementer's attention on two
things: the ABNF and the subtags that fit into them. In that regard
draft-langtags will simplify the lives of implementers who do not read the
text (in the same way that having a registry for character encoding
names--"charsets"--does).
> appropriate than "account."

The IETF process is not really my concern. I will note that many IETF and
non-IETF standards folks have participated in the process of developing and
reviewing draft-langtags, though. I don't know if a wider audience should
have been invoked earlier in the process. Mark and I welcome comments and
questions on the technical suitability of our draft. I think that we have
fully and carefully considered the potential impact and, in fact, have
helped to stabilize language tags, not just now but for the future as well.

Peter made the argument that future I-D authors could write a draft that
does whatever they please with regard to language tags. Which is true.
However, draft-langtags lays down a framework that should guide the
activities of these authors and constrain the changes they make in a manner
that is completely compatible with implementations of draft-langtags (not to
mention RFC 3066 and RFC 1766). I think that a guarantee of future
stability---in implementations (including current ones), extensions, and the
tags (data) themselves---is of great benefit to related and/or affected IETF
standards.

Best Regards,

Addison

Addison P. Phillips
Director, Globalization Architecture
http://www.webMethods.com

Chair, W3C Internationalization Working Group
http://www.w3.org/International

Internationalization is an architecture.
It is not a feature.

---

From: "Addison Phillips [wM]" <[email-masked]>
Date: Tue, 14 Dec 2004 09:28:18 -0800

Mark and I have both worked extensively with time zone issues, so we're aware of the potential problems.

RFC 3339 would be an appropriate substitute: its "full-date" production describes the ISO 8601 profile used by the draft.

I would also tend to agree that lack of a timezone would be ambiguous in most applications. However, for this use I think that:

  a) the dates indicate the date of accession of each subtag to the registry. These dates will all be in the past. Since the registry itself is versioned and has its own date record, the question of time zone is probably not important because implementations will use their registry date and not an arbitrary date to determine compatibility. That is: the dates will all be used in the same context with one another.

  b) we can safely assume (or explicitly state) the use of UTC time based on the above.

Best Regards,

Addison

Addison P. Phillips

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]]On Behalf Of Joe Abley
> Sent: 2004年12月13日 17:51
> To: Peter Constable
> Cc: [email-masked]; [email-masked]
> Subject: Re: New Last Call: 'Tags for Identifying Languages' to BCP
> 
> 
> 
> Ietf-languages mailing list
> [email-masked]
> http://www.alvestrand.no/mailman/listinfo/ietf-languages

---

From: Vernon Schryver <[email-masked]>
Date: Tue, 14 Dec 2004 12:04:07 -0700 (MST)

> From: John Cowan

> BNF.  As far as the BNF is concerned, this is good sound C:

In contexts other than UNIX applications with modern compilers,
that fragment is perfectly sound, if not something I'd write.  An
example context is before typing of formal args and in what ANSI/ISO
9899-1990 calls a "freestanding environment" where main() is not
special.  I've suppressed most of the memories, but I seem to recall
that what Microsoft calls threaded WIN32 applications are such
things, or were before the POSIX additions.

Besides, I didn't say that one should ignore the English, but that
implementors give precedence to the ABNF.  When you are writing an RFC
that you hope will be implemented, you MUST remember that programmers
are lazy.  We transliterate the ABNF to build the parser and so implement
the syntax and read the English to figure out and so build the semantics.
As I said, if you must have contradictions between your ABNF and your
English, you must accept the fact that most technical people will
assume your ABNF is right and your English is wrong.  That fact seemed
to me to conflict with statements in this thread, and that suggests a
problem in your working group and your RFC.