Subject: Re: New Last Call: 'Tags for Identifying Languages' to BCP

From: Bruce Lilly <[email-masked]>
Date: Sun, 12 Dec 2004 22:40:20 -0500

>  Date: 2004-12-12 19:20
>  From: Mark Crispin <[email-masked]>
>  To: [email-masked], [email-masked]
>  
> On Sun, 12 Dec 2004, Bruce Lilly wrote:
> > If by international agreement, 'yz' becomes the designation
> > for that country, then it is rather silly to stick one's
> > fingers in one's ears and shout "NA-NA-NA-NA-NA I don't want
> > to hear you".
> 
> What is silly is saying that every language tag has to have a date/time 
> attribute associated with it so that computer software managing that text 
> knows the language of that text.

In the specific cases of the core Internet protocols that
I have mentioned, there *is* a date/time attribute in the
form of an RFC [2]822 Date field.  If we're talking about
some file stored on some machine, every OS that I know of
has a date/time stamp associated with that file.  If you
have something else in mind, a concrete description and/
or example might help.
 
> It is a disaster for language identifiers to get recycled.  Something has 
> to make those identifiers unique.  Your notion will force the inclusion of 
> a date/time stamp in language tags, to restore the uniqueness that you are 
> so excruciatingly eager to abolish.

I'm not "eager to abolish" "uniqueness".  There never was
any guarantee that codes would never change. Both RFCs
1766 and 3066 specifically mention changes as a fact of
life.
 
> > Never
> > mind the shortcomings of that particular example; consider
> > "de-DE" -- does that mean Germany as it exists today, West
> > Germany as it existed 25 years ago, Germany as it existed
> > in the 1930s, the 1900s, ...?
> 
> For the 98% case, it does not matter at all.
> 
> But it does matter if, one day, "DE" becomes Denmark.

In either case, to understand precisely what geographical
area is referred to requires knowing the date to more or
less degree of accuracy.
 
> > As far as I can tell, the draft pretends that the meaning
> > of "CS" hasn't changed, and would in fact change the meaning
> > of the currently valid RFC 3066 language tag "sr-CS".
> 
> No, it restores the previous meaning of sr-CS.

But what of the current meaning under the current
standard (RFC 3066 + ISO 639 + ISO 3166)?  Surely
the draft would change the meaning of that valid
RFC 3066 language-tag.

> > It is very different; under the proposed draft, there is only
> > an English definition, somebody wishing to provide a French
> > definition finds that he has none and must resort to an
> > unofficial translation.
> 
> Why is the situation for French different from someobody wishing to 
> provide a Lower Slobbobian definition?

French is an official language used by the ISO in its
publications.  "Lower Slobbobian" is probably about as
meaningful as "BLURDYBOOP".
 
> > SO where are the French definitions?
> 
> Ask a person who is bilingual in English and French to provide one.

That would lack definitiveness which characterizes the
ISO lists.
 
> > Well, sure. But the name is an important thing by itself.
> > It is rather pointless to ask a user to indicate the
> > language of a piece of text by selecting from a list "AB, ACE,
> > ACH,..., ZHA, ZUL, ZUN" -- the user doesn't normally refer to
> > languages by codes. It's quite a different matter to ask the
> > user to select from "Abkhaze, Aceh, Acoli,..., Zhuang (Chuang),
> > Zoulou, Zuni".
> 
> Abkhaze, Aceh, Acoli,..., Zhuang (Chuang), Zoulou, and Zuni are not 
> language tags.  So what's your point?

They are the human-readable names corresponding to codes.
For interoperability, it is insufficient to label any and
all languages as "ZZ" with no definition of what "ZZ"
means. Moreover, it is necessary for two (or more) communicating 
parties to *agree* on the meaning of "ZZ"; that is done
by assigning the code "ZZ" to an agreed-upon name.  The
code "ZZ" is nothing more than shorthand for that agreed-upon
name.  If one produces some text in the BCP 18 sense of "text"
(spoken, written, signed, etc.), it is useful to indicate
the language of that text; languages are known to humans by
names of languages -- the codes are, as noted, merely
shorthand for those names.  Likewise, somebody presented
with some text may desire or need to know the language of
that text; informing that person that the language has code
"QZ" is unlikely to mean anything to most people -- only
the name corresponding to the shorthand code is likely to
be meaningful to persons other than those involved in
standardizing the codes.

> >> Note that the RFC 3066 specifies a registry that does not include French
> >> language names. I suggest that this issue should be dropped.
> > Yes, the current IANA registry has that problem for
> > the non-ISO-based tags only. If the registry is to be
> > changed to subsume ISO codes as well, that defect should
> > be remedied.
> 
> Why is it a problem?  Why is it a defect?

Because it unnecessarily reduces by 50% the information
content currently available.
 
> > On the contrary, it is preposterous to suggest that codes
> > will be attached to text by magic
> 
> Here is where you are misled.  Many of these tags are embedded within the 
> text itself.  That text may long outlive its author in an archive.

Which is precisely why the code by itself is meaningless
without the associated language name.  If I write "blurfl
(lang=QZ)" in a hypothetical diary, that will be
incomprehensible unless the meaning of "QZ" is known.

You have not explained how the code came to be "embedded
within the text itself" -- surely the author didn't say
(or write, or sign) "this text is in language QZ"; most
likely the language was indicated by name, or by some proxy
representing the name (such as a locale).

_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf

---

From: Bruce Lilly <[email-masked]>
Date: Mon, 13 Dec 2004 00:14:03 -0500

>  Date: 2004-12-12 20:57
>  From: "Peter Constable" <[email-masked]>
> > From: [email-masked] [mailto:ietf-languages-
> > [email-masked]] On Behalf Of Bruce Lilly
> 
> 
> > > That is not at all the aim here wrt stability; rather, the aim is
> that a
> > > symbolic identifier used for metadata in IT systems not change
> because
> > > some government on a whim says, "We would now prefer to use 'yz'
> rather
> > > than 'xy' to designate our country."
> > 
> That misses the point entirely. The point is that IDs used by political
> administrations may change for any number of reasons, and those
> admministrations may have no qualms with such changes;

For such changes to become enshrined in an ISO standard
requires a bit more than a mere whim on the part of one
party; in the case of the particular ISO standards under
discussion, it requires convincing the duly appointed
maintenance authority to make the change.

> but in IT 
> systems, we cannot afford changes that break existing implementations
> and data.

Any implementations that depend on country/language codes
never changing are by definition broken implementations,
since there was never any guarantee that codes would never
change.  Change happens, and IT knows how to cope; it's a
versioning problem, and that's not a particularly difficult
problem.  Now I fully agree that in hindsight the ISO and
its appointed MAs could have provided a better record of
changes.

> If for whatever reason ISO and the UN decided that "US" should 
> be used to designate the country of France, I doubt you'd expect every
> software vendor to update all of their deployed installations to use
> "fr-US" instead of "fr-FR", and for every user to go through every data
> repository they manage to make such changes in their data.

The only way that would be likely to happen would be if
there were no longer a "US" *and* if the ISO and UN
representatives of France were to initiate a request for
such a change.  One would presume that they would have
good reason to do so, and could explain said reasons in
order to convince their ISO and UN counterparts to agree
to the change.  Under those hypothetical circumstances, I
can only assume that software vendors who care about such
matters would either agree with the hypothetical reasons
or would have acted to convince those in favor of the change
of reasons to avoid the change.  And while I would not
expect users to retroactively change documents any more than
I would expect coins and paper money to be reissued with old
dates but new designations of country name, I would expect
that as of the agreed-upon effective date of the change that
new documents would be prepared in accordance with the new
standard.  It's difficult to be more precise about such a
wild hypothetical, but consider similar changes made to
time zones...

> The people that maintain time zone definitions may have their means for
> changing times; that's fine for them. They are not dealing with the same
> concerns as we are dealing with.

Sure they are; it's another instance of the same sort of
versioning problem, with the same root causes, viz. items
which are changed (more frequently than some would like)
by politicians.

> The group here that has focused 
> specifically on language-tagging issues for several years has evaluated
> issues that affect language tags and the impact of changes and has
> decided what is best practice for *this* domain, and it is to maintain
> stability of data rather than cater to whims of political
> administrations.

Now that the horses have all run away, you'd better make
sure the stable doors are locked. :-)  There was never
any guarantee of stability of country codes or of language
codes.  Declaring at some time in the future that today's
meaning of sr-CS never meant what it in fact does mean
doesn't create stability; it creates instability -- it
doesn't make the versioning problem go away; it adds yet
a third version to the existing two.
 
> > "Designed" or not, country codes *are* read by humans; they
> > appear in top-level domain names.  Currently the ISO 639
> > 2-letter codes mean the same thing as the last component of
> > a domain name
> 
> I think you mean ISO 3166 2-letter codes.

Yes, my error.
 
> > and as the second component of a language-tag.
> > It's rather silly to change that correspondence simply because
> > a few people are piqued that international agreement has been
> > reached to change a few 2-letter codes.
> 
> The usability flaw in treating ISO 639 and ISO 3166 as human-readable is
> evident in the confusion between ja and JP (or is it jp and JA?), and GB
> vs UK.

Without looking I can easily tell that jp and uk are country
codes precisely *because* they are well-known as TLDs.

> As for what is silly, if the UN country ID for Canada changed to 
> CN (and that for PRC changed to something else), I'm sure it would cause
> far greater problems for users to have to change the last two letters in
> domain names than for them to keep doing what they always did.

And it is precisely because of such problems that it is
as unlikely to happen as your hypothetical FR->US change.

> In fact, 
> I would have thought it would create a rather significant problem on the
> Internet if such a change were made. (URIs don't come with versioning
> dates for domain names, so how would a DNS server know what the "cn"
> meant?)

URIs aren't guaranteed to be persistent; the Foo company buys
the Bar company and after a while bar.com URIs stop working
because they've been changed to foo.com URIs.  That sort of
thing happens all the time, and people adapt.

> > > Neither RFC 1766 or RFC 3066 has ever presented "official"
> translations;
> > 
> > Both defer to the ISO lists for definitions (not "translations")
> > of the various codes.
> 
> Definitions; not language names for display use.

Feh. Whatever. The human-readable stuff that corresponds
to the code which you say shouldn't be read.   The stuff
without which codes are meaningless.  The stuff without
which two communicating parties cannot agree on the meaning
of "XX".

> > > this is no different for RFC 3066bis.
> > 
> The more you press this, the more silly it seems. RFC 3066 does not
> anywhere discuss display names

If I have used the term "display" in conjunction with the
language/country names it is only incidental.  Effective
communication, including language tagging, is an end-to-end
process (RFC 1958).  Without agreement on what a code *means*
there is no effective communication.  And the ISO lists
provide the definitions that attach meaning to otherwise
meaningless combinations of letters.  The stuff that the
humans at each end of the communications channel use to
identify the language.

> The source ISO standards are every bit as accessible as they ever
> were, and just as RFC 3066 gave the user no option but to refer to the
> source ISO standard, so users should and can continue to do so.

So, you're saying that the ISO definition of "CS" as
"Serbia and Montenegro" will continue to be valid, with
that meaning, in a language-tag?
 
> After this response, I will not waste my time any further on this
> foolishness.

The foolishness is your insistence on trying to tie
the definitions to a localization issue.  While definitions
in some language can be localized, the definitions can
also be used directly, and indeed must be used in some form
into to make sense of the shorthand codes that represent
the definitions.  The issue isn't about localization at all;
it's about having continued accessibility to the definitions
in at least as many languages as is currently the case.

> > I'm willing to postpone the discussion
> > (other problems with the proposed registry format dictate
> > a broader solution which could easily have provision for
> > an arbitrary number of descriptions).
> 
> I strongly object to the suggestion that progress on this draft be
> delayed to deal with this non issue that caters to implementation issues
> that are well beyond the scope of either RFC 3066 or its proposed
> replacement.

Your characterization of the issue is inaccurate.  No matter;
let's deal instead with the issue of retaining the current
meaning of sr-CS (Serbian as used in Serbia and Montenegro).
How does the draft proposal retain the meaning of that valid
RFC 3066/ISO 639/ISO 3166 language tag?

> > No, you are overlooking the fact that a set of codes with
> > no corresponding definitions is useless.  RFC 3066 defers
> > the code/definition pairs to ISO, which provides multilingual
> > definitions. The proposed draft would remove that multilingual
> > characteristic.
> 
> What if the registry provide no name, just the ID? Then people would
> have to refer to the source ISO standard as they did in the past, and we
> would be able specify which ISO IDs were or were not valid.

You'd still have a versioning problem because of the multiple
meanings of "CS". Trying to sweep any of those multiple
meanings under the rug (by declaring them "not valid") isn't
going to make them go away; what about existing data using the
current meanings? I doubt you'd expect every software vendor...
etc. The only way to solve a versioning problem is with an
appropriate versioning solution.

> > > Display names for languages and countries are not within the scope
> of
> > > RFC 1766 or RFC 3066. It is preposterous to suggest that this draft
> is
> > > not compatible with existing implementations of RFC 3066 on that
> basis.
> > 
> > will be attached to text by magic; some human somewhere,
> > somehow is going to have to indicate the language to
> > something, and it certainly isn't going to be by way of
> > a 2- or 3-letter code without some reference to what those
> > codes *mean*.  And at the present time, the meaning of
> > those codes is defined -- bilingually -- in the ISO
> > lists.
> 
> RFC 3066 did not even discuss let alone provide a means for attaching
> display text to codes. It *is* preposterous to suggest that this draft
> is incompatible with RFC 3066 on that basis. Again, the more you press
> this, the more silly it seems.

I haven't specifically discussed "display names"; that is your
assertion, and not my basis for objection.  I refer to the
definitions and the need to map to and from those definitions
at either end of the communications channel.  Whether or not
that happens by "display" is incidental to the issue of the
number of languages that the definitions are provided in.

> > No, I am complaining about removal of internationalized
> > definitions associated with language tag components.
> 
> No definitions are removed. The draft points to the source ISO standards
> just as RFC 3066 does.

So, again I ask for confirmation; the ISO definition of "CS"
as "Serbie et Monténégro" will not be removed from use with
that meaning in language tags by the draft proposal?

> > "Localization" would be translation of the French definition
> > into some other language.  That is not my concern. My concern
> > is the elimination of the French definition in the first place.
> 
> No, you have not commented on definitions; you have repeatedly commented
> on stings to present to users.

No, presentation is incidental.  I have repeatedly referred to
the definitions, and the draft proposal's effect of removing
50% of those definitions.

> Please accept that your arguments on this 
> matter are empty.

Please accept that your empty caricature is not the same as my
argument.
 
> > As mentioned, under RFC 1766/3066 review/registration rules,
> > excessively long tags would certainly raise objections. That's
> > no coincidence -- it's an intentional design feature.
> 
> But excessive is not defined anywhere in RFC 1766/3066, and if there was
> a very good reason presented why a tag of x characters long were needed,
> it would have to be considered.

Considered, yes, of course.  But not rubber-stamp approved.
And if not approved, not usable.  And unlikely to be approved
if incompatible with core Internet protocols.

---

From: Keld Jørn Simonsen <[email-masked]>
Date: Mon, 13 Dec 2004 14:20:45 +0100

On Mon, Dec 13, 2004 at 01:37:04AM -0800, Mark Crispin wrote:
> 
> When I retrieve a file via FTP, HTTP, etc. the time stamp of that file on 
> my computer is the date/time of retrieval, not the date/time of the file 
> on the source.
> 
> Unless, of course, both systems are running TOPS-20 and thus use that 
> wonderful XTP mode that copies file metadata.  Now, if you want to mandate 
> that all UNIX and Windows systems be replaced with TOPS-20, I might 
> support that... :-)

Actually some ftp and http transfer programs, incl wget and ncftp keep
the original date stamp.

With Xmas greetings
Keld

---

From: John Cowan <[email-masked]>
Date: Mon, 13 Dec 2004 08:44:43 -0500

> which two communicating parties cannot agree on the meaning

Two communicating parties can unquestionably agree on the meaning of
"XX" without both English and French definitions.  Either will suffice.
Indeed, if either definition provided a nuance not available to the
other, they would not be interchangeable, and one would have to be
the authentic definition and the other a mere aide-memoire.

-- 
[W]hen I wrote it I was more than a little              John Cowan
febrile with foodpoisoning from an antique carrot       [email-masked]
that I foolishly ate out of an illjudged faith          www.ccil.org/~cowan
in the benignancy of vegetables.  --And Rosta           www.reutershealth.com

---

From: Mark Crispin <[email-masked]>
Date: Mon, 13 Dec 2004 01:37:04 -0800 (PST)

Silliness aside, the file may well have embedded language tags in the text 
of the file.  Have you forgotten Plane 14?

> 1766 and 3066 specifically mention changes as a fact of

That's what's now being fixed.

> publications.

Why is this vestige of colonialism important in the IETF context?

> That would lack definitiveness which characterizes the

What magic attribute is there to French that provides "definitiveness" 
that is absent in English, or Mandarin, or Hindi, all of which are far 
more significant languages to the world?

> content currently available.

A mandatory French translation to an English definition does not 
significantly increase the information content, and certainly does not 
double it.

The only increase in the information content would be to those individuals 
who comprehend French but not English.  This is a very small number of 
individuals.

If there is to be a mandatory translation into a second language to 
increase information content, then that language should be Mandarin. 
Among individuals who do not comprehend English, far more comprehend 
Mandarin than comprehend French.

If there is to be a mandatory translation into a third language, that 
would probably be Hindi.

> representing the name (such as a locale).

Plane 14.

HTML and other markups.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.