Subject: [[email-masked]: Last Call: UTF-16, an encoding of ISO 10646 to Proposed]

From: [email-masked]
Date: Mon, 13 Sep 1999 01:38:42 +0200

> The IESG has received a request to consider UTF-16, an encoding of ISO
> 10646 <draft-hoffman-utf16-04.txt> as Proposed Standard. This has
> been reviewed in the IETF but is not the product of an IETF Working
> Group.
> 
> The IESG plans to make a decision in the next few weeks, and solicits
> final comments on this action.  Please send any comments to the
> [email-masked] or [email-masked] mailing lists by September 13, 1999.
> 
> Files can be obtained via
> http://www.ietf.org/internet-drafts/draft-hoffman-utf16-04.txt

Here is my response to last call on UTF-16 draft

I appreciate the draft and I agree to the purpose of
registering the MIME-charsets UTF-16 UTF-16LE and UTF-16BE.

But I have some objections to publishing it as an RFC in its
current form.

1. The RFC should not be standards track, but rather an informative RFC.
IETF should not define character sets normatively, and I believe
this was actually set as an IAB policy some years ago.

2. Instead RFCs on charset should refer to the normative
specifications defining the charsets, and to refer to 
ISO specifications whenever possible. The draft gives preference
to an industry standard (Unicode) where the ISO standard (10646)
gives the same specification, and should be preferred over the industry
standard. This would be in line with how other charsets are defined,
for example the ISO 8859 series, where the preceding and technically
equivalent ECMA standards are not referenced. I think it is appropiate,
though, to mention the Unicode standard the first time the ISO
standard is introduced. The charsets should be defined with reference
to the ISO standard, and not do its own definition. An explanation
(but not definition) of the standard could be appropiate.

3. Referencing the ISO standard would also mean that ISO terminology
should be used in the draft. In the current draft a mixture of
ISO and Unicode terminology, together with newly invented terminology
is used, furthering the confusion about charset concepts.

4. The draft mentions UTF-8 being variable width, and does not mention
that UTF-16 is also variable-width. I think it woul be prudent to
also mention that UTF16 is variable width, the reader could get
the impression that UTF-16 is not variable width.

5. ISO/IEC 19646 has recently adapted more than 12 amendments,
so the references section may be updated in this respect.
But anyway that note will be inaccurate when new amendments are
approved.

I am willing to assist to reword the draft along the lines
stipulated above.

Best regards
Keld Simonsen

---

From: Ned Freed <[email-masked]>
Date: Sun, 12 Sep 1999 18:04:22 -0700 (PDT)

> this was actually set as an IAB policy some years ago.

There are two independent issues here: Whether the IETF should define charsets
normatively and whether or not charset registration RFCs should be on the
standards track.

As to the first issue, a clear consensus emerged from the discussions
that led to RFC 2279 that the IETF has no choice but to make such
normative definitions. I see no material diference in the situation
surrounding UTF-16 that suggests that the alternative Keld suggests is viable.

Specifically, the approach taken in RFC 2279 is to define UTF-8 as aligned
with ISO 10646 plus any published amendments unless an amendment appears
that isn't backwards compatible. I believe this is the approach to take
with UTF-16 as well.

Now, insofar as standards track status is concerned, again we see that a
precedent has been established for this by RFC 2279, which is standards track.
I see no reason to deviate from this approach.

> (but not definition) of the standard could be appropiate.

I agree with the principle that ISO specifications should be referenced when
possible (in keeping with RFC 2279). However, I'm not sure I see a case in the
current document where Unicode is referenced preferentially over ISO 10646. I'd
like to see specific examples of something that needs changing here.

> is used, furthering the confusion about charset concepts.

The implicit assumption here is that ISO terminology is consistent. It isn't.
Nor do I think that ISO terminology is always better, or less confusing. It is
neither.

Again I suggest that specific examples be given of problematic text and how it
should be changed. I am much more comfortable with specific examples than I am
with arguments of the form "ISO good, Unicode bad".

> the impression that UTF-16 is not variable width.

I agree that this should be made explicit in the document -- the implication
currently is that UTF-16 is fixed width, and it isn't.

> But anyway that note will be inaccurate when new amendments are

The RFC 2279 approach should be used here, where the specification
automatically aligns to admendments as long as they are backwards compatible.
However, I do agree that the references should list all amendments that have
been made thus far.

				Ned

---

From: Ned Freed <[email-masked]>
Date: Sun, 12 Sep 1999 18:04:22 -0700 (PDT)

Ned

---

From: John Stracke <[email-masked]>
Date: Mon, 13 Sep 1999 12:19:08 -0400

> However, I do agree that the references should list all amendments that have

It occurs to me that there could be a risk that UTF-16 and UTF-8 could get out of
sync in terms of what amendments get incorporated, which would be a headache.
(For example, if the UTF-16 RFC does copy-and-paste from 2279, and then 2279 gets
updated, UTF-16 might have to be updated at the same time.) Perhaps UTF-16 could
reference UTF-8, rather than referencing ISO-10646 directly?

--
/==============================================================\
|John Stracke    | http://www.ecal.com |My opinions are my own.|
|Chief Scientist |=============================================|
|eCal Corp.      |I have strong opinions about ambivalence.    |
[email-masked]|                                             |
\==============================================================/

---

From: John Stracke <[email-masked]>
Date: Mon, 13 Sep 1999 12:19:08 -0400

---

From: [email-masked]
Date: Mon, 13 Sep 1999 12:26:05 -0400

> this was actually set as an IAB policy some years ago.

This sounds reasonable, but...

> ISO specifications whenever possible. The draft gives preference

Now, I have *no* idea what the *actual* status of the corresponding ISO
documents is, but while  doing some research, I found
this text in RFC2026, sections 7.1.[12]:

7.1.1  Incorporation of an Open Standard

   An Internet Standard TS or AS may incorporate an open external
   standard by reference.  For example, many Internet Standards
   incorporate by reference the ANSI standard character set "ASCII" [2].
   Whenever possible, the referenced specification shall be available
   online.

7.1.2  Incorporation of Other Specifications

   Other proprietary specifications may be incorporated by reference to
   a version of the specification as long as the proprietor meets the
   requirements of section 10.  If the other proprietary specification
   is not widely and readily available, the IESG may request that it be
   published as an Informational RFC.

Section 10.3.3 then goes on to talk about "reasonable and non-discriminatory"
terms.

What similar checks are there that section 7.1.1 "open standards" are
easily accessible?  I ask because many standards organizations have a
reputation (deserved or not) of requiring the exchange of money for a
copy of the standard.  For instance, I've often been at a loss when wanting
to cite the ANSI standard for C chapter-and-verse, or *exactly* what
the Posix standard says (as opposed to what my vendor's documentation
claims it says, as usually I'm arguing with my vendor ;)

It looks like the IETF makes it own checks for 7.1.2, but takes the word
of the other standards body for 7.1.1 - is this correct?  Or is this
something for a case-by-case discussion during the Last Call?

-- 
				Valdis Kletnieks
				Computer Systems Senior Engineer
				Virginia Tech

---

From: [email-masked]
Date: Mon, 13 Sep 1999 12:26:05 -0400

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Tue, 14 Sep 1999 01:31:24 +0200

>    Whenever possible, the referenced specification shall be available

......

>claims it says, as usually I'm arguing with my vendor ;)

We gave up on "open" having to mean "free" a *very* long time ago.
We've even given up on "online" meaning "free" - see ITU.

"open" means something like (IMHO):

- the content is accessible to all in some fashion (which may involve
   tossing money at the problem); everyone gets the same content
- the procedure for changing it is known, and administered by some body that
   is regarded as "neutral" under some (sometimes contorted) meaning
   of that term

that's how far we've gotten in this world, methinks.

this horse has made a career out of being dead.

                     Harald

--
Harald Tveit Alvestrand, Maxware, Norway
[email-masked]

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Tue, 14 Sep 1999 01:31:24 +0200

......

                     Harald

---

From: Dave Crocker <[email-masked]>
Date: Mon, 13 Sep 1999 18:35:08 -0700

>this horse has made a career out of being dead.

and in some other venues, people are starting to make a career out of 
burying it.

in fact, it is looking like a growth industry...

d/

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Dave Crocker                                         Tel: +1 408 246 8253
Brandenburg Consulting                               Fax: +1 408 273 6464
675 Spruce Drive                             <http://www.brandenburg.com>
Sunnyvale, CA 94086 USA                 <mailto:[email-masked]>

---

From: Dave Crocker <[email-masked]>
Date: Mon, 13 Sep 1999 18:35:08 -0700

d/

---

From: [email-masked]
Date: Tue, 14 Sep 1999 11:40:14 -0400

> We've even given up on "online" meaning "free" - see ITU.

A bad choice of examples on my part (although recent mail from Francois Yergeau 
indicates that cost *is* an issue:

  Given the price and accessibility of ISO standard, and especially of 10646,
  it is a fact that the Unicode terminology is much more widely known and

>    tossing money at the problem); everyone gets the same content

There's an implicit assumtion here that the standard that you throw money
at is (a) functional and (b) still aligned with what the RFC intended.
Given that some standards (such as ISO-3166) are by definition moving
targets, this is a potential problem...

>    is regarded as "neutral" under some (sometimes contorted) meaning

Read RFC1829, with its normative references to FIPS-46.
Now ponder the implications if the US Government gets even more
silly in its stance regarding the export of cryptographic information.

The point I made badly was that RFC2026 makes no discussion of what
to do if a section 7.1.1 "standards body" becomes unable, for whatever
reason, to act in a "neutral" manner, *or* manages to break a standard
that we've included as a normative reference.

This isn't purely hypothetical - we've already seen allegations that
ISO3166 has been used as a political tool for not recognizing new
countries.

Seen in RFC1101 (April 1989):

   Note that "YP" is NOT a valid country code under [ISO 3166] (although
   we may want to worry about the future), and the existence of a

I think if RFC2026 is ever overhauled, section 7.1.1 needs some more
text regarding what to do if normative reference is made to an external
standard that may in some way become broken over time.  Possible solutions
include requiring citing specific releases/modifications/versions,
or requiring a discussion of what to do if a moving-target standard
such as 3166 or one of the charset standards ecomes broken.
-- 
				Valdis Kletnieks
				Computer Systems Senior Engineer
				Virginia Tech

---

From: [email-masked]
Date: Tue, 14 Sep 1999 11:40:14 -0400

---

From: John Day <[email-masked]>
Date: Tue, 14 Sep 1999 13:26:15 -0400

>> We've even given up on "online" meaning "free" - see ITU.

The real problem of the cost of standards is that ITU and ISO are still
stuck in an early 20th C mode which perceived standards as a premium item.
A company only needed to have one or two copies of any standard in the
corporate library for someone to be able to refer to infrequently to ensure
that the latest molds for screw threads met the standard.  Even for
telephone equipment there was not much demand for the kind of frequent
reference the development current systems require.

Both ITU and ISO have failed to understand that standards of this ilk are
*commodity* items and that they would make far more money by charging far
less for them.  IEEE figured this out.  You can buy IEEE standards at more
or less text book prices and have a nice bound book on your shelf you can
refer to.  Well within reason.  But to charge $100 or more for a xerox copy
is ludicrous.  We tried to make this argument to them 15 years ago but they
were just too dense to get it.  There are still areas of standardization
(not in anything having to do with our industry) where standards are still
premium product and perhaps should be treated that way.  But ITU and ISO
could probably make up the difference by greatly lowering their price and
selling far more copies.

Worse, they don't understand marketing.  A couple of years ago, an ITU
person asked me if I thought $1000 for all ITU recommendations on CD was
reasonable.  I told him that I thought companies might by one.  However, if
you put all ISDN or ATM or whatever on CD and charged $50.  You would sell
thousands.  (Not that *I* would buy them!)  Needless to say, they didn't do
it.

Basically, I don't necessarily see charging for a copy of the standard to
be a barrier to openness. However, the kinds of prices ISO and ITU charge
reflect a different time and a different midset to what would make sense
today.

But then that is also what you expect.

---

From: John Day <[email-masked]>
Date: Tue, 14 Sep 1999 13:26:15 -0400