Subject: Re: Reviving a dormant IANA registry

From: Florian Weimer <[email-masked]>
Date: Thu, 03 Sep 2020 21:20:28 +0200

* John C. Klensin:

> --On Wednesday, August 26, 2020 20:33 +0200 Florian Weimer
> <[email-masked]> wrote:
>
>>...
>>> If the charset name code you are interested in registering is
>>> defined in an RFC, you are presumably want the first, but it
>>> would be helpful for you to confirm that.
>> 
>> The charset is currently unnamed, as far as I can see: it's
>> the UTF-7 variant in section 5.1.3 of RFC 3501.
>
> I have not been following IMAP work carefully in recent years
> (others here may be able to easily fill in the blanks) but my
> impression is that, as Unicode encoded in UTF-8 has taken over
> as the generally preferred form for transmission of non-ASCII
> characters over the Internet, UTF-7 has generally fallen out of
> use even if it has not been explicitly deprecated.  In
> particular, its use is strongly discouraged in fully
> internationalized email contexts ... see the discussion in  RFC
> 6855 for IMAP and RFC 6530ff for the more general case.

I see.  Is RFC 6855 widely implemented?  The IMAP server at work does
not seem to offer the UTF8=ACCEPT capability, if I read the debug
output from the mail client correctly.

<https://imapwiki.org/Specs> does not list any compatible
implementations, if I read that table correctly.

>> The charset is already defined and widely implemented.  It's a
>> transformation format of Unicode.
>
> But a transformation form that was invented in the IETF, more or
> less specifically for one particular element of IMAP, not one
> standardized or recognized by the Unicode Consortium.   That
> might be hair-splitting except for the problem below.
>
>>  It's just presently unnamed.
>> Having a standardized name for it would help implementing
>> support in the POSIX iconv framework.
>
> Ah.  Up to you -- and something to be worked out with the
> designated experts and on the appropriate lists (once it is
> sorted out) -- but some free (and maybe worth what you pay for
> it) advice: you'd get much better support and interoperability
> by dropping UTF-7 and converting the relevant applications to
> use UTF-8.  That is particularly important because there are
> apparently several web applications and form processors that
> will ignore the charset parameter (either entirely or with any
> values they don't recognize) and try to deduce the charset or
> encoding form heuristically.  And they will not accurately
> detect UTF-7 for reasons that should be obvious.

It seems to me that we need this UTF-7 variant for a few years to
come.  Implementations already have come up with ad-hoc names such as
"utf-7-imap" or "mUTF-7".

Telling implementors to use UTF-8 instead, when it does not provide
interoperability with existing software, does not seem particularly
useful to me.

---

From: John C Klensin <[email-masked]>
Date: Thu, 03 Sep 2020 17:02:04 -0400

--On Thursday, September 3, 2020 21:20 +0200 Florian Weimer
<[email-masked]> wrote:

>>>> If the charset name code you are interested in registering
>>>> is defined in an RFC, you are presumably want the first,
>>>> but it would be helpful for you to confirm that.
>>> 
>> characters over the Internet, UTF-7 has generally fallen out
>> of use even if it has not been explicitly deprecated.  In
>> internationalized email contexts ... see the discussion in
>> RFC 6855 for IMAP and RFC 6530ff for the more general case.
> 
> I see.  Is RFC 6855 widely implemented?

In places (Ned and others may have more to say about this).  One
on the problems (somewhat discussed in RFCs 6857 and 6858) is
that, if an SMTP delivery server and associated IMAP server are
going to be dealing with clients that have not been upgraded,
there may be perceived advantages in either not implementing
that option, leaving it turned off, or not even accepting
SMTPTUF8 at the SMTP server.

>  The IMAP server at
> work does not seem to offer the UTF8=ACCEPT capability, if I
> read the debug output from the mail client correctly.

>>> The charset is already defined and widely implemented.  It's
>>> a transformation format of Unicode.
>> 
>> But a transformation form that was invented in the IETF, more
>> or less specifically for one particular element of IMAP, not
>> one standardized or recognized by the Unicode Consortium.
>> That might be hair-splitting except for the problem below.

 
> It seems to me that we need this UTF-7 variant for a few years
> to come.  Implementations already have come up with ad-hoc
> names such as "utf-7-imap" or "mUTF-7".
 
> Telling implementors to use UTF-8 instead, when it does not
> provide interoperability with existing software, does not seem
> particularly useful to me.

Much as I might wish otherwise, I tend to agree.

best,
   john

---

From: "John Levine" <[email-masked]>
Date: 3 Sep 2020 22:40:04 -0000

In article <[email-masked]>,

>output from the mail client correctly.

At this point very few. The Courier open source mail package does, and
the Coremail service in China. The popular Dovecot and Cyrus IMAP servers do not.
Gmail and Hotmail/Outlook support SMTPUTF8 on the SMTP side but not
for POP or IMAP.

>"utf-7-imap" or "mUTF-7".

Agreed.

>interoperability with existing software, does not seem particularly

I've talked to some IMAP developers who say that yeah, going to UTF-8
would be great but cleaning up a decade of old character set cruft will
be a lot of work.

-- 
Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly