Subject: What do we mean when we standardize something?

From: John C Klensin <[email-masked]>
Date: Wed, 29 May 2013 13:23:51 -0400

Hi.  A number of recent discussions, specifically including the
Last Calls on DKIM and standardizing RRTYPEs and even to some
extent the meeting location ones, have started me thinking that
perhaps we need to review what business the IETF is actually in
and to be sure that we actually still agree about it.  The note
that follows is an attempt to take the isolated parts (and
symptoms) of that discussion up a level and discuss it as a
general issue.

The key issues go to the very nature of standardization.  While
there are many variations on each, there are two fundamentally
different models for what standards are about.  Each is
legitimate in its own way and each has its advocates, but they
are different.  One is a focus on quality: an engineering
consensus on the best technical solution given physics or
consensus goals and boundary conditions.  The other is
sometimes called "consensus of industry practice" in which a
standard reflects what is already deployed, possibly picking
among different implemented options in a few cases but mostly
requiring that a common model appears before standards are
issued.  

Two variations on the second theme were extremely common in the
earlier days of computer and telecommunciations standardization
but are less popular today (at least in theory) due to IPR and
antitrust concerns.  Both start with a single vendor (or closed
consortium) implementation.  That implementation might then be
offered for standardization (sometimes with a "no major
changes" restriction) and adopted more generally or endorsed by
traditional SDOs.  Or that single implementation might be
reverse-engineered by others and then the result treated as
common industry practice.

Despite the occasional claim that a strange (to me) idea called
"anticipatory standardization" is a variation on the second
model, that combination makes standards in the absence of, or
prior to, implementation and deployment an impossible
contradiction.  It is basically incompatible with innovation of
any type.

The IETF has traditionally chosen the first model and a great
deal of our thinking and public rhetoric are based on it. Even
when we have adopted proposals, sometimes implemented ones,
from elsewhere, we have insisted on change control and usually
taken that seriously.  One could claim that an adaptation of
the second model would make the Internet more uniform, but a
consensus of existing practice cannot aspire to "working
better" except in the narrow sense of substitutability of
conforming products.

It is interesting that the multi-stage model of RFC 2026
(particularly the original "Proposed Standard" definition), the
IEEE's "Trial-Use" standards (the current Section 5.7 of their
Standards Board Operations Manual,
http://standards.ieee.org/develop/policies/opman/sect5.html#5.7,
makes instructive reading), and ISO "Technical Specification"
all try to combine the two models by allowing a preliminary
specification --one that is intended for trial implementation
but not deployment-- to be designed on an engineering basis and
then standardized only when implementations exist and can be
compared (our original "Draft Standard").  In theory, only when
both an industry common practice and consensus about value
emerges does true standardization (including our original
definition for "Internet Standard") move forward.  We know how
well that has worked for us in practice.  While there have been
exceptions, I don't believe it has worked much better for other
SDOs.

If we are not going to move significantly in the direction of
"consensus of industry practice", then it seems to me that we
need to be careful about the arguments that are made (or at
least those that are considered legitimate) in support of
advancement into or on the standards track.  

For example, if we agree on a relaxed registration procedure
for some protocol parameters, it is not reasonable to later
turn around and ask that that those parameters be standardized,
unchanged, because they are already registered (and maybe
deployed).  For standardization, the IETF generally needs not
only change control but an opportunity to comment on and affect
the actual design and specification of whatever is being
standardized.  

Similarly, we sometimes hear it argued that we should accept a
specification for Proposed Standard unchanged because it has
been extensively developed and reviewed elsewhere.  That may be
reasonable in some cases although I'd hope we wouldn't make it
a common practice.  But, if a specification adopted for
Proposed Standard on that basis is then proposed for
advancement to Internet Standard, I think the review should be
comprehensive --perhaps even more comprehensive than the usual
such review-- because the Internet Standard is unambiguously an
IETF product and recommendation not that of the other body.  If
we allow a specification to advance to Proposed Standard
without an open and critical review because it was developed
and reviewed comprehensively by outside expert organizations
and then allow the Last Call review for Internet Standard to be
constrained by existing deployment, we would have gone into the
rubber stamping business that we regularly say is incompatible
with our standardization model.  That doesn't mean, of course,
that changes during review for Internet Standard are likely to
be a good idea.  Those who propose changes to the spec, and
especially to the protocol, should have to convince the
community that those changes are important enough to overcome
the costs, including those of incompatibility with deployed
versions.  But I don't think we can reasonably exclude them
from raising the issues and making those arguments.

thanks,
   john

---

From: Melinda Shore <[email-masked]>
Date: Wed, 29 May 2013 09:38:16 -0800

I think this is one of the best discussions of what we're
about that I've seen anywhere, and I'm grateful to John
for working this through.

One thing I'd like to take up further is this:

On 5/29/13 9:23 AM, John C Klensin wrote:
> IETF product and recommendation not that of the other body. 

I'd actually much prefer to see these go to informational, if
they're to be published.  Otherwise I agree - if something's going
to be an IETF standard it needs to go through the IETF standards
development review and revision process, which is probably not
what the authors want.

Melinda

---

From: Peter Saint-Andre <[email-masked]>
Date: Wed, 29 May 2013 11:42:09 -0600

On 5/29/13 11:38 AM, Melinda Shore wrote:
> I think this is one of the best discussions of what we're about
> that I've seen anywhere, and I'm grateful to John for working this
> through.
> 
>> specification for Proposed Standard unchanged because it has been
>> extensively developed and reviewed elsewhere.  That may be 
>> reasonable in some cases although I'd hope we wouldn't make it a
>> common practice.  But, if a specification adopted for Proposed
>> Standard on that basis is then proposed for advancement to
>> Internet Standard, I think the review should be comprehensive
>> --perhaps even more comprehensive than the usual such review--
>> because the Internet Standard is unambiguously an IETF product
>> and recommendation not that of the other body.
> 
> development review and revision process, which is probably not what
> the authors want.

/me wonders if we need a separate series for informational documentation

Peter

- -- 
Peter Saint-Andre
https://stpeter.im/

---

From: Dave Cridland <[email-masked]>
Date: Wed, 29 May 2013 21:04:57 +0100

> /me wonders if we need a separate series for informational documentation

Or maybe multiple paths, with multiple entry points.

Perhaps instead of Proposed Standard, we have a Engineering Proposal for an
engineering consensus, and a Submitted Proposal for an industry submission.
Both would move to Internet Standard from there, if appropriate.

I admit to picking names without the word standard in on purpose, but
that's because I think we should reclaim PS... I know I'm in the rough.

Dave.

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 30 May 2013 08:44:43 +1200

> Or maybe multiple paths, with multiple entry points.

We already do have exactly that, and there are many instances
of proprietary or local protocols being documented, but not
standardised, as Informational RFCs in the Independent stream.
Sometimes they squeeze through as Informational RFCs in the
IETF stream.

We also have BCPs, when there is robust consensus on good
operational practice. Again, we have Informational RFCs when
somebody wants to document current practice without seeking
consensus.

I'm not sure what we need to change.

Where we get into trouble seems to be when people want a rubber
stamp for something that doesn't make the cut for IETF consensus.
We have a little trouble saying "No." But I think we have a duty
to say "No" when something works but is believed to be bad for
the Internet as a whole.

    Brian

---

From: John C Klensin <[email-masked]>
Date: Thu, 30 May 2013 01:10:11 -0400

Melinda, Peter, Dave, and Brian,

I wanted to try to describe some fundamental models and their
implications but avoid leaping head first into either the "fix
the standards process" or "change the categories or content of
the RFC Series" rat holes.    However...

--On Wednesday, May 29, 2013 09:38 -0800 Melinda Shore
<[email-masked]> wrote:

>> such review-- because the Internet Standard is unambiguously
>> an IETF product and recommendation not that of the other
>> body. 
> 
> they're to be published.  Otherwise I agree - if something's
> going to be an IETF standard it needs to go through the IETF
> standards development review and revision process, which is
> probably not what the authors want.

While I agree with the preference, I didn't mention it because I
assumed, for the purpose of my analysis, that the choice of
"standards track" or not was outside my logical scope. I suggest
we've done it both ways, e.g., with NFS, we published Version 3
as Informational (RFC 1813) and then produced Version 4 on the
standards track (RFC 3010).   HTTP was pretty much the same: 1.0
was pretty much taken over from W3C and published as
Informational (RFC 1945); 1.1 was a Proposed Standards in RFC
2068.  DKIM, IIR, went directly to Proposed Standard (RFC
4871ff) based mostly on an external spec (RFC 4870 appears to
represent a somewhat different protocol).   There are probably
better examples of the latter.  My guess is that cases will
differ and trying to make a hard rule would just get us into
trouble.

> /me wonders if we need a separate series for informational

I'm not sure what you are suggesting.  If it is "time to get
informational documents out of the RFC Series", the community
has considered variations on that theme endless times and been
unable to reach consensus that it is a good idea (and has
sometimes reached consensus that it isn't).  In particular,
putting them in a separate series would complicate the model of
publishing externally-produced specifications as Informational
and then following up with Standards Track, IETF designed and
vetted, specs discussed above.

If it is time to rethink the categories of RFCs and/or those of
the standards track as Dave suggests...  well, I've tried
floating several of those proposals and gotten absolutely no
traction.  If you want to try, I promise to cheer supportively
from the sidelines.

--On Thursday, May 30, 2013 08:44 +1200 Brian E Carpenter
<[email-masked]> wrote:

> stamp for something that doesn't make the cut for IETF
> consensus. We have a little trouble saying "No." But I think
> we have a duty to say "No" when something works but is
> believed to be bad for the Internet as a whole.

Yes, and the latter is implicit in my long note.  The even more
difficult problems arise when someone comes to us with a spec
that might be ok but isn't how we would do it and tries to say
"you can have this and we will turn over change control as long
as you don't really want to make any changes".  When a statement
equivalent to that is justified on the basis of either being in
a hurry or not invalidating existing implementations, we find
ourselves in the middle of the contradiction between "consensus
of industry practice" and "best engineering solution" for
standardization.

  best,
     john

---

From: "Hannes Tschofenig" <[email-masked]>
Date: Thu, 30 May 2013 09:11:27 +0200 (CEST)

Hi John,

this is a great summary.

Regarding the question about the type of standardization we want I would argue for a mixture of both since in practice there are, of course, a lot of grey areas. I suspect that setting the expectations right at the beginning of starting the work (in a group or on a specific document) are important. I guess it is just the "surprises" that folks get concerned about when they suddenly realize that the last call period isn't actually something were they can change a lot (even if they dislike something) because the technology is widely deployed already and any change in the spec is likely going to cause a disconnect with the deployment.

Ciao

Hannes

Gesendet: Mittwoch, 29. Mai 2013 um 19:23 Uhr

Von: "John C Klensin" <[email-masked]>

An: [email-masked]

Betreff: What do we mean when we standardize something?

general issue.

issued.

any type.

conforming products.

SDOs.

standardized.

thanks,

john

---

From: SM <[email-masked]>
Date: Thu, 30 May 2013 10:43:46 -0700

Hi John,
>better" except in the narrow sense of substitutability of

The note is thoughtful.  It's the first time I see a message where 
the "working better" angle is discussed.

>advancement into or on the standards track.

If the IETF were to move in the direction of "consensus of industry 
practice" it would be moving towards consensus of industry practice 
based on the opinions of people in the industry who are active in the 
IETF.  Michael Richardson asked the following question:

   "Given that the [Country X] mandated them, why wasn't the IETF involved
    earlier?  The regulator really should have reached out to the IETF here.

If people from Country X do not come to the IETF with the problem the 
IETF won't know about the problem.  If people from Country X do not 
provide input about their industry practice the IETF will know about it.

>the actual design and specification of whatever is being

Country X would also have to allow the IETF to have change control on 
the specification or else it is like asking the IETF to rubber-stamp 
the specification.

If I am not mistaken the relaxed registration procedure is so that 
there can be a registry of what is being used on the Internet.  The 
difference here is that the IETF does not review the design, or it 
may set some minimal criteria for the review before allocating the code point.

>from raising the issues and making those arguments.

If an intended Proposed Standard has actually been developed and 
reviewed elsewhere the Last Call and IESG Evaluation is not a 
problem, i.e. any problem in the specification has already been 
identified and addressed.  One of the issues here is the rhetoric 
(the undue use of exaggeration, or from an IETF perspective, making 
unpleasant comments to discourage people from identifying problems).

It is difficult to have an open and critical review of an Internet 
Standard.  There is a significant cost for any change made at that 
level as it may cause interoperability issues.  What "we" mean when 
we standardize something is that "we" will think carefully before 
making a change as "we" would like the specification to be stable 
even if that means not fixing some minor errors.  Internet Standard 
could have been used to say "we" are not going to tinker with this 
specification and it is your best bet if you want to deploy the code 
and forget about it for a few years.

As an unrelated comment, what could "we" mean?  It could mean:

  1. doing what is right to make you look good
  2. doing what is right for your company
  3. doing what is right for a country
  4. doing what is right for the IETF

Regards,
-sm

---

From: Abdussalam Baryun <[email-masked]>
Date: Fri, 31 May 2013 06:37:13 +0200

> of industry practice" and "best engineering solution" for

If the standards proposed are reviewed well I don't think there will
be contradiction, I don't recommending always sticking to best
engineering solutions, because it is difficult to guarantee best
solutions in present/future (best practices ok). For industry request
work, IMO its better that our standards get into between *best
engineering practices* and *good engineering practices*, that will not
contradict with *consensus* and  *industry practices*.

AB