Subject: Newtrk and ISDs (was: Re: Facts, please not handwaving [Re: Its about mandate RE: Why cant the IETF embrace an open Election Process]

From: John C Klensin <[email-masked]>
Date: Thu, 21 Sep 2006 08:08:26 -0400

--On Wednesday, 20 September, 2006 08:23 -0400 Scott Bradner
<[email-masked]> wrote:

> Spencer remembered:
>> My understanding (as author of three of the proposals) was
>> that for most of  the time newtrk was in existence, the
>> working group's attention was focused  on ISDs as a way of
>> avoiding the need to tackle the 3 stage process. So I'm  not
>> sure there was even a call for consensus on adopting any of
>> the  proposals as a working group document.
> 
> almost
> 
> the newtrk chair (at least) felt that ISD would remove the
> issue of how many stages by taking a different path

So did another one of the authors.

Having seen the consequences of one-step standards processes,
especially in environments in which the standards designers are
not very closely tied to products that are shipping or ready to
ship, I remain strongly committed to a standards model that
separates "consensus proposals for implementation" and
"consensus standards that have been tested -- for both
specification quality and demonstrable experience with
interoperability and scalability-- by implementation and
deployment.

>From that point of view, there are three problems with our
present multi-step model:

	(1) The complexity of our standards has risen
	sufficiently that attaching a single category label
	(with only three choices) as the maturity level of an
	entire standard just doesn't make sense, if it ever did.
	
	(2) The IESG has both responded to and aggravated the
	difficulties of moving from one step to the next by
	continually raising the threshold for the first step,
	making it harder to get to that step. This creates less
	differentiation from the subsequent steps as well as
	more exhaustion, both of which deter getting there.
	While the rate of deterioration has been slower than one
	might expect, this is a positive feedback loop.
	
	(3) If we had wanted to choose a term for the first step
	that was guaranteed to cause confusion outside the
	community, we could not have done much better than
	"proposed standard".   With apologies to the IEEE
	"Standard for trial use" or even "Preliminary
	specification for trial use" would have come much closer
	to what we have historically thought we were doing.

I find myself in complete agreement with Dave Cridland's posting
and analysis of 20 September, 2006 10:17 +0100.  I think that we
are more or less saying the same thing in different language and
that his explanation is better than anything I could come up
with during the period in which ISDs were a live issue.

ISDs were intended to change that situation, not by having an
extended argument over "two" or "three" (or even "one") but by

	*  removing both the fixed categories and the notion
	that the same classification needed to apply to an
	entire document with many features while
	
	* continuing to be very clear about the IETF's beliefs
	about the degree to which the specification had been
	validated by implementation, interoperability testing or
	demonstrations, and deployment.

They were also intended to do another job but, from the
perspective of this discussion, it is merely a useful
side-effect.  That was to make it far more clear which
specifications were necessary or recommended to effectively
implement a particular function, answering the "what standard"
question of John Loughney's original I-D on the subject and
providing much of the function of the applicability statements
we never get around to writing.

>From my point of view, ISDs were a proposal for a fairly radical
change in the IETF's standards model and process, albeit one
that brought us closer to what I see as what the community
culture intends than what we have been doing in recent years.
They were much more about that change than about its expression
in specific document formats, organization, or titles.  Part of
the community that argued for and against ISDs never understood
that ("non-normative ISD" is about as close to an oxymoron as
one can get).  Others understood it perfectly well but were
unwilling or unable to consider anything that wasn't tiny-step
incrementalism.

As far as the "running code" part of this story was concerned,
several drafts were produced that showed what ISDs might look
like and how they might fit in.  They were mostly ignored.  To
the extent to which they were not, unimportant details of the
examples became the focus of attention, not the goals and
principles of the ISD model.

So we get stuck.  Over and over and over and over and... again.

I felt (and feel) that seriously considering and refining a
change in the IETF's model for talking about standards would be
far more productive than an extended debate over "two" versus
"three" (or "one").  I believe that we will never get consensus,
except possibly by exhaustion, on "two" versus "three", mostly
because the change wouldn't solve any significant problem, and
that we will never get consensus on "one" because too much of
the community fundamentally understands the risks of speculative
(or "anticipatory" to use the official term) standards.

    john

p.s. to avoid a separate note, Brian wrote...

> There was consensus to put forward the ISD proposal, which the
> IESG kicked back, with an explanation of its issues, which you
> can find in the newtrk archive. That didn't lead to a revised
> ISD proposal.

Obviously, interpretations of this differ.  From the standpoint
of at least one of the newtrk authors and proponents, there was
no explanation from the IESG that anyone could satisfy.  The
combination of the written explanation and the oral comments was
a general impression that the IESG simply did not like the
proposal because (i) it was too radical, (ii) it was a proposal
for significant change, rather than some incremental tuning,
and/or (iii) it might increase IESG workload for a while.  The
message --as received, regardless of what was intended -- was no
proposal with any of those properties, much less all three of
them, was going anywhere as long as the IESG got to decide.  And
the logical conclusion from that was that the precondition for
serious process changes was to change the IESG's ability to
decide, not to try to disguise ISDs into something that would
not solve any of the key problems to which they were addressed.

Just my opinion
   john

_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf

---

From: Fred Baker <[email-masked]>
Date: Thu, 21 Sep 2006 16:14:46 -0700

On Sep 21, 2006, at 5:08 AM, John C Klensin wrote:
> especially in environments in which the standards designers are not  
> very closely tied to products that are shipping or ready to ship, I  
> remain strongly committed to a standards model that separates  
> "consensus proposals for implementation" and "consensus standards  
> that have been tested -- for both specification quality and  
> demonstrable experience with interoperability and scalability-- by  
> implementation and deployment.

No argument. But I don't think that our current model works either.  
The fact is that we don't take the final step (Full Standard, or what  
I call "Obsolete Standard") very frequently, and we don't document  
interoperability (which is what Draft Standard does) very frequently.  
What we in fact do is take documents to Proposed Standard ("we agree  
that this is what we will build, and if you're lucky some of us have  
partially implemented it when we said that") and leave them there.  
And it's not that we don't ask the question; it's that the answer  
doesn't come back in the form that RFC 2026 recognizes.

I would really like to see interoperability documented, and lack of  
use/interoperability documented.

Collapsing Draft Standard and Standard together into what Mike O used  
to call a "good 'nuff standard", and making one of the ways to  
demonstrate interoperability be demonstrated multi-vendor  
interoperation in the field for a period of in excess of two years,  
is IMHO a good thing. The number of years could be argued, and if  
someone wants to say "three years" or "four years" I'm not going to  
complain. We have a Proposed Standard dated 1975; that's silly. Pick  
one. Is it obsolete or in current use, and if the latter, which  
Telnet implementations implement the Extended ASCII option? We should  
be able to bless it or not at this point.

We would really like interoperability testing in the sense that PPP  
did for many years and the SCTP community does now, with multiple  
vendors and open source implementers getting together in a room and  
each proving interoperability with every other, and then reporting  
back with issues found and successes experienced. That's a good  
thing, very good, and some of us do it. Far more commonly, for  
example in BGP, interoperability testing is done bilaterally between  
vendors (our routing guys and the Juniper guys have been known to get  
together and test specific features, for example), or more commonly  
yet it is done under real life conditions in operational networks.  
Live insertion is not as controlled (understatement), and the reports  
back tend to be to the vendor's TAC. But trust me, it happens. Since  
the operators and end users don't demand Full Standard status to use  
the protocols, the vendors have no incentive to spend the money for  
interoperability demos. Hence, I think the IETF needs to find a way  
to capitalize on what really happens rather than attempt to impose a  
solution that no-one else seems to find useful.

When I sent this note this morning, I attached a file listing 416  
candidate RFCs. I guess, at 174K, the file was too large for the  
exploder. So I have instead posted the suggestions broken out by  
decade or year, and telnet separately; the URLs, with the count of  
candidate RFCs they suggest, are at the end of this note. I think  
these should each be either moved to Historic ("we don't use any  
more, or we don't recommend people use it any more"), Informational  
("in use but not commonly implemented"), or Standard ("this is  
something widely implemented and proven operationally"). I'll bet in  
each case the relevant AD or working group chair could make that call  
without thinking very hard, and in any event it could be sorted out  
with a short email thread on each relevant list. If there were a  
feature to remove in standardization it could be done by publishing a  
"profile" RFC that says "we think RFC 12345 works operationally for  
the following reasons, but the opening twelve way handshake is not  
widely implemented, substituting a three-way handshake, and we  
consider that sufficient" that updates the RFC in question and is  
taken to "standard" with the relevant RFC.

For your amusement:

6	ftp://ftpeng.cisco.com/fred/ietf-ancient-rfc/1970s.txt
16	ftp://ftpeng.cisco.com/fred/ietf-ancient-rfc/1980s.txt

25	ftp://ftpeng.cisco.com/fred/ietf-ancient-rfc/early-1990s.txt
20	ftp://ftpeng.cisco.com/fred/ietf-ancient-rfc/1994.txt
13	ftp://ftpeng.cisco.com/fred/ietf-ancient-rfc/1995.txt
36	ftp://ftpeng.cisco.com/fred/ietf-ancient-rfc/1996.txt
54	ftp://ftpeng.cisco.com/fred/ietf-ancient-rfc/1997.txt
77	ftp://ftpeng.cisco.com/fred/ietf-ancient-rfc/1998.txt
88	ftp://ftpeng.cisco.com/fred/ietf-ancient-rfc/1999.txt

105	ftp://ftpeng.cisco.com/fred/ietf-ancient-rfc/2000.txt
69	ftp://ftpeng.cisco.com/fred/ietf-ancient-rfc/2001.txt
100	ftp://ftpeng.cisco.com/fred/ietf-ancient-rfc/2002.txt
106	ftp://ftpeng.cisco.com/fred/ietf-ancient-rfc/2003.txt
89	ftp://ftpeng.cisco.com/fred/ietf-ancient-rfc/2004.txt

30	ftp://ftpeng.cisco.com/fred/ietf-ancient-rfc/telnet.txt

---

From: Jefsey_Morfin <[email-masked]>
Date: Fri, 22 Sep 2006 01:42:30 +0200

Fred,
you talk about interoperability between vendors, this is good. Let 
not forget interoperability with users, i.e. our own IETF document 
interoperability with the external standard we leverage and the user 
demand. Waiting for industrial products not to excite the public is 
too long and chancy.
jfc

At 01:14 22/09/2006, Fred Baker wrote:
I would really like to see interoperability documented, and lack of

---

From: Frank Ellermann <[email-masked]>
Date: Sun, 24 Sep 2006 06:31:12 +0200

> 16 ftp://ftpeng.cisco.com/fred/ietf-ancient-rfc/1980s.txt

Frank