Subject: RE: Facts, please, not handwaving [Re: Its about mandate RE: Why cant the IETF embrace an open Election Process]

From: "Gray, Eric" <[email-masked]>
Date: Tue, 19 Sep 2006 14:02:49 -0400

Eliot/Brian,

	This, I think, is part of the problem.

	To say that "it is well understood that the Internet mainly 
runs on Proposed Standards," is to indulge in over-simplification. 
And to say "we are not actually following our documented process"
- is to focus on the meta-issues when the problem is with what is
documented in the RFCs themselves, in many cases.

	One of the useful features of the current "process" is that 
it means the community has one or two opportunities to insist that
what is technically documented is in alignment with reality.  For
many of the most important RFCs, it takes more than competence and
an RFC to implement what's actually going to work in the Internet.
A significant amount of the time it either takes lab testing, some
snooping on what goes on the wire, some reverse engineering and at
least one more iteration of lab testing - or it takes being on the
"inside track" in dominant (or early) implementation.

	One of the reasons why it _looks_ like "the Internet mainly 
runs on Proposed Standards," is that the people who know about the
difference between what's technically done, and what's technically
documented, have no real incentives to get involved in efforts to
document what they know - thus letting future competitors have the
information for free.  This is - I believe - a key factor in why
most Proposed Standards do not progress to Draft Standard.

	This factor has come up repeatedly in the efforts to advance
Proposed Standards that I've been involved in, and it usually adds
years to the process.

	But another factor is that we (as in the IETF) aren't usually
satisfied with just documenting what is really being done.  For one
reason or another, many of us would like to "fix" the Standard to -
for example - make it consistent with protocols and specifications
we've developed since then, even if that means documenting protocol
behaviors that are inconsistent with what is really being done.  

	These factors just adds to the burden.  Because it is not easy 
to progress along the standards track, many (if not most) Proposed 
Standards only roughly document what the Internet runs on.

	I would argue that "Proposed Standard" as the end-of-the-line
in our standardization process is just wrong.  I certainly can see
an argument for merging "Proposed" and "Draft" - but there are lots
of indications that even the simplified one-step process of moving 
from Draft to full Standard would not get done much of the time.

	There are examples of ways to deal with this difficulty from 
other standards groups.  The issue we need to decide as a group is
whether we're happy with our (slightly) imperfect process, or we
would like to adopt (and/or modify) someone else's.

--
Eric

--> -----Original Message-----
--> From: Eliot Lear [mailto:[email-masked]] 
--> Sent: Monday, September 18, 2006 4:13 AM
--> To: Brian E Carpenter
--> Cc: [email-masked]
--> Subject: Re: Facts, please, not handwaving [Re: Its about 
--> mandate RE: Why cant the IETF embrace an open Election Process]
--> 
--> Brian E Carpenter wrote:
--> > Phill,
--> >
--> >> As a result the IETF is a standards body with 2000 active
--> >> participants that produces on average less than 3 
--> standards a year
--> >> and typically takes ten years to produce even a specification.
--> >
--> > It is well understood that the Internet mainly runs on Proposed
--> > Standards,
--> > so the appropriate metric is how many Proposed Standards the IETF
--> > produces
--> > a year. I think you will find that is more than three.
--> 
--> I have to agree.  Going back to the numbers I posted to 
--> NEWTRK in March:
--> 
--> Status	1999	2000	2001	2002	2003	2004	2005
--> -------------------------------------------------------------
--> PS	102	119	71	105	103	131	169
--> 
--> 
--> 
--> This represents a tremendous amount of work by many people.  And so
--> while I *do* believe we have process problems, they are 
--> largely of the
--> form that we are not actually following our documented 
--> process.  I also
--> believe that we need to be more nimble when it comes to changing our
--> processes.  I would like to see both of those problems 
--> corrected in due
--> course.
--> 
--> I have not done the work to review velocity from -00 to 
--> RFC, but perhaps
--> Bill Fenner has.
--> 
--> Eliot
--> 
--> _______________________________________________
--> Ietf mailing list
--> [email-masked]
--> https://www1.ietf.org/mailman/listinfo/ietf
--> 

_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf

---

From: Dave Crocker <[email-masked]>
Date: Tue, 19 Sep 2006 14:31:40 -0700

> from Draft to full Standard would not get done much of the time.

The reality of the current IETF is that we view publishing a Proposed 
specification as an indication of success.  That is, we think it is an 
ending, rather than a beginning.

A minor problem with this is that we have no sense of what IETF actually 
gets deployed and used, and what doesn't. This leaves us exposed to 
developing  very unrealistic feeling about our current impact on the 
Internet.

(The status of Historic is not relevant to this question, since it is 
applied only erratically and long after a specification has demonstrated 
that it is not useful...any more.)

What we need is a more immediate basis for assessing current utility of 
recent IETF work.

That's why I keep suggesting that we set a time-limit for deployment and 
use of Proposed specifications.  Those failing to garner the necessary 
community support automatically go to Historic.  Those that succeed go 
to Full.

Of course, such a mechanism would require that the IETF community 
actually step up and take affirmative responsibility for whether our 
output gets used.

d/

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: Dave Cridland <[email-masked]>
Date: Wed, 20 Sep 2006 10:17:41 +0100

On Tue Sep 19 22:31:40 2006, Dave Crocker wrote:
> The reality of the current IETF is that we view publishing a 
> Proposed specification as an indication of success.  That is, we 
> think it is an ending, rather than a beginning.
> 
> A minor problem with this is that we have no sense of what IETF 
> actually gets deployed and used, and what doesn't. This leaves us 
> exposed to developing  very unrealistic feeling about our current 
> impact on the Internet.
> 
> (The status of Historic is not relevant to this question, since it 
> is applied only erratically and long after a specification has 
> demonstrated that it is not useful...any more.)
> 
> What we need is a more immediate basis for assessing current 
> utility of recent IETF work.
> 
> That's why I keep suggesting that we set a time-limit for 
> deployment and use of Proposed specifications.  Those failing to 
> garner the necessary community support automatically go to 
> Historic.  Those that succeed go to Full.

Well, I think there's a lot of confusion between the statement "We, 
as engineers trying to maintain our scientific integrity as a whole, 
consider this specification a good thing and recommend it", and "We, 
as disinterested engineers trying to be practical and document what 
gets used, note this specification is widely used". At the moment, 
we've tied the two statements into one - we cannot advance something 
down the standards track that's well deployed, but has arguble flaws 
(say, like HTTP/1.1, although you can subsitute your own contentious 
example here), because it implicitly implies we think it's flawless, 
which I think is a practical problem.

I'm not saying the two are utterly disconnected - it's good and fair 
that we require a substantial degree of implementation and deployment 
experience for us to consider a specification "good" - but neither 
are they one and the same thing, and I think they pull in different 
directions.

I suspect there's a third metric about a specification, too, which is 
concerned with how stable its foundation is - how "good" we think the 
dependencies are in aggregate, as they affect this specification. So, 
we might acknowledge that ACAP has a dependency on CRAM-MD5, but we 
acknowledge that CRAM-MD5, whilst not particularly good, is very well 
deployed, and moreover has little practical impact on how good ACAP 
is. (And says nothing at all about how well deployed ACAP is, of 
course, which is an entirely different matter)

At the moment, we require that we recommend a specification only as 
much as it is deployed, and never more than its dependencies. We 
never admit to deployment of protocols we do not approve, nor do we 
openly approve of protocols that are not deployed. Once deployed and 
approved, then we're fine - only if deployment drops off, we don't 
admit to that either.

Imposing a simple time limit does not reflect either the technical 
goodness - which is not to say that new techniques make older ones 
look foolish sometimes - nor does it aid in deployment.

Given a standpoint that much of the problem with Standards Track 
advancement is in the multiple implications of advancement (and in 
the inability to move backwards along the Standards Track), I 
strongly suspect that reducing it to a one-step would intensify the 
differences between deployment and recommendation, and therefore make 
the problem worse, and not better.

Dave. (Currently feeling in danger of being branded another loon by 
posting twice on ietf-politics within the same week)
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

---

From: Jefsey_Morfin <[email-masked]>
Date: Wed, 20 Sep 2006 14:18:42 +0200

At 11:17 20/09/2006, Dave Cridland wrote:
>gets used, note this specification is widely used".

The first statement best matches RFC 3935, except that "we" should be 
read as the IETF "leaders" (what is realistic in a rough consensus 
approach). The problem ("the near-impossibility of getting an 
informed consensus opinion on a complex subject out of a community of 
several thousand people in a short time") and this soliution is 
documented in part 3.

Also, "The Internet isn't value-neutral, and neither is the IETF. We 
want the Internet to be useful for communities that share our 
commitment   to openness and fairness.  We embrace technical concepts 
such as decentralized control, edge-user empowerment and sharing 
of   resources, because those concepts resonate with the core values 
of the IETF community.  These concepts have little to do with 
the   technology that's possible, and much to do with the technology 
that we choose to create." However,"we are also constrained by the 
principle of competence: Where we do not have, and cannot gather, the 
competence needed to make technically sound standards, we should not 
attempt to take the leadership." because "Sometimes the IETF 
leverages standards that are defined and maintained by other 
organizations; we continue to work with those organizations on their 
standards and do not attempt to take them over."

This creates many difficulties in world reality which is under 
distributed control, and therefore edge-user centric, working by 
subsidiarity, using concerted procedures and consensus, while we may 
chose to select parts of external standards and contradict their 
inner logic because they are not based on the same core values, or do 
not share our vision of these values.

The second statement seems therefore more appropriate. However this 
is only a reporting mission and we miss the IEB (Internet Engineer's 
Book) publication adequate system. We should therefore target some 
intermediate statement between "influence" and "report" and consider 
the RFC 3935 principles of responsiblity and competence, and the most 
inportant sentence "the benefit of a standard to the Internet is in 
interoperability - that multiple products implementing a standard are 
able to work together in order to deliver valuable functions to the 
Internet's users" remembering the quote above "Sometimes the IETF 
leverages standards that are defined and maintained by other 
organizations; we continue to work with those organizations on their 
standards and do not attempt to take them over". Interoperability 
must not be only within our own standards, but with the standards we 
leverage" we have the duty to respect and to know.

jfc

---

From: Sam Hartman <[email-masked]>
Date: Wed, 20 Sep 2006 20:54:00 -0400

>>>>> "Jefsey" == Jefsey Morfin <[email-masked]> writes:

I think the following is  a good summary of our quandary.

    >> gets used, note this specification is widely used".

    Jefsey> The first statement best matches RFC 3935, except that "we" should be
    Jefsey> read as the IETF "leaders" (what is realistic in a rough consensus
    Jefsey> approach). The problem ("the near-impossibility of getting an
    Jefsey> informed consensus opinion on a complex subject out of a community of
    Jefsey> several thousand people in a short time") and this soliution is
    Jefsey> documented in part 3.

    Jefsey> Also, "The Internet isn't value-neutral, and neither is the IETF. We
    Jefsey> want the Internet to be useful for communities that share our
    Jefsey> commitment   to openness and fairness.  We embrace technical concepts
    Jefsey> such as decentralized control, edge-user empowerment and sharing
    Jefsey> of   resources, because those concepts resonate with the core values
    Jefsey> of the IETF community.  These concepts have little to do with
    Jefsey> the   technology that's possible, and much to do with the technology
    Jefsey> that we choose to create." However,"we are also constrained by the
    Jefsey> principle of competence: Where we do not have, and cannot gather, the
    Jefsey> competence needed to make technically sound standards, we should not
    Jefsey> attempt to take the leadership." because "Sometimes the IETF
    Jefsey> leverages standards that are defined and maintained by other
    Jefsey> organizations; we continue to work with those organizations on their
    Jefsey> standards and do not attempt to take them over."

    Jefsey> This creates many difficulties in world reality which is under
    Jefsey> distributed control, and therefore edge-user centric, working by
    Jefsey> subsidiarity, using concerted procedures and consensus, while we may
    Jefsey> chose to select parts of external standards and contradict their
    Jefsey> inner logic because they are not based on the same core values, or do
    Jefsey> not share our vision of these values.

    Jefsey> The second statement seems therefore more appropriate. However this
    Jefsey> is only a reporting mission and we miss the IEB (Internet Engineer's
    Jefsey> Book) publication adequate system. We should therefore target some
    Jefsey> intermediate statement between "influence" and "report" and consider
    Jefsey> the RFC 3935 principles of responsiblity and competence, and the most
    Jefsey> inportant sentence "the benefit of a standard to the Internet is in
    Jefsey> interoperability - that multiple products implementing a standard are
    Jefsey> able to work together in order to deliver valuable functions to the
    Jefsey> Internet's users" remembering the quote above "Sometimes the IETF
    Jefsey> leverages standards that are defined and maintained by other
    Jefsey> organizations; we continue to work with those organizations on their
    Jefsey> standards and do not attempt to take them over". Interoperability
    Jefsey> must not be only within our own standards, but with the standards we
    Jefsey> leverage" we have the duty to respect and to know.

    Jefsey> jfc

    Jefsey> _______________________________________________
    Jefsey> Ietf mailing list
    Jefsey> [email-masked]
    Jefsey> https://www1.ietf.org/mailman/listinfo/ietf