Subject: Re: PS Characterization Clarified

From: Jari Arkko <[email-masked]>
Date: Mon, 2 Sep 2013 17:23:47 +0300

Olaf, Scott,

Apologies for a late reply on this (I was on vacation after the IETF). But thank you for writing this draft. My general comment is that the draft makes what in my mind is an accurate correction to our documents, aligning the documents to the current reality. I'd be happy to take the document forward. In fact, I think we need to make this change even if we made other, more long term changes.

There is at least one ongoing effort right now that has the potential to reclassify a large set of Proposed Standard RFCs that form the basis of widely used technology. These types of efforts can have a relatively big effect on the standards status of the most commonly used RFCs. Do we want to do more? Can we do more?

Secondly, the other obvious action we could take is to go back to the original mode of operation, i.e., making PS RFCs truly early and somewhat untested specifications. I am personally opposed to that on the following grounds. First, it would not change the fact that a large part of Internet technology today runs on PS RFCs, and Olaf's problem with getting these RFCs recognised would continue. Second, while I think we need to keep adjusting the level of review performed by the IESG and in IETF Last Call (we sometimes overdo it), I think broad review is actually useful.

But enough about my opinions. What do the rest of you think?

In terms of specific text, I also wrote a few observations, below. These are purely personal comments.

First, I think you assumed this but never made it explicit. While the new characterisation recognises the often final role of PS RFCs, it does not take away the possibility of publishing Internet Standard specifications. Can this be clarified?

> In the two decades after publication of RFC 2026 [RFC202] the IESG
> has evolved its review processes of Proposed Standard RFCs and thus
> RFC 2026 section 4.1.1 no longer accurately describes IETF Proposed
> Standards.

I'd prefer saying "the IETF review processes Proposed Standard RFCs have evolved". And leave the details to Section 2.

> 2.  IESG Reveiew of Proposed Standards

Review

> In response,
> the IESG strengthened its review of Proposed Standards, basically
> operating as if the Proposed Standard was the last chance for the
> IESG to ensure the quality of the technology and the clarity of the
> standards document. 

That is part of it, but I think the situation is more complicated than that. The world changed around us, and suddenly Internet was big business, global, and we got more careful about impacts to it. The process has evolved, including the number of steps in the ladder. Review practices in general have changed quite a lot, we now have a fairly broad review of RFCs.

Progression has also varied, mostly downwards. But as noted, it also seems very much affected by specific initiatives. 

Here's what I'd say:

   Initially it was assumed that most IETF technical specifications
   would progress through a series of maturity stages starting with
   a relatively early Proposed Standard, then progressing to Draft Standard then, finally,
   to Internet Standard (see RFC 2026 section 6).  Over time, for a
   number of reasons, this progression became less common.  At the same time,
   the review for Proposed Standard RFCs was strengthened.
   This strengthening was partially a response by the IESG for the above,
   and in part a consequence of the growth in the importance of the
   Internet and broader interest in reviewing new Internet technology.

   At the time of this writing, the IETF operates
   as if the Proposed Standard was the last chance for the
   to ensure the quality of the technology and the clarity of the
   standards document.  The result is that IETF Proposed Standards
   approved over the last decade or more have had extensive review.
   Because of this change in review assumptions, IETF Proposed Standards
   should be considered to be at least as mature as final standards from
   other standards development organizations.  In fact, the IETF review
   is more extensive than is done in other SDOs due to the cross-area
   technical review performed in the IETF.

Jari

---

From: Scott O Bradner <[email-masked]>
Date: Mon, 2 Sep 2013 14:09:15 -0400

> There is at least one ongoing effort right now that has the potential to reclassify a large set of Proposed Standard RFCs that form the basis of widely used technology. These types of efforts can h…

seems like a quite bad idea (as Randy points out)

take extra effort and get some interoperability data

> Secondly, the other obvious action we could take is to go back to the original mode of operation, i.e., making PS RFCs truly early and somewhat untested specifications. I am personally opposed to t…

imo - not a chance in hell of the IESG going back to the original meaning of PS
it is not in the IESG genetics, nor has it been for quite a while

>   technical review performed in the IETF.

wfm

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 03 Sep 2013 08:07:40 +1200

Hi Jari,

On 03/09/2013 02:23, Jari Arkko wrote:

...
> At the time of this writing, the IETF operates as if the
> Proposed Standard was the last chance for the to ensure the
> quality of the technology and the clarity of the standards
> document.  

There's a point that I think should be made here, something like:

In practice, interoperable implementations are commonly based on
Proposed Standard documents, so whatever design defects those
documents have tend to become part of the interoperable network,
perhaps in the form of work-arounds. Similarly, in today's
Internet, any security defects tend to be exploited at an early
stage. Fixing design and security issues in widely deployed code
may be difficult or impossible in practice. Therefore, there is
now very strong pressure to make the Proposed Standard as mature
as possible, rather than being just good enough to meet the RFC
2026 requirements.

> The result is that IETF Proposed Standards 

Exactly.

   Brian

---

From: John C Klensin <[email-masked]>
Date: Mon, 02 Sep 2013 16:14:15 -0400

--On Monday, 02 September, 2013 14:09 -0400 Scott O Bradner
<[email-masked]> wrote:

>> There is at least one ongoing effort right now that has the
>> potential to reclassify a large set of Proposed Standard RFCs
>> that form the basis of widely used technology. These types of
>> efforts can have a relatively big effect on the standards
>> status of the most commonly used RFCs. Do we want to do more?
>> Can we do more?
> 

More than that.  Unless we want to deserve the credibility
problems we sometimes accuse others of having, nothing should be
a full standard, no matter how popular, unless it reflects good
engineering practice.  I think there is more flexibility for
Proposed Standards, especially if they come with commentary or
applicability statements, but I believe that, in general, the
community should consider "bad design" or "bad engineering
practice" to fall into the "known defect" category of RFC 2026.
If RFC 6410 requires, or even allows, that we promote things
merely because they are popular, then I suggest there is
something seriously wrong with it.

   john

---

From: Jari Arkko <[email-masked]>
Date: Tue, 3 Sep 2013 06:44:31 +0300

> as possible, rather than being just good enough to meet the RFC

Yes, that is pretty accurate.

Jari

---

From: Olaf Kolkman <[email-masked]>
Date: Tue, 3 Sep 2013 15:00:22 +0200

> something seriously wrong with it.

John,

+1

All, 

In fact, going back to the language of RFC2026 for Full (now Internet) Standard. It confirms that popularity (significant implementation) is one necessary but not sufficient criterium.

4.1.3  Internet Standard

   A specification for which significant implementation and successful
   operational experience has been obtained may be elevated to the
   Internet Standard level.  An Internet Standard (which may simply be
   referred to as a Standard) is characterized by a high degree of
   technical maturity and by a generally held belief that the specified
   protocol or service provides significant benefit to the Internet
   community.

I would hope that any concerns about technical maturity or significant benefit to the Internet community are taken into account when making the decision. If it is the case that members of the community assess that a specification lacks interoperability that should be sufficient grounds to not advance until data proofs otherwise.

And for what its worth. One of the concerns most seen are those of IPR. The stamp of Internet Standard is a confirmation of the community that any IPR on the specification can be death with, that is an important piece of information on layer 9.
 
= On a more generic note.

The reason I took initiative for this draft is mainly because I believe we need to do what we document and document what we do. As discussed in this thread the practice for the approval of PS has changed, the bar is much higher than 20 years ago. In this case it is good that we document what we do.

That shouldn't be a motivation to not do what we document: namely be serious about the maintenance of our standards. And I would hope that somehow we as a community find the energy needed to advance our specification in a way that truly assesses the requirements of RFC2026 sect 4.1.3

* significant implementation
* successful operational experience
* technical maturity
* significant benefit to the Internet community.

--Olaf

---

From: Scott Brim <[email-masked]>
Date: Tue, 3 Sep 2013 09:17:13 -0400

+1. Well said.

---

From: Jari Arkko <[email-masked]>
Date: Tue, 3 Sep 2013 16:20:01 +0300

Olaf, John, Scott,

> In fact, going back to the language of RFC2026 for Full (now Internet) Standard. It confirms that popularity (significant implementation) is one necessary but not sufficient criterium.

Sorry. I was careless when I wrote about the effort. I didn't mean to suggest that we have an effort to classify standards merely based on popularity. What I meat that we have an effort to move a particular set of specifications to Internet Standard, and will use the usual criteria when deciding whether the documents are ready. While that particular set of specifications happens to be popular, that was just an observation, not a (sole) reason of moving them forward.

Hope this clarifies.

> I would hope that any concerns about technical maturity or significant benefit to the Internet community are taken into account when making the decision. If it is the case that members of the commu…

Yes, of course.

Jari

---

From: Scott O Bradner <[email-masked]>
Date: Tue, 3 Sep 2013 13:29:19 -0400

thank you - clarity does help

but such an effort will not remove the need for this document imo

Scott

---

From: Barry Leiba <[email-masked]>
Date: Tue, 3 Sep 2013 16:40:01 -0400

I mostly agree with this draft, but I have a concern.  Let's anchor
that concern off of this bit that Jari said:

> Secondly, the other obvious action we could take is to go back to the original
> mode of operation, i.e., making PS RFCs truly early and somewhat untested
> specifications. I am personally opposed to that on the following grounds. First,
> it would not change the fact that a large part of Internet technology today runs
> on PS RFCs, and Olaf's problem with getting these RFCs recognised would
> continue. Second, while I think we need to keep adjusting the level of review
> performed by the IESG and in IETF Last Call (we sometimes overdo it), I think
> broad review is actually useful.

It's certainly clear to all of us that most PS specs are far more
mature than what we thought about when we wrote RFC 2026.

The only concern I have is that once we do this -- declare that PS is
always more mature than that -- we can't go back.  Do we *really* want
to say that we will never again approve a PS spec that's partially
baked?  This is painting us into the room where PS is mature and
robust.  If we like being in that room, that's fine.  But it removes
the "IESG can put fuzzy stuff out as PS if it thinks that's the right
thing to do" option.

It says that IETF PS specs are "at least as mature as final standards
from other" SDOs.  Mostly, that's true.  But it doesn't have to be.
After this, it would have to be, always, for every PS spec.  Are we
*sure* that's what we want?

Barry

---

From: "Scott O. Bradner" <[email-masked]>
Date: Tue, 3 Sep 2013 16:42:57 -0400

fwiw - I would love for the IESG to exercise flexibility here 
but I have not seen that in many years - so I think we are already there
without a discernible path back

Scott

---

From: Ted Lemon <[email-masked]>
Date: Tue, 3 Sep 2013 21:14:21 +0000

On Sep 3, 2013, at 4:40 PM, Barry Leiba <[email-masked]> wrote:
> baked? 

It seems as if we already do this.   It's not unusual to publish things as experimental when we don't think they're baked, and that seems entirely appropriate.

The fact is that anything that has PS on it at this point has had _very_ thorough review.   Frequently PS have multiple implementations.   I think that what you are describing is actually the status quo—it would be very hard to get something really half-baked published as PS at this juncture.   That doesn't mean that every PS is high quality, but neither are they half-baked.

---

From: Olaf Kolkman <[email-masked]>
Date: Wed, 4 Sep 2013 07:02:38 +0200

Barry,

Question, in-line.

> the "IESG can put fuzzy stuff out as PS if it thinks that's the right

Wouldn't such spec come with an applicability statement of sorts? (today, in practice?)
 
> *sure* that's what we want?

This draft is mostly targeted to document what we do, not what we want. Although I can see how you want to keep the door open.

--Olaf.

---

From: Barry Leiba <[email-masked]>
Date: Wed, 4 Sep 2013 10:34:35 -0400

> Wouldn't such spec come with an applicability statement of sorts? (today, in practice?)

That's a good point; probably yes.

So if the text here can say something that allows a PS spec to *say*
that it's less mature, and that that's being done on purpose, my
concern is satisfied.

> This draft is mostly targeted to document what we do, not what we want. Although
> I can see how you want to keep the door open.

As a specific current example, I have the sense that the documents
coming out of the repute working group are specifying a protocol
that's somewhat less mature than what we usually do -- more comparable
to the 2026 version of PS than to this one.  Yet I absolutely think
they should be PS, *not* Experimental.

Barry

---

From: Scott Brim <[email-masked]>
Date: Wed, 4 Sep 2013 12:14:21 -0400

> that it's less mature, and that that's being done on purpose, my

Not the spec itself but an associated statement about it?

> they should be PS, *not* Experimental.

OK, somebody has to say it.  Maybe we should have another state,
something like draft standard.

---

From: Randy Bush <[email-masked]>
Date: Thu, 05 Sep 2013 01:32:11 +0900

> something like draft standard.

[ sob alluded to a private message from me which said ]

while i really like the idea of pushing well-tested interoperable
documents to full standards, i think tested interop is key here.
hence i liked the old interop tests for ds and am not all that happy
to see it go (yes, i whined privately to russ).

our protocols are rarely based on any testable formalism, computer
science, mathematics, ...  we live on a pile of crap hacks.  interop
is about all that keeps us from entropic degeneration.

randy

---

From: Spencer Dawkins <[email-masked]>
Date: Wed, 04 Sep 2013 11:38:03 -0500

> Not the spec itself but an associated statement about it?

There was a proposal to provide an alternative way of publishing 
applicability statements, in 
http://tools.ietf.org/html/draft-ietf-newtrk-repurposing-isd-04 (now 
expired).

> something like draft standard.

There were a couple of proposals to provide a way of saying "the working 
group thinks they're finished, but lets hold off on PS until they see 
how the protocol works". The one I co-authored was at 
http://tools.ietf.org/html/draft-dawkins-newtrk-wgs-00, Scott Bradner's 
was at http://tools.ietf.org/html/draft-bradner-ietf-stds-trk-00, in 
Section 5. Both are now expired.

Perhaps those drafts would be helpful background for folks thinking 
about this? (especially for folks who were too busy doing protocol work 
to follow NEWTRK? :-)

Spencer

---

From: Olaf Kolkman <[email-masked]>
Date: Fri, 13 Sep 2013 16:56:34 +0200

Colleagues

[I have added a number of people who were active in the discussion previously to the CC, my apologies if that is bad etiquette but I wanted to make you explicitly aware of this.]

Based on the discussion so far I've made a few modifications to the draft.  I am trying to consciously keep this document to the minimum that is needed to achieve 'less is more' and  my feeling is that where we are now is close to the sweetspot of consensus.

This is the summary of these changes:
*   Added "Updates 2026" and added Sean's initial"

*   Copied the whole characterization pararaph for Internet Standards
   from 2026, instead of only the line that is the actual
   characterization itself.

*   Added the Further Consideration section based on discussion on the
   mailinglist.
See:
http://www.ietf.org/id/draft-kolkman-proposed-standards-clarified-01.txt

For diff:
http://tools.ietf.org/rfcdiff?url2=draft-kolkman-proposed-standards-clarified-01.txt

--Olaf

---

From: S Moonesamy <[email-masked]>
Date: Fri, 13 Sep 2013 10:17:45 -0700

Hi Olaf,
At 07:56 13-09-2013, Olaf Kolkman wrote:
>Based on the discussion so far I've made a few modifications to the 
>draft.  I am trying to consciously keep this document to the minimum 
>that is needed to achieve 'less is more' and  my feeling is that 
>where we are now is close to the sweetspot of consensus.

The intended status would have to be BCP instead of 
Informational.  In Section 3.1:

   "A specific action by the IESG is required to move a
    specification onto the standards track at the "Proposed Standard"
    level."

I suggest "standards" instead of "specific" action if you (and the 
other authors) decide that BCP is appropriate.  The last paragraph in 
Section 3.1 is okay.  I don't think that Section 4 is 
necessary.  Please note that I do not have a strong opinion about 
this.  I leave it to your discretion.

The two references in Section 7 would have to be normative references.

I have reason to believe that you mean it when you say "we document 
what we do".  draft-kolkman-proposed-standards-clarified-01 proposes 
that the IETF does that and I think that it is a fine idea.

Regards,
S. Moonesamy

---

From: Barry Leiba <[email-masked]>
Date: Fri, 13 Sep 2013 13:49:42 -0400

> Based on the discussion so far I've made a few modifications to the draft.
> I am trying to consciously keep this document to the minimum that is needed
> to achieve 'less is more' and  my feeling is that where we are now is close
> to the sweetspot of consensus.

Section 4 makes me happy.  I think it's ready to go.

Barry

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 13 Sep 2013 20:03:11 +0200

> *   Added the Further Consideration section based on discussion on the

I believe the current document is fine for a major part of the IETF standards activities.

It is, however, important to keep in mind that the IETF is not a homogeneous organization, not even within each of the quite different areas.  Section 4 seems to try to open up the straightjacket created by section 3 a little bit again, but the way it does this is probably the wrong approach.

Instead of writing a long diatribe here, let me just point to section 2.7 of https://www.iab.org/wp-content/IAB-uploads/2011/04/Bormann.pdf (see the references to the other sections there, too).

Grüße, Carsten

---

From: Olaf Kolkman <[email-masked]>
Date: Fri, 13 Sep 2013 20:32:45 +0200

On 13 sep. 2013, at 19:17, S Moonesamy <[email-masked]> wrote:

> The intended status would have to be BCP instead of Informational.  

Correct….  fixed on trunk.

> In Section 3.1:

> I suggest "standards" instead of "specific" action if you (and the other authors) decide that BCP is appropriate.  
> 

I have used exactly the same term as RFC2026. I have no idea if 'standards action' is defined somewhere.

> The two references in Section 7 would have to be normative references.

Yes. (see PS)

Thanks, and best,

--Olaf

PS. I think this is xml2rfc playing up. The xml contains this:

<back> 
 <references title='Normative References'>
    &rfc2026;
    &rfc6410;
  </references>
 <section title="Acknowledgements">
…..

But it seems to not want to translate . If anybody has suggestions, off-list please.

---

From: Scott O Bradner <[email-masked]>
Date: Fri, 13 Sep 2013 14:35:25 -0400

> I have used exactly the same term as RFC2026. I have no idea if 'standards action' is defined somewhere.

I do not think we should move away from the ted used in RFC 2026

Scott

---

From: Olaf Kolkman <[email-masked]>
Date: Fri, 13 Sep 2013 20:50:08 +0200

> It is, however, important to keep in mind that the IETF is not a homogeneous organization, not even within each of the quite different areas.  Section 4 seems to try to open up the straightjacket c…

Note this is not trying to change… I is trying to document what we do now. 

On https://www.iab.org/wp-content/IAB-uploads/2011/04/Bormann.pdf

I am trying to see what one gets if one translates the fallacies into positive actions, or answer the question on how do you cope with the fallacy. I notice that your draft observes but doesn't seem to recommend. That is not a value judgement on the text btw but it doesn't give me insight in if 'this is probably wrong' what is the right way? And more important, is there any indication we can get there? I believe we had several tries in doing something different (better, was the intention of all those that took part in that debate) but we never reached consensus. That is why this is not trying to change, but tries to document the realities.

Have a nice weekend.

--Olaf

---

From: John C Klensin <[email-masked]>
Date: Fri, 13 Sep 2013 14:57:50 -0400

--On Friday, September 13, 2013 16:56 +0200 Olaf Kolkman
<[email-masked]> wrote:

>...
> Based on the discussion so far I've made a few modifications
> to the draft.  I am trying to consciously keep this document
> to the minimum that is needed to achieve 'less is more' and
> my feeling is that where we are now is close to the sweetspot
> of consensus.

Olaf,

I'm afraid I need to keep playing "loyal opposition" here.

> *   Added the Further Consideration section based on
> discussion on the    mailinglist.

Unfortunately, IMO, it is misleading to the extent that you are
capture existing practice rather than taking us off in new
directions.  You wrote:

> While commonly less mature specifications will be published as
> Informational or Experimental RFCs, the IETF may, in
> exceptional cases, publish a specification that does not match
> the characterizations above as a Proposed Standard.  In those
> cases that fact will be clearly communicated on the front page
> of the RFC e.g. means of an IESG statement.

On the one hand, I can't remember when the IESG has published
something as a Proposed Standard with community consensus and
with an attached IESG statement that says that they and the
community had to hold our collective noses, but decided to
approve as PS anyway.  Because, at least in theory, a PS
represents community consensus, not just IESG consensus (see
below), I would expect (or at least hope for) an immediate
appeal of an approval containing such as statement unless it
(the statement itself, not just the opinion) matched community
consensus developed during Last Call.

Conversely, the existing rules clearly allow a document to be
considered as a Proposed Standard that contains a paragraph
describing loose ends and points of fragility, that expresses
the hope that the cases won't arise very often and that a future
version will clarify how the issues should be handled based on
experience.   That is "no known technical omissions" since the
issues are identified and therefore known and not omissions.  In
the current climate, I'd expect such a document to have a very
hard time on Last Call as people argued for Experimental or even
keeping it as an I-D until all of the loose ends were tied up.
But, if there were rough consensus for approving it, I'd expect
it to be approved without any prefatory, in-document, IESG notes
(snarky or otherwise).

The above may or may not be tied up with the "generally stable"
terminology.  I could see a spec with explicit "this is still
uncertain and, if we are wrong, might change" language in it on
the same basis as the loose end description above.  Such
language would be consistent with "generally stable" but, since
it suggests a known point of potential instability, it is not
consistent with "stable".

Additional observations based on mostly-unrelated recent
discussions:  

If you are really trying to clean 2026 up and turn the present
document into something that can be circulated to other groups
without 2026 itself, then the "change control" requirement/
assumption of RFC 2026 Section 7.1.3 needs to be incorporated
into your new Section 3.  It is not only about internal debates,
it is our rule against why we can't just "endorse" a standard
developed elsewhere as an IETF standards track specification.

Along the same lines but more broadly, both the sections of 2026
you are replacing and your new text, if read in isolation,
strongly imply that these are several decisions, including those
to approve standardization, that the IESG makes on its own
judgment and discretion.  I think it is fairly clear from the
rest of 2026 (and 2028 and friends and IETF oral tradition) that
the IESG is a collector and interpreter of community consensus,
not a body that is someone delegated to use its own judgment.  I
believe that, if an IESG were ever to say something that
amounted to "the community consensus is X, but they are wrong,
so we are selecting or approving not-X", we would either see a
revolution of the same character that brought us to 2026 or the
end of the IETF's effectiveness as a broadly-based standards
body.  

More important --and related to some of my comments that you
deferred to a different discussion-- the "IESG as final
_technical_ review and interpreter of consensus" model is very
different from that in some other SDOs in which the final
approval step is strictly a procedural and/or legal review that
is a consensus review only in the sense of verifying that the
process in earlier stages followed the consensus rules and is
not technical review at all.  I don't think you need to spend
time on that, but you need to avoid things that would make your
document misleading to people who start with that model of how
standards are made as an initial assumption.

    best,
     john

---

From: "Adrian Farrel" <[email-masked]>
Date: Fri, 13 Sep 2013 20:02:05 +0100

Hey Olaf,
 
Thanks for stubbornly pushing on with this.
 
Comments (sorry I haven't read the thread to see if others have already made
these comments)...
 
Abstract
   This document clarifies the description of the review performed on
   and the maturity level of IETF Proposed Standard RFCs and updates RFC
   2026
Clarifies the description found where?
Review performed by whom?
 
Probably replace as...
   RFC 2026 describes the review performed by the IESG on IETF
   Proposed Standard RFCs and states the maturity level of those
   documents. This document clarifies those descriptions and updates
   RFC 2026 by providing a new characterization Proposed Standards.
 
---
 
Section 1
 
OLD
  standard maintenance procedures
NEW (maybe?)
  procedures for the maintenance of Standards Track documents
END
 
I'm sure you did not mean that the procedures are standard.
I think you did not intend to limit the discussion to Standards.
 
---
 
Section 2
 
   clarity of the standards document
 
Prefer "Standards Track document"
 
---
 
Section 2
 
   over the last decade or more have had extensive review.
 
...by the IESG?
...by or on behalf of the IESG?
 
---
 
Section 2
 
   technical review performed by the IESG.
 
I wonder whether you should add
 
...a position that is further strengthened by the implementation and running
code that is often present before publication as a Proposed Standard.
 
---
 
Section 3
 
   Section 3.1 updates RFC 2026 Section 4.1.1. Section 3.2 is a verbatim
   copy of the characterization of Internet Standards from RFC 2026
   Section 4.1.3.
 
Suggest...
 
   Section 3.1 of this document replaces RFC 2026 Section 4.1.1.  Section
   3.2 of this document is a verbatim copy of the characterization of
   Internet Standards from RFC 2026 Section 4.1.3 and is provided for
   convenient reference.
 
---
 
Section 3.1
 
   Usually, neither implementation nor operational experience is
   required for the designation of a specification as a Proposed
   Standard.  However, such experience is highly desirable, and will
   usually represent a strong argument in favor of a Proposed Standard
   designation.
 
You could add 
 
... and may be tracked and reported as described in [RFC6982]
 
---
 
Section 3.1
 
Two paragraphs seem to enjoy some duplication in their final sentences.
 
   A Proposed Standard specification is stable, has resolved known
   design choices, is well-understood, has received significant
   community review, and appears to enjoy enough community interest to
   be considered valuable.  However, as with all technical standards,
   further experience might result in a change or even retraction of the
   specification in the future.
 
and
 
   A Proposed Standard will have no known technical omissions with
   respect to the requirements placed upon it.  Proposed Standards are
   of such quality that implementations can be deployed in the Internet.
   However, as with all technical specifications, Proposed Standards may
   be revised if problems are found or better solutions are identified,
   when experiences with deploying implementations of such technologies
   at scale is gathered.
 
---
 
Section 4
 
   While commonly less mature specifications will be published as
   Informational or Experimental RFCs,
 
Maybe...
 
   While less mature specifications will  commonly be published as
   Informational or Experimental RFCs,
 
---
 
Section 4
 
s/ e.g. means of an IESG statement./ e.g., by means of an IESG statement./
 
---
 
Cheers,
Adrian

 
From: [email-masked] [mailto:[email-masked]] On Behalf Of Olaf
Kolkman
Sent: 13 September 2013 15:57
To: IETF list
Cc: SM Moonesamy; John Klensin; Barry Leiba; Scott O Bradner
Subject: Re: PS Characterization Clarified

Colleagues
 
[I have added a number of people who were active in the discussion previously to
the CC, my apologies if that is bad etiquette but I wanted to make you
explicitly aware of this.]

Based on the discussion so far I've made a few modifications to the draft.  I am
trying to consciously keep this document to the minimum that is needed to
achieve 'less is more' and  my feeling is that where we are now is close to the
sweetspot of consensus.

 
 
For diff:
http://tools.ietf.org/rfcdiff?url2=draft-kolkman-proposed-standards-clarified-01
.txt

--Olaf

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 13 Sep 2013 21:09:47 +0200

On Sep 13, 2013, at 20:50, Olaf Kolkman <[email-masked]> wrote:

> I am trying to see what one gets if one translates the fallacies into positive actions, or answer the question on how do you cope with the fallacy. I notice that your draft observes but doesn't seem to recommend.

Indeed, the 2011 document is heavy on diagnosis, and very light on therapy.
The hope was that "Selbsterkenntnis ist der erste Schritt zur Besserung" (self-awareness is the first step to improvement), so it tries to create some of that awareness.
How to go from personal awareness to organizational awareness is indeed the question*).

Documenting the status quo *does* change the status quo, by making it more rigid.
(Unless one also documents why exactly that wasn't intended.)

But my main point is that "Proposed Standard" is turning into a Procrustes bed, and fixing that will require more work than section 4 seems to want making one believe.

Grüße, Carsten

*) (Going through the process of turning that 2011 workshop submission into a consensus RFC might be one step towards that, and I apologize for not having attempted that yet.)

---

From: Olaf Kolkman <[email-masked]>
Date: Mon, 16 Sep 2013 15:56:27 +0200

On 13 sep. 2013, at 21:02, Adrian Farrel <[email-masked]> wrote:

> Comments (sorry I haven't read the thread to see if others have already made these comments)…

This is to acknowledge I took the suggestions that I am not quoting.

> Prefer "Standards Track document"

This is a change to the original 2026 language, but I support it it makes really clear what we are taking about.

> ...by or on behalf of the IESG?

That is already explained in the paragraphs above. I propose to just keep the focus on the document having had review and not make an addition to that sentence.

On the other hand, elsewhere in the thread John Klensin made the case that we should stress that the IESG does this quality assurance since that is different from other SDOs, this might be a good place to add that extra emphasis.

As said, I keep the text as is for now but I will take editorial guidance if this is a serious issue.

> ...a position that is further strengthened by the implementation and running code that is often present before publication as a Proposed Standard.
>  

Added, but with the word "Interoperable" added. Could you review if this is still proper English and contact me off-list if it isn't:

      In fact, the
      IETF review is more extensive than is done in other SDOs due to
      the cross-area technical review performed by the IESG, a
      position that is further strengthened by the regular precense of
      interoperable implementation and running code before publication
      as a Proposed Standard.

> ... and may be tracked and reported as described in [RFC6982]

Hmmm.. I want to be careful with that reference. It is an experimental document and increases the referential complexity for little benefit.

Until serious pushback otherwise I propose to not take over this suggestion.

>    when experiences with deploying implementations of such technologies

Fixed by removing that final sentence at first occurrence.

I'll wait a few days to see discussion converge before posting 02 in the mean time:
pre 02 is at:
http://www.secret-wg.org/Secret-Media/draft-kolkman-proposed-standards-clarified-pre02-2013091600.txt
with a diff at:
http://tools.ietf.org/rfcdiff?url1=http://tools.ietf.org/id/draft-kolkman-proposed-standards-clarified-01.txt&url2=http://www.secret-wg.org/Secret-Media/draft-kolkman-proposed-standards-clarified-pre02-2013091600.txt

Thanks,

--Olaf

---

From: Olaf Kolkman <[email-masked]>
Date: Mon, 16 Sep 2013 15:58:35 +0200

[Barry added explicitly to the CC as this speaks to 'his' issue]

On 13 sep. 2013, at 20:57, John C Klensin <[email-masked]> wrote:

[… skip …]

> directions.  

Yeah it is a thin line. But the language was introduced to keep a 
current practice possible (as argued by Barry I believe).

> You wrote:

I see where you are going. 

<draft Proposed rewrite>

While commonly less mature specifications will be published as
Informational or Experimental RFCs, the IETF may, in
exceptional cases, publish a specification that still contains areas for improvement or 
certain uncertainties about whether the best engineering choices are made.  In those
cases that fact will be clearly communicated in the document prefereably on the front page
of the RFC e.g. in the introduction or a separate statement.

</draft>

I hope that removing the example of the IESG statement makes clear that this
is normally part of the development process.

> standards are made as an initial assumption.

So noted. 

As actionable for this draft I take that I explicitly mention that Section 4.1 2026 is exclusively updated.

--Olaf

---

From: Barry Leiba <[email-masked]>
Date: Mon, 16 Sep 2013 10:43:06 -0400

> current practice possible (as argued by Barry I believe).

Yes, that was my concern.

> is normally part of the development process.

I think that version is just fine too, with the typo in "preferably" corrected.

John, to give you the context again:
I agree that we're normally requiring much more of PS documents than
we used to, and that it's good that we document that and let external
organizations know.  At the same time, we are sometimes proposing
things that we know not to be fully baked (some of these came out of
the sieve, imapext, and morg working groups, for example), but we *do*
want to propose them as standards, not make them Experimental.  I want
to be sure we have a way to continue to do that.  The text Olaf
proposes is, I think, acceptable for that.

Barry

---

From: John C Klensin <[email-masked]>
Date: Mon, 16 Sep 2013 11:31:08 -0400

--On Monday, September 16, 2013 15:58 +0200 Olaf Kolkman
<[email-masked]> wrote:

>> Unfortunately, IMO, it is misleading to the extent that you
>> are capture existing practice rather than taking us off in new
>> directions.  
> 
> Yeah it is a thin line. But the language was introduced to
> keep a  current practice possible (as argued by Barry I
> believe).

Understood.  Barry and I are on the same page wrt not wanting to
accidentally restrict established existing practices.

>> You wrote:
>> 
>>> While commonly less mature specifications will be published
>>> as Informational or Experimental RFCs, the IETF may, in
>...

> exceptional cases, publish a specification that still contains
> areas for improvement or  certain uncertainties about whether
> the best engineering choices are made.  In those cases that
> fact will be clearly communicated in the document prefereably
> on the front page of the RFC e.g. in the introduction or a
> separate statement.
> 
> </draft>
> 
> I hope that removing the example of the IESG statement makes
> clear that this is normally part of the development process.

Yes.

Editorial nits:

* "While commonly less mature specifications will be
published..." has "commonly" qualifying "less mature".  It is
amusing to think about what that might mean, but it isn't what
you intended.  Try "While less mature specifications will
usually be published...".  Replace "usually" with "commonly" or
"normally" if you like, but I think "usually" is closest to what
you are getting at.

* "prefereably" -> "preferably"

>> Along the same lines but more broadly, both the sections of
>> 2026 you are replacing and your new text, if read in
>> isolation, strongly imply that these are several decisions,
>> including those to approve standardization, that the IESG
>> makes on its own judgment and discretion.  I think it is
>...
>> approval step is strictly a procedural and/or legal review
>> that is a consensus review only in the sense of verifying
>...
> So noted. 
> 
> As actionable for this draft I take that I explicitly mention
> that Section 4.1 2026 is exclusively updated.

While I understand your desire to keep this short, the pragmatic
reality is that your non-IETF audience is likely to read this
document (especially after you hand it to them) and conclude
that it is the whole story.  Since the natural question that
immediately follows "why should we accept your standards at all"
is "why can't you hand them off to, e.g., ISO, the way that many
national bodies and organizations like IEEE do with many of
their documents".  

Suggestion in the interest of brevity: in addition to mentioning
the above, mention explicitly that there are requirements in
other sections of 2026 that affect what is standardized and how. 

By the way, while I understand all of the reasons why we don't
want to actually replace 2026 (and agree with most of them),
things are getting to the point that it takes far too much
energy to actually figure out what the rules are.  Perhaps it is
time for someone to create an unofficial redlined version of
2026 that incorporates all of the changes and put it up on the
web somewhere.   I think we would want a clear introduction and
disclaimer that it might be be exactly correct and that only the
RFCs are normative, but the accumulation of changes may
otherwise be taking us too far into the obscure.  If we need a
place to put it, it might be a good appendix to the Tao.  And
constructing it might be a good job for a relative newcomer who
is trying to understand the ins and outs of our formal
procedures.

best,
   john

---

From: John C Klensin <[email-masked]>
Date: Mon, 16 Sep 2013 12:04:33 -0400

--On Monday, September 16, 2013 10:43 -0400 Barry Leiba
<[email-masked]> wrote:

>...
> I agree that we're normally requiring much more of PS
> documents than we used to, and that it's good that we document
> that and let external organizations know.  At the same time,
> we are sometimes proposing things that we know not to be fully
> baked (some of these came out of the sieve, imapext, and morg
> working groups, for example), but we *do* want to propose them
> as standards, not make them Experimental.  I want to be sure
> we have a way to continue to do that.  The text Olaf proposes
> is, I think, acceptable for that.

In case it wasn't clear, I have no problems with that at all.  I
was objecting to three things that Olaf's newer text has fixed:

(1) It is a very strong assertion to say that the above is
"exceptional".  In particular, "exceptional" would normally
imply a different or supplemental approval process to make the
exception.  If all that is intended is to say that we don't do
it very often, then "commonly" (Olaf's term), "usually", or
perhaps even "normally" are better terms.

(2) While it actually may be the common practice, I have
difficulty with anything that reinforces the notion that the
IESG makes standardization decisions separate from IETF
consensus.  While it isn't current practice either, I believe
that, were the IESG to actually do that in an area of
significance, it would call for appeals and/or recalls.   Olaf's
older text implied that the decision to publish a
not-fully-mature or incomplete specification was entirely an
IESG one.   While the text in 2026, especially taken out of
context, is no better (and Olaf just copied the relevant bits),
I have a problem with any action that appears to reinforce that
view or to grant the IESG authority to act independently of the
community.

(3) As a matter of policy and RFCs of editorially high quality,
I think it is better to have explanations of loose ends and
not-fully-baked characteristics of standards integrated into the
document rather than using IESG Statements.  I don't think
Olaf's new "front page" requirement is correct (although I can
live with it) -- I'd rather just say "clearly and prominently
communicated in the document" and leave the "is it clear and
prominent enough" question for Last Call -- but don't want to
see it _forced_ into an IESG statement.

I do note that "front page" and "Introduction" are typically
inconsistent requirements (header + abstract + status and
copyright boilerplate + TOC usually force the Introduction to
the second or third page).  More important, if a real
explanation of half-baked features (and why they aren't fully
baked) may require a section, or more than one, on it own.  One
would normally like a cross reference to those sections in the
Introduction and possibly even mention in the Abstract, but
forcing the text into the Introduction (even with "preferably"
given experience with how easily that turns into a nearly-firm
requirement) is just a bad idea in a procedures document.  We
should say "clearly", "prominently", or both and then leave
specifics about what that means to conversations between the
authors, the IESG and community, and the RFC Editor.

    best,
    john

---

From: S Moonesamy <[email-masked]>
Date: Mon, 16 Sep 2013 22:49:49 -0700

Hi John,
>web somewhere.   I think we would want a clear introduction and

I posted draft-moonesamy-stds-process-00 (expired) [1] in 2010.  I 
have to update the draft as it does not take into account the 
two-track change.  I would not post a revision on the web as the IETF 
Trust might not like it.  In my opinion it might be related to the 
original negotiating position of CNRI.

Regards,
S. Moonesamy

1. http://tools.ietf.org/html/draft-moonesamy-stds-process-00

---

From: Olaf Kolkman <[email-masked]>
Date: Tue, 17 Sep 2013 11:47:59 +0200

Based on the conversation below I converged to:

   <t>
      While less mature specifications will usually be published as
      Informational or Experimental RFCs, the IETF may, in exceptional
      cases, publish a specification that still contains areas for
      improvement or certain uncertainties about whether the best
      engineering choices are made.  In those cases that fact will be
      clearly and prominently communicated in the document e.g. in the
      abstract, the introduction, or a separate section or statement.
    </t>

I will be publishing version 02 shortly.

--Olaf

[This is a top-post, nothing but context below]

---

From: Olaf Kolkman <[email-masked]>
Date: Tue, 17 Sep 2013 12:14:37 +0200

> other sections of 2026 that affect what is standardized and how. 

Second paragraph of the introduction now reads:

      This document exclusively updates the characterization of
      Proposed Standards from RFC2026 Section 4.1.1 and does not speak
      to or alter the procedures for the maintenance of Standards
      Track documents from RFC 2026 and <xref target="RFC6410">RFC
      6410</xref>. For complete understanding of the requirements for
      standardization those documents should be read in conjunction
      with this document.
    
> is trying to understand the ins and outs of our formal

I guess this is a call for volunteers.

--Olaf

---

From: Dave Cridland <[email-masked]>
Date: Tue, 17 Sep 2013 11:32:19 +0100

>       abstract, the introduction, or a separate section or statement.
I read John's message as being against the use of the phrase "in
exceptional cases". I would also like to avoid that; it suggests that some
exceptional argument may have to be made, and has the implication that it
essentially operates outside the process.

I would prefer the less formidable-sounding "on occasion", which still
implies relative rarity.

Dave.

---

From: Alexey Melnikov <[email-masked]>
Date: Tue, 17 Sep 2013 11:52:33 +0100

On 17/09/2013 11:32, Dave Cridland wrote:
> On Tue, Sep 17, 2013 at 10:47 AM, Olaf Kolkman <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>
> exceptional cases". I would also like to avoid that; it suggests that 
> some exceptional argument may have to be made, and has the implication 
> that it essentially operates outside the process.
>

+1.

---

From: Scott Brim <[email-masked]>
Date: Tue, 17 Sep 2013 07:06:25 -0400

implies relative rarity.
> Dave.

Exceptions and arguments for and against are part of the process. Having a
process with no consideration for exceptions would be exceptional.

---

From: John C Klensin <[email-masked]>
Date: Tue, 17 Sep 2013 07:44:13 -0400

--On Tuesday, September 17, 2013 11:47 +0200 Olaf Kolkman
<[email-masked]> wrote:

>       While less mature specifications will usually be
> published as       Informational or Experimental RFCs, the
> IETF may, in exceptional       cases, publish a specification
> that still contains areas for       improvement or certain
> uncertainties about whether the best       engineering choices
> are made.  In those cases that fact will be       clearly and
> prominently communicated in the document e.g. in the

I suggest that "communicated in the document e.g. in..." now
essentially amounts to "... communicated in the document, e.g.
in the document." since the examples span the entire set of
possibilities.   Consequently, for editorial reasons and in the
interest of brevity, I recommend just stopping after
"prominently communicated in the document.".  But, since the
added words are not harmful, I have no problem with your leaving
them if you prefer.

   john

---

From: John C Klensin <[email-masked]>
Date: Tue, 17 Sep 2013 08:36:38 -0400

--On Tuesday, September 17, 2013 11:32 +0100 Dave Cridland
<[email-masked]> wrote:

> I read John's message as being against the use of the phrase
> "in exceptional cases". I would also like to avoid that; it
> suggests that some exceptional argument may have to be made,
> and has the implication that it essentially operates outside
> the process.

Exactly.

> I would prefer the less formidable-sounding "on occasion",
> which still implies relative rarity.

And "on occasion" is at least as good or better than my
suggestions of "usually", "commonly", or "normally" although I
think any of the four would be satisfactory.

--On Tuesday, September 17, 2013 07:06 -0400 Scott Brim
<[email-masked]> wrote:

>...
> Exceptions and arguments for and against are part of the
> process. Having a process with no consideration for exceptions
> would be exceptional.

Scott, it an IETF technical context, I'd completely agree,
although some words like "consideration for edge cases" would be
much more precise if that is actually what you are alluding to.
But part of the intent of this revision to 2026 is to make what
we are doing more clear to outsiders who are making a good-faith
effort to understand us and our standards.  In that context,
what you say above, when combined with Olaf's text, is likely to
be read as:

	"We regularly, and as a matter of course, consider
	waiving our requirements for Proposed Standard entirely
	and adopt specifications using entirely different (and
	undocumented) criteria."  

That is misleading at best.  In the interest of clarity, I don't
think we should open the door that sort of interpretation if we
can avoid it.

I don't think it belongs in this document (it is adequately
covered by Olaf's new text about other sections), but it is
worth remembering that we do have a procedure for making
precisely the type of exceptions my interpretation above
implies: the Variance Procedure of Section 9.1 of 2026.   I
cannot remember that provision being invoked since 2026 was
published -- it really is "exceptional" in that sense.  Its
existence may be another reason for removing "exceptional" from
the proposed new text because it could be read as implying that
we have to use the Section 9.1 procedure for precisely the cases
of a well-documented, but slightly incomplete, that most of us
consider normal.  In particular, it would make the approval of
the specs that Barry cited in his examples invalid without
invoking the rather complex procedure of Section 9.1.  I'd
certainly not like to have text in this update that encourages
that interpretation and the corresponding appeals -- it would
create a different path to the restriction Barry is concerned
about.

    john

---

From: Olaf Kolkman <[email-masked]>
Date: Tue, 17 Sep 2013 18:27:05 +0200

FYI.

I just posted the third version of the draft at:
http://tools.ietf.org/html/draft-kolkman-proposed-standards-clarified-02

Diff with the previous document:
http://tools.ietf.org/rfcdiff?url2=draft-kolkman-proposed-standards-clarified-02.txt

I will let Jari know that I believe we converged and that as far as I am concerned its last call time.

--Olaf

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 18 Sep 2013 08:33:57 +1200

On 17/09/2013 17:49, S Moonesamy wrote:
> I posted draft-moonesamy-stds-process-00 (expired) [1] in 2010.  I have
> to update the draft as it does not take into account the two-track
> change.  I would not post a revision on the web as the IETF Trust might
> not like it.  In my opinion it might be related to the original
> negotiating position of CNRI.

For some years I've maintained http://www.ietf.org/about/process-docs.html
"to assist IETF participants in navigating the labyrinth." It
does carefully avoid red-lining or commentary, and I think it also
shows the complexity that we have created.

   Brian

---

From: Pete Resnick <[email-masked]>
Date: Tue, 17 Sep 2013 17:48:09 -0500

> http://tools.ietf.org/html/draft-kolkman-proposed-standards-clarified-02

I would like to change "IESG" to "IETF" in five places:

Section 1:

"the IESG has evolved its review processes"

Section 2:

"IESG Reveiew of Proposed Standards"
"the IESG strengthened its review"
"last chance for the IESG to ensure the quality"
"cross-area technical review performed by the IESG"

The IETF as a whole, through directorate reviews, area reviews, doctor 
reviews, *and* IESG reviews, has evolved, strengthened, ensured, etc., 
its reviews.

Saying "the IESG" in these places implies precedent setting that I think 
would be bad. If the IETF capitulated to the IESG changing the rules on 
its own in the past, so be it, but I think it would be bad to indicate 
in a BCP that we think it's OK for the IESG to do so unilaterally.

pr

-- 
Pete Resnick<http://www.qualcomm.com/~presnick/>
Qualcomm Technologies, Inc. - +1 (858)651-4478

---

From: "Scott O. Bradner" <[email-masked]>
Date: Tue, 17 Sep 2013 18:52:55 -0400

1/ I believe that change would be factually incorrect

2/ I do not see that being factually correct about what happened says anything about
    the community opinion about any future IESG decision to change processes.

Scott

On Sep 17, 2013, at 6:48 PM, Pete Resnick <[email-masked]> wrote:

> The IETF as a whole, through directorate reviews, area reviews, doctor reviews, *and* IESG reviews, has evolved, strengthened, ensured, etc., its reviews.
> 
> Saying "the IESG" in these places implies precedent setting that I think would be bad. If the IETF capitulated to the IESG changing the rules on its own in the past, so be it, but I think it would be bad to indicate in a BCP that we think it's OK for the IESG to do so unilaterally.
> 
> pr
> 
> --

---

From: Pete Resnick <[email-masked]>
Date: Tue, 17 Sep 2013 18:10:44 -0500

On 9/17/13 5:52 PM, Scott O. Bradner wrote:
> On Sep 17, 2013, at 6:48 PM, Pete Resnick<[email-masked]>  wrote:
>
>    
> I believe that change would be factually incorrect

Which part of the above do you think is factually incorrect?

pr

---

From: John C Klensin <[email-masked]>
Date: Tue, 17 Sep 2013 19:54:59 -0400

Pete,

I generally agree with your changes and consider them important
-- the IESG should be seen in our procedural documents as
evaluating and reflecting the consensus of the IETF, not acting
independently of it.

Of the various places in the document in which "IESG" now
appears, only one of them should, IMO, even be controversial.
It is tied up with what I think is going on in your exchange
with Scott:

--On Tuesday, September 17, 2013 18:10 -0500 Pete Resnick
<[email-masked]> wrote:

>>> The IETF as a whole, through directorate reviews, area
>>> reviews, doctor reviews, *and* IESG reviews, has evolved,
>>> strengthened, ensured, etc., its reviews.
>>>      

The issue here --about which I mostly agree with Scott but still
believe your fix is worth making-- is that the impetus for the
increased and more intense review, including imposing a number
of requirements that go well beyond those of 2026, did not
originate in the community but entirely within the IESG.  It
didn't necessarily originate with explicit decisions.  In many
cases, it started with an AD taking the position that, unless
certain changes were made or things explained to his (or
occasionally her) satisfaction, the document would rot in the
approval process.  Later IESG moves to enable overrides and
clarify conditions for "discuss" positions can be seen as
attempts to remedy those abuses but, by then, it was too late
for Proposed Standard.  And, fwiw, those changes originated
within the IESG and were not really subject to a community
consensus process either.

However, because the document will be read externally, I prefer
that it be "IETF" in all of the places you identify.  If we have
to hold our noses and claim that the community authorized the
IESG actions by failing to appeal or to recall the entire IESG,
that would be true if unfortunate.  I would not like to see
anything in this document that appears to authorize IESG actions
or process changes in the future that are not clearly authorized
by community consensus regardless of how we interpret what
happened in the past.

    john

---

From: Olaf Kolkman <[email-masked]>
Date: Wed, 18 Sep 2013 10:59:55 +0200

> evaluating and reflecting the consensus of the IETF, not acting

Agreed….

> by community consensus regardless of how we interpret what

But one of the things that we should try to maintain in making that change is the notion that the IESG does  have a almost key-role in doing technical review. You made the point that that is an important distinction between 'us' and formal SDOs. 

Therefore I propose that that last occurrence reads:

> cross-area technical review performed by the IETF, exemplified by technical review by the full IESG at last stage of specification development.

I think that this language doesn't set precedence and doesn't prescribe how the review is done, only that the IESG does do review.

In full context:

    In fact, the IETF review is more extensive than that done in other
    SDOs owing to the cross-area technical review performed by the
    IETF,exemplified by technical review by the full IESG  at last stage of
    specification development. That position is further strengthened
    by the common presence of interoperable running code and
    implementation before publication as a Proposed Standard.

Does that work?

--Olaf

---

From: Scott O Bradner <[email-masked]>
Date: Wed, 18 Sep 2013 07:25:47 -0400

John covered why I said that Pete's assertion is factually incorrect

that said, I agree that being accurate here (that the IESG is the final decider and the body that 
changed the review from what was described in RFC 2026)  may be counter productive when 
the document is reviewed outside of the IETF so changing most of the IESG reverences to 
read "IETF" is the right thing to do

the one that should not be changed is the one that Olaf calls out at the end of the included message

Scott

---

From: John C Klensin <[email-masked]>
Date: Wed, 18 Sep 2013 08:19:49 -0400

--On Wednesday, September 18, 2013 10:59 +0200 Olaf Kolkman
<[email-masked]> wrote:

>> However, because the document will be read externally, I
>> prefer that it be "IETF" in all of the places you identify.
>> If we have to hold our noses and claim that the community
>> authorized the IESG actions by failing to appeal or to recall
>> the entire IESG, that would be true if unfortunate.  I would
>> not like to see anything in this document that appears to
>> authorize IESG actions or process changes in the future that
>> are not clearly authorized by community consensus regardless
>> of how we interpret what happened in the past.
>...

> But one of the things that we should try to maintain in making
> that change is the notion that the IESG does  have a almost
> key-role in doing technical review. You made the point that
> that is an important distinction between 'us' and formal SDOs.

It doesn't affect the document but can we adjust or vocabulary
and thinking to use, e.g., "more traditional" rather than
"formal".  There is, IMO, too little that we do that is
"informal" any more, but that isn't the point.

> I think that this language doesn't set precedence and doesn't
> prescribe how the review is done, only that the IESG does do
> review.
>...
> 
> In full context:
> 
>     In fact, the IETF review is more extensive than that done
> in other     SDOs owing to the cross-area technical review
> performed by the     IETF,exemplified by technical review by
> the full IESG  at last stage of     specification development.
> That position is further strengthened     by the common
> presence of interoperable running code and     implementation
> before publication as a Proposed Standard.

> Does that work?

The new sentence does work and is, IMO, excellent.

I may be partially responsible for the first sentence but, given
other comments, suggest that you at least insert "some" so that
it ends up being "...more extensive than that done in some other
SDOs owing...".  That makes it a tad less combative and avoids a
potentially-contentious argument about counterexamples.

The last sentence is probably ok although, if we were to do an
actual count, I'd guess that the fraction of Proposed Standards
for which implemented and interoperability-tested conforming
running code exists at the time of approval is somewhat less
than "common".

    john

---

From: Pete Resnick <[email-masked]>
Date: Wed, 18 Sep 2013 07:59:57 -0500

> potentially-contentious argument about counterexamples.

All sounds good to me.

pr

---

From: Jari Arkko <[email-masked]>
Date: Wed, 18 Sep 2013 16:05:40 +0300

Olaf, John, Pete,

> of the RFC e.g. in the introduction or a separate statement.

I like it much better than earlier versions in the discussion. The review that we perform for documents is not really just the IESG review. There is a plenty of WG and IETF Last Call and directorate and specialist and AD review going on… Secondly, for a long time the preferred IESG approach to saying something about the document's applicability and status has been to put the words in the document itself, abstract, introduction, separate section…. as opposed to an IESG statement. And we'd really like WGs and authors to be the source of those words, rather than the IESG. Of course there may be special situations that demand an IESG statement, but that is another matter.

> I would like to change "IESG" to "IETF" in five places:

And given my explanation above, I fully agree with his suggestion. And I see that you seem to have converged on that change, too. Good.

Jari