Subject: RE: "Discuss" criteria

From: "Hallam-Baker, Phillip" <[email-masked]>
Date: Sat, 30 Dec 2006 06:24:50 -0800

More recalls?

How many have we had?

I looked into what it would take to engage the recall process. I don't think it is possible to use it without tearing the whole organization appart.

With reference to John's recent campaigns I note that we still have a situation where IETF practice is to employ a two stage standards process but the process documents describe a mythical three stage process.

The IESG appears to be unwilling to either change the process document to reflect reality or to begin applying the three stage process. And I don't even have visibility into the process to know which individuals are the holdouts. The only response I am ever going to get back is the passive voice 'people on the IESG were not happy with the proposal'.

This is a real business issue for me, not a theoretical one. I spend too much time having to counter FUD claims that some IETF protocol or other is 'merely' draft and that it should not therefore be considered. People in the Internet area understand the mendacity but this is not the case in banking. I can explain the fact that according to the IETF HTTP 1.1 is still a draft standard but in doing so I have to conceed the fact that the IETF processes are broken at which point the proprietary FUD peddled chips in.

There are cases where consensus does not work. This is one of them. There is clearly no consensus in the IESG to either follow the process document or to fix it to match current practice. So we have the organization stuck in a decade long deadlock.

This is where you need to have leadership (another thing that the NOMCON process is expressly designed to exclude).

> -----Original Message-----
> From: John C Klensin [mailto:[email-masked]] 
> Sent: Saturday, December 30, 2006 8:57 AM
> To: [email-masked]; [email-masked]
> Cc: [email-masked]
> Subject: Re: "Discuss" criteria
> 
> Dave, Scott,
> 
> At the risk of repeating what a few others have said in 
> different form, a few observations.  Please understand that 
> these comments come from someone who has been more 
> consistently and loudly critical about even a hint of IESG 
> arrogance and assertions of their power than either of you 
> and who has formally proposed a significant number of ways of 
> dealing with those problems --real or imagined-- than, I 
> think, anyone else in the community... none of which 
> proposals have gone anywhere.
> 
> I believe that, ultimately, the IETF has to pick IESG members 
> who can do the job of evaluating documents and consensus 
> about it and then let them to do that job.  And we had better 
> pick people to do that job who technical judgment and good 
> sense we trust.  If we can't do that, then we are in 
> big-time, serious,
> trouble: trouble from which no set of rules or procedures can 
> rescue us.   Much as it makes me anxious, I think we ultimately 
> need to let an AD raise a Discuss because he or she has a bad 
> feeling in his or her gut... and pick people who will use 
> that particular reason with considerable care and who will 
> challenge each other and work to understand the objection and 
> either better document it or remove it as appropriate.
> 
> If that discussion is abused in particular cases, I think it 
> means that we need more appeals and, if there is a pattern, more 
> recalls.   In a long-term tradition of the IETF that we seem to 
> be losing, we may also need more specific, focused, public 
> abuse (in plenaries and otherwise) from the community, not just from 
> regular complainers and microphone-hogs.   What we don't need is 
> more rigid rules that either try to anticipate every 
> circumstance or that give too strong a presumption to the 
> wisdom of a too-homogeneous WG, especially at a time when 
> fewer and fewer documents seem to be getting widespread 
> community review during Last Call.
> 
> In that context, I can only applaud this document, not as a 
> set of rules that the IESG has to follow, but as one that 
> informs the community about the mechanisms the IESG is using. 
> Information is good.  And, if the IESG discovers that it 
> needs to update that information every time its membership 
> changes (or every time they discover something isn't working 
> and make an adjustment according), I'd consider that a sign 
> of good health: 
> at least it would show that, at least in this area, 
> historical rules and behavior patterns are not constraining 
> current thinking to the extent that replacing IESG members 
> doesn't bring about change.
> 
> At the risk of giving a sales pitch, my other proposals have 
> been intended to reinforce the model above: I think it is 
> always going to be hard, in our community, to find IESG 
> members who are good at doing these kinds of technical 
> evaluations and sensitive to the issues involved and who are 
> also outstanding managers, cat-herders, bureaucrats, finance 
> experts, and experts on 
> organizational behavior.   So I have sought to separate some of 
> those roles.  I think that long terms on the IESG tend to 
> breed detachment from the community and a tendency to put 
> IESG judgment ahead of that of the community and I don't 
> think we can solve that with more rules about IESG behavior.  
> So I have sought to give Nomcoms guidance about terms, to 
> change the nomination/appointment model, and to make the 
> recall mechanism 
> more effective in practice.   And I have sought ways to simplify 
> the job and reduce the workload in the hope that we can go 
> back to treating a term or two on the IESG as an obligation 
> that the right sorts of people owe the community, rather than 
> a position to be sought and in the more general hope of 
> broadening the pool of people who are willing to serve.
> 
> The fact that my proposals for change have not been 
> instituted tells me that the community does not see a serious 
> problem and doesn't believe that changes are needed.  While I 
> believe that the lack of acceptance of changes has been IESG 
> recalcitrance and efforts to protect the authority and ways 
> of working with they are familiar and comfortable, I don't 
> think that changes the conclusion: the community has ways, 
> however unpleasant, for imposing changes that have community 
> consensus but that it IESG doesn't like and has chosen to not 
> use them.  I disagree, but I think the consensus-in-practice 
> is fairly clear and I have to accept that.
> 
> To me, it is in the areas of adjusting IESG scope, 
> responsiveness, and membership that we need to do our tuning, 
> not by trying to restrict the IESG to particular ways of 
> doing its technical evaluations or the statements ADs can 
> make about specifications submitted for approval and 
> especially what arguments an AD can use for forcing the rest 
> of the IESG to take a harder look and initiate an in-depth 
> discussion (internally and, if appropriate, with the 
> community).  More hard rules about how the IESG does its 
> technical evaluation work won't, IMO, help us in the common, 
> ordinary, cases and, when an exceptional one arrives, such 
> rules are likely to force the IESG into making the wrong 
> decisions and doing the wrong things and thereby hurt the 
> IETF and the Internet.
> 
>     john
> 
> 
> _______________________________________________
> Ietf mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/ietf
> 

_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf

---

From: Michael Thomas <[email-masked]>
Date: Sat, 30 Dec 2006 07:09:21 -0800

So what occurs to me is that a reasonable question to ask is whether
there are some legitimate success stories where a DISCUSS has actually
found big or reasonably big problems with a protocol that would have
wreaked havoc had they not been caught. I ask because it seems to me
that the main things that wreak havoc with protocols tend to be rather
subtle and not likely to be very visible to someone whose sum experience
with the work is their assignment to read the current draft. That's not a
slap at the people whose job is to review, only that it seems to me to
be asking for super-human abilities.

 From my limited experience with DISCUSS -- and last call for
that matter -- is that the focus is far more geared toward wordsmithing
than actual mechanics of the protocol (from relatively disinterested
parties, that is). While I have no issue with tightening up drafts for
publication, it doesn't seem reasonable to be holding up the works
for endless amounts of time _just_ because  somebody -- or some
faction of bodies -- isn't convinced that a draft is the prose they
deign worthy.

The other thing that occurs to me -- and I know this has been brought
up in many different forms -- is that if an AD _was_ following the
working group to some degree, why is it legitimate for them to wait
for IESG evaluation to voice comments that affect the protocol's
operation? That is, why aren't they held just like anybody else to voice
those in WG last call when the WG is far more responsive to dealing
with issues? These "IESG Surprises" really hurt the community by
leading to the general perception that the IESG is capricious in a
royally anointed kind of way.

       Mike

---

From: Julian Reschke <[email-masked]>
Date: Sat, 30 Dec 2006 16:49:27 +0100

Michael Thomas schrieb:
> ...

And things get even worse when the response to the "IESG Surprise" is: 
we will fix that during AUTH48. There's absolutely no transparency in 
that, and no checking by the community.

I'd really like to see the IESG stop sending documents with known 
defects to the RFC Editor.

Best regards, Julian

---

From: "Steven M. Bellovin" <[email-masked]>
Date: Sat, 30 Dec 2006 11:14:26 -0500

On Sat, 30 Dec 2006 07:09:21 -0800
Michael Thomas <[email-masked]> wrote:

> operation? 

Most ADs don't follow most WGs in any detail -- they can't possibly do
so.  They should (and generally do) follow their own WGs, possibly the
others in their area, plus one or two elsewhere that they're personally
interested in.

		--Steve Bellovin, http://www.cs.columbia.edu/~smb

---

From: John C Klensin <[email-masked]>
Date: Sat, 30 Dec 2006 12:45:39 -0500

> royally anointed kind of way.

Michael,

If an AD who was responsible for a WG came up with an issue 
about that WG's work and raised it only during or after Last 
Call, I'd expect either a really good explanation or a 
resignation.  I certainly would not expect it to happen often. 
But, IMO, we have an IESG and, indeed, an IETF rather than a 
collection of different organizations addressing per-area issues 
precisely to increase the odds of catching serious cross-area 
and cross-perspective issues before things go out.  Like it or 
not, unless the relevant WG makes an effort to get broad reviews 
at critical "early" points, there will always be a risk of late 
surprises as someone --in the community during Last Call or on 
the IESG-- suddenly wakes up and says "but how does this relate 
to XYZ".

With regard to textual nit-picking and evaluation of worthiness 
of prose, I tend to agree with what I think you are saying. 
However, if a document is too badly written to permit 
interoperable implementations to be constructed without 
clarifying conversations among implementers, authors, and/or the 
WG, then the document is a failure and needs pushback.  As with 
late surprises, somewhat more proactive effort on the part of 
WGs could prevent many of the problems we see, but...

(1)  While I think the worst problems got fixed, RFC 4714 can be 
seen as an effort to either de-skill the RFC Editor function or 
to recognize that our expectations of that function a decade ago 
are no longer consistent with today's realities.  If we are not 
to publish documents that are incomprehensible, then someone has 
to have the responsibility and authority to either fix them or 
push back on them.  If it is going to be the RFC Editor, then we 
should be pushing back every time an IESG member complains about 
an editorial matter smaller than "the document is 
incomprehensible and cannot be evaluated or implemented" ... and 
we should (again) be giving the RFC Editor the authority to 
refuse to publish an IESG-approved document because it cannot be 
safely edited into comprehensible form.   If it isn't going to 
be the RFC Editor, then it probably needs to be the IESG --or 
some significant process changes-- and we should stop 
complaining.

(2) My comments about the expectation of good sense and 
community willingness to abuse and/or fire those who don't 
exercise it are very much applicable to the above.   I don't 
have any idea how to write rules that will improve the situation 
without very high risk of tying ourselves in knots _and_ not 
solving the underlying problem.

     john

---

From: Michael Thomas <[email-masked]>
Date: Sat, 30 Dec 2006 18:00:51 -0800

John C Klensin wrote:
>
> If an AD who was responsible for a WG came up with an issue about that 
> WG's work and raised it only during or after Last Call, I'd expect 
> either a really good explanation or a resignation.  I certainly would 
> not expect it to happen often. But, IMO, we have an IESG and, indeed, 
> an IETF rather than a collection of different organizations addressing 
> per-area issues precisely to increase the odds of catching serious 
> cross-area and cross-perspective issues before things go out.  Like it 
> or not, unless the relevant WG makes an effort to get broad reviews at 
> critical "early" points, there will always be a risk of late surprises 
> as someone --in the community during Last Call or on the IESG-- 
> suddenly wakes up and says "but how does this relate to XYZ".

I was obviously not clear enough on my first post because I wasn't referring
to the shepherding AD or the poor AD's who have paid no attention at all
who are then tapped to do their job. It's really the in between AD's that I
think really exasperate the wg participants -- if they've paid enough 
attention
to attend the wg meetings and perhaps even subscribe to the mailing list and
registered concerns, I really think they owe it to the working group to 
either
get them out when the working group is in active issue resolving mode, or
to... well, just live with it. Or something. Frankly it does a lot of 
damage to
the IESG's reputation when you know that there's likely going to be 
trouble,
but there is nothing you can do to fend it off ahead of time because those
AD's have only given vague allusions to their non-amusement. It promotes
the vision of the unitary IESG which I think is a bad thing.
>
> With regard to textual nit-picking and evaluation of worthiness of 
> prose, I tend to agree with what I think you are saying. However, if a 
> document is too badly written to permit interoperable implementations 
> to be constructed without clarifying conversations among implementers, 
> authors, and/or the WG, then the document is a failure and needs 
> pushback.  As with late surprises, somewhat more proactive effort on 
> the part of WGs could prevent many of the problems we see, but...

I was using "wordsmithing" rather broadly. My probably idiosyncratic meaning
of "wordsmithing" here was "will this DISCUSS change the mechanics of the
protocol or not". If the answer is no, we're really just making the document
more ready for publication IMO. Something that does bring that possibility
is obviously a lot more serious. It's been my admittedly limited 
experience that
my version of "wordsmithing" is a lot more common, and the source of a lot
of delay to varying degrees of dubiousness.

       Mike

---

From: John C Klensin <[email-masked]>
Date: Sat, 30 Dec 2006 21:11:04 -0500

--On Saturday, December 30, 2006 6:00 PM -0800 Michael Thomas 
<[email-masked]> wrote:

> is obviously a lot more serious. It's been my admittedly
> limited experience that
> my version of "wordsmithing" is a lot more common, and the
> source of a lot of delay to varying degrees of dubiousness.

My own personal, long-standing view, is that "clear enough to 
implement" together with "comprehensible, even with some effort" 
ought to be the requirement for Proposed, with anything more 
than that either after Last Call or immediately before it just 
being a waste of time and a poor use of resources.  Personally, 
I'd apply a similar standard to what I expect the RFC Editor to 
do.   While the formal procedures say nothing about this 
particular issue, I've got a much higher threshold for Draft: I 
think those documents should not require more effort to read 
than the underlying protocol itself requires.

>From your comment above, I think that puts us in near-, if not 
complete, agreement.

    john