Subject: Re: Last Call: <draft-resnick-on-consensus-05.txt> (On Consensus and Humming in the IETF) to Informational RFC

From: Ted Hardie <[email-masked]>
Date: Mon, 7 Oct 2013 12:03:55 -0700

First, I am always happy when folks take the time to think deeply about the
IETF's processes and share those thoughts with the community.  I think the
conversation this has already started has been useful, and I hope that the
last call conversation is the same.

That said, I do not think this document is ready for publication as an RFC,
and I personally suspect that it wouldn't be the best fate for it even it
were.  On the second point, the truth is that informational RFCs are
treated as actual requests for comments much any more, but are taken as
fixed; if the points this raises are to be maintained as items of
conversation (which is my personal preference), then incorporating pieces
of it into the Tao, Edu Team documents, or WG training may be appropriate
instead.  That is, put this into some form where folks will not take it as
an item of dogma, but as the start of a conversation, and the community
will be better served.  Even as an Informational document, if it is
published as an RFC by a sitting AD via the IETF stream, it may not get
that treatment.

If the IESG does believe that this should be published as an RFC, I think
it continues to need serious work before publication.  At the very base, I
think it needs a much stronger sense of its audience (advice to WG chairs
and those that deal with them? new participants in the IETF? those who want
to understand the workings of the IETF from the outside?) and a structure
which relates to that audience.  I note, for example, that the document
references none of the IETF's process documents or discussions that arose
out previous work in this area; that's fine for some audiences but is going
to be missed by folks who might get handed this as an explanatory document
on how things work (or ought to) in the IETF.

Once it has that audience, some of the issues with the current document may
fall out, but among those with which I have personal disagreements:

The document consistently describes a test for objection model of document
processing; that's a fair summary of how the IETF works, but shoe-horning
the description into "rough consensus" because we like the slogan is not
really helping folks understand the IETF.  The core of the IETF is a
participatory process which is meant to be cooperative rather than
adversarial; to that extent it is fair to describe it in consensus terms.
Beyond that, however, we are really not seeking the consent of all parties,
even as an ideal.  We are trying to make sure that there are no known
technical issues with a proposal and that there is sufficient support to
believe that it will be implemented and deployed to the good of the
network.  Encouraging folks to believe that the ideal is full consensus is
highly problematic, for exactly the reasons that Pete later raises with
regards to voting:  it is very difficult to determine when you have the
consent of all parties if you cannot limit the parties in some way.  We
cannot name our members, so we cannot either count votes *or count all of
them as consenting*.

The document has vastly improved its description of compromise over its
previous iterations, for which Pete should be commended.  But it remains
problematic in its description of participation. In the section "Five
people for
and one hundred people against might still be rough consensus", the
document describes what can occur when one proposal is favoured over
another by a working group's active participants:  one set of participants
may recruit new voices to add to their preferences.  The difficulty with the
given description is that it is a strawman compared to that actual
complexity
of dealing with this situation, because it posits sales and marketing folks
being the new voices.  The far more difficult reality is that the voices
often
come from members of an impacted technical community who generally do
not attend or participate in the IETF.  Were they known participants in the
IETF process, it is clear that their message "I agree with X's points" would
be heard in one way, where them being new participants causes them, in
this approach, to be discounted entirely if they raise no new points.
Novelty
is the wrong test here, as is length of participation; the test that WG
chairs
must use is the much messier judgement call of what the impact of the
objections is on the likely implementation and deployment of the system
under
discussion.

Lastly, I think Pete has failed to capture that one reason for using
humming or hands is that it is easy for very active participants to
dominate a conversation
but much less easy for them to pretend to be a large group.  Particularly
in BoFs, using those methods to indicate the likely breadth of interest is
critical.  The same method can be used, with some of the caution Pete
recommends, to gauge whether an issue which appears to be contentious based
on a mic line is actually a problem.  It can also, in some cases, be a
valuable method of reinforcing the resolve a room that has already likely
come to a broad agreement.  That does not contravene Pete's point that this
should not be used to silence objections, but there are cases where it is
important in its own right.

Again, I believe this is a valuable conversation and one that ought to keep
going; my recommendation against publication should also be taken as a
recommendation both to Pete to keep working on these ideas and to the
General Area Director to support a forum for that discussion.

regards,

Ted Hardie

On Mon, Oct 7, 2013 at 9:48 AM, The IESG <[email-masked]> wrote:

>
> The IESG has received a request from an individual submitter to consider
> the following document:
> - 'On Consensus and Humming in the IETF'
>   <draft-resnick-on-consensus-05.txt> as Informational RFC
>
> The IESG plans to make a decision in the next few weeks, and solicits
> final comments on this action. Please send substantive comments to the
> [email-masked] mailing lists by 2013-11-04. Exceptionally, comments may be
> sent to [email-masked] instead. In either case, please retain the
> beginning of the Subject line to allow automated sorting.
>
> Abstract
>
>
>    The IETF has had a long tradition of doing its technical work through
>    a consensus process, taking into account the different views among
>    IETF participants and coming to (at least rough) consensus on
>    technical matters.  In particular, the IETF is supposed not to be run
>    by a "majority rule" philosophy.  This is why we engage in rituals
>    like "humming" instead of voting.  However, more and more of our
>    actions are now indistinguishable from voting, and quite often we are
>    letting the majority win the day, without consideration of minority
>    concerns.  This document is a collection of thoughts on what rough
>    consensus is, how we have gotten away from it, and the things we can
>    do in order to really achieve rough consensus.
>
>       Note (to be removed before publication): This document is quite
>       consciously being put forward as Informational.  It does not
>       propose to change any IETF processes and is therefore not a BCP.
>       It is simply a collection of principles, hopefully around which
>       the IETF can come to (at least rough) consensus.
>
>
>
>
> The file can be obtained via
> http://datatracker.ietf.org/doc/draft-resnick-on-consensus/
>
> IESG discussion can be tracked via
> http://datatracker.ietf.org/doc/draft-resnick-on-consensus/ballot/
>
>
> No IPR declarations have been submitted directly on this I-D.
>
>
>

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 08 Oct 2013 08:34:39 +1300

On 08/10/2013 08:03, Ted Hardie wrote:
...

> were.  On the second point, the truth is that informational RFCs are [not]
> fixed; 

I've inserted the "not" that Ted certainly intended. But I think he raises
an important point. If the phrase "Request For Comments" no longer means
what it says, we need another RFC, with a provisional title of
"Request For Comments Means What It Says".

We still see comments on RFC 791 reasonably often, and I see comments
on RFC 2460 practically every day. That's as it should be.

So I'd like to dispute Ted's point that by publishing a version of
resnick-on-consensus as an RFC, we will engrave its contents in stone.
If that's the case, we have an even deeper problem than misunderstandings
of rough consensus.

otoh Ted's specific points on the draft are all valuable.

    Brian

---

From: Ted Lemon <[email-masked]>
Date: Mon, 7 Oct 2013 16:35:03 -0400

> If that's the case, we have an even deeper problem than misunderstandings

Right, I think what Ted is describing is a BCP, not an Informational RFC.

---

From: John Leslie <[email-masked]>
Date: Mon, 7 Oct 2013 17:08:12 -0400

Brian E Carpenter <[email-masked]> wrote:
> 
> ... If the phrase "Request For Comments" no longer means what it says,
> we need another RFC, with a provisional title of

   ;^)

> on RFC 2460 practically every day. That's as it should be.

   Absolutely!

> resnick-on-consensus as an RFC, we will engrave its contents in stone.

   Well, of course, all RFCs have always been "archival" -- to that
extent, they _are_ "engraved in stone"...

> If that's the case, we have an even deeper problem than misunderstandings

   Alas, IMHO, we do. :^(

   Ted and Dave Crocker _have_ made very good comments on this I-D which
is proposed to be an Informational RFC. For the most part, these comments
could just as well come _after_ it is published as such.

   The _problem_ is that an RFC clearly labeled as an individual
contribution _should_ have value _as_ an individual contribution. It
should not have to pass muster of our famous peanut gallery before
being published.

   The story is completely different for working-group documents
published on the Standards Track. For those, our quality-control process
is quite necessary.

   Perhaps the problem comes from the boilerplate for Informational:
] 
] Status of This Memo
] 
]  This document is not an Internet Standards Track specification; it is
]  published for informational purposes.
] 
]  This document is a product of the Internet Engineering Task Force
]  (IETF).  It represents the consensus of the IETF community.  It has
]  received public review and has been approved for publication by the
]  Internet Engineering Steering Group (IESG).  Not all documents
]  approved by the IESG are a candidate for any level of Internet
]  Standard; see Section 2 of RFC 5741.
] 
]  Information about the current status of this document, any errata,
]  and how to provide feedback on it may be obtained at
]  http://www.rfc-editor.org/info/rfc7017.

   The only differences between this boilerplate and the boilerplate
for standards track is that the Standards Track has a different first
paragraph
]
] This is an Internet Standards Track document.

and the Standards Track boilerplate omits the sentence
] 
]  Not all documents approved by the IESG are a candidate for any level
]  of Internet Standard; see Section 2 of RFC 5741.

replacing it with
] 
]  Further information on Internet Standards is available in Section 2
]  of RFC 5741.

   (In fact, the IESG review is quite different for these categories.)

   We see that all these categories state, "It represents the consensus
of the IETF community." In fact, if there is an easy way to tell from
the published RFC whether an Informational RFC represents an individual
contribution or a Working Group output, it escapes me at the moment. :^(

   Thus perhaps Ted and Dave are right to hold this draft to a high
"consensus of the IETF community" standard.

   I just wish that were not so...

--
John Leslie <[email-masked]>

---

From: John Leslie <[email-masked]>
Date: Mon, 7 Oct 2013 19:23:14 -0400

> Right, I think what Ted is describing is a BCP, not an Informational RFC.

   Oh my! I just saw the IESG agenda, and this _is_ proposed for BCP.

   I retract anything I said which might criticize Ted and/or Dave Crocker
for being too picky!

---

From: Peter Saint-Andre <[email-masked]>
Date: Mon, 07 Oct 2013 17:32:01 -0600

On 10/7/13 5:23 PM, John Leslie wrote:
>>> So I'd like to dispute Ted's point that by publishing a version
>>> of resnick-on-consensus as an RFC, we will engrave its contents
>>> in stone. If that's the case, we have an even deeper problem
>>> than misunderstandings of rough consensus.
>> 

It might not be the first time that an IESG agenda has the wrong text
for intended status.

Peter

- -- 
Peter Saint-Andre
https://stpeter.im/

---

From: Pete Resnick <[email-masked]>
Date: Mon, 7 Oct 2013 22:49:50 -0500

On 10/7/13 6:23 PM, John Leslie wrote:
>     Oh my! I just saw the IESG agenda, and this_is_  proposed for BCP.
>    

No, it's not. I'm just prolific this month. What you see on the agenda 
is draft-resnick-retire-std1, not this document. That one *is* for BCP, 
but draft-resnick-on-consensus isn't.

Remain calm. All is well.

pr

-- 
Pete Resnick<http://www.qualcomm.com/~presnick/>
Qualcomm Technologies, Inc. - +1 (858)651-4478

---

From: Ted Hardie <[email-masked]>
Date: Tue, 8 Oct 2013 08:36:24 -0700

On Mon, Oct 7, 2013 at 12:34 PM, Brian E Carpenter <
[email-masked]> wrote:

> [not]
> I've inserted the "not" that Ted certainly intended.

Indeed, thanks for the correction.

> But I think he raises

> on RFC 2460 practically every day. That's as it should be.
And what are the RFC numbers for the comments?  If none, as I suspect, then
the comments aren't the same status as the documents--that's fine for RFC
791 and 2460, but it is not clear that Pete's document falls into the same
class.  I would argue it does not.

> If that's the case, we have an even deeper problem than misunderstandings
Archival may not mean "engraved in stone", but it does impute status.  If
we want, as a community, to create an archival document on this topic, then
we should take on the work.  Pete's document is a good spark for the
conversation that might kick off that work, but I personally don't think it
is a good output document for that; if it is meant to be a spark, I don't
see why moving it into
an archival series is useful for us at this stage.

regards,

Ted

---

From: Ted Hardie <[email-masked]>
Date: Tue, 8 Oct 2013 08:39:23 -0700

On Mon, Oct 7, 2013 at 1:35 PM, Ted Lemon <[email-masked]> wrote:

> To be clear here, I did not think Pete's document was going for BCP.  I
remain concerned that the publication of this document as an an
AD-sponsored Informational RFC will impute status to it as a community
conclusion, rather than the start of a conversation (or, rather, the
continuation of one).  Some of the comments of "I've wanted something like
this to hand to..." are part of what cause me to believe that.

And, to re-iterate, I do not think the document is ready, even if the IESG
believes that a document of this type should be published; it needs a much
clearer sense of audience as well as attention to the other points that
have been raised.

regards,

Ted

---

From: Dave Crocker <[email-masked]>
Date: Tue, 08 Oct 2013 08:47:56 -0700

On 10/8/2013 8:36 AM, Ted Hardie wrote:
> And what are the RFC numbers for the comments?  If none, as I suspect,
> then the comments aren't the same status as the documents--that's fine
> for RFC 791 and 2460, but it is not clear that Pete's document falls
> into the same class.  I would argue it does not.

Unfortunately the concern you are raising has often been applied to all 
sorts of IETF work.  Many bits of IETF work are subject to on-going 
comments and often reach the practical status of de facto -- or, in the 
case of the errata mechanism, IETF de jure -- modifications to the 
published document.

In fact, the line of argument you raise has frequently been lodged 
against the BCP construct.  Yet we keep finding BCPs useful to create.

    1.  Does the IETF need a modern, thorough, community-approved 
statement of it's consensus model and the application of the model? 
That is, both theory and practice.

        So far, it looks as if the community certainly thinks we do, and 
  strongly agree.  In fact I think we suffer greatly by not having it. 
And as we've gone through multiple generations of participants, we've 
tended towards reliance on catch-phrases, without a shared understanding 
of their deeper meaning and specific practice.  So folks invent their 
own meanings as best they can.  Something like Pete's draft is needed to 
provide shared substance to what we mean when we talk about rough consensus.

    2.  Should the statement be an RFC or something more malleable (and 
therefore ephemeral)?

        Why would we not want something this essential to be available 
through our formal publishing and archiving mechanism?  To the extent 
that later discussions prompt modifications, that's what the errata 
mechanism is for.  And eventual revision to the RFC.  Unless someone 
thinks that this core construct for the IETF is going to be subject to 
constant and fundamental modification???

d/

-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: Peter Saint-Andre <[email-masked]>
Date: Tue, 08 Oct 2013 09:53:47 -0600

>   <draft-resnick-on-consensus-05.txt> as Informational RFC

I would like to perform a thorough review and provide more detailed
feedback, but time is short right now. Here are a few questions in the
meantime...

The Abstract states:

Does that mean you aim to make *this* I-D an IETF stream RFC that will
itself have consensus, or do you instead aim to use this document to
generate discussion that might result in consensus in the future (e.g.,
as a BCP)? If the latter, publishing in the Independent Stream seems
sensible. If the former, then I think we need to have more discussion
(along the lines of other Last Call feedback I've glanced at).

The Introduction states:

   Our ideal is full consensus, but we don't
   require it: Full consensus would allow a single intransigent person
   who simply keeps saying "No!" to stop the process cold.

By "full consensus" do you mean "unanimity"?

And do you think that unanimity or "full consensus" is our ideal,
although an ideal that's not always reachable in practice? Or is our
ideal actually rough consensus (i.e., something like general agreement
without unanimity)?

If unanimity or "full consensus" is our ideal then we might expend more
energy to win over instransigent persons or those who are "in the rough"
than we would if rough consensus were our ideal. So I think it's
important to be clear on what we're aiming for.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Ted Lemon <[email-masked]>
Date: Tue, 8 Oct 2013 12:31:27 -0400

On Oct 8, 2013, at 11:39 AM, Ted Hardie <[email-masked]> wrote:
> To be clear here, I did not think Pete's document was going for BCP. 

Indeed, but you are speaking of it as if it were, which was my point.

---

From: Ted Hardie <[email-masked]>
Date: Tue, 8 Oct 2013 10:49:13 -0700

Some comments in-line.

On Tue, Oct 8, 2013 at 8:47 AM, Dave Crocker <[email-masked]> wrote:

> In fact, the line of argument you raise has frequently been lodged against
> the BCP construct.  Yet we keep finding BCPs useful to create.
>
>
>    1.  Does the IETF need a modern, thorough, community-approved statement
> of it's consensus model and the application of the model? That is, both
> theory and practice.
>
>  strongly agree.  In fact I think we suffer greatly by not having it. And
> as we've gone through multiple generations of participants, we've tended
> towards reliance on catch-phrases, without a shared understanding of their
> deeper meaning and specific practice.  So folks invent their own meanings
> as best they can.  Something like Pete's draft is needed to provide shared
> substance to what we mean when we talk about rough consensus.
>
>
If the community believes that we need a community-approved statement of
its  decision-making model and how it is applied, then we should develop one
in a community process.  It may at that point be something that becomes a
BCP.

As an input draft to that discussion or community process, I think Pete's
draft is very useful--it has sparked multiple rounds of discussion.  But I
don't think it is clear that this is its intended function (that "unclear
on the audience bit") and I think it might be read to be a proposed output
document from that process. I don't think it is anywhere near ready to be
considered as an output document, for reasons I already laid out.

> through our formal publishing and archiving mechanism?  To the extent that
> later discussions prompt modifications, that's what the errata mechanism is
> for.  And eventual revision to the RFC.  Unless someone thinks that this
> core construct for the IETF is going to be subject to constant and
> fundamental modification???
>
>
Again, I think this is the question of the document's audience and
function.  If the aim is to use it to spark debate, than ephemeral is
better than fixed.  If it is meant to be a community statement of our
process, in theory and practice, it should be fixed--but this document is
not that community statement in its current form.

regards,

Ted

---

From: "Fred Baker (fred)" <[email-masked]>
Date: Tue, 8 Oct 2013 19:05:01 +0000

On Oct 7, 2013, at 12:03 PM, Ted Hardie <[email-masked]> wrote:

> Lastly, I think Pete has failed to capture that one reason for using humming or hands is that it is easy for very active participants to dominate a conversation
> but much less easy for them to pretend to be a large group.  Particularly in BoFs, using those methods to indicate the likely breadth of interest is critical.  The same method can be used, with some of the caution Pete recommends, to gauge whether an issue which appears to be contentious based on a mic line is actually a problem.  It can also, in some cases, be a valuable method of reinforcing the resolve a room that has already likely come to a broad agreement.  That does not contravene Pete's point that this should not be used to silence objections, but there are cases where it is important in its own right.

In my working group, that is the principal use of a hum. It is useful when a set of people have made a viewpoint known, and the chairs are trying to see if that represents a general view, or whether a focused minority is speaking.

---

From: S Moonesamy <[email-masked]>
Date: Tue, 08 Oct 2013 13:56:14 -0700

>[email-masked] mailing lists by 2013-11-04. Exceptionally, comments may be

I found the review by Dave Crocker [1] interesting.  Instead of 
reading the latest revision of the draft I wrote a draft [2].  I read 
what Pete Resnick said about consensus after that to compare notes.

The intended status of draft-resnick-on-consensus-05 is 
Informational.  What we have here is a document about consensus which 
will reflect the consensus of the IETF.  Should the document reflect 
the consensus of the IETF or not?

In Section 1:

   'Our ideal is full consensus, but we don't require it: Full consensus
    would allow a single intransigent person who simply keeps saying
    "No!" to stop the process cold.'

The above introduces the term "full consensus".  I took a quick look 
and I found at least one occurrence of the term in IETF discussions 
[3].  However, none of the IETF process documents use that term.

   "If the chair of a working group determines that a technical issue
    brought forward by an objector has been truly considered by the
    working group, and the working group has made an informed decision
    that the objection has been answered or is not enough of a
    technical problem to prevent moving forward, the chair can declare
    that there is rough consensus to go forward, the objection
    notwithstanding."

The word "objector" emphasizes that there is one person who 
dissents.  My preference is to consider the objection and address it 
instead of viewing the issue as dissent from one person.

    "This document attempts to explain some features of rough consensus,
     explain what is not rough consensus, and suggest ways that we might
     achieve rough consensus and judge it in the IETF."

The title of the document is "on consensus and humming in the 
IETF".  According to the above sentence the document attempts to 
discuss about rough consensus.  In my opinion there a nuance between 
"consensus" and "rough consensus".  As a quick reaction I would say 
that rough consensus provides a faster path to shape up a 
proposal.  It helps to cut down on document delivery time to the 
IESG.  The consensus part is sought by getting a broader perspective.

Section 2 sounds like objection-based processing.  A binary choice 
(re. technical question) can end up polarizing a discussion.  The 
section provides a good discussion about lack of disagreement.

One of the questions I wondered about is whether the person making 
the determination should use technical judgement or whether the 
person should only make a determination about the comments 
received.  I mentioned in an unrelated discussion that if it is the 
consensus of the group that the sky is green, that's what goes in the 
document.  The person making the determination can flag it as an 
issue as a matter of technical judgement.  I'll highlight a point 
from Section 3:

   "Remember, if the objector feels that the issue is so essential that it
    must be attended to, they always have the option to file an appeal."

There are very few people who exercise that option.

According to the title of Section 4 humming should be the start of a 
conversation, not the end.  BCP 25 states that:

   "Consensus can be determined by a show of hands, humming, or any
    other means on which the WG agrees (by rough consensus, of course)."

I am not sure whether hums are for a starting point or not.  It can 
be argued in different ways, for example, see Section 4.  Humming 
helps to get a sense of the room without people making a decision 
under duress.  It is a useful way to resolve a controversy.  I would 
say that it ties in Section 5, i.e. consensus is the path, not the 
destination.  A show of hands is a disguised way to vote [4].

Section 5 identifies a few issues with the way people talk about 
"consensus".  I understand what Pete Resnick wrote as he has 
explained that to me in an unrelated discussion.  The text discusses 
about "making the call".  I don't know whether the reader will easily 
understand the meaning of that.

Section 6 and Section 7 attempt to explain that a high percentage of 
votes in a direction does not necessarily mean that there is rough 
consensus for that.  I agree with the conclusion in Section 8.

Regards,
S. Moonesamy

1. http://www.ietf.org/mail-archive/web/ietf/current/msg82843.html
2. http://tools.ietf.org/html/draft-moonesamy-consensus-00
3. http://www.ietf.org/mail-archive/web/isms/current/msg00943.html
4. http://www.ietf.org/mail-archive/web/ietf/current/msg25014.html

---

From: "Fred Baker (fred)" <[email-masked]>
Date: Tue, 8 Oct 2013 23:21:07 +0000

On Oct 8, 2013, at 1:56 PM, S Moonesamy <[email-masked]>
 wrote:

> I am not sure whether hums are for a starting point or not.  It can be argued in different ways, for example, see Section 4. Humming helps to get a sense of the room without people making a decision under duress. 

Personally, I think focusing on Jeff Case's hums is missing the point. The point is the meaning of the term "rough consensus", and how that plays out in working group process. The manner of measurement is a secondary issue.

To my small and somewhat naive mind, the difference between rough consensus on a topic and a vote on the same topic is something about winners and losers. In a purely political process, when a set of parties vote on something and the preponderance (by some definition of "preponderance") say something, the views of the losing set of parties are deemed irrelevant. In IETF process, and hopefully in any technical process, there is understanding that the parties who disagree may have valid reasons to disagree, and a phase of negotiation. When we talk about "rough consensus", I understand it to mean - and would like to believe that we all understand it this way - that we investigate the reasons for disagreement, perhaps discover that some of them are valid, and address those issues to the satisfaction of those who raised them. As a result, the ultimate solution, even though it may not be the specific solution we would all have designed or selected, is one that in fact addresses all known issues. While we may not all agree, we don't disagree.

I think the document on the table tries to address that. There are points of phraseology that I might express differently, but it's close enough that I don't disagree.

---

From: Melinda Shore <[email-masked]>
Date: Tue, 08 Oct 2013 19:23:01 -0800

On 10/8/13 3:21 PM, Fred Baker (fred) wrote:
> To my small and somewhat naive mind, the difference between rough
> consensus on a topic and a vote on the same topic is something about
> winners and losers. In a purely political process, when a set of
> parties vote on something and the preponderance (by some definition
> of "preponderance") say something, the views of the losing set of
> parties are deemed irrelevant. In IETF process, and hopefully in any
> technical process, there is understanding that the parties who
> disagree may have valid reasons to disagree, and a phase of
> negotiation. When we talk about "rough consensus", I understand it to
> mean - and would like to believe that we all understand it this way -
> that we investigate the reasons for disagreement, perhaps discover
> that some of them are valid, and address those issues to the
> satisfaction of those who raised them. As a result, the ultimate
> solution, even though it may not be the specific solution we would
> all have designed or selected, is one that in fact addresses all
> known issues. While we may not all agree, we don't disagree.

I've done a lot of work on consensus over the years and I think
this is fundamentally correct, although I'd amend the last sentence
to something along the lines of "While we may not all agree, those
who disagree can live with it."  That is to say, it's not a binary
question, and sometimes things we disagree with just aren't
showstoppers.  (I'd like to see people take that position more
often - for some reason a lot of people seem to take disagreement
as a reason to block a decision even when it doesn't matter that
much).

Melinda

---

From: "Fred Baker (fred)" <[email-masked]>
Date: Wed, 9 Oct 2013 04:08:21 +0000

> as a reason to block a decision even when it doesn't matter that

wfm

---

From: Loa Andersson <[email-masked]>
Date: Wed, 09 Oct 2013 13:20:35 +0800

All,

FWIW - my personal way of thinking about consensus vd. rough consensus,
please note that it my personal view not a definition.

Consensus - An agreement by everyone in a group that a proposed
             solution is the best of all of all possible solutions

Rough consensus - An agreement by almost everyone that the proposed
             solution is good enough for everyone to live with.

/Loa

-- 

Loa Andersson                        email: [email-masked]
Senior MPLS Expert                          [email-masked]
Huawei Technologies (consultant)     phone: +46 739 81 21 64

---

From: Melinda Shore <[email-masked]>
Date: Tue, 08 Oct 2013 21:30:23 -0800

> Rough consensus - An agreement by almost everyone that the proposed

That's a lot like voting, I think.

Melinda

---

From: Loa Andersson <[email-masked]>
Date: Wed, 09 Oct 2013 13:53:59 +0800

On 2013-10-09 13:30, Melinda Shore wrote:
>> FWIW - my personal way of thinking about consensus vs. rough consensus,
> That's a lot like voting, I think.

Well - if you say so, but then we don't have even a rough consensus on
what consensus and rough consensus means.

Note I talked about what a group need to reach to be able to say that
there is "consensus" or "rough consensus". Not how it is measured, in
IETF we trust the wg chairs to correctly judge if we have reached a
rough consensus.

/Loa

--

---

From: Eliot Lear <[email-masked]>
Date: Wed, 09 Oct 2013 06:20:42 -0400

Pete,

As usual, I really like your writing style, and I think you're
addressing a very important issue.  There are two aspects that I would
suggest require further exploration, both having to do with the role of
the chair (the whole document has to do with the role of the chair, I
suppose):

1.  No matter how you slice the definition of rough consensus, if the
chair does not act in a fair and balanced way, the outcome will be
incorrect.  This is what the appeals process is for, and I would suggest
mentioning it, perhaps in some detail.

2.  The case of Section 7 is, as you say, a mind bender.  I would
suggest adding another use case: what if those 100 people write their
own draft.  Can they use the exact same process to get the draft adopted
and approved, so long as they answer the technical issues that arise? 
In other words, if there are multiple valid alternatives, and one suits
one vendor group and another suits another, can there be just one
standard?  At the neck of the hourglass, perhaps so?  What happens in
this case, from your point of view?  What makes group (a) more special
than group (b)?

Eliot

---

From: Ted Lemon <[email-masked]>
Date: Wed, 9 Oct 2013 08:35:14 -0400

> That's a lot like voting, I think.

It's worse than voting, because it encourages people to invite their friends to sway the consensus.   At least with voting you have the transparency of knowing who the electorate is.   So if this were what "rough consensus" meant, rough consensus would most often be won by whoever has the most friends, which in practice is probably whoever works for the biggest company.  So whatever "rough consensus" means, it can't mean "only a few people disagree."

---

From: Loa Andersson <[email-masked]>
Date: Wed, 09 Oct 2013 22:04:36 +0800

> It's worse than voting, because it encourages people to invite their friends to sway the consensus.   At least with voting you have the transparency of knowing who the electorate is.   So if this w…

OK - I don't know what is wrong, but either
- I can't write; or
- some leading members of our community can't or don't care to read

Ted,

what you are describing is is not a "group", it is a group or a loosely
collection of people subject to manipulation.

There is no room for that type of manipulation if we shall be able to
talk about consensus/rough consensus (at least we seem to agree on
that).

If we have wg-chairs, ADs or chairs that are not able to detect this
kind of manipulation we are in bad shape.

What I tried to talk about is the the state of mind the (un-manipulated) 
group needs to be in before someone can make a
consensus call; again not a definition, just a way of thinking about
it.

With that I rest my case.

/Loa

--

---

From: Melinda Shore <[email-masked]>
Date: Wed, 09 Oct 2013 09:22:29 -0800

> It's worse than voting, because it encourages people to invite their
> friends to sway the consensus. 

Excellent point.

Melinda

---

From: Jari Arkko <[email-masked]>
Date: Thu, 10 Oct 2013 23:42:25 +0300

FWIW, on the issue of Informational RFCs seen as cast in stone:

I think I've seen that problem occasionally. I.e. people assigning a far too high value to a document, just because it is an RFC. The world changes, our understanding changes, and as Dave pointed out processes evolve… RFCs need to change now and then as well. 

I agree that the issue is much more serious for BCPs. But I think we really need to get over the issue. The best defense for that problem is to be seen publishing and revising RFCs. If we start to be afraid of publishing an RFC… I think we've already lost too much. I do not want to go there. 

Just a personal opinion, of course.

Jari

---

From: Pete Resnick <[email-masked]>
Date: Sun, 27 Oct 2013 04:28:15 -0500

Finally getting back to this. Overall, I hope that the new version which 
I'm about to submit (Jari has given clearance to post during the 
blackout given that this is not being discussed in Vancouver) addresses 
these (and others') comments. Some specifics:

On 10/8/13 3:03 AM, Ted Hardie wrote:
> First, I am always happy when folks take the time to think deeply 
> about the IETF's processes and share those thoughts with the 
> community.  I think the conversation this has already started has been 
> useful, and I hope that the last call conversation is the same.

I appreciate that.

> That said, I do not think this document is ready for publication as an 
> RFC, and I personally suspect that it wouldn't be the best fate for it 
> even it were.  On the second point, the truth is that informational 
> RFCs are treated as actual requests for comments much any more, but 
> are taken as fixed; if the points this raises are to be maintained as 
> items of conversation (which is my personal preference), then 
> incorporating pieces of it into the Tao, Edu Team documents, or WG 
> training may be appropriate instead.  That is, put this into some form 
> where folks will not take it as an item of dogma, but as the start of 
> a conversation, and the community will be better served.  Even as an 
> Informational document, if it is published as an RFC by a sitting AD 
> via the IETF stream, it may not get that treatment.

I will leave it to Jari and the other members of the IESG to make the 
call on this. However, I do think some of these points have solidified 
to the point that having a stable reference is good. I wouldn't object 
in principle to it being published in the the form of a web page a la 
the Tao, or some Edu Team documents, but I think having an Informational 
RFC does give it some wider viewing. I've already heard from people who, 
due to the Last Call, are looking at this document in other 
organizations and find the discussion useful and interesting. I also 
don't harbor the fears that Ted does about it being treated as dogma. It 
is true that people latch on to all sorts of things as dogma, but they 
already have plenty of them that disagree with the notions in this 
document, so the counterbalance doesn't seem so bad. But again, I only 
have a personal leaning at this point toward publication, and I'm 
inclined to leave it to Jari to make the call.

> If the IESG does believe that this should be published as an RFC, I 
> think it continues to need serious work before publication.  At the 
> very base, I think it needs a much stronger sense of its audience 
> (advice to WG chairs and those that deal with them? new participants 
> in the IETF? those who want to understand the workings of the IETF 
> from the outside?) and a structure which relates to that audience.

I hope I have this clarified. Here's the paragraph I've now got in the 
intro:

    This document attempts to explain some features of rough consensus,
    explain what is not rough consensus, discuss some new ways to think
    about rough consensus, and suggest ways that we might achieve rough
    consensus and judge it in the IETF. Though this document describes
    some behaviors of working groups and chairs, it does so in broad
    brushstrokes and it does not specific procedures. Rather, this
    document is intended to foster understanding of the underlying
    principles of IETF consensus processes. While it may be of general
    interest to anyone interested in the IETF consensus processes, the
    primary audience for this document is expected to be those who have
    experience working in the IETF, and it is particularly aimed at
    generating thought and discussion among those who might lead a
    consensus discussion. Although most of the examples in this document
    talk about working group chairs, these principles are meant to apply
    to any person who is trying to lead a group to rough consensus,
    whether a chair, a design team leader, a document editor, an IESG
    area director, or any community member who is facilitating a discussion.

> The document consistently describes a test for objection model of 
> document processing; that's a fair summary of how the IETF works, but 
> shoe-horning the description into "rough consensus" because we like 
> the slogan is not really helping folks understand the IETF.  The core 
> of the IETF is a participatory process which is meant to be 
> cooperative rather than adversarial; to that extent it is fair to 
> describe it in consensus terms.  Beyond that, however, we are really 
> not seeking the consent of all parties, even as an ideal.  We are 
> trying to make sure that there are no known technical issues with a 
> proposal and that there is sufficient support to believe that it will 
> be implemented and deployed to the good of the network.

Some of this is addressed by the things I have done to address Dave's 
concerns: I've tried to make it clear what is a new view of consensus 
versus what we've actually been doing. However, in one sense I actually 
believe the opposite of what Ted says here: To claim that the what we 
are "really" doing at core is simply to make sure that there are no 
known technical issues and that there is "sufficient" support is to 
reject that idea that consensus (rough or otherwise) is important at 
all. All you need to determine that things are for the "good of the 
network" is to have a (perhaps benevolent) president or king. We reject 
that. We believe that coming to a meeting of the minds (of whoever is 
participating at the moment) over the technical issues is actually 
important.

Now, at the end of the day, I don't think Ted and I are really that far 
apart. In the end, we expect everyone to be working toward no known 
technical issues and the good of the network, so putting all of those 
heads to get a consensus is pretty likely to get the right technical 
outcome. But I think more often than not it *is* the objections that 
cause a group to rethink the consensus technical decision it may have 
arrived at (in good faith), and it is the re-forming of consensus around 
the objection, or the acceptance of the objection and placing it "in the 
rough" that really should be how we view our process.

Either way, I've tried to adjust the language a bit along these lines. I 
don't know that we're going to completely agree, but see if the changes 
make things any better in your view.

> Encouraging folks to believe that the ideal is full consensus is 
> highly problematic, for exactly the reasons that Pete later raises 
> with regards to voting:  it is very difficult to determine when you 
> have the consent of all parties if you cannot limit the parties in 
> some way.  We cannot name our members, so we cannot either count votes 
> *or count all of them as consenting*.

This I disagree with. Or we might be in violent agreement: The ideal is 
to come up with a solution that *everyone who might come along* agrees 
is the right technical solution. You don't need to count heads (or know 
which heads you're counting) to do that. Of course this is a theoretical 
ideal: You can't know that you've ever satisfied everyone who might come 
along. But that's OK. The only thing we can do is say, "We've addressed 
all known technical problems and made all known correct engineering 
tradeoffs. If others come along, we'll deal with them." That's the only 
criteria we can really use. I don't see how the theoretical ideal of 
everyone agreeing is a problematic concept.

> In the section "Five people for and one hundred people against might 
> still be rough consensus", the document describes what can occur when 
> one proposal is favoured over another by a working group's active 
> participants:  one set of participants may recruit new voices to add 
> to their preferences.  The difficulty with the given description is 
> that it is a strawman compared to that actual complexity of dealing 
> with this situation, because it posits sales and marketing folks being 
> the new voices.  The far more difficult reality is that the voices 
> often come from members of an impacted technical community who 
> generally do not attend or participate in the IETF.

Yup. I've added a paragraph to talk about just that.

> dominate a conversation but much less easy for them to pretend to be a 
> large group.  Particularly in BoFs, using those methods to indicate 
> the likely breadth of interest is critical.  The same method can be 
> used, with some of the caution Pete recommends, to gauge whether an 
> issue which appears to be contentious based on a mic line is actually 
> a problem.  It can also, in some cases, be a valuable method of 
> reinforcing the resolve a room that has already likely come to a broad 
> agreement.  That does not contravene Pete's point that this should not 
> be used to silence objections, but there are cases where it is 

I've added a bit about this. But I am also somewhat wary of going too 
far with it. As I say in the document, hums (or shows of hands) have the 
potential to sway a working group due to herd mentality: Someone may be 
dissuaded from raising what could be an essential concern if they 
somehow feel that the group is overwhelmingly for a particular position, 
or some folks may decide to "hum along with the crowd" even though 
they're not committed to the outcome. I've mentioned the positive and 
the negative of this case.

> Again, I believe this is a valuable conversation and one that ought to 
> keep going; my recommendation against publication should also be taken 
> as a recommendation both to Pete to keep working on these ideas and to 
> the General Area Director to support a forum for that discussion.

Thanks for that, and all of the comments.

pr

---

From: Pete Resnick <[email-masked]>
Date: Sun, 27 Oct 2013 04:28:49 -0500

I believe the status of the document issue was addressed elsewhere. On 
the other point:

On 10/8/13 10:53 AM, Peter Saint-Andre wrote:

> do you think that unanimity or "full consensus" is our ideal,

Given the context of the document, that consensus is based on whether 
all issues are addressed, I do think that unanimity is our theoretical 
ideal: We do try to convince people that we *have* answered their issues 
and give up (i.e., declare rough consensus) when we feel we have but 
they're still not convinced. If rough consensus were the ideal, I think 
it would encourage a habit of declaring things in the rough that have 
not gotten real airing. ("Well, all we're really aiming for is rough 
consensus, and I'm pretty sure Pete's issue has been addressed, so let's 
just move along.")

pr

---

From: Pete Resnick <[email-masked]>
Date: Sun, 27 Oct 2013 05:40:01 -0500

> mentioning it, perhaps in some detail.

I do allude to this at the end of section 3, and I've highlighted it a 
bit better with one of the edits Dave suggested. But every time I tried 
to elaborate the "detail", I found myself talking more about the theory 
of appeals rather than the theory of consensus. I'm open to some 
suggested text if you have some.

> this case, from your point of view?  What makes group (a) more special

Though I think your example adds delightfully to the mind bender, I'm 
inclined to avoid it. At one level, 100 proposed drafts is no different 
than 100 proposed objections, though I would contend that if the drafts 
are actually well written, they would have to have good reasoning as to 
why the WG's original choice was not the best one. One way to sort your 
proposed mess is for the chair to go back to the charter of the WG and 
see which proposal actually meets the charter requirement. Or the chair 
can make the question about who the assigned document editor is. But 
really what your proposing is just a step worse into the "pathological 
WG" problem. And really we could do this ad infinitum. (I'm reminded of 
the old George Carlin routine about his days in Catholic school and "Ask 
the priest" day: "So, Father, say you haven't received your Easter 
communion. And it's Pentecost Sunday, the last day. And you're on a ship 
at sea. And the chaplain goes into a coma. But you wanted to receive. 
And then it's Monday, too late. And then you cross the international 
dateline...".) The fact is that though rough consensus is a resilient 
process, it's not bulletproof to every attack. This document can only go 
so far into how it deals with problems. I've added a good bit to that 
section to address Ted's comments. Hopefully you find that satisfying.

pr

---

From: Eliot Lear <[email-masked]>
Date: Sun, 27 Oct 2013 22:14:17 +0100

Hi Pete,

Thanks for your reply.  Please see below.

On 10/27/13 11:40 AM, Pete Resnick wrote:
> bit better with one of the edits Dave suggested. But every time I
> tried to elaborate the "detail", I found myself talking more about the
> theory of appeals rather than the theory of consensus. I'm open to
> some suggested text if you have some.

If you've got an updated version I'd take another look.  On further
reflection, tho, detail might lead to precisely what you're concerned
about.  What I was really aiming at was this:

> The fact is that though rough consensus is a resilient process, it's
> not bulletproof to every attack. This document can only go so far into
> how it deals with problems. I've added a good bit to that section to
> address Ted's comments. Hopefully you find that satisfying.
>

Those sentences are excellent.  I think your point about looking at the
charter is key, because there is advice to area directors hidden there
about being clear on what is supposed to be done.

Eliot

---

From: Pete Resnick <[email-masked]>
Date: Tue, 29 Oct 2013 11:33:24 -0500

Last of the messages in my list:

> the consensus of the IETF or not?

See other messages. I think the intention is to make it a consensus 
Informational document.

> ...Section 1...introduces the term "full consensus".  I took a quick 
> look and I found at least one occurrence of the term in IETF 
> discussions [3].  However, none of the IETF process documents use that 
> term.

See my reply to PSA. I don't think that there's a real need for a formal 
definition. I've thrown in the word "unanimity" to clarify.

> The word "objector" emphasizes that there is one person who dissents.  
> My preference is to consider the objection and address it instead of 
> viewing the issue as dissent from one person.

That's what the example above says. It is the *objection* that is 
answered, not the objector. It is, after all, a hypothetical.

> The title of the document is "on consensus and humming in the IETF".  
> According to the above sentence the document attempts to discuss about 
> rough consensus.  In my opinion there a nuance between "consensus" and 
> "rough consensus".

It's more than an nuance. It is the distinction made between sections 2 
and 3.

> As a quick reaction I would say that rough consensus provides a faster 
> path to shape up a proposal.  It helps to cut down on document 
> delivery time to the IESG.  The consensus part is sought by getting a 
> broader perspective.

I'm not clear on what if any change you intend here.

> section provides a good discussion about lack of disagreement.

See my response to Ted on this point. I think objection-based processing 
is part of how we come to consensus (rough or otherwise). And I think 
this section does talk about the problems of posing binary choices. I've 
added text to section 5 on this.

> One of the questions I wondered about is whether the person making the 
> determination should use technical judgement or whether the person 
> should only make a determination about the comments received.  I 
> mentioned in an unrelated discussion that if it is the consensus of 
> the group that the sky is green, that's what goes in the document.  
> The person making the determination can flag it as an issue as a 
> matter of technical judgement.

I think section 3 is pretty clear that the chair must exercise technical 
judgment in many cases. Your hypothetical is the extreme case not 
mentioned in the document. If the group says that the sky is green, 
*and* nobody questions that assertion, *and* the chair suspects the sky 
is not green, I believe it is reasonable for the chair to pose the 
question to the WG: "Are we sure the sky is green?" If the working group 
simply says "Yes" without explanation, I think the chair is within 
bounds to say, just as they are if someone else had raised the issue, "I 
don't think the group has truly considered and weighed this issue such 
that I can declare (rough) consensus on this point." But this must be 
done *extremely* carefully. Chairs who insert their technical judgment 
too often or too heavy-handedly risk losing the confidence of the group 
as an impartial consensus caller. Better might be for the chair to 
(perhaps quietly) solicit a review from a sky color expert from another 
working group, asking for them to comment on this sky being green issue. 
But having the document go forward with a technical issue that the chair 
knows about but that the rest of the working group hasn't noticed is, I 
think, to shirk responsibility as a member of the community.

All that said, I wonder if it's necessary to say this in the document. 
It's a more general comment about the chair role rather than about 
consensus per se.

> I'll highlight a point from Section 3:
>

I think (hope) that if we start thinking about consensus the way the 
document describes, appeals (at least in the early form, where it's 
simply discussing the issue with the chair or the AD) would not be 
nearly so fraught.

> I am not sure whether hums are for a starting point or not.  It can be 
> argued in different ways, for example, see Section 4.  Humming helps 
> to get a sense of the room without people making a decision under 
> duress.  It is a useful way to resolve a controversy.  I would say 
> that it ties in Section 5, i.e. consensus is the path, not the 

Yup. See my response to Russ, and some of the changes I've made in the 
document. I think this is handled more clearly.

> "consensus".  I understand what Pete Resnick wrote as he has explained 
> that to me in an unrelated discussion.  The text discusses about 
> "making the call".  I don't know whether the reader will easily 

I've added a bit to section 5. Send text if you think it is not sufficient.

> consensus for that.  I agree with the conclusion in Section 8.

Excellent.

With that, I will send out a new version ASAP. As I said earlier, Jari 
has pushed the magic "please let it through" buttons so that I can post 
this week.

pr

---

From: S Moonesamy <[email-masked]>
Date: Thu, 31 Oct 2013 00:06:29 -0700

Hi Pete,
>See other messages. I think the intention is to make it a consensus 

Ok.  As the document is Informational I'll read it according to BCP 9.

>I think section 3 is pretty clear that the chair must exercise 
>technical judgment in many cases. Your hypothetical is the extreme 
>case not mentioned in the document. If the group says that the sky 
>is green, *and* nobody questions that assertion, *and* the chair 
>suspects the sky is not green, I believe it is reasonable for the 
>chair to pose the question to the WG: "Are we sure the sky is 
>green?" If the working group simply says "Yes" without explanation, 
>I think the chair is within bounds to say, just as they are if 
>someone else had raised the issue, "I don't think the group has 
>truly considered and weighed this issue such that I can declare 
>(rough) consensus on this point." But this must be done *extremely* 
>carefully. Chairs who insert their technical judgment too often or 
>too heavy-handedly risk losing the confidence of the group as an 
>impartial consensus caller. Better might be for the chair to 
>(perhaps quietly) solicit a review from a sky color expert from 
>another working group, asking for them to comment on this sky being 
>green issue. But having the document go forward with a technical 
>issue that the chair knows about but that the rest of the working 
>group hasn't noticed is, I think, to shirk responsibility as a 
>member of the community.
>
>All that said, I wonder if it's necessary to say this in the 
>document. It's a more general comment about the chair role rather 
>than about consensus per se.

I preferred to use a case which is not related to any IETF 
discussion.  The reason I brought this up is because the Chair may 
not be viewed as impartial (re. above comment).  Thanks for 
explaining the above.

>simply discussing the issue with the chair or the AD) would not be 

Yes.

>I've added a bit to section 5. Send text if you think it is not sufficient.

I'll send text off-list if I notice anything.

Regards,
S. Moonesamy