Subject: Thoughts on the nomcom process

From: Lakshminath Dondeti <[email-masked]>
Date: Sat, 15 Mar 2008 20:10:38 -0700

Folks,

I spent the last five days listening to the debates on the nomcom 
candidate confirmation process and the proposals to fix the process.  As 
much as possible, I tried to stay away from the debates so I can 
carefully listen, understand and reflect.  It appears that opinions on 
the causes ranged from 'everyone messed up' to 'everyone did the right 
thing, under the circumstances.'

After listening to the many opinions, I further thought about the 
proceedings of the past 9 months and reflected on what went wrong. 
Next, I went back and reread various notes and emails.  Andrew, this 
year's nomcom adviser, sent an email to me on July 24, 2007, titled 
"Notes to next-year's chair" in which he said the following confirmation 
process:

"I'll be happy to chat about the confirmation process well ahead of time 
so you know what to expect."

We had that conversation in Chicago, and I recall Andrew noting that 
there were difficulties in the confirmation process with the IAB, and 
that he'll help out when the time comes.  I recall noting that I will be 
extra careful in ensuring that we do all the due diligence in candidate 
selection and in preparation of testimony.  However, I stopped there; in 
retrospect I should have exhibited more common sense and curiosity and 
asked Andrew more questions.  But, I did not do that!  I guess I can 
think of many excuses, but after all, I accepted the position and 
clearly did not do a good job.  I can never explain how hard it is for 
me to say this, but I am sorry for my mistake.  I know I made at least 
one more mistake in running the process.

I think it is unfair to blame the entire nomcom in this case.  The 
volunteer voting members deserve the thanks and appreciation that the 
community at large showed them at the plenaries last week, but none of 
the blame.  Please direct any blame on the nomcom at me.  I accepted the 
position of chairing this nomcom and the mistake of not catching this 
issue early enough is mine.

So, what should we do now?  First, I don't believe what Harald has 
suggested is the right direction.  I believe that the IAB's 
interpretation of 3777 on the matter of the confirmation process sets a 
dangerous precedence whereby one of the confirming bodies could require 
that the nomcom provide (samples of) verbatim feedback.  One could 
interpret the same text that the IAB cites -- all information and any 
means acceptable to them -- as being in support of that requirement as 
well.  The community tends to express their lack of confidence in the 
process by not participating, for instance, not volunteering or not 
providing feedback.

So, what is the best direction?  I think a revision to 3777 is in fact 
needed.  There are at least three things to do:
1. Define the roles and responsibilities of the people and the bodies 
involved in the process, more clearly.
2. Define the expectations on the confidentiality of the information 
collected.
3. Specify the nature information sharing within the nomcom process.

Once we arrive at consensus conclusions on those, state those and revise 
3777 to delete text that is inconsistent with those conclusions.  Unlike 
other parts of our process, nomcoms are under hard time constraints and 
much of the work happens in secrecy.  We are already late for any 
revision to be useful for the next nomcom.

thanks,
Lakshminath
_______________________________________________
IETF mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ietf

---

From: "Spencer Dawkins" <[email-masked]>
Date: Sun, 16 Mar 2008 06:52:30 -0500

Hi, Lakshminath,

> revision to be useful for the next nomcom.

I'm speaking as someone who has filled out NomCom questionnaires in the 
past, multiple times, and someone who never dreamed that I should be looking 
on the IAB website for clues about what information would be shared outside 
the NomCom...

You did something very GOOD this time - you invoked the never-before-invoked 
arbitration process, instead of secretly resolving an "irresistable force 
meets immovable object" dispute. Please accept credit for this good 
judgement, especially if you are assigning blame to yourself.

Obviously, we're not going to revise RFC 3777 before the next NomCom is 
seated. The discussion to date has already identified an improvement that 
doesn't require RFC 3777bis in order to be implemented - clearly divide the 
questionnaire into "this information may be shared with confirming bodies" 
and "this information will not be shared outside NomCom". I think any sane 
incoming NomCom chair would plan to implement that improvement, based on 
this year's experiences, which past NomCom chairs have congratulated you for 
publicizing broadly.

So, I don't think you guys did too badly. ;-)

And thanks for your service to the community.

Spencer

---

From: Dave Crocker <[email-masked]>
Date: Sun, 16 Mar 2008 13:06:16 -0700

> judgement, especially if you are assigning blame to yourself.

+1

Dispute does not have to mean nastiness; for any interesting human 
collaboration, some dispute is inevitable.

Everything I have heard about how this resolution process went indicates that it 
was handled diligently and constructively.  Independent of one's opinion of 
Scott's decision, it is clear that this matter got serious and deep thought, 
with real concern for longer-term implications.

That's as much as anyone can reasonably ask for.

d/

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net
_______________________________________________
IETF mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ietf

---

From: Michael StJohns <[email-masked]>
Date: Sun, 16 Mar 2008 17:19:12 -0400

This isn't a dangerous precedent, this is the IAB doing its due diligence.

One of the possible outcomes here could have been the IAB simply rejecting those candidates for which it could not form an opinion due to lack of information.  Confirmation is an "affirmative" action, not simply a lack of a reasons to reject.  The confirming bodies should not be concerned with the way the Nomcom got to the point of nominating someone (at least not during the process), but they are there to examine the nomination and nominee and to determine if - in the confirming body's best judgement - the nominee is acceptable for the position.

The IAB, as early as 2003, drafted a document that described the information they require to do their job of confirming IESG candidates.  As I understand this dispute, the Nomcom for some reason either never saw the document, or decided they were just going to ignore it and hope for the best. I'm not sure which was the case even from reading your document multiple times.  And given that you were a member of the previous Nomcom, I am surprised to hear that you hadn't seen it at some point.

If the community wants the confirming bodies to be a rubber stamp, they should limit the deliberations of the confirming bodies in a manner which, absent errors of process, will result in the confirming bodies confirming the Nomcom's choices.  [However, I suggest anyone reading this consider and attempt to describe publicly how any external entity, not privy to the internal deliberative process of the Nomcom, could make such a judgement in a manner that was anything but a coin toss] 

If the community wants to continue with quality selections of its leadership, it should probably continue with "strong" confirming board model.

With respect to your FUD comment below " I believe that the IAB's 
interpretation of 3777 on the matter of the confirmation process sets a 
dangerous precedence whereby one of the confirming bodies could require 
that the Nomcom provide (samples of) verbatim feedback." - HOG WASH.   The specific language in the IAB document is:

>     1. Resume or CV of the candidate
>     2. Summary of the IETF feedback on the candidate
>     3. Summary of the IETF community feedback on the state of the Area
>        and it's current needs
>     4. The Nominating Committee's conclusions of the qualifications
>        required for the position
>     5. Nominating Committee's view of the qualifications of the
>        candidate to undertake the role associated with the position
>     6. Candidate's statement to the Nominating Committee on the
>        position, conveyed to the IAB with the candidate's knowledge and
>        assent

As someone who's been in all 4 nomcom related positions (Chair, Voting Member, Liason(past chair), Confirming Body (IAB)),  (and the guy who actually wrote the first nomcom Questionnaire with the idea of gathering in an orderly manner the information we were going to need to provide to the confirming bodies), I find none of this either inappropriate or over-reaching.  

And finally, a closer reading of RFC3777 will reveal that this dispute between the IAB and Nomcom probably shouldn't have gone to 3777 dispute resolution.  

    * Section 6 says "The dispute resolution process described here is to be used as indicated elsewhere in this document.  Its applicability in other circumstances is beyond the scope of this document."  
    * The four cases I find are: 
        * [Disputes between Liasons and the Nomcom] Liaisons from the IESG, IAB, and Internet Society Board of Trustees (if one was appointed) are expected to review the operation and executing process of the nominating committee and to report any concerns or issues to the Chair of the nominating committee immediately.  If they can not resolve the issue between themselves, liaisons must report it according to the dispute resolution process stated elsewhere in this document. 
        * [Disputes between current and prior chairs] The prior year's Chair is expected to review the actions and activities of the current Chair and to report any concerns or issues to the nominating committee Chair immediately.  If they can not resolve the issue between themselves, the prior year's Chair must report it according to the dispute resolution process stated elsewhere in this document. 
        * [Related to random selection of the members] The community must have at least 1 week during which any member may challenge the results of the random selection.(P) The challenge must be made in writing (email is acceptable) to the Chair.  The Chair has 48 hours to review the challenge and offer a resolution to the member.  If the resolution is not accepted by the member, that member may report the challenge according to the dispute resolution process stated elsewhere in this document. 
        * [Disputes between members and the Chair] Any member of the committee who has an issue or concern regarding the Chair should report it to the prior year's Chair immediately.   The prior year's Chair is expected to report it to the Chair  immediately.  If they can not resolve the issue between themselves, the prior year's Chair must report it according to the dispute resolution process stated elsewhere in this document. 
But fait is accompli so... :-) 

And continuing - the issues that sparked the IAB document occurred during the term (and almost at the end) of the WG that did 3777, not prior to it.  I think 3777 had gone through several last calls and there was all that pesky secrecy stuff - we couldn't actually tell the WG WHY certain changes should really be made.  That resulted in a document that did not include the IAB requirements and mentioned above.

Jim Galvin can probably do a better job of describing the timeline.  The fact that we didn't turn right around and reopen the process to add these requirements (or prohibit them) is mostly due to extreme exhaustion after both the WG and the confirming process.

Later, Mike

---

From: Ole Jacobsen <[email-masked]>
Date: Sun, 16 Mar 2008 14:46:44 -0700 (PDT)

You said:

" The confirming bodies should not be concerned with the way the 
  Nomcom got to the point of nominating someone (at least not during 
  the process), but they are there to examine the nomination and
  nominee and to determine if - in the confirming body's best judgement
  - the nominee is acceptable for the position."

And yet you believe that ALL the information provided by a candidate 
to the nomcom should be freely available to the confirming bodies?

The only thing at issue here is: What information did the candidate 
think would be forwarded to the confirming body and what information
did he/she have a reasonable expectation would stay within the 
committee?

This may not require a new process RFC, it may simply require a 
questionnaire with a "confidential" section.

The rest of your note isn't worth responding to since you choose to 
use such phrases as "FUD" and "HOG WASH". Sorry Mike, I would have 
expected better from you!

Ole

Ole J. Jacobsen
Editor and Publisher,  The Internet Protocol Journal
Cisco Systems
Tel: +1 408-527-8972   Mobile: +1 415-370-4628
E-mail: [email-masked]  URL: http://www.cisco.com/ipj

---

From: Michael StJohns <[email-masked]>
Date: Sun, 16 Mar 2008 18:28:51 -0400

>to the nomcom should be freely available to the confirming bodies?

That's not actually what I said.  What I said was the list of information requested by the IAB in the referenced document was reasonable and appropriate.  But let's break down "ALL" for a minute

A candidate provides data that is either a) relevant to his selection by the Nomcom or b) is not relevant.  If the Nomcom relies on a piece of information provided to it by the candidate (lots of possibilities, but lets try something like "I currently work for ABC company - and I realize the other AD also works there, however I'm leaving my current job on X date and moving to Y.  My new employer has agreed to sponsor me."),then its relevant.  I'm unsure how the confirming body confirms the candidate without also being apprised of this information.  I can do more examples, but my personal belief is that any information a candidate provides that the Nomcom relied upon to select the candidate is fair game for the confirming body.  In general though, I think the IAB struck a good balance with what it wanted.

>did he/she have a reasonable expectation would stay within the 

Not really.  As has been pointed out numerous times before, the confirming bodies are within the "cone of silence" of the nominations process. This interpretation of the confirming bodies as adversaries to the Nomcom that shouldn't ever see the raw material is fairly recent - 5-6 years maybe.

Ask any given candidate about whether or not they anticipated their responses to the questionnaire were or were not fair game for the confirming body and I expect you'll get a "huh"?   I know if I submit something to the process, I expect it will be used where it needs to be used to get me confirmed.  Why, given any reasonable reading of 3777 (or its predecessors), would I think otherwise?

>questionnaire with a "confidential" section.

A "Please don't tell the IAB but you should know..." section?  And what exactly would you expect a candidate to put in this section?  I'm not saying its a bad idea, but I'm having problems conceiving of information that the Nomcom should consider that the Confirming body should not.

If you're talking about information provided in confidence to the Nomcom as commentary by one candidate on another - that's a whole other matter and has its own set of problems.  But I think that's not what you're talking about here?

And I would have thought better of you as well.  I would have expected you to consider my terms, then consider the tone of Dondeti's text.  He said:

>I believe that the IAB's 
>that the nomcom provide (samples of) verbatim feedback. 

I identified this as FUD and included my opinion (strongly stated) that his opinion was unsupported by the facts.  I cited the particular items the IAB had requested, and noted that nothing in those items were "dangerous" (and were in fact due dilligence) and there was no indication that the IAB had asked for, would ask for, or even cared about the above.  It is fear mongering to support a particular point of view (FUD) and it is unsupported by facts (HOG WASH). 

I stand by both terms.

Mike

---

From: Dave Crocker <[email-masked]>
Date: Sun, 16 Mar 2008 16:18:30 -0700

>      I'm
> unsure how the confirming body confirms the candidate without also being
> apprised of this information.  

This seems to go to the heart of a long-standing dilemma in the IETF:

    Is it the job of a reviewing body to pre-empt lengthy and diligent work or 
is it the job of a reviewing body to the work was done diligently and competently?

These are very different jobs.

Whether Nomcom or a working group, a decision process over a long period of time 
represents extensive research, deliberation, and balancing among trade-offs. 
This is something that simply cannot be replicated by another person or body 
spending a few days or even weeks on "review".

If they are not replicating the decision process, they are doing something else.

What should it be?

If a reviewing body that is doing no original research and is working over a 
very brief period can legitimately second-guess the protracted effort of a 
working group or a Nomcom, then why bother with the first group?  It would be 
dramatically easier and faster to simply have the "reviewing" bodies make all 
the choices in the first place.

The necessary difference, here, is between reviewing that a primary body did its 
work competently, versus re-doing the underlying work.

Certainly we want the review process to be substantive.  That is, meaningful. 
So what questions should it be asking?  It's easy to go through some pro forma 
material to establish that the primary group really was diligent.  Fine.  But 
what about going to deeper issues?

Perhaps a review of the primary group's competence requires going to all the 
source material?  Why?  that really reduces the primary group to merely 
assembling source data.  Is that what we want?  Probably not.

What if the reviewing body believes it has essential data that might have been 
missed or considered inadequately?  Now we essentially go to the possibility 
that the reviewing body really does its own source of research information or 
insight to it.  After all, the reviewing body comprises knowledgeable folk.

It seems to me that that extra insight should serve to ask the primary group to 
explain how it considered the special issues.  In fact, that kind of dialogue 
can be a very good way to establish the substance of the primary group's 
diligence.

Primary groups tend to be defensive about this dialogue and reviewing bodies 
tend to be indelicate and/or vague when asking these questions.  And the 
exchange adds delay, but I think that these are the price of having meaningful 
review. Just as the primary group's effort needs to be respected, so does the 
reviewing body's.

As long as we have no consensus about the nature of the job to be done by a 
reviewing body, we are going to suffer with its thinking can can reasonably 
second-guess primary bodies.

d/
-- 

   Dave Crocker

---

From: Michael StJohns <[email-masked]>
Date: Sun, 16 Mar 2008 21:10:22 -0400

>is it the job of a reviewing body to the work was done diligently and competently?

I think you're missing a "decide if" before "the work" in the second line?

I think this is kind of a slanted (sorry) statement of the problem. I'd put it more like:

"Is it the job of the reviewing body to make an independent decision on the candidates suitability, or is it the only job of the reviewing body to protect the process irrespective of the actual nominations?"

>spending a few days or even weeks on "review".

The Nomcom has to winnow through a pile of candidates, discussion, gathering information, discarding and ultimately selecting the one person (or for IAB group of persons) that it is recommending for selection. That takes lots of time and effort.  

Taking the information which applies only to those candidates, reviewing it, and making a decision, hopefully takes less time given the appropriate documentation.

Put another way, the Nomcom is a search committee, but the hiring authority resides in the confirming bodies.

>If they are not replicating the decision process, they are doing something else.

The rest of this message is sort of ignoring the whole "winnowing" process done by the Nomcom.  The CBs don't repeat that, they can only act on the candidates provided to them.  The CBs provide a check and balance, not the original research.

Since I mostly don't agree with the premise the reviewing bodies are "repeating" the Nomcom's job if they consider candidates qualifications, I don't really have comments on the rest of the message.

>second-guess primary bodies.

And on this we agree. 

Mike

---

From: Ralph Droms <[email-masked]>
Date: Sun, 16 Mar 2008 18:16:55 -0700 (PDT)

On Sun, 16 Mar 2008, Michael StJohns wrote:
> [...]
> Put another way, the Nomcom is a search committee, but the hiring 
> authority resides in the confirming bodies.

Mike - I fundamentally and strongly disagree.  In my opinoin, the Nomcom 
is the hiring committee; the confirming body is the oversight and sanity 
check body.  The Nomcom is selected from the IETF as a whole to select the 
management for the IETF, who then serve at the pleasure of the IETF as a 
whole.  The confirming bodies do not form a hierarchical management or 
hiring organization; rather, they perform a final check and review of the 
process.

- Ralph

---

From: "Spencer Dawkins" <[email-masked]>
Date: Sun, 16 Mar 2008 20:22:29 -0500

I have misunderstood before, but one point of view I've heard expressed was 
that

- NomCom is supposed to choose the best candidate, while

- the confirming body is supposed to make sure NomCom chose a good candidate

does this remotely map onto either Mike's or Ralph's point of view in this 
thread?

Spencer, who volunteers for every NomCom, and who has never been selected to 
serve in any capacity...

---

From: Dave Crocker <[email-masked]>
Date: Sun, 16 Mar 2008 18:31:24 -0700

> - the confirming body is supposed to make sure NomCom chose a good candidate

That sounds like exactly the opposite of a rational sequence.

Nomcom's always do an early filter against clear inadequacy.  If that leaves no 
candidates, they search for more.

This is like doing late-stage reviews, after a working group has been operating 
for two years, and raising strategic concerns about their entire apporach:  Good 
issues, but at the wrong time.

d/
-- 

   Dave Crocker

---

From: Michael StJohns <[email-masked]>
Date: Sun, 16 Mar 2008 21:42:06 -0400

:-) 

This is a slight misquote of my 

"The Nomcom's goal should be to select the best qualified candidates from the pool of volunteers. The confirming bodies should confirm any candidate they believe to be fully qualified."

N.B.; Reasonable people can differ on whether any given candidate is fully qualified and the fact the Nomcom considers someone qualified is simply input into the CBs deliberations as opposed to a given fact.

---

From: "Joel M. Halpern" <[email-masked]>
Date: Sun, 16 Mar 2008 21:50:54 -0400

I know that when 3777 was being written, the question of what the 
confirming bodies should do was discussed.  No clear answer was available.
However, my perception of what happened included rulting out two 
possible answers:
1) The confirming bodies are not supposed to be a rubber stamp.  They 
are supposed to apply judgment, and actually review the nominations 
(including, but not limited to, the process.)
2) Equally, the confirming bodies are not supposed to repeat the 
nominating committees activities.  Whatever the confirmation is, it is 
NOT asking "is this the answer the confirming body would come up with."

There is a line to be walked balancing the various interests.
I am aware that by the nature of this process, those of us outside do 
not have enough information to really judge what has happened. 
Unfortunately, it is those of us outside who have to decide if the 
current rules work, or if different rules should be applied.

Mike, whatever your personal opinion, based on the public information 
many people have concluded in good faith that something went wrong. 
Asserting that the problems are FUD does not help anyone resolve this.

At the same time, for all the concerns (some of which I share) it is 
quite clear to me that the IAB was acting in good faith.  They were 
trying their best to do what we had asked, namely to perform meaningful 
review of the slate they were handed.

I have my doubts about the utility of actually re-opening 3777.  For one 
thing, I don't actually think we are in a better position to actually 
write a definition of the confirmation process.  And while I would like 
to make candidate names public at a suitable point in the process, I 
lost that argument last time and do not see that much has changed to 
justify re-opening it.  (We usually insist that folks can not revisit a 
WG decision without an indication of change.)

On the other hand, making clear what parts of the questionnaire may be 
shared with the confirming bodies seems like a very good idea.  I know 
that I would have been very surprised if someone said that the IAB was 
going to see the questionnaires in full.  While I have heard the 
argument that the nomcom can extend the confidentiality umbrella as far 
as they want, it seems to me that extending it that far would be a mistake.

Yours,
Joel M. Halpern

---

From: Michael StJohns <[email-masked]>
Date: Sun, 16 Mar 2008 21:58:09 -0400

At 09:16 PM 3/16/2008, Ralph Droms wrote:

>Mike - I fundamentally and strongly disagree.  In my opinoin, the Nomcom is the hiring committee; the confirming body is the oversight and sanity check body.  The Nomcom is selected from the IETF as a whole to select the management for the IETF, who then serve at the pleasure of the IETF as a whole.  The confirming bodies do not form a hierarchical management or hiring organization; rather, they perform a final check and review of the process.
>
>- Ralph

And I think this is the nut of the matter. 

If the IETF agrees with you, the best thing they can do is rewrite 3777 to eliminate the confirmation process, as there is no possible way to do any reasonable sanity and oversight without a complete end-to-end view of the process.  E.g. leave it to the liaison's and past-chairs to deal with those "is the process good?" matters.

But the text of all the Nomcom process documents back to the first (RFC2027) says 

"review the candidates...and then consent to some, all or none of the candidates".

Review the "candidates" - NOT "review the process by which the candidates were selected".

This is the black letter reading of ALL of versions of the chain of Nomcom documents and I believe it is in opposition to your opinion.

I'm going to self-initiate a personal 24 hour hold down.  I'm going to refrain from any more comments on this topic or the other chain until tomorrow night just to see if anyone else has anything to say.. :-) 

Later, Mike

---

From: Michael StJohns <[email-masked]>
Date: Sun, 16 Mar 2008 22:36:18 -0400

My apologies, I was going to leave this alone, but this ... chastisement .. is off-target.

>many people have concluded in good faith that something went wrong. 

I agree with this. Something went wrong.

>Asserting that the problems are FUD does not help anyone resolve this.

I'm sorry you didn't actually read what I wrote. I did not refer to the problems as FUD.  I called one specific statement by LD FUD and hogwash. The statement was an attempt to use an emotional response to an unlikely or improbable action by the IAB sometime in the future to gain an outcome (e.g. don't let this dangerous precedent stand) that matched his personal belief.

What would you call it if not FUD?  Never mind.  Substitute "an emotional appeal" for FUD and "This is an absurd extrapolation of what the IAB may do in the far future" for hogwash and see if you like the text better.

Mike

---

From: "Steven M. Bellovin" <[email-masked]>
Date: Mon, 17 Mar 2008 02:42:10 +0000

On Sun, 16 Mar 2008 18:31:24 -0700
Dave Crocker <[email-masked]> wrote:

> 
> 
> Spencer Dawkins wrote:
> > I have misunderstood before, but one point of view I've heard
> > expressed was that
> > 
> Nomcom's always do an early filter against clear inadequacy.  If that
> leaves no candidates, they search for more.
> 
> This is like doing late-stage reviews, after a working group has been
> operating for two years, and raising strategic concerns about their
> entire apporach:  Good issues, but at the wrong time.
> 
So what do you see as the role of the confirming body?

		--Steve Bellovin, http://www.cs.columbia.edu/~smb
_______________________________________________
IETF mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ietf

---

From: Dave Crocker <[email-masked]>
Date: Sun, 16 Mar 2008 19:53:12 -0700

> I think you're missing a "decide if" before "the work" in the second line?

Yeah.  See.  We can start with something we agree on.  But then, agreeing on my 
having missed something is no challenge.

> "Is it the job of the reviewing body to make an independent decision on the
> candidates suitability, or is it the only job of the reviewing body to
> protect the process irrespective of the actual nominations?"

Glossing over the linguistic subtleties, I think that the recent reality behind 
"make an independent decision" is that it is functionally identical to 
"second-guess".  My choice of language is certainly biased, but then I was (and 
am) trying to emphasize the impact on the entire process.

People who do line work are undermined if their work is constantly subject to 
reversal, unless the work is legitimately flawed.  It's not the job of the 
review process to replicate the work, but to confirm it's quality.  There is a 
difference.

Perhaps by "independent" you meant that it must have substance rather than be an 
automatic approval.  If so, we certainly agree.

Where we differ is about methods.

>> Whether Nomcom or a working group, a decision process over a long period of
>> time represents extensive research, deliberation, and balancing among
>> trade-offs. This is something that simply cannot be replicated by another
>> person or body spending a few days or even weeks on "review".
> 
> The Nomcom has to winnow through a pile of candidates, discussion, gathering
> information, discarding and ultimately selecting the one person (or for IAB
> group of persons) that it is recommending for selection. That takes lots of
> time and effort.
> 
> Taking the information which applies only to those candidates, reviewing it,
> and making a decision, hopefully takes less time given the appropriate
> documentation.

Sampling error.  Experimenter bias.  Something like that...

To pursue your model, the review committee is given a subset of the source data 
and none of the history involved in doing the winnowing, and is somehow supposed 
to be able to perform as well or better than the primary group?

These sorts of decision processes are dominated by trade-offs.  The review body 
has none of that protracted context that is leads to the decision among the 
trade-offs. Nor can that context be replicated easily or even very well.  That's 
why a review has to have more to do with verifying that the process of the 
primary group was diligent. You don't get that by looking over source data.

I do add that a review body might believe it holds special knowledge -- that is, 
source data -- that it deems important, and it well might use that in its 
dialogue with the primary group.  This is more than mere process review and I, 
for one, think it entirely appropriate. As a basis for dialogue.

In the case of Nomcom work, it's pretty rare, however, for that dialogue to 
result in a reversal, since Nomcoms really are typically quite diligent -- 
and/or the review body does not focus on the right basis for reversal...

(I think "quite rare" translates to "never" but of course we'll never know.)

> resides in the confirming bodies.

Yes, that is the myth.  And that's why it is labelled "Nominating" Committee. 
But it really is only a myth, since the repercussions of an outright rejection 
are pretty onerous.

Even if we skip over my potentially unpleasant assessment of reality, I'll 
repeat my above observation about the effect of undermining folks who do primary 
work.

> The rest of this message is sort of ignoring the whole "winnowing" process
> done by the Nomcom.

I was certainly not intending to and am pretty sure I didn't.  It's all about 
that protracted context.  That's more than simply forwarding a nomination.  It's 
about juggling trade-offs.

What I suspect does tend to get ignored in these kinds of discussions, is how 
the IETF is different from the places we work as a day job:

    In the IETF the people who are in charge are not really in charge.  All the 
approval stuff distracts from the reality that work is done by people who 
volunteer and those people are not "hired" by the decision-makers.

    Yeah, those decision-makers name people to particular jobs, and those jobs 
are important and difficult.  But all of this hinges on a community 
constituency's deciding to work on the topic. The decision-makers come in to 
play only after that community constituency has developed.

    It is an inherently grass-roots model, quite the opposite of "working for" 
an organization.

    The other thing that is missed is that the Nomination process does not 
forward multiple candidates for a position.  It forwards one.

    This is entirely different from the hiring activities most of us are used 
to.  Again, it should serve as a very bright flag that this process is quite 
different from the model we see in companies.

> Since I mostly don't agree with the premise the reviewing bodies are
> "repeating" the Nomcom's job if they consider candidates qualifications, I
> don't really have comments on the rest of the message.

Offhand, I don't think that there is a problem for the review body to "consider 
candidate's qualifications".  It is a question of what material is reasonable 
for them to use to make that assessment.

>>  reviewing body, we are going to suffer with its thinking can can
>> reasonably second-guess primary bodies.
> 
> And on this we agree.

Begin and end in agreement.  This is goodness.

d/

-- 

   Dave Crocker

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 17 Mar 2008 16:17:46 +1300

> hiring organization; rather, they perform a final check and review of the 

To put it very slightly otherwise, the nomcom is supposed to represent
the whole community in the process of appointing people - maybe it would
be better named as the "appointments committee". The confirming bodies
are supposed to provide a check that due process has been followed and
that the proposed appointees are suitable, but they are clearly doing
that as guardians of the process.

I believe that it's appropriate for the confirming bodies to ask for
additional information if they have reason to doubt that due proces
has been followed or that some of the proposed appointees are suitable.
I agree that they are inside the confidentiality boundary, too, and
this should be made clear to all concerned. What I don't like about
http://www.iab.org/documents/docs/2003-07-23-nomcom.html
is that the materials are requested a priori, as if *every* NomCom
choice is suspect. I think these are questions that should only be
asked if the confirming body has specific reason to query a choice.
(With one exception: it is quite reasonable to request a resume or CV
a priori.)

    Brian

---

From: Ralph Droms <[email-masked]>
Date: Mon, 17 Mar 2008 03:55:54 -0700 (PDT)

Agreeing with Brian's dislike of 
http://www.iab.org/documents/docs/2003-07-23-nomcom.html, it was drafted, 
as far as I know, before RFC 3777 was published.  RFC 3777 defines the 
process, with the consensus of the IETF community as a whole.  I suggest 
that the IAB at least review its requirements document within the process 
defined by RFC 3777.  I think it would be more appropriate and more in 
keeping with RFC 3777 for the IAB to publish minimal or no a priori 
requirements, leaving to the Nomcom the responsibility for the provision 
of adequate documentation in support of its nominaions.  As Brian writes, 
the IAB can ask for specific additional information in those cases where 
it finds that information is necessary to complete its due diligence.

- Ralph

---

From: Stewart Bryant <[email-masked]>
Date: Mon, 17 Mar 2008 15:14:10 +0000

> has been followed or that some of the proposed appointees are suitable.
Isn't one of the roles of the liaisons to ensure that due process is 
followed to the extent required by the body they represent, and to give 
advanced notice when the choice of candidate is likely to be 
unacceptable to their body?

Stewart

---

From: SM <[email-masked]>
Date: Mon, 17 Mar 2008 08:34:04 -0700

>has been followed or that some of the proposed appointees are suitable.

The nominating committee selects candidates which, in their opinion, 
fits the requirements they have been given and which they believe 
will be able to contribute in such as way as to improve the 
body.  The confirming body are there to determine whether the 
requirements are met.  They may have to understand the rationale for 
the selection if they find it inappropriate.

>this should be made clear to all concerned. What I don't like about

There is an expectation that the information provided to the 
nominating committee is confidential.  The confirming body needs some 
information to determine whether the candidate fits the stated requirements.

>(With one exception: it is quite reasonable to request a resume or CV

That's the expectations of the IAB in support of a nomination.  It is 
reasonable for the nomination committee to provide  a brief resume to 
the confirming body.  Turning the expectations of the confirming body 
into preconditions for the nominating committee's selection may be 
perceived as undermining the role of the nominating committee.

Regards,
-sm

---

From: Danny McPherson <[email-masked]>
Date: Mon, 17 Mar 2008 10:10:46 -0600

> unacceptable to their body?

Yes, and it's stated somewhat redundantly in S. 7 of RFC 3777,
but if there are issues with process the liaison should address
them first with the chair, and if to no avail, invoke dispute  
resolution.
If I were liaison and believed there were ANY process issues that
couldn't be resolved with the chair, I'd be reluctant to provide any
slate to my confirming body.

I.e., as IAB liaison this year I had no problem providing the
NomCom's IESG slate to the IAB for confirmation.  This should
serve as a testament that I believe process was followed and
properly executed by the NomCom.  However, this is orthogonal
of the IAB's consideration of that slate as confirming body, which
as an insider there I can assure you is something they take far
more seriously than I'd ever previously thought, which I believe
to be a good thing.

I believe the crux of the issue here is indeed what the role of the
confirming body is.  IMO, the current model ensures the NomCom
feels second-guessed, which I assure you weighs heavy on
NomCom members after an extraordinary amount of work.   And
the confirming bodies must take the confirmation process
seriously, and pushback from the NomCom seems unreasonable.

I've seen this from all perspectives, and I'm not sure where the
right balance is.  The dispute resolution this year is a symptom
of a larger issue, IMO.

Beyond that, my personal opinion is that both the NomCom and
the IAB were acting in good faith this year, and it's unfortunate
that the IETF NomCom framework is clouding the product of that
work.  It is clear that it needs to be readdressed.

-danny

---

From: Dave Crocker <[email-masked]>
Date: Mon, 17 Mar 2008 09:17:02 -0700

> unacceptable to their body?

Hmmm.

RFC 3777, Section 4. #7 certainly does seem to say yes to your question.

I wonder how familiar folks -- including liaisons -- are with the explicit list 
of their duties.

For the last Nomcom I was on (before 3777) they acted exactly like regular 
members except for the formal votes. They participated in interviews and were 
full participants (advocates) in the committee decision debates.

d/
-- 

   Dave Crocker

---

From: "Hallam-Baker, Phillip" <[email-masked]>
Date: Mon, 17 Mar 2008 09:45:45 -0700

I think you have the whole confirmation process backwards.

If you start from the premise that the absolute priority is to keep control in the hands of the establishment you naturally arrive at a need for at least two bodies arranged so that each acts as a guardianship council to the other.

Having arrived at the need for a confirmation process you then have to work out how to explain why they should exercise veto power over the consensus of the body as a whole. Nomcon provides a surrogate for consensus but in a form that ensures that there is no mandate.

I know that people can find arguments as to why this is better than Nomcon qualified IETF participants having a vote. I got rather tired of arguments of the form 'this is better than democracy' after hearing them used to defend the 'need' to build the Berlin wall.

-----Original Message-----
From: [email-masked] on behalf of Ralph Droms
Sent: Sun 16/03/2008 9:16 PM
To: Michael StJohns
Cc: IETF Discussion
Subject: Re: Confirming vs. second-guessing

- Ralph

---

From: Danny McPherson <[email-masked]>
Date: Mon, 17 Mar 2008 10:49:30 -0600

>> unacceptable to their body?

I overlooked the "and to give advanced notice..." part of this in my
previous response.  I don't consider it appropriate for a liaison to
divulge any sensitive information learned through the NomCom
process to a confirming body without the consent of the NomCom.
If the information is not already provided in the testimony provided
by the NomCom to the confirming body, that which the liaison is
in part responsible for helping to prepare, then to do so would seem
only to sabotage the process, and breach the confidence of the
NomCom.

I believe the current 3777 text surrounding this is sufficient.

-danny

---

From: Stephen Kent <[email-masked]>
Date: Mon, 17 Mar 2008 15:36:30 -0400

Mike,

I have to disagree with your characterization of the proper role of 
the IAB with regard to the NOMCOM process.

I have been on three NOMCOMs, including the one prior to this, so I 
too have some experience in the process.

My feeling is that the IAB may have been trying to assert too much 
authority in the process recently.  That certainly was my perception 
with the previous NOMCOM, and the report about this year's activities 
suggests that the problem continues.

RFC 3777 is ambiguous wrt some details of the process, especially the 
IAB's purview re confirming IESG candidates selected by the NOMCOM. 
One could read 3777 so as to allow the IAB to effectively supplant 
the NOMCOM, e.g., by refusing to approve a slate of candidates until 
one candidate, acceptable to the IAB, is named for a given position. 
This is clearly not what 3777 intends, as undermines the NOMCOM role, 
yet I have seen behavior that comes close to this.

I think the preferred way forward is to clarify 3777 to make clear 
that the IAB must not engage in actions that are tantamount to 
usurping the rile of the NOMCOM.

As for the IAB publication you cited in your message, I don't recall 
seeing the RFC number for it.  If it was just an informational RFC, I 
don't believe that has standing relative to 3777, which is the result 
of the usual IETF process.  The IAB is not empowered to write an 
interpretation of an RFC to expand the IAB's role unilaterally.

Steve

---

From: Lakshminath Dondeti <[email-masked]>
Date: Mon, 17 Mar 2008 14:06:39 -0700

Mike,

Thanks for your note.

Are you saying that there is text within 3777 that says that confirming 
bodies should not ask for verbatim feedback but could ask for verbatim 
questionnaire responses?

Consider this: what if the next nomcom were to be asked to provide 
verbatim feedback by one of the confirming bodies, what should they do? 
  The supporting information that was cited from 3777 is very generic 
(with specific information provided later, if I may point out again) and 
so I don't see a reason why it cannot be cited in the context of that 
request.

For the nomcom process to work, a number of people in the community 
ought to believe in it sufficiently to volunteer their time and a larger 
number of people need to believe in it to provide feedback.

What am I hearing from my vantage point (having been a nomcom member for 
3 out of the past 4 years) is what guides me to work on this; my goal is 
to maintain and if possible increase the level of confidence in the 
process.  When I ask for information, I provide assurances that the 
information is for nomcom's eyes only (for instance, this was 
specifically the case when we interviewed people).  3777 also says any 
dissemination of private information by the nomcom must be minimal.  So, 
I for one cannot just hand out private information without clearer text 
in 3777.

Finally, when I make a mistake, I am more than ready to own up to that. 
  In this case, my choice of words is quite appropriate.

regards,
Lakshminath

On 3/16/2008 3:28 PM, Michael StJohns wrote:
>> nominee and to determine if - in the confirming body's best
>> judgement - the nominee is acceptable for the position."
>> 
>> And yet you believe that ALL the information provided by a
>> candidate to the nomcom should be freely available to the
>> confirming bodies?
> 
> That's not actually what I said.  What I said was the list of
> information requested by the IAB in the referenced document was
> reasonable and appropriate.  But let's break down "ALL" for a minute
> 
> A candidate provides data that is either a) relevant to his selection
> by the Nomcom or b) is not relevant.  If the Nomcom relies on a piece
> of information provided to it by the candidate (lots of
> possibilities, but lets try something like "I currently work for ABC
> company - and I realize the other AD also works there, however I'm
> leaving my current job on X date and moving to Y.  My new employer
> has agreed to sponsor me."),then its relevant.  I'm unsure how the
> confirming body confirms the candidate without also being apprised of
> this information.  I can do more examples, but my personal belief is
> that any information a candidate provides that the Nomcom relied upon
> to select the candidate is fair game for the confirming body.  In
> general though, I think the IAB struck a good balance with what it
> wanted.
> 
>>  think would be forwarded to the confirming body and what
>> information did he/she have a reasonable expectation would stay
>> within the committee?
> 
> Not really.  As has been pointed out numerous times before, the
> confirming bodies are within the "cone of silence" of the nominations
> process. This interpretation of the confirming bodies as adversaries
> to the Nomcom that shouldn't ever see the raw material is fairly
> recent - 5-6 years maybe.
> 
> Ask any given candidate about whether or not they anticipated their
> responses to the questionnaire were or were not fair game for the
> confirming body and I expect you'll get a "huh"?   I know if I submit
> something to the process, I expect it will be used where it needs to
> be used to get me confirmed.  Why, given any reasonable reading of
> 3777 (or its predecessors), would I think otherwise?
> 
> A "Please don't tell the IAB but you should know..." section?  And
> what exactly would you expect a candidate to put in this section?
> I'm not saying its a bad idea, but I'm having problems conceiving of
> information that the Nomcom should consider that the Confirming body
> should not.
> 
> If you're talking about information provided in confidence to the
> Nomcom as commentary by one candidate on another - that's a whole
> other matter and has its own set of problems.  But I think that's not
> what you're talking about here?
> 
> And I would have thought better of you as well.  I would have
> expected you to consider my terms, then consider the tone of
> Dondeti's text.  He said:
> 
>> I believe that the IAB's interpretation of 3777 on the matter of
>> the confirmation process sets a dangerous precedence whereby one of
>> the confirming bodies could require that the nomcom provide
>> (samples of) verbatim feedback.
> 
> I identified this as FUD and included my opinion (strongly stated)
> that his opinion was unsupported by the facts.  I cited the
> particular items the IAB had requested, and noted that nothing in
> those items were "dangerous" (and were in fact due dilligence) and
> there was no indication that the IAB had asked for, would ask for, or
> even cared about the above.  It is fear mongering to support a
> particular point of view (FUD) and it is unsupported by facts (HOG
> WASH).
> 
> I stand by both terms.
> 
> Mike
> 
> 
>

---

From: Lakshminath Dondeti <[email-masked]>
Date: Mon, 17 Mar 2008 15:38:06 -0700

On 3/16/2008 7:36 PM, Michael StJohns wrote:
>> Mike, whatever your personal opinion, based on the public
>> information many people have concluded in good faith that something
>> went wrong.
> 
> I'm sorry you didn't actually read what I wrote. I did not refer to
> the problems as FUD.  I called one specific statement by LD FUD and
> hogwash. The statement was an attempt to use an emotional response to
> an unlikely or improbable action by the IAB sometime in the future to
> gain an outcome (e.g. don't let this dangerous precedent stand) that
> matched his personal belief.
> 
> What would you call it if not FUD?  Never mind.  Substitute "an
> emotional appeal" for FUD and "This is an absurd extrapolation of
> what the IAB may do in the far future" for hogwash and see if you
> like the text better.

I guess I should respond to this.  Why would the extrapolation be 
absurd?  Where is it stated that a confirmation body cannot seek such 
information?  One of the IAB requirements is to provide a summary of 
feedback on a candidate.  From there to asking for verbatim feedback is 
not a stretch.  I am not saying IAB would ask for this.  I am saying one 
of the confirmation bodies could ask for this and the nomcom would be in 
the same situation.

Expressing incredulity does not work in such situations.  I have tried.

regards,
Lakshminath

---

From: Fred Baker <[email-masked]>
Date: Mon, 17 Mar 2008 16:55:52 -0700

> information to determine whether the candidate fits the stated  

There is a simple solution to that. The nomcom asks the candidates a  
number of questions. A new form needs to be developed that informs  
the candidate that the information given will be available to the  
confirming bodies, and gives the candidate a way to say things that  
are confidential to the nomcom. For example, every question including  
a new "do you have anything else you would like to add" question  
needs to have two slots, one confidential to the nomcom and one  
confidential to the nomcom plus the confirming body.

---

From: Dave Crocker <[email-masked]>
Date: Mon, 17 Mar 2008 17:28:12 -0700

> confidential to the nomcom plus the confirming body.

How about something even simpler:

The form is not confidential.  If a candidate feels the need to share something 
confidential, they submit it separately and clearly marked.

And in order to make the confidentiality issue more concrete (ie, real) would 
folks offer some examples of what falls under it.  Obviously I'm not asking 
about a traceable example, but something that moves this from a theoretical 
exercise to something where the community can agree it really is worth the 
effort to maintain confidentiality.

d/

-- 

   Dave Crocker

---

From: "Dan Wing" <[email-masked]>
Date: Mon, 17 Mar 2008 18:22:51 -0700

> confidential to the nomcom plus the confirming body.

I was on NOMCOM this year, and Fred's suggestion would solve this
problem perfectly.

-d

---

From: "Dan Wing" <[email-masked]>
Date: Mon, 17 Mar 2008 18:33:07 -0700

> And in order to make the confidentiality issue more concrete 
> (ie, real) would folks offer some examples of what falls under 
> it. 

"I accept the nomination of area director.  The current area
director, Mr. J. Sixpack, has been attempting to impose his
opinion that beer should contain rice.  This is causing a rift
in the working groups within the area.  I would follow the area
consensus that we should outlaw rice in beer and thus my 
appointment as new area director would achieve peace and 
harmony within the area."

-d

---

From: Christian Huitema <[email-masked]>
Date: Mon, 17 Mar 2008 18:44:49 -0700

> harmony within the area."

Why should such a statement be confidential?

-- Christian Huitema

---

From: "Dan Wing" <[email-masked]>
Date: Mon, 17 Mar 2008 18:51:02 -0700

> Why should such a statement be confidential?

Imagine it is complaining about an IAB member, and the IAB is
the confirming body.  That outgoing IAB member is part of the
IAB until the new IAB is seated.  That outgoing IAB member may
well take offense to someone thinking rice is unsuitable for 
beer.

-d

---

From: "Steven M. Bellovin" <[email-masked]>
Date: Tue, 18 Mar 2008 02:08:15 +0000

On Mon, 17 Mar 2008 18:44:49 -0700

> Why should such a statement be confidential?
Try this one, quite non-hypothetical: a candidate for the IESG is
contemplating switching jobs.  His or her current employer does not yet
know this.  It has a clear bearing on whether or not that person can do
the job of AD, but equally clearly should not be broadcast to the world.

---

From: Danny McPherson <[email-masked]>
Date: Mon, 17 Mar 2008 20:17:43 -0600

> the job of AD, but equally clearly should not be broadcast to the  

Indeed.  And this is one of the concerns with the confirming body
seeing information as well.  For example, assume that a willing
nominee for a given AD position wishes to complain about the
incumbent AD in their questionnaire and where the incumbent
went wrong with the area, and they both work same employer.
The willing nominee decides that providing some detailed
information on the issues at hand to the NomCom is fine because
it will be kept in confidence.  However, what they don't realize is
that there are folks on the confirming body that also work for that
employer, and they'd be reluctant (for any of an array of reasons)
to share that feedback if they knew those folks within their company
would see it..   This would apply equally in SMB's scenario.

Same sort of issue, quite plausible, methinks.

-danny

---

From: Dave Crocker <[email-masked]>
Date: Mon, 17 Mar 2008 19:58:30 -0700

> Why should such a statement be confidential?

Actually, it's an interesting example, because it has almost nothing to do with 
the candidate and a great deal to do with criticizing a sitting AD.

So the statements "There is a rift within the area and I promise to follow area 
consensus" surely have nothing sensitive in them, while ""attempting to impose 
his opinion" surely does.

Whether such criticism should be subject to public scrutiny very reasonably 
ought to be the decision of the critic.

There is a very real possibility -- and some would argue even the history -- 
that criticizing a member of IETF management can damage one's ability to get 
things done.  So concern about confidentiality of criticism strikes me as 
reasonable.

Whether one really believes that such confidential statements really remain 
fully confidential is an entirely separate matter.

d/

-- 

   Dave Crocker

---

From: Lixia Zhang <[email-masked]>
Date: Mon, 17 Mar 2008 20:10:08 -0700

> the job of AD, but equally clearly should not be broadcast to the  

- it could be the case that one of NOMCOM members were in the same  
company
   withe the candidate
- and the confidentiality rule protects the candidate, right?
- shouldn't/isn't the confirming body bounded by the same
   confidentiality rule?

Lixia

---

From: Theodore Tso <[email-masked]>
Date: Mon, 17 Mar 2008 23:52:14 -0400

> the job of AD, but equally clearly should not be broadcast to the world.

A lot of whether you think information shared with NOMCOM should be
confidential depends on whether you frame the process from the point
of view of a governance issue, or from the point of view of personnel
process.

I think most poeple would agree that if you consider Nomcom as being
more of a "performance review" or a "hiring/firing process", then like
most personnel issues, the information used to make these sorts of
decisions should be treated as confidential.  Certainly if I give
feedback about my manager or vice president as part of some 360 review
process, I would feel *quite* betrayed if that information was shared
any further than it needed to be, lest that information get back to
the person in question --- or to a close friend of the person in
question.

If you think of NOMCOM as being more of a governance question, then
especially if you are from the United States, and in particular from
those localities that have "Open Meeting" or "Open Door" laws that
mandate complete transparency in governance where *all* meetings
between any 2-3 goverment officials must be adequately noticed so the
public can attend and minutes taken which are publically posted, then
you'll probably also share the opinion that NOMCOM should run without
any confidentiality at all.  (As an aside, I've noticed that this is
not true in all cultures, and there is variance on this depending on
where you're from.  I've been on committees where we debated this
issue, and I recall some Europeans saying at this absolute insistance
on complete transparency was quite daft --- their words --- and not
the norm from their experience.)

The problem is that NOMCOM process can be viewed through either lens
equally well, and I suspect that's one of the reasons we don't have
consensus on this issue.

My personal bias, as a former NOMCOM chair, is to view NOMCOM as being
more of a personnel process, and thus I believe that most information
about the process should be kept confidential, *especially* if making
it public were might dissuade some talented individuals from throwing
their hat in the ring.

As for the related issue concerning the role of the confirming body, I
personally believe that the confirming body should act more as a
*sanity check* than anything else.  That is, it should question any
obvious process violations that it noticed or had brought to its
attention (perhaps through the Laison) and if some choice has some
obvious problems that might harm the IETF if said selection should go
forward, the confirming body should ask questions.  However, if there
are two obvious, viable candidates, and the choice of one or the other
is a 60/40 or a 55/45 question, I do **not** believe it is the place
of the confirming body to request so much information so they can
second-guess the NOMCOM and determine whether they made that 60/40 or
45/55 call correctly.

This is more than rubber-stamp, in that the confirming body should
call into question obvious mistakes in the result or process (or what
might be appear to obvious mistakes).  But there is a far cry between
that and guaranteeing that the NOMCOM "made the correct decision".  In
order to do the latter the confirming body would need a lot more
information and effective redo the work of the NOMCOM in order to
effectively "check their sums"; and I don't believe that is a healthy
or useful use of the confirming body's time.

However, I don't believe (although I would be delighted if I was
wrong) that we have consensus on this point, either...

       	       	    	      	      	     - Ted

---

From: "Dan Wing" <[email-masked]>
Date: Mon, 17 Mar 2008 21:07:04 -0700

> Whether one really believes that such confidential statements 
> really remain fully confidential is an entirely separate matter.

As the number of people with access to confidential information
increases, the chances of an inadvertant (or malicious) disclosure
of the information increases.

(And nothing is 'fully confidential'.  If you want to keep a 
secret you keep the information to yourself.)

-d

---

From: Lixia Zhang <[email-masked]>
Date: Mon, 17 Mar 2008 22:05:42 -0700

> well take offense to someone thinking rice is unsuitable for

First of all, I fully agree with others it should be the candidate's  
choice about what to disclose to whom.

having said that, if I were the candidate in question and had a  
dispute with an IAB member, I'd want the confirming body to know  
everything to make a good judgment call.

Call me an idealist:), I personally believe, generally speaking, it is  
better to put everything on the table, rather than partial info,  
between nomcom and confirming body.

Step up a level: wonder where this discussion is leading to?
Exactly how to revise 3777?

Lixia

---

From: Fred Baker <[email-masked]>
Date: Mon, 17 Mar 2008 23:38:20 -0700

On Mar 17, 2008, at 10:05 PM, Lixia Zhang wrote:

> Call me an idealist:), I personally believe, generally speaking, it  
> is better to put everything on the table, rather than partial info,  

It sounds like you would rather get rid of the nomcom and have the  
confirming body do the work from the start.

I have heard it said that the IETF, in the most recent discussion  
that failed up update that portion of what we now call 3777, had a  
90/10 consensus and didn't come to a perfect consensus. I think we  
have to say what the role and reach of the confirming body is, which  
may require us to think hard about what it means to have "rough  
consensus".
_______________________________________________
IETF mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ietf

---

From: Lixia Zhang <[email-masked]>
Date: Tue, 18 Mar 2008 04:32:46 -0700

> confirming body do the work from the start.

Actually to the opposite: I firmly believed it is the nomcom who makes  
the selection.

If you quote my full messages, I said

     First of all, I fully agree with others it should be
     the candidate's choice about what to disclose to whom.

Just that personally and for myself, I would not mind whoever I had  
concern with to know about it.

> 90/10 consensus and didn't come to a perfect consensus.

I did not participate in 3777 formation.  If above is the case, my own  
vote would be that 90/10 is a lot more than a "rough consensus", and  
we should just write down precisely what that is.

> I think we have to say what the role and reach of the confirming  
> body is, which may require us to think hard about what it means to  
> have "rough consensus".

---

From: Theodore Tso <[email-masked]>
Date: Tue, 18 Mar 2008 08:18:17 -0400

> confirming body do the work from the start.

It's interesting to note that this would mean reverting our processes
back to the pre-1993 days, back when the IAB *did* pick the IESG.  One
of the reasons why the Nomcom was created, way back then, was because
of the July 4, 1992 fireworks (as I believe Vint Cerf termed them) on
the IETF list.  I've also heard it referred to as the "Boston Tea
Party" of the IETF.

To quote from Christian Huitema's, "Network Protocols and Standards"
as to what happened:

     We thought that our wording was very careful, and we were
     prepared to discuss it and try to convince the Internet
     community. Then, everything accelerated. Some journalists got the
     news, an announcement was hastily written, and many members of
     the community felt betrayed. They perceived that we were selling
     the Internet to the ISO and that headquarters was simply giving
     the field to an enemy that they had fought for many years and
     eventually vanquished. The IAB had no right to make such a
     decision alone. Besides, CLNP was a pale imitation of IP. It had
     been designed 10 years before, and the market had failed to pick
     it up for all those years. Why should we try to resurrect it?
     The IAB announcement was followed by a tremendous hubbub in the
     Internet's electronic lists. The IAB draft was formally withdrawn
     a few weeks later, during the July 1992 meeting of the Internet
     Engineering Task Force (IETF). The incident triggered a serious
     reorganization of the whole IETF decision process, revising the
     role of managing bodies such as the Internet Engineering Steering
     Group (IESG) or the Internet Architecture Board, the new
     appellation of the IAB.

At the time, there was a feeling that the IAB was "out of touch", and
that the Nomcom selection process was a better way of getting
community consensus about the engineering leadership than the previous
scheme where the IAB was a self-perpetuating body which selected the
IESG.  For a long time, given how hard the IAB had been slapped down,
it pretty much let the Nomcom do most of the work, and when I
submitted the 2001-2002 Nomcom report to the confirming bodies, what
we provided was a brief resume and a short "testament" summarizing the
deliberations and reasoning behind the choice (in most cases it was a
paragraph or two).  At least for the years when I was on Nomcom, the
IAB did not request access to any of the questionaires or comments
from the community; all we provided was 2-3 paragraphs describing some
of the concerns and summarizing at a high level what the concerns
which drove us to replace an incumbent and why we chose a particular
new AD.

It seems that since then, the IAB has been more assertive about
wanting more information, and I really think we need to consider where
the line is between performing due diligence and "redoing the work of
the NOMCOM".  I personally like the line that we drew in the early
2000's; we told the confirming bodies the issues about why IESG or IAB
members were not returned to the body, and why we picked new members.
The confirming bodies asked us if we had considered certain issues,
and we had to draft a response when went into more detail, but that
was about it.

> may require us to think hard about what it means to have "rough  

I'm not sure it was 90/10 consensus; at least in this recent
discussion, there certainly have been a rather wide range of opinions
on this list, from people like Mike St. John's with one view, and
Steve Kent with another.

						- Ted

---

From: "Hallam-Baker, Phillip" <[email-masked]>
Date: Tue, 18 Mar 2008 06:47:02 -0700

Lets look at this from a security usability point of view.

The whole nomcon process is opaque, all meetings and discussions are secret. Requests for comment are solicited in confidence. Given those circumstances it is a reasonable assumption for a participant to make that all nomcon actions are strictly confidential. In fact that is by far the most reasonable assumption to make.

When you have a process that is vested in such a high degree of secrecy you will inevitably end up with a very high degree of suspicion. Secret processes are antithetical to accountability.

The worst failure mode here is not that the nomcon is going to make the wrong choices and the IAB is unable to rescue them. The worst failure mode is that information that is released with a reasonable expectation of confidentiality is then disclosed.

I would much prefer to have a process that is completely open except in regard to actual balloting. To paraphrase Dave Crocker: Why would we expect to be experts in the area? We do bits on the wire, design of political institutions is certainly not an area in which competency has been demonstrated. 

But a process that is assumed to be more confidential than it actually is would appear to be the worst of all cases.

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 18 Mar 2008 10:26:17 -0400

The inner comment, does not match my memory of the discussions.

Theodore Tso wrote:
> Attributed to Fred Baker:

There were a number of issues on which no consensus was reached, or on 
which there was not consensus to make a change.  I don't think any of 
those were anywhere near as close as 90/10.  Some of the "don't change" 
conclusions were probably a significant majority against the change. 
But I don't think there was ever a case that I saw where I thought 80 
(much less 90) percent of the room wanted something, but the chairs 
ruled that there was no consensus.
My concern about re-opening the document is in fact that opinions were 
very divided.  Getting agreement on any change is going to be a lot of 
work, if it succeeds at all.  Try to get rough consensus on clear words 
on something as divisive as how much oversight the confirming bodies 
should perform seems a recipe for failure.

Yours,
Joel

---

From: "Hallam-Baker, Phillip" <[email-masked]>
Date: Tue, 18 Mar 2008 07:28:09 -0700

But what is an 'obvious mistake in the result'?

All I can think of is that the Nomcon would make a choice that the confirming body found controversial. While that might be because the ten momcon members somehow managed to remain ignorant that a particular person was incompetent, had a felony conviction or whatever, it seems much more likely that the confirming body is going to object to candidates who are insufficiently committed to the prevailing groupthink.

This is somewhat more of a problem on the IAB than the IESG. As I pointed out at the plenary, the fact that the IETF transition plan for IPv6 is bogus has been apparent to many of us for many years. This is the type of observation that should be coming from the IAB. Instead the IAB is yet more inertia that has to be overcome.

If you look at the IETF mailing list threads on IPv6 NAT beginning with the decision to kill NAT-PT you will see that at the start of the conversation I had virtually no support for my position that NAT represents an essential component in the IPv6 transition. Continuing to hold that position was certainly not the popular thing to do, but at this point there is a very clear consensus that IPv4-6 transition has to be transparent for the end user.

The same thing happened on email authentication. It was certainly not a popular view in the anti-spam community when it was first proposed. The authentication faction was pretty isolated, we didn't get invited to speak at conferences in the early days.

Telling people that everything is just splendid is going to make you so much more popular than trying to get people to face the brutal facts.

A justification for the nomcon process would be if it actually increased the chance that unpopular dissident views would be represented in the IETF management. But the exact opposite is the case, every step in the process is designed to minimize the risk of contrarian views. And the IAB selection criteria in particular make the peculiar assertion that compatibility with the group consensus is considered an asset rather than a liability.

-----Original Message-----
From: [email-masked] on behalf of Theodore Tso
Sent: Mon 17/03/2008 11:52 PM
To: Steven M. Bellovin
Cc: Christian Huitema; 'Fred Baker'; Dan Wing; 'IETF Discussion'
Subject: Re: Confirming vs. second-guessing

       	       	    	      	      	     - Ted

---

From: "Hallam-Baker, Phillip" <[email-masked]>
Date: Tue, 18 Mar 2008 07:39:23 -0700

Well there you hit the problem of the status-quo veto. The most effective aspect of the IETF constitution is that it is essentially impossible to overturn the status-quo without arriving at a 'consensus', the existence of which is judged by the incumbent establishment.

We cannot apparently arrive at a consensus on fixing the three stage standards track despite the incontrovertible facts that we are not following the three stage process today and the failure to advance specifications to full standard is harming the IETF reputation.

Worse than that, we cannot apparently be told who the objectors are or even the grounds for the objection.

---

From: Dave Crocker <[email-masked]>
Date: Tue, 18 Mar 2008 08:24:39 -0700

Theodore Tso wrote:
>      community. ...
>          The IAB had no right to make such a
>      decision alone. Besides, CLNP was a pale imitation of IP.

This wasn't about careful wording or reporters getting ahold of the story.  This 
was about a premature and preemptive decision by the IAB.

The nature of the need for a revision to IP had been under consideration by an 
assigned committee, for some time.  The discussion about possible solutions had 
had almost no discussion at all.

In fact, CLNP was a serious candidate.  While some folk rejected it because of 
its ISO genes, that wasn't much of a focus at the time.  Of more concern was the 
significant lack of large scale experience with it.  That made the question more 
of a classic make-vs-buy issue.  Could the IETF satisfy the needs of expanded 
address space with a small change to IPv4?

>      Group (IESG) or the Internet Architecture Board, the new

A phrase like "serious reorganization" leads folk to miss how small the changes 
were, structurally.  The existing structure of the IETF was retained.  From the 
standpoint of organization structure, the changes were minimal, although of 
course they had huge impact.

There were only two changes:

  1.  Decisions previously made by the IAB would now be made by the IESG

  2.  A formal and independent selection process for the IAB and IESG would be 
instituted.

The IAB was retained but careful to avoid anything that looked like an attempt 
to exercise power.  Over time, if found very useful tasks for itself.

The question, today, seems to be whether it is moving too far into an exercise 
of powers it ought not to have.  This isn't anything like the Boston Tea Party 
situation -- the organizational change was made at the IETF in Danville, whereas 
the offending decision was made in Kobe Japan -- since it is incremental and is 
clearly being reviewed as things change.

The discussion taking place on this list would either not have taken place or 
would have been an exercise in futility.  (I suppose it still might be, but for 
different reasons...)

+1

Working group could go through their entire process of developing a 
specification and consensus around it, only to have the IAB reject it out of hand.

>    At least for the years when I was on Nomcom, the
> the NOMCOM".  

Right.  The current discussion should try to specify what exactly the boundaries 
and requirements for a confirming body are and what input is reasonable for them 
to have.

> on this list, from people like Mike St. John's with one view, and

There is a wide range of opinion in our community, for pretty much any topic.
That's one of the reasons we do not require unanimity.

Saying that there was a 90% consensus is a very different datum. The IETF tends
to let a few noisy folk veto rough consensus.

While such folk sometime have concerns that are useful to address, trying to
attend to those concerns is a different task from insisting that the concerns be
satisfied.

d/

-- 

   Dave Crocker

---

From: Theodore Tso <[email-masked]>
Date: Tue, 18 Mar 2008 14:02:40 -0400

On Tue, Mar 18, 2008 at 08:24:39AM -0700, Dave Crocker wrote:
>
> This wasn't about careful wording or reporters getting ahold of the story.  
> This was about a premature and preemptive decision by the IAB.

I quoted Christian's story because it was the kindest towards the IAB.
There were of course far uglier spins on the IAB that were running
around, and the truth is somewhere between the two, I think.  It's not
really that important.  I quoted it becuase I thought it might be
useful to consider the history around the creation of the Nomcom.

> A phrase like "serious reorganization" leads folk to miss how small the 
> changes were, structurally.  The existing structure of the IETF was 
> retained.  From the standpoint of organization structure, the changes were 
> minimal, although of course they had huge impact.
>
>  2.  A formal and independent selection process for the IAB and IESG would 
> be instituted.
>
> The IAB was retained but careful to avoid anything that looked like an 
> attempt to exercise power.  Over time, if found very useful tasks for 
> itself.

We can dispute how small the changes actually were, but the key point
was that IAB had nearly all of its power stripped from it, aside from
the power to make recommendations and, and that was moved to the IESG.
That was a pretty earth-shaking change to the power hierarchy, even if
it was only executed using a few small changes.  (As we all know,
changing even a few lines of code can make a huge difference in how a
program functions.)

I suspect, but am not 100% sure, that the very early NOMCOM's, in
1993-1996, probably had their decisions close to rubber-stamped, given
how badly the IAB had been slapped down by the events of the July,
1992.  Eight years later, right after the turn of the century, it
seemed they were exerting themselves more (and I think that was
appropriate), and it seems to me that more recently, the pendulum has
swung even further to the right.  So perhaps it's not surprising that
we're all over the map, since if you look at past practice over time
we've been all over the map as well.

> The question, today, seems to be whether it is moving too far into an 
> exercise of powers it ought not to have.  This isn't anything like the 
> Boston Tea Party situation -- the organizational change was made at the 
> IETF in Danville, whereas the offending decision was made in Kobe Japan -- 
> since it is incremental and is clearly being reviewed as things change.

I agree that it's nothing like what happened in July, 1992, and we
*are* having this discussion.  The question indeed is what is the
right level of powers is most appropriate for the IAB; ranging from
nearly all powers stripped from it in 1993, to now where it is
requesting access to substantially more documents than it had
historically, and where a few are aruging that the trust boundary for
the Nomcom and the confirming bodies should be the same (i.e., that
the confirming bodies get to see all or nearly all of what the Nomcom
gets to see.)

> Right.  The current discussion should try to specify what exactly the 
> boundaries and requirements for a confirming body are and what input is 
> reasonable for them to have.

And furthermore, give more clarifications about when a confirming body
should try to act.  I believe, for example, that if a confirming body
were to say, "yes, we believe that Person A would do an adequate job,
but Person B will do the job 10% better, and we will reject the slate
until you select person B", that this would be an abuse of the
confirming body's powers.

If instead the feedback is, "we believe this person is totally
unqualified", or "if you select this person half of the volunteer IETF
members in the area will walk off in a huff", that's a very different,
and appropriate feedback from the confirming body.

In between these two areas, of couse, is a rather large grey area...

   	   	     	       	      	 - Ted

---

From: Jari Arkko <[email-masked]>
Date: Tue, 18 Mar 2008 23:59:12 +0200

Spencer, Laksminath,

> judgement

+1

INDEED. Thanks, Laksminath.

And as for what to do about this problem for the future - I would
suggest that we should not focus too much on this particular incident or
even IAB's role. There are different confirming bodies and there are
different situations.

I think a good first step is to make sure the questionnaire templates
are clear about what information stays strictly within the nomcom and
what may be shown to confirming bodies. And clarify that feedback from
the community shall never leave the nomcom, if that's not already clear.

But even after doing this, the basic question of what confirming means
stays open. I think the intent is that the confirming body makes a final
sanity check and under normal circumstances that should be it. I
understand the need for requesting some amount of information; it may
well be that the candidates are not all personally known to the
confirming body. CVs would be useful in this process, for instance. This
is what we have been talking about on the list. However, presumably we
have the confirming process to catch problems, so the more difficult
case is the problem case. Hopefully this is extremely rare or
non-existent. I would expect the confidentiality rules on, e.g.,
feedback to hold even in such a case. But it would seem a dialog between
the nomcom and the confirming body would be needed, and some information
would have to be exchanged -- perhaps in summarized form. "Yes, we
realize A has these issues but the only other candidate B had such and
such problem" or "He escaped from the institution last month? Oops,
perhaps we need to go with our backup candidate instead."

I have no idea how to specify the process on this so that it works under
all circumstances. But my main expectation for the confirming bodies is
that they do a thorough job without resorting to repeating nomcom's work
or questioning the results unless a truly serious problem can be seen.

Jari

P.S. Since the dispute resolution results are public, I would be
interested to know what kind of a report Scott would have been able to
give in the plenary, had the dispute been about a person rather than the
process. I hope we never have to see that. But maybe some food for
thought about the confidentiality rules of this whole thing...

---

From: "Hallam-Baker, Phillip" <[email-masked]>
Date: Tue, 18 Mar 2008 15:36:21 -0700

The minutes of that meeting are online, its pretty much the point at which the system then broke for many years:

http://www.iab.org/documents/iabmins/IABmins.1992-06-18.html

Understanding this particular history is rather helpful if you want to understand the early history of the World Wide Web Consortium.

-----Original Message-----
From: [email-masked] on behalf of Theodore Tso
Sent: Tue 18/03/2008 2:02 PM
To: Dave Crocker
Cc: IETF Discussion
Subject: Re: Confirming vs. second-guessing

   	   	     	       	      	 - Ted

---

From: Ole Jacobsen <[email-masked]>
Date: Tue, 18 Mar 2008 16:38:46 -0700 (PDT)

Yep, they didn't even know how to spell FOREWORD.

Ole

Ole J. Jacobsen

On Tue, 18 Mar 2008, Hallam-Baker, Phillip wrote:

> The minutes of that meeting are online, its pretty much the point at 
> which the system then broke for many years:
> 
> Understanding this particular history is rather helpful if you want 
> to understand the early history of the World Wide Web Consortium.

---

From: Michael StJohns <[email-masked]>
Date: Tue, 18 Mar 2008 21:39:22 -0400

At 05:06 PM 3/17/2008, Lakshminath Dondeti wrote:
>Mike,
>
>Thanks for your note.
>
>Are you saying that there is text within 3777 that says that confirming bodies should not ask for verbatim feedback but could ask for verbatim questionnaire responses?

No.  There is no text in 3777 which prohibits the CBs from asking for anything they want.  There's also nothing in 3777 that requires the Nomcom to give it to the CB.  And there's nothing in 3777 that requires the CB to approve a candidate in the absence of information they consider essential.

>Consider this: what if the next nomcom were to be asked to provide verbatim feedback by one of the confirming bodies, what should they do?  The supporting information that was cited from 3777 is very generic (with specific information provided later, if I may point out again) and so I don't see a reason why it cannot be cited in the context of that request.

Consider this:  What if the next Nomcom were to provide to the CB's ONLY the "brief statement for qualifications for the position" and testimony "as to how [the] candidate meets the qualifications" consisting of the single statement "The candidate has confirmed to the Nomcom that he meets all the qualifications and we believe him."?  [This is all required by 3777 section 5 para 14]

Both of these are outcomes are completely possible AND permitted within the meaning of 3777 but neither are within the spirit of 3777 or this process.  Both are absurd and extreme readings of 3777.  

By verbatim feedback, I am assuming you mean comments by anyone on specific candidates?  (E.g. if I sent you a comment on say Russ Housley?)

The IAB at least clarified 5 years ago in the form of a written document its expectations for the information that would be provided to it by the Nomcom. I've see no evidence that this IAB asked for, or will ask for the "verbatim" information.  I can't predict what future IABs will do. I also can't predict what future Nomcom chairs will do.

>For the nomcom process to work, a number of people in the community ought to believe in it sufficiently to volunteer their time and a larger number of people need to believe in it to provide feedback.
>
>What am I hearing from my vantage point (having been a nomcom member for 3 out of the past 4 years) is what guides me to work on this; my goal is to maintain and if possible increase the level of confidence in the process.  When I ask for information, I provide assurances that the information is for nomcom's eyes only 

And that is not required to 3777 and may actually be harmful to the nominations process.

>(for instance, this was specifically the case when we interviewed people).  3777 also says any dissemination of private information by the nomcom must be minimal.  So, I for one cannot just hand out private information without clearer text in 3777.

I finally figured out why there's so much disagreement over this.

3777 got an expanded Security Considerations section over 2727.  That's where you're getting the "minimal" language from.    This appeared in the first draft by Jim Galvin of 2727bis and I don't know what discussion if any there was over this section.  (The archives from that group appear to be off line right now). 

A couple of things:  Security Consideration sections tend to be Informative, not Normative.  This section I believe was basically a "use caution in handling confidential information" and "if you need to seek outside help, please share the minimum" cautionary tale.  If it had been meant to be Normative it would have been included in Section 3 subsection 6.

I believe (and I hope the archives will reflect this) that the addition of this section was never meant to reflect on intra-process communications.  I think it was written as it was (e.g. didn't mention the CBs) because the Nomcom (and ONLY the Nomcom) had hooks for external consultation - the "ask someone about a list" consultation and the "dispute" consultation and because Jim was getting heat over a too short security considerations section. 

Yet another thing to clean up in the next version....

---

From: Russ Housley <[email-masked]>
Date: Wed, 19 Mar 2008 11:38:19 -0400

The archives of the NomCom WG that generated RFC 3777 are now online:

http://lists.elistx.com/archives/ietf-nomcom/