Subject: Re: Complaining about ADs to Nomcom (Re: Voting (again))

From: Brian E Carpenter <[email-masked]>
Date: Wed, 04 May 2005 17:04:30 +0200

Please understand the argument that was made strongly while
RFC 3777 was in WG discussion: there is reason to believe that
a substantial fraction of the potential candidates would *not*
volunteer if they were entering a public race. It's hard to
judge the validity of that argument, but it's certain that
publishing the names would change the whole process in
unpredictable ways.

One way to open up the process would be to allow any participant
to personally request a list of candidates from Nomcom, against
a personal non-disclosure promise. (Not my idea; this was suggested
during last week's IESG retreat.)

     Brian

Soliman, Hesham wrote:
> Spencer, 
> 
> You hit an important issue. Having a small group of people
> choose who they want feedback from (granted everyone else is welcome 
> to send feedback without knowing the options for ADs) is not 
> a recipe for a successful process. I haven't yet seen a good reason
> for not publicising the names of people nominated for an AD position.
> Until this is done, we can't claim an open process for AD selection.
> 
> Why have a "selected" group of people knowing all the information necessary
> to choose an AD? And on what basis are these people selected? I know
> people that are not chairing any WGs that were asked for input by the nomcom
> while others weren't. 
> 
> I can see pros and cons in publicising the list of nominees, but in 
> the absence of a clear advantage I choose openness, because eventually, an
> open process should be self-correcting. I can't say the same about the 
> current process.
> 
> Hesham
> 
> 
>  > Spencer Dawkins wrote:
>  > 
>  > > Just to agree with JohnL,
>  > >
>  > >>
>  > >> NOMCOM has been good about soliciting feedback, but I still think 
>  > >> that we miss out on useful feedback because IETF members cannot 
>  > >> reliably say who is a candidate and who is not.  Some 
>  > candidates have 
>  > >> sent around BCC: mails, from time-to-time, saying that they are a 
>  > >> candidate & would appreciate folks to send comments to 
>  > NOMCOM. This 
>  > >> doesn't seem like a good way for getting information 'public.'
>  > >
>  > >
>  > > I've served as a WG chair, and as a member of the General Area 
>  > > Directorate, so I'm one of the people that NOMCOM was 
>  > actively seeking 
>  > > input from ("this is a list of the people who have been 
>  > nominated for 
>  > > X Area Director, plus a couple of ringers, please tell us what you 
>  > > think we should know that would help us make a better decision").
>  > >
>  > > My point is that I *have* seen a complete list of nominations, 
>  > > including a couple of ringers, for specific AD positions, 
>  > and I *have* 
>  > > seen a complete list of nominations for IAB positions.
>  > >
>  > > This is not a bad thing (the AD positions were in areas I 
>  > was working 
>  > > closely with). The less-than-desirable part is that my input on 
>  > > unannounced candidates was based on more information about who was 
>  > > being considered than was generally available. I could say "that 
>  > > person would be the greatest disaster for the IETF since ...", and 
>  > > people who disagreed with my input don't even know the 
>  > person is being 
>  > > considered ... unless they also got the same list, or unless that 
>  > > specific person happened to ask others to provide NOMCOM input.
>  > >
>  > > Hardly seems fair, does it?
>  > >
>  > > For the last two years, I've met with the NOMCOM 
>  > representatives and 
>  > > gone through the list of every IAB/IESG member that's up 
>  > for renewal, 
>  > > so it's not like people don't TRY to provide good input 
>  > (positive and 
>  > > negative). But it's difficult to provide input on unannounced 
>  > > candidates, unless they "out" themselves.
>  > >
>  > > Are we supposed to send a list of notorious Internet 
>  > kooks, just in 
>  > > case one of them has been nominated for something?
>  > >
>  > >> In the absence of facts, there are lots of rumors about whether a 
>  > >> specific IESG / IAB member is stepping down or not; reasons why; 
>  > >> etc.  This doesn't seem to be an optimal process, IMO.
>  > >
>  > >
>  > > I've also had conversations with ADs who decided to step 
>  > down, then 
>  > > decided not to, then decided to step down (lather, rinse, repeat). 
>  > > Private sampling doesn't seem to provide reliable 
>  > information, even 
>  > > when the sampling is direct-to-the-intermittent-candidate.
>  > >
>  > > ... and, to go a tiny bit closer toward the edge ... 
>  > saying that we 
>  > > believe someone can serve successfully as an AD for two years (and 
>  > > hopefully for four, since it takes a year to come up to 
>  > speed, we are 
>  > > told), but is too sensitive to be nominated publically for the 
>  > > position, seems silly. If someone cares what people think 
>  > that much, 
>  > > how can the same person serve with integrity after being selected?
>  > >
>  > > Spencer
>  > >
>  > >
>  > > _______________________________________________
>  > > Ietf mailing list
>  > > [email-masked]
>  > > https://www1.ietf.org/mailman/listinfo/ietf
>  > >
>  > >
>  > 
>  > 
>  > 
>  > _______________________________________________
>  > Ietf mailing list
>  > [email-masked]
>  > https://www1.ietf.org/mailman/listinfo/ietf
>  > 
> 
> ===========================================================
> This email may contain confidential and privileged material for the sole use
>  of the intended recipient.  Any review or distribution by others is strictly
>  prohibited.  If you are not the intended recipient please contact the sender
>  and delete all copies.
> ===========================================================
> 
> 
> _______________________________________________
> Ietf mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/ietf
> 

_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf

---

From: Jari Arkko <[email-masked]>
Date: Thu, 05 May 2005 00:20:29 +0300

Hi Brian,

> publishing the names would change the whole process in

Like Hesham, I am also aware of this argument and do not
necessarily agree with it. (In fact, one could make the point
that not being able to tell you have volunteered sounds a bit
wimpy compared to what kind of public visibility and pressures
the folks need to deal with if they are actually selected,
particularly to the IESG.)

Also, to give you a datapoint on potential candidates -- my name's
been given to nomcom a couple of times and I certainly would not be
bothered by folks knowing who has volunteered and who has not.
And I think John L said something similar as well.

--Jari

---

From: John C Klensin <[email-masked]>
Date: Sat, 07 May 2005 09:18:19 -0400

--On Wednesday, 04 May, 2005 17:04 +0200 Brian E Carpenter
<[email-masked]> wrote:

>...

Let me follow this up a bit.

I've been encouraging people to try to sort through reasons and
things that would make it different on another thread, but I
think we have a "choice of potential candidates" problem today.
The IESG and IAB received "very few real choices" reports from
several nomcoms while I was serving there.  Possibly things have
gotten better, but I have my doubts.  

Whatever the reasons, we don't seem to have enough plausible
candidates to provide reasonable turnover on the IESG (which,
personally, I think would be healthy).

Even assuming that publishing candidate lists would result in
better-quality feedback and permit the Nomcom to make better
choices among plausibly-appropriate candidates, please look at
the other side.   There are people in the community who, for
whatever reassons, find the prospect of a "volunteer, have that
public, and then not be selected" process sufficiently painful
to prevent them from volunteering... or certainly from
volunteering more than once or twice.  There are also subtle
differences in how one can volunteer that can be expressed in
confidence to the Nomcom: "I don't really want to do this, but
will serve if you conclude that it is important and I'm the best
choice" or "I can't work with X and would accept the position
only if X were not selected" are comments that can be made
today, but which don't show up on public lists.   I believe that
many of the people who would semi-volunteer with such conditions
would decline to volunteer at all if their names would go on an
undifferentiated public list.

So, those of you who strongly advocate a public list...  What
percentage of the already-too-small potential candidate pool are
you willing to lose?   Are you convinced that anyone with
sensitivities or conditions similar to those outlined above
would make a bad AD if selected?   Do you think the tradeoffs
are worth it?

      john

---

From: "Joel M. Halpern" <[email-masked]>
Date: Sat, 07 May 2005 11:13:57 -0400

You raise two questions about making the candidate list public.
You raise the question of whether we can afford the loss of candidates from 
those people not willing to be seen as losing.  I will admit to not being 
sure I understand the driver for people who both have that concern and 
could do the job of AD.  (I can see it for IAB membership.)  I can't swear 
that we won't lose one or two vialbe candidates, but I tend to doubt it.

You also ask about the nuances.  The details / constraints / parameters for 
a volunteer would, I think, still be confidential.  I can't see any problem 
with that information being only in the hands of the nomcom.  The public 
will not be able to tell that the nomcom might have chosen a different 
person but for some constraints.  (Heck, that is probably always the case.)

There is one reason I have heard occasionally that I wonder about.  If the 
list is made public, will this cause public second guessing based on 
insufficient information after the results are announced.  That, I think, 
would be a serious problem.

And there is some risk (small, I think) of people pushing others to endorse 
them.  This would seem easier with a public list, because the nomcom is not 
left wondering why they got the supportive email.

On balance, I think we would be significantly better off with a public list 
because of the ability to get much wider feedback.

Yours,
Joel

At 09:18 AM 5/7/2005, John C Klensin wrote:
>... initial discussion of publishing candidates elided ...

---

From: Margaret Wasserman <[email-masked]>
Date: Sat, 7 May 2005 11:43:17 -0400

Hi John,

>personally, I think would be healthy).

What is "reasonable turnover" for the IESG?

I haven't been on a nomcom, but (from the outside) most of them seem 
to start with the assumption that they should not change more than 3 
IESG members at a time.  If that is considered prudent, then we are 
talking about  a situation where a maximum of 1/4 of the IESG will be 
intentionally replaced each cycle. Factoring in mid-term resignations 
and the possibility that the nomcom may occasionally make a poor 
choice requiring quicker turnover, successful ADs who are willing to 
continue serving will probably be in-office for an average of 8-10 
years (4-5 terms).  This seems to match existing practice.

What level of turnover do you think would be healthy?  And what would 
be the impacts of having more new ADs each year?

Margaret

---

From: Geoff Huston <[email-masked]>
Date: Mon, 09 May 2005 03:33:49 +1000

>And there is some risk (small, I think) of people pushing others to 
>endorse them.  This would seem easier with a public list, because the 
>nomcom is not left wondering why they got the supportive email.

A risk not without quite extensive precedent over the years, and the 
concept of overt electioneering is one that personally I find a strange 
perversion of an already somewhat strange process. Are we after the the 
judgement of a few as to the best qualified individual for the role, or the 
one who is seen as being the "most popular" on the basis of a concerted 
campaign of electioneering? What is this body again? What is its purpose? 
Why does it exist? etc.

Geoff

---

From: "Joel M. Halpern" <[email-masked]>
Date: Sun, 08 May 2005 13:52:05 -0400

I agree that electioneering is extremely undesirable.
And it does currently agree to some degree.
The question is whether publishing the list would actually cause a 
significant increase in that behavior.  If we conclude that publishing 
would indeed result in such an increase, then that is a good reason to not 
publish the list of candidates.

But given that electioneering already goes on, and that the feedback the 
nomcom receives is somewhat affected by the question of who can manage to 
discern the candidates from the leaky information flow, I am doubtful as to 
whether it would result in a significant increase.

Yours,
Joel M. Halpern

At 01:33 PM 5/8/2005, Geoff Huston wrote:

>judgement of a few as to the best qualified individual for the role, or 
>the one who is seen as being the "most popular" on the basis of a 
>concerted campaign of electioneering? What is this body again? What is its 
>purpose? Why does it exist? etc.
>
>Geoff
>
>
>

---

From: Jari Arkko <[email-masked]>
Date: Mon, 09 May 2005 22:34:54 +0300

Hi John,

>would make a bad AD if selected?   Do you think the tradeoffs
There are possible tradeoffs in this decision. But let
me turn the questions around for you: What percentage
of the candidate pool are you willing to lose because
other potential candidates don't realize that the set of
realistic alternatives is small? Do you want to risk
selecting someone that the nomcom didn't realize had
issues but the IETF at large knew about it? Are you
sure that you want to exclude input from the whole
IETF? What image do we want to project from our
process to the participants? Are you convinced that
shyness from public eye isn't being used to guide the
process by a set of insiders? Do you have any hard data
about sensitivities, anything to back up the default (to
me) assumption of being open? And if there is hard
data, can you show that it is not an issue already,
in the partial publicity of list-receivers and friends of
candidates etc?

--Jari

---

From: Sam Hartman <[email-masked]>
Date: Fri, 13 May 2005 23:56:41 -0400

>>>>> "Brian" == Brian E Carpenter <[email-masked]> writes:

    Brian> Please understand the argument that was made strongly while
    Brian> RFC 3777 was in WG discussion: there is reason to believe
    Brian> that a substantial fraction of the potential candidates
    Brian> would *not* volunteer if they were entering a public
    Brian> race. It's hard to judge the validity of that argument, but
    Brian> it's certain that publishing the names would change the
    Brian> whole process in unpredictable ways.

Seems fairly easy to judge the validity of that argument to me.  ASk
the nomcom to ask volunteers whether they would have volunteered if
their name was gonig to be made public.  Collect statistics.

--Sam

---

From: Dave Crocker <[email-masked]>
Date: Fri, 13 May 2005 23:02:35 -0700

>  their name was gonig to be made public.  Collect statistics.

Sam, 

Sorry, no.

As I posted earlier, that sort of methodology relies on what survey 
researchers call "self-report". 

It is very good for assessing attitudes and very bad for assessing 
actual behavior.

For example, what you are likely to get are responses that indicate 
whether the people would like to have nominations be public. 

It does not guarantee -- and well might not even correlate with -- 
whether they really would run or not run, depending on the public-ness 
of the nomination.

It is one thing to ask simple questions about simple issues.  As soon as 
we get into something more "political" the psychodynamics get messy.

  d/
  ---
  Dave Crocker
  Brandenburg InternetWorking
  +1.408.246.8253
  dcrocker  a t ...
  WE'VE MOVED to:  www.bbiw.net

---

From: Will McAfee <[email-masked]>
Date: Sat, 14 May 2005 20:22:00 -0400

I think the minimum time before a document can pass to another
standards-track state is ridiculously long.  If an rfc is huge, I can
understand that.  But to sweep that over all of them?  A two-page
proposed standard can take an absolutely ridiculous amount of time to
pass through!  I say we have variations based on how long the document
is.

---

From: "James M. Polk" <[email-masked]>
Date: Sat, 14 May 2005 21:03:36 -0500

At 08:22 PM 5/14/2005 -0400, Will McAfee wrote:
>pass through!

each of the normative references need to be at that higher level too 
though... even for your 2 page PS-RFC

>I say we have variations based on how long the document
>is.
>

cheers,
James

                                 *******************
                 Truth is not to be argued... it is to be presented.

                      Alas, few *truths* exist without the math.

                             ...all else is a matter of perspective

---

From: Brian E Carpenter <[email-masked]>
Date: Sun, 15 May 2005 13:55:44 +0200

This would be a good topic for the newtrk WG, I think,
since it is so specific.

    Brian

---

From: Bill Manning <[email-masked]>
Date: Sun, 15 May 2005 08:57:56 -0700

perhaps....   but i think (based on personal experience w/ the 
"discover" draft
first submitted in 1998 - still in "process") that the reason for the 
increased
"slowness" in getting documents through the RFC Editor is the 
extra-ordinary
burden placed on the RFC editor staff to coordinate w/ and through the
IETF's IAB/IESG functions and the creation of onerous "objective" checks
that must be done before the RFC editor can proceed with document
publication.  Historically, the RFC editor had broad subjective power to
publish or not - since the community then trusted the technical 
grounding of
the RFC editor.  The Editor has not changed - the community has lost 
trust in
the RFC editor to do the proper/correct thing and tries to 
"second-guess" the
RFC editor at every step of the way... which is the real reason for the 
extra-long
time it takes to get a document published.

IMHO, if the RFC editor was given the same latitude it had in 1997, 
publication would
take weeks, not months or years.  of course YMMV.

--bill

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 16 May 2005 10:08:15 +0200

Bill, the thing that can create unbounded delay on RFC publication
is a normative reference to work in progress. But apart from that,
it's dangerous to generalize. For many years, the RFC Editor has
only had complete discretion for non-IETF documents (for which there
is now a 4 week timeout on the IESG review, see RFC 3932).

    Brian

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 16 May 2005 10:52:58 +0200

You've seen Danny's message with the results of asking the
question in a straightforward way - 20% of IESG nominees
say they would not have volunteered. Unlike Dave, I am
willing to believe them.

fwiw I responded "Yes" to Danny's question, but not
without careful thought and some hesitation.

    Brian

---

From: [email-masked]
Date: Mon, 16 May 2005 09:42:51 -0400

there have been cases where names were published by the iab or iesg, 
e,g, in selecting for the iaoc and the isoc bot.

while i know it is probably impossible to know how many people refused 
to be considered because of the publication (don't remember if there 
was a opt-out of publication opportunity - if so it would be 
interesting to know how many chose it), what i would be interested in 
knowing is whether the I* received many comments from a wide cross 
section of the ietf community on those occasions.

thanks
a.

---

From: Dean Anderson <[email-masked]>
Date: Mon, 16 May 2005 10:20:07 -0400 (EDT)

The size of an RFC has nothing whatsoever to do with its impact. It takes
time for people review, test, document problems, and propose alterations.
That's the point of delay and a gradual process.

Anyone can give you a one page document that will break just about 
everything. Because its only one page doesn't mean it should be less well 
tested and reviewed. Nor does it mean that its somehow easier to test and 
review. 

		--Dean

-- 
Av8 Internet   Prepared to pay a premium for better service?
www.av8.net         faster, more reliable, better service
617 344 9000

---

From: Dave Crocker <[email-masked]>
Date: Mon, 16 May 2005 08:22:10 -0700

>  say they would not have volunteered. Unlike Dave, I am

Unfortunately Brian, this has nothing to do with my personal "beliefs".  

It has to do with decades of hard-learned methodology realities, in the field 
of survey research.

The convenience afforded by the concept of "straight-forward" does not exist 
in survey research. 

To the extent that you care to consider this issue as a matter of technical 
knowledge, rather than personal opinion, take a look at, for example, at 
<http://www.sysurvey.com/tips/introduction_to_survey.htm>. 

  d/
  ---
  Dave Crocker

---

From: "JFC (Jefsey) Morfin" <[email-masked]>
Date: Mon, 16 May 2005 12:25:08 +0200

Could these two comments of yours introduce some solutions?

At 10:08 16/05/2005, Brian E Carpenter wrote:
>Bill, the thing that can create unbounded delay on RFC publication is a 
>normative reference to work in progress. But apart from that, it's 
>dangerous to generalize. For many years, the RFC Editor has only had 
>complete discretion for non-IETF documents (for which there is now a 4 
>week timeout on the IESG review, see RFC 3932).

Does that mean that specialised SDO/TFs, meeting the IETF general values 
and interests, could be a better vehicle to publish RFCs for information 
which could be market tested; and then reinjected in the standard track if 
positively accepted? This would be a way to help innovation without risking 
derailing the IETF.

At 10:52 16/05/2005, Brian E Carpenter wrote:
>You've seen Danny's message with the results of asking the question in a 
>straightforward way. - 20% of IESG nominees say they would not have 
>volunteered. Unlike Dave, I am willing to believe them. fwiw I responded 
>"Yes" to Danny's question, but not without careful thought and some hesitation.

Did the [email-masked] see it. Question is the motivations of the "No" and 
of hesitations. Could they be a good filter of the problem the IETF meets?

It is interesting that the IETF at large does not applies to itself the 
excellent analysis carried on WG mistakes in RFC 3774 (one is missing 
however: the _organisation_ of consensus by exhaustion). In particular, the 
IETF has no precise yearly Charter and no Road Map.

jfc

---

From: John C Klensin <[email-masked]>
Date: Mon, 16 May 2005 14:22:37 -0400

> is now a 4 week timeout on the IESG review, see RFC 3932).

Brian,

Part of what I find troubling about these discussions is the 
disconnect between theory --or maybe "aspirations" would be a 
better term-- and practice.  Let's look at "complete discretion 
for non-IETF documents" as an example.   In the years between 
the publication of 2026 (or earlier) and that of 3932, the rules 
were that the IESG's review applied only to conflicts with IETF 
work, serious risks to the Internet, etc.   By agreement between 
the IESG and RFC Editor, that review was conducted against a two 
week timeout.  That timeout was extended to four weeks when it 
became clear that the IESG was _never_ meeting the schedule and 
kept needing to ask for extensions.  For many documents, the 
main consequence of the shift from a two-week timeout to a 
four-week timeout is that the IESG simply ignored the deadline 
rather than bothering to ask for extension.  And the RFC Editor, 
presumably out of professional courtesy and respect for the 
smooth functioning of the IETF, never (at least as far as I 
know) turned down an extension request or enforced a timeout 
(i.e., got tired of waiting for the IESG and just went ahead and 
published).

In theory, 3932 changed almost nothing.   The IESG asserted that 
it was not going to do what it had been barred from doing all 
along, which was holding up individual submissions ("non-IETF 
documents") until they were rewritten to match the tastes and 
preferences of any AD who cared.  The principle that quality 
control for those documents was an RFC Editor responsibility was 
clarified.    And the four-week timeout was formalized.

In practice, it changed even less.   Since 3932 was published, 
there have been at least severa instances of "discuss" positions 
in then IESG over substantive (not conflicts with IETF work) 
objections, and I'm aware of at least one or two over editorial 
matters.  The four-week cutoff is still ignored, apparently 
routinely, and the RFC Editor apparently does not consider it 
wise to take a "the IESG has not responded within four weeks, so 
they don't have an objection" position.

So, much as I remain optimistic that 3932 --which, IMO and given 
the language in 2026 never should have been necessary-- will 
eventually be taken seriously by all parties, the evidence is 
that hasn't happened yet.

    john

---

From: John C Klensin <[email-masked]>
Date: Mon, 16 May 2005 14:58:21 -0400

In the light of this and Dave's comments, and since I used to 
teach people how to design survey questions so that the 
questions were as non-reactive as possible and the answers could 
be interpreted.  There is nothing inherently wrong with a 
self-report question.  We ask them all the time and normally 
expect truthful answers.  The tricky part is understanding which 
questions people may not want to answer truthfully, the reasons 
why, and, if the person who is reluctant to answer provides 
_some_ answer, how either that or a pattern of non-response is 
likely to bias the results.

For example, if one asks a large sample of 10-year-olds how old 
they are, the answers will, predictably, be mostly truthful: 
there are few incentives to lie and mistakes will tend to be 
nearly randomly distributed (slightly fatter tail to the 
"younger" side because of forgetting birthdays).   If one asks 
the same question of 60 year olds, the answer pattern would 
probably be different, and it is important, if one is trying to 
interpret validity, to understand those differences and their 
likely impact, rather than assuming either that all population 
groups are the same or that all self-report answers are invalid.

Coming back to the question at hand, if the nomcom asks people 
whether they would have accepted nominations if their names 
would become public, why would someone lie?  And, if they did, 
then which way would the report be biased.   I would think that 
people who are inclined to give incorrect answers would be more 
inclined to answer "no problem" given the community's biases 
about openness and unwillingness to admit that they require 
secrecy.  Maybe I'm wrong about that, but, if I'm not, the 
results Danny reported would, if anything, underestimate the 
number of people who would not be willing to be considered if 
their names were public.

We now return you to the regularly-scheduled religious arguments 
on the subject.

        john

--On Monday, May 16, 2005 10:52 +0200 Brian E Carpenter 
<[email-masked]> wrote:

>>> Seems fairly easy to judge the validity of that argument to
>>> me.  ASk the nomcom to ask volunteers whether they would
>>> have volunteered if their name was gonig to be made public.
>>> Collect statistics.
>>
>>
>>
>> Sam,
>>
>> Sorry, no.
>>
>> As I posted earlier, that sort of methodology relies on what
>> survey  researchers call "self-report".
>>
>> It is very good for assessing attitudes and very bad for
>> assessing  actual behavior.
>>
>> For example, what you are likely to get are responses that
>> indicate  whether the people would like to have nominations
>> be public.
>>
>> It does not guarantee -- and well might not even correlate
>> with --  whether they really would run or not run, depending
>> on the public-ness  of the nomination.
>>
>> It is one thing to ask simple questions about simple issues.
>> As soon as  we get into something more "political" the
>> psychodynamics get messy.
>>
>>
>>   d/
>>   ---
>>   Dave Crocker

---

From: "Spencer Dawkins" <[email-masked]>
Date: Mon, 16 May 2005 21:48:09 -0500

> say they would not have volunteered.

It's not my intent to develop BCP text on [email-masked], but I do feel 
the need to say that we've had a previous suggestion that we could ask 
people if it's OK that their names be published and respect both "yes" 
and "no" answers.

Could the 20 percent who wish to remain anonymous do so without the 80 
percent also having to be anonymous to the community (modulo IESG and 
IAB members who are currently serving the community)?

Spencer

---

From: Dave Crocker <[email-masked]>
Date: Mon, 16 May 2005 21:10:42 -0700

>  inclined to answer "no problem" given the community's biases

It is to a candidate's advantage to limit the amount of information provided 
to the nomcom, since the more obscure sources are more likely to have negative 
feedback about the candidate.

In any event, this is less a question of "lieing" and more a question of 
"preference".  

The question that was asked more than likely elicited a response to "tell us 
if you strongly prefer to have nominations be kept secret."

The interviewees had no cost in giving the answer.  They are not held to their 
responses.  Hence their answer is about preferences, not guarantees that they 
will not run.

What is most fascinating about this sequence is the apparent belief that those 
participating in a political process do not try to game it.

d/
  d/
  ---
  Dave Crocker

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 17 May 2005 16:45:32 +0200

John C Klensin wrote:
...
> In theory, 3932 changed almost nothing.   The IESG asserted that it was 
> not going to do what it had been barred from doing all along, which was 
> holding up individual submissions ("non-IETF documents") until they were 
> rewritten to match the tastes and preferences of any AD who cared.  The 
> principle that quality control for those documents was an RFC Editor 
> responsibility was clarified.    And the four-week timeout was formalized.
> 
> In practice, it changed even less.   Since 3932 was published, there 
> have been at least severa instances of "discuss" positions in then IESG 
> over substantive (not conflicts with IETF work) objections, and I'm 
> aware of at least one or two over editorial matters.  The four-week 
> cutoff is still ignored, apparently routinely, and the RFC Editor 
> apparently does not consider it wise to take a "the IESG has not 
> responded within four weeks, so they don't have an objection" position.

Well, there are always going to be judgement calls about whether something
is or isn't an end-run, which is where I would expect "discuss"
positions to come from on such documents. It's also a fact that
with o(15) drafts on the IESG agenda every two weeks, respecting the
four week cycle can be tough if the draft in question is of any
length and complexity. All I can say is the IESG is supposed to
follow 3932 and is very much aware of that.

    Brian

---

From: Thomas Narten <[email-masked]>
Date: Tue, 17 May 2005 13:30:55 -0400

> positions to come from on such documents.

Process-wise, this isn't right, IMO (which is where I suspect John is
coming from). Process-wise, the thing to do is respond to the RFC
editor with one of response 4 or 5 in Section 3 of RFC 3932 (e.g.,
potential end run, this needs to be reviewed by the IETF', to make it
clear exactly what state the document is.

The decision of whether something is an "end run" should be relatively
fast. One can always air on the conservative side if in doubt and say
"looks like end run", while getting more detailed reviews.

Clearly, getting the reviews and resolving issues stemming from such
reviews can take time. But we should be able to pretty quickly decide
which pile a document falls into.

Thomas

---

From: Joe Touch <[email-masked]>
Date: Tue, 17 May 2005 13:17:51 -0700

> "looks like end run", while getting more detailed reviews.

If in doubt, it's more productive say "MIGHT be an end run" and ask for
more time. Throwing tar isn't a substitute for hesitating before throwing.

Joe

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 18 May 2005 10:50:07 +0200

Joe Touch wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> 
> 
> Thomas Narten wrote:
> 

You're correct about the process. I was trying to explain what may
happen in reality...

> more time. Throwing tar isn't a substitute for hesitating before throwing.

3932 isn't explicit about asking for more time. It says:

    The IESG will normally have review done within 4 weeks from the RFC
    Editor's notification.  In the case of a possible conflict, the IESG
    may contact a WG or a WG chair for an outside opinion of whether
    publishing the document is harmful to the work of the WG and, in the
    case of a possible conflict with an IANA registration procedure, the
    IANA expert for that registry.

Would it be better if the process required an explicit request for
more time?

     Brian

---

From: Bill Sommerfeld <[email-masked]>
Date: Wed, 18 May 2005 11:55:54 -0400

> Would it be better if the process required an explicit request for

In the face of variable workload it makes no sense to expect
constant-time response from the IESG.

My understanding is that there is no load-levelling on the IESG agenda
-- documents ready for review go on the next agenda regardless of how
full it is.

A review body I'm involved with at Sun has a somewhat different approach
to reviewer time management.

Oversimplifying wildly:  proposals are split into "fast-track" and
"full" review.

Fast-tracks are reviewed by email with a confirming step in our
meetings.

If the email discussion converges we typically spend about 15-30 seconds
of meeting time per fast-track.  if the discusion doesn't converge, it
may get turned into a full review.

We sharply limit the amount of meeting time we spend discussing
fast-tracks, and also limit the number of full reviews per meeting.  

Because the full reviews are scheduled at least a meeting or more in
advance into a specific time slot within the meeting, the folks making
the proposals can attend the review meeting/concall and can often
quickly resolve issues 
which in IESG terms might wind up as a DISCUSS.

						- Bill

---

From: Dave Crocker <[email-masked]>
Date: Wed, 18 May 2005 12:17:57 -0700

> > >  Process-wise, this isn't right, ...
> > >

Brian,

The problem is that the IESG, in reality, treats these deadlines as 
optional.  Hence, mechanisms that are designed to balance trade-offs 
between timeliness and oversight have become skewed wholly towards the 
oversight, losing the timeliness.

Please imagine someone running a network that is chronically congested 
and simply shrugging at the problem. 

That is what the IETF experiences today with the process at document 
completion.  

Users of a chronically congested network will switch to an alternate 
provider.  Not surprisingly, that is what we are seeing happen to the 
network we call the IETF.

It does not seem at all reasonable to stay with a process that is 
chronically congested.  We simply must find alternative mechanisms.

  d/
  ---
  Dave Crocker

---

From: Keith Moore <[email-masked]>
Date: Wed, 18 May 2005 15:45:20 -0400

> The problem is that the IESG, in reality, treats these deadlines as optional.  

When there's no limit on the amount of work that can be imposed on IESG
from external sources, treating such deadlines as optional (or at least
relaxing them) is common sense.

If IESG charters too many working groups, any delays due to excessive
workload are arguably IESG's own fault.   They should not have
chartered so many WGs and should have held those WGs to their
timetables.  OTOH, when too many people send in half-baked individual
submissions, that's beyond IESG's control. It's silly to blame IESG for
delays caused by a workload that other people impose on them.

A policy that creates an expectation that an individual submission will
be published and receive the implicit endorsement of IETF (disclaimers
to the contrary notwithstanding), taking up resources from the RFC
Editor and/or IESG, is even sillier.  Or at best anachronistic.

Keith

---

From: Dave Crocker <[email-masked]>
Date: Wed, 18 May 2005 13:08:39 -0700

> >  optional.

This is a very good example of the approach that many folks have taken, in 
thinking about these problems.  It is really pretty amazing, given the nature 
and history of this community.  It is not immodest to assert that we are a 
community of relatively self-actualized participants, with a considerable 
history of solving difficult protocol (process) problems.  Yet in the matter 
of solving serious deficiencies in our organizational processes, we shrug and 
give up, rather than look for changes that solve the bottlenecks, improve 
quality and retain the underlying culture of the original IETF process. 

For example, as you note, the IESG approves working groups and working group 
charters.  So the IESG does, in fact, have the ability to control the later 
demand placed on it.

Another example is the kind of mission creep that John Klensin noted.  The 
IESG review is supposed to be for some vary narrow purposes, but the scope has 
greatly expanded.

>  Editor and/or IESG, is even sillier.  Or at best anachronistic.

Many of these problems derive from the IESG believing that it is solely 
capable of being responsible for doing all the reviews for all documents.  The 
official scope of the IESG review of non standards-track individual 
submissions is supposed to be pretty darned limited.  Equally, the reasonable 
scope of the IESG for work that is from an experienced working group 
reasonably ought to be limited.

If one reviews the IETF mailing list postings, there is a extremely 
consistent pattern of responses from the IESG.  

Every proposal for any change that might, in any way, affect its 
authority, is dismissed quickly and completely.  And these dismissals 
tend to be based, at best, on very weak conjectures about possible 
human behaviors, and without any attention to the downside of keeping 
things the way they are.

So we have a process that isn't working.  

We have at least 3 years in which the community forcefully sounded the 
the alarm. 

Yet we have no changes that make our output more timely, relevant and 
useful.

Working groups need to do work that is more relevant, focused and 
timely to current Internet needs.  

The IESG needs to be more timely and constructive in helping working 
groups achieve these improvements.

The usual reference to fear of releasing bad specifications onto the 
net nicely ignores the dangers of releasing no specifications, or of 
having the specifications get developed elsewhere.  

Yet these latter outcomes are what is happening.

  d/
  ---
  Dave Crocker

---

From: Keith Moore <[email-masked]>
Date: Wed, 18 May 2005 16:26:48 -0400

> quality and retain the underlying culture of the original IETF process. 

Dave, that's a ridiculous statement.  I've made several suggestions in 
numerous fora, including when I was on IESG, for how to minimize
bottlenecks and improve quality.  I was one of the first to suggest an
I-D tracker.  I may have been the first to draw up a state diagram for
the IESG's document handling process and use that to look for ways to 
minimize states in which documents could get hung.  Lots of improvements
have been made due to suggestions from myself and lots of other people.

For you to use my observation that unbounded delays are a natural
result of an unbounded workload as evidence that "we shrug and give up,
rather than look for changes" is both insulting and completely without
merit.

Keith

---

From: [email-masked]
Date: Wed, 18 May 2005 13:34:25 -0700 (PDT)

> charters.  So the IESG does, in fact, have the ability to control the later

Well, sort of. For one thing, note your own use of the word "later". The IESG
that creates a WG is often not the one that ends up having to deal with its
output.

Second, in the apps area at least, we have in fact seen a dramatic reduction
of the number of WGs. I just checked, and when I left the IESG there 
were 25 apps area groups. This number is now down to 14. And while there
are obviously lots of other factors, close to a factor of 2 reduction in the
number of groups should have had some effect. Have you noticed any?

> IESG review is supposed to be for some vary narrow purposes, but the scope has

I'm afraid I have to agree with this assessment, and I think it causes
timeliness problems, although I confess that I do not have empirical evidence
to support this belief.

> submissions is supposed to be pretty darned limited.

All this assumes that there are in fact lots of fairly problematic individual
submissions for the IESG to deal with. (I think those of us who have been there
all agree that these are the documents that are far and away the most costly in
terms of time.) Checking the datatracker for the two apps ADs, I didn't find
any documents I would say qualified. Indeed, the number of truly individual
submissions (as opposed to submissions that happen to be under someone's name
but are in fact closely related to longstanding IETF activities, such as moves
to draft) seems to be pretty low.

I therefore have to question whether we are talking about the actual factors
that are _currently_ creating delays.

				Ned

---

From: Joe Touch <[email-masked]>
Date: Wed, 18 May 2005 14:44:40 -0700

...
> Would it be better if the process required an explicit request for

yes, IMO.

---

From: Dave Crocker <[email-masked]>
Date: Wed, 18 May 2005 15:21:05 -0700

> >  This is a very good example of the approach that many folks have taken,
>  Dave, that's a ridiculous statement.  I've made several suggestions 

I don't recall citing any individual's actions, so i do not know how you came 
to view my statement as a direct criticism of you.

>   I was one of the first to suggest an I-D tracker.

the kinds of delays people have complained about pertain to rather more than 
difficulty with 'tracking'.

  d/
  ---
  Dave Crocker

---

From: Keith Moore <[email-masked]>
Date: Wed, 18 May 2005 18:28:10 -0400

> to view my statement as a direct criticism of you.

you cited my statement as an example of an approach that many folks have
taken and then immediately claimed that we shrug and give up rather than
looking for improvements.  it doesn't really matter whether you meant this
as direct criticism of me or merely criticism of that statement.  the inference
you made is ridiculous and unsupportable.

> difficulty with 'tracking'.

heaven forbid someone should actually analyze a problem before proposing
solutions for it.

Keith

---

From: David Kessens <[email-masked]>
Date: Wed, 18 May 2005 17:05:03 -0700

Dave,

> charters.  So the IESG does, in fact, have the ability to control the later

Are you suggesting that we start disapproving new working groups
since they cause work for the IESG ?

We can help to fine tune charters. We can engage the community to
scope the work better. However, we are explicitly not chartered to
review charters for the amount of work that they cause for the IESG.

David Kessens
---

---

From: Dave Crocker <[email-masked]>
Date: Wed, 18 May 2005 17:22:34 -0700

David,

>  since they cause work for the IESG ?

That does sound pretty silly, doesn't it?  

But, then, it sounds pretty silly to have a pattern of filling up with working 
groups when we cannot provide adequate support to them.  (Note, for example, 
that having too many working groups means we have too many parallel sessions 
and cannot get the breadth and depth of participation we need.)

As I said, we do not tolerate that sort of continued congestion in an Internet 
service, so why do we fail to attend to the problem in the IETF?

>  review charters for the amount of work that they cause for the IESG.

Working groups are expensive.  Very expensive.

Not just in iesg time, of course, but in quite a variety of resources 
consumed.  Having lots of working groups seems to mean that a) we give very 
poor support to them, and b) we hassle them when they are done.

If the IESG is really concerned about quality, then it needs to both faciliate 
and apply pressure at the beginning of the working group and continue these 
during the life of the working group.  

The silliest thing we can do is permit poor charters, floundering working 
groups and then veto their documents after 3-10 years.

So, what sort of support?

1. Much, much better charters.  For example, we do not even try to enforce the 
requirements specified in the Working Group Guidelines RFC.

2. Much better oversight of new working group chairs, to ensure that the 
group gets traction on its effort.

3. Insistence on review of major decisions along the way.  

4. Reorganization of IETF management to handle the current load and 
diversity of participation better, with a particular eye on finding ways 
to prevent single-person vetos.

  d/
  ---
  Dave Crocker

---

From: Margaret Wasserman <[email-masked]>
Date: Wed, 18 May 2005 20:50:30 -0400

>since they cause work for the IESG ?

Yes!  In fact, if I closed down all of those pesky INT area WGs, I 
would definitely be able to process those individual submissions on 
time!  :-)

Seriously, I am operating under the assumption that timely review and 
processing of WG charters and WG documents is more important to the 
community than timely processing of RFC Editor individual 
submissions, and the fact that I do not always manage to process 
individual submissions on time reflects that prioritization.

Are there really a significant number of people who think it should 
be otherwise?

Margaret

---

From: David Kessens <[email-masked]>
Date: Wed, 18 May 2005 18:22:08 -0700

Dave,

> That does sound pretty silly, doesn't it?

Yes, it does.

> and cannot get the breadth and depth of participation we need.)

The issues that you bring up are very real. However, I miss a proposal
for a solution.

As I said in my previous mail, I really don't have a mandate to tell
the community that a working group is not going to happen because it
causes work for me as an AD. In addition, making the charter better
does not in any way imply that the working group will actually cause
less work for the IESG.

> during the life of the working group.

You seem to assume that the IESG doesn't do this already. What you
describe above is exactly what the IESG does. 

Do you want us to apply more pressure to working groups that don't
achieve milestones ? What kind of pressure do you want us to apply ?
Quite frankly, I don't think that many working group chairs and the
community would tolerate much more pressure than what we already apply.

David Kessens
---

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 19 May 2005 13:41:44 +0200

Bill,

> may get turned into a full review.

I've been involved in similar mechanisms.

To get some facts on the table, in the last IESG telechat,
14 documents were on the agenda. Here's how the time was spent:

1 minute each: 5 documents
2 min. : 3 documents
3 min. : 2 documents
4 min. : 1 documuent
5 min. : 2 documents
7 min. : 1 document.

for a total of 38 minutes. The IESG typically spends a lot more time
on chartering discussions and complicated management issues than
on document review. Our practice is to delegate the resolution
of discusses to the document shepherd.
> fast-tracks, and also limit the number of full reviews per meeting.  

That is a serialization bottleneck right there.

> which in IESG terms might wind up as a DISCUSS.

Only if the protagonists happen to be available, which is low probability
in the IETF case. And it only works if the DISCUSS is really a request
for clarification; those very often get resolved before the telechat.
If it is something that requires a WG consensus, there is no way it can
be done during the telechat.

Bottom line, I dodn't think the solution lies here.

    Brian

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 19 May 2005 13:57:09 +0200

Ned,

> that are _currently_ creating delays.

There is a long tail in the delay distribution that worries me. As an
indicator of this tail, there are 34 open DISCUSSes created by
former ADs. (er, 3 created by you...).

You make an important point - a common error in organizational change
is to measure its effect too soon, and therefore conclude that it
hasn't worked. Today's long tail is probably a result of the way the
process was operating sometime in 2004 or earlier.

But I get a message that the IESG needs to analyze this long tail some more.

    Brian

---

From: Keith Moore <[email-masked]>
Date: Thu, 19 May 2005 08:32:18 -0400

On May 18, 2005, at 8:50 PM, Margaret Wasserman wrote:

> Seriously, I am operating under the assumption that timely review  
> and processing of WG charters and WG documents is more important to  
> the community than timely processing of RFC Editor individual  

Individual submissions are an important part of our process, and in  
many cases it is important for IESG to review them.  I think it is  
necessary for IESG to manage its WG workload in such a way that it  
has some amount of time left over to process individual submissions.   
In other words, a discipline that gives strict priority to WG work  
might not be appropriate.

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 19 May 2005 14:50:02 +0200

Margaret Wasserman wrote:
 
> Are there really a significant number of people who think it
> should be otherwise?

Sometimes an apparently "individual submission" might represent
the views of a "community" which just prefers to be no IETF WG
for its own reasons (e.g. because a WG was closed).  Bye, Frank
-- 
<http://purl.net/xyzzy/home/test/spf-council.htm>

---

From: Thomas Narten <[email-masked]>
Date: Thu, 19 May 2005 09:20:03 -0400

Hi Dave.

> groups achieve these improvements.

The above is spot on. I doubt anyone would disagree.

But let's get to the bottom line and suggest something
constructive. Name 3 things we (the IETF/IESG/indviduals/WGs/etc.)
should and can be doing to address the above?

Thomas

---

From: Thomas Narten <[email-masked]>
Date: Thu, 19 May 2005 11:18:03 -0400

> 1. Much, much better charters.  For example, we do not even try to
> enforce the requirements specified in the Working Group Guidelines
> RFC.

> 2. Much better oversight of new working group chairs, to ensure that
> the group gets traction on its effort.

> 3. Insistence on review of major decisions along the way.

This is all spot on.

Where  I think we are not actually doing enough of a good job is in
steps 1-2). But, I'd go a bit further.

When I think of successful WGs, they have a number of things going for
them, that go beyond the charter. They include:

1) Having a shared view/understanding of the problem and the general
   outline of a solution. If you don't have this, the WG is saddled
   from the start with a fundamental lack of consensus on a solution,
   becuase they can't ever agree on whether a proposed solution
   satisfies _their_ idea of what the problem is.

2) Building/having a "team", that serves as the critical mass and
   "core competency" for the WG. This teams needs to have at least the
   following set of skills (collectively):

   a) project manager (i.e., puts together a pragamtic schedule --
      something much more detailed than project milestones with
      deadlines and nags folk to deliver, etc.)
      
   b) good editors, who can translate what the WG is saying into text
      that finds a good balance between readabilty and addressing the
      concerns of the WG.
      
   c) someone who follows the mailing list in real time and "manages"
      discussion by closing off ratholes, summarizing on a regular
      basis what the consensus seems to be, etc.
      
   d) someone who is familiar enough with basic internet architecture
      (and what other WGs are doing) to prevent the WG from making
      decisions that are unlikely to be well-received by folk outside
      the WG (and by implication the IESG). Also, a person that knows
      what type of outside expertise needs to review the WG's work in
      order for it to get the necessary broader cross-area approval. For
      example, its a tad depressing  how many times I've actually
      _fought_ with WG chairs about the need for them to get a review
      from, e.g., a  radius or DNS expert. Well, if you don't get that
      review early,  you can be darn sure the IESG will demand it
      later  ... with obvious implications.
      
   e) quality reviewers
   
   f) A real push from a "customer" of the work to ensure that the
      problem being addressed is a real problem and not one that is a
      research problem that no vendor/user really cares about. It
      really helps to have a clear customer saying "no, that doesn't
      solve my problem" and "please decide issue X, it doesn't matter
      what way the WG goes, but we do need the document to get
      finished RSN"

   I would assert that when one considers WGs that are successful, the
   above pieces are usually all there (even if by accident), and where
   WGs are off track, they are missing out on on one or more of the
   above in rather big way.

Thus, a good charter is _necessary_ (usually), but by no means
sufficient by any means. And where we have the biggest problems in
practice is where the WG doesn't understand the above, but is willing
to go along with any charter the IESG allows in order for it to get
started.
     
Thomas

---

From: Dave Crocker <[email-masked]>
Date: Thu, 19 May 2005 08:30:59 -0700

Thomas,

>    outline of a solution. ...
>    "core competency" ...
>    a) project manager ...
>    b) good editors, ...
>       discussion by closing off ratholes...
>       the WG...
>    f) A real push from a "customer" of the work...

I like your list.  It also happens to be in line with the contents of (I think 
Section 2) of the Working Group Guidelines RFC, although there are bits that 
go beyond it nicely.

Although they often are essential, reviewers are not required 100% of the 
time. Working groups often have sufficient expertise and focus, as well as 
other forms of feedback, to get away without formal review.  However many 
working groups suffer from lack of such sanity checking early in their 
efforts.

The benefit of having an architecture expert is both essential and dangerous.  
Essential for the reasons you cite, as well as to improve the likelihood of 
developing a solution that will work long-term and in large-scale.  Dangerous 
because it has the potential to insert a block -- the reviewer -- who is too 
abstracted from the actual demands of the topic, thereby missing the adequacy 
of a proffered solution.

Item c is probably the most seriously missing component from problematic 
working groups.  Given the diversity of current participation, list 
discussions DO need to be managed, in order to assure some assemblance of 
focus and (forward) progress.

>  sufficient by any means. 

right.

  d/
  ---
  Dave Crocker

---

From: Keith Moore <[email-masked]>
Date: Thu, 19 May 2005 13:01:38 -0400

> should and can be doing to address the above?

1. develop guidelines for how to do good protocol engineering for
the Internet environment
2. adapt our working group process to follow those guidelines, 
to include external review at appropriate points (especially at
the point a WG decides to commit to working on a particular
proposal), and introduce a system of tracking and reporting 
changes (along with their justification) to a proposal once a 
working group and IESG have accepted it
3. rethink the division of labor between IESG, working group chairs,
and the RFC Editor.

---

From: "Tom Petch" <[email-masked]>
Date: Thu, 19 May 2005 20:05:57 +0200

Only one. Pay more attention to the choice of Working Group chair, in particular
their non-technical, non-engineering capabilities.

When everything goes well, engineering excellence is all we need.  When things
go wrong- a WG gets stuck, goes round in circles, produces unacceptable drafts
and such like - it is non-technical skills that are needed to resolve the
situation; sometimes they are present in good quality, sometimes otherwise.

Tom Petch

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Thu, 19 May 2005 18:08:48 -0400

On Thursday, May 19, 2005 08:32:18 AM -0400 Keith Moore <[email-masked]> 
wrote:

> Individual submissions are an important part of our process, and in  many
> cases it is important for IESG to review them.  I think it is  necessary
> for IESG to manage its WG workload in such a way that it  has some amount
> of time left over to process individual submissions.   In other words, a
> discipline that gives strict priority to WG work  might not be
> appropriate.

OK; before, I thought there might be some confusion; now I'm sure of it.

I'm pretty sure Margaret's point is that she believes the community 
considers processing of things that are part of the IETF process to be more 
important than what RFC3978 calls "RFC Editor Contributions"; that is, 
documents which are submitted directly to the RFC Editor, outside the IETF 
process.

What Keith is trying to say, I think, is that it is an important part of 
our process that it be possible to submit documents to the IESG which do 
not originate in a working group, and that the IESG needs to manage its 
workload to be able to handle such documents.

Unfortunately, they're both using the term 'individual submissions' to mean 
two very different things.  The same is true of a lot of other messages in 
this thread; I'm only picking on Keith and Margaret because their messages 
make a good example.

Can we stop using the unqualified term "individual submissions", and 
substitute something more precise, like "RFC Editor Contribution" or 
"non-working-group IETF document" ?

-- Jeff

---

From: Eliot Lear <[email-masked]>
Date: Mon, 23 May 2005 22:06:25 +0200

> Working groups are expensive.  Very expensive.

To me this discussion shows that documents are as expensive as working 
groups, and maybe more so.  Unless the document is relatively straight 
forward, it's easier for someone doing an individual draft to be 
funneled to a working group so that there's a first round of review.

In fact, a working group that doesn't produce anything seems pretty darn 
cheap, especially if it doesn't meet.

Eliot