Subject: draft status links on the wg pages?

From: Michael Thomas <[email-masked]>
Date: Thu, 14 Dec 2006 14:22:24 -0800

First I have to say that I really like the draft tracker, and kudos for
those responsible for making it happen. In fact, I like it so much that
it seems to me that it would be nice to have a link directly to it from
the working group page with its list of drafts. Ie:

draft-foowg-bardraft-00.txt (link to draft) sts (link to the status tracker)
Even nicer might be to display the actual tracker state as, say, the button
itself.

       Mike

_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf

---

From: "Bill Fenner" <[email-masked]>
Date: Sun, 17 Dec 2006 16:33:11 -0800

Mike,

  Check out http://tools.ietf.org/wg/<wgname> and see if that gives
you the view you're looking for.

  Bill

---

From: Dave Crocker <[email-masked]>
Date: Sun, 17 Dec 2006 18:09:36 -0800

> you the view you're looking for.

Bill, thanks.  I suspect that's what Michael has in mind, except for one minor 
enhancement:  Put a link to that page on the working group's main IETF page.

A working group's main page really is its home page and, therefore, ought to 
bring together references to all relevant information.

Given that the Tools folk have created yet-another useful mechanism, making each 
working group's page have a link to its related status information now becomes a 
trivial effort, with substantial benefit.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: Jari Arkko <[email-masked]>
Date: Mon, 18 Dec 2006 14:23:31 +0200

Dave,
> Given that the Tools folk have created yet-another useful mechanism,
> making each working group's page have a link to its related status
> information now becomes a trivial effort, with substantial benefit.
This would be useful. (FWIW, I've already moved to using
only the tools page -- you get to everywhere where you
need from the tools page, including the official charter
page.)

--Jari

---

From: Alexey Melnikov <[email-masked]>
Date: Mon, 18 Dec 2006 12:55:40 +0000

> information now becomes a trivial effort, with substantial benefit.

I agree about the benefits.
One thing I like about a tools wg page is that it clearly separates 
document which are already approved from the documents which are still 
being worked on.

---

From: Frank Ellermann <[email-masked]>
Date: Mon, 18 Dec 2006 17:20:08 +0100

Dave Crocker wrote:
 
> minor enhancement:  Put a link to that page on the working group's
> main IETF page.

There's a link to the Charter, isn't that the old main page ?

> A working group's main page really is its home page and, therefore,
> ought to bring together references to all relevant information.

The WG pages on the tools' site offer links to more relevant
information, therefore I consider this as their new main page.

Frank

---

From: Michael Thomas <[email-masked]>
Date: Mon, 18 Dec 2006 08:28:48 -0800

Dave Crocker wrote:
> Bill, thanks.  I suspect that's what Michael has in mind, except for 
> one minor enhancement:  Put a link to that page on the working group's 
> main IETF page.
>
Right, this is really cool -- although a link from the ietf wg page would be
nice, the tools version almost looks like wg page v2.0 with a few additions
(like the charter, milestones, etc.)

cheers, Mike

---

From: Henrik Levkowetz <[email-masked]>
Date: Mon, 18 Dec 2006 17:32:07 +0100

Hi Dave, Frank,

>> minor enhancement:  Put a link to that page on the working group's

Good idea.  And actually already implemented some time ago.  Looking
at for instance the mip4 charter page, at 

    http://www.ietf.org/html.charters/mip4-charter.html

there is a line right above the listing of the Chairs which provides
an URL and link:

    "Additional information is available at tools.ietf.org/wg/mip4"

I think that's what you were looking for?

> There's a link to the Charter, isn't that the old main page ?

This refers to the link in the opposite direction, from tools.ietf.org
to www.ietf.org, which is also good to have, but not what Dave was
asking for, I think...

	Henrik

---

From: Dave Crocker <[email-masked]>
Date: Mon, 18 Dec 2006 09:17:02 -0800

Michael Thomas wrote:
>> Given that the Tools folk have created yet-another useful mechanism, making
>> each working group's page have a link to its related status information now
>> becomes a trivial effort, with substantial benefit.
>> 

I hadn't thought of merging the existing wg page with the tools one.  I also 
never noticed that a pointer to the tools page is already on the official wg 
page.  It is right above "Chairs", slipping past the eye, after the Last 
Modified date. I also had entirely missed just how rich the tools page is.

The Status page really has two innovations.  One is the status information.

The other is the page format -- heading anbd side-column.  Most IETF pages are 
organized by information "topics".  The user gets to figure out what topic to 
query.  And, of course, this is often obvious.  Other times, not so much.

Although this one's heading uses topic terminology, it actually is organized for 
working group participant *activity*.  True activity-based design tends towards 
use of verbs rather than nouns, but given the list of links at the page heading, 
I think it covers all the things a participant would want to get at. (The fact 
that the "charter" link is in the middle of the list, rather than the beginning, 
is a pretty strong indicator that this is more oriented towards participants 
than towards folks trying to find out about the wg.  Of course, it serves the 
latter folks just fine.)

Activity-based GUI design has become a point of focus in the Usability field and 
seems to be far more productive that the classic style, which leaves it up to 
the user to figure out how to combine info queries together, to get work done.

      This page header should become the standard for all pages
      relating to a working group.

(Whether to also make the left-hand column universal for wg-related pages could 
be debated a bit. My first reaction was not to keep it, but the ability to jump 
between wgs does seem pretty appealing, and the display real estate cost of the 
feature isn't onerous.)

So my suggestions are:

1. Move these Status pages out of the "tools" development area and make them an 
official part of a working group's "official" pages.

2. Make the header standard for *all* working-group specific pages.

3. Remove the documents list from the WG charter page, since it is redundant 
with the Status page and less complete.

d/

-- 

   Dave Crocker

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 19 Dec 2006 10:00:04 +0100

Dave Crocker wrote:
...
> So my suggestions are:
> 
> 1. Move these Status pages out of the "tools" development area and make 
> them an official part of a working group's "official" pages.

That isn't simply a matter of wishing it to be so. It actually
involves work (to make the tools that generate these pages ready
to hand over) and expense (since this would be new work for
the Secretariat).

> 3. Remove the documents list from the WG charter page, since it is 
> redundant with the Status page and less complete.

However, it contains different information - it "knows" which
drafts are official WG drafts, and it includes "obsoleted by/
updated by" information for RFCs.

There's also a whole other topic: what gets archived when
a WG is closed.

     Brian

---

From: Dave Crocker <[email-masked]>
Date: Tue, 19 Dec 2006 06:51:04 -0800

Brian E Carpenter wrote:
> That isn't simply a matter of wishing it to be so. It actually involves work 
> (to make the tools that generate these pages ready to hand over) and expense 
> (since this would be new work for the Secretariat).

And this is distinguished from any other recommendation for change made in the
IETF how?

It is certain to be true that moving any of the Tools work into ongoing
production use is going to require Secretariat expense. Are you arguing against
ever doing that?

(In other words, Brian, what is your point in stating this, here?
It almost would seem to be an argument against ever making any changes
involving the Secretariat, and I suspect you don't mean that.)

By way of anticipating a plausible answer:

     If you are encouraging some form of cost/benefit consideration, beyond what
has taken place so far, then I think that is an excellent idea for most work
involving the IETF.

     With respect to the benefit of the specific suggestions I am making: My
reference to activity-based GUI design was intended to provide a bit of benefit
justification.  To extend that a bit more: The new page header -- and the left
column has grown on me enough so that I'll include that, too -- is an enormous
improvement for participant navigation.  The Status-specific content has had
plenty of postings noting its benefit to make the justification of its movement
to production use equally clear.

     But, Brian, the concern for costs ought to extend much farther, such as to
the kinds of issues raised by an IESG Discuss so that the AD provides an
explanation of the benefit that justifies the considerable cost in delay and wg
effort they are imposing...

> However, it contains different information - it "knows" which drafts are 
> official WG drafts, and it includes "obsoleted by/ updated by" information 
> for RFCs.

Good point.  Then I suggest that the Tools folks upgrade the Status page to
include those features.

> There's also a whole other topic: what gets archived when a WG is closed.

My suggestion:  Everything.  The benefits of having an extensive archival record 
of IETF efforts is pretty clear, and I have heard that storage is pretty 
inexpensive, these days.

d/
-- 

   Dave Crocker

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 19 Dec 2006 16:27:16 +0100

Dave Crocker wrote:
> 
> 
> Brian E Carpenter wrote:
> 
>>> 1. Move these Status pages out of the "tools" development area and 
>>> make them an official part of a working group's "official" pages.
>>
>>
>> involves work (to make the tools that generate these pages ready to 
>> hand over) and expense (since this would be new work for the 
>> Secretariat).
> 
> 

Not at all. In fact, the IAD wants to do that; but we have to choose
which tools are first in the queue.

> to production use equally clear.

There's an element of taste and personal working habits involved,
but basically I agree with you, as long as we keep our pages clean
and simple, and meet accessibility requirements.

> effort they are imposing...

Hence the DISCUSS criteria document, which is a work in progress (and
likely to become an ION).
>> However, it contains different information - it "knows" which drafts 
>> are official WG drafts, and it includes "obsoleted by/ updated by" 
>> information for RFCs.
> 
> 

Makes sense to me.
> My suggestion:  Everything.  The benefits of having an extensive 
> archival record of IETF efforts is pretty clear, and I have heard that 
> storage is pretty inexpensive, these days.

True, and for that to be useful, you need a version managament system for
the archived material.

    Brian

---

From: Dave Crocker <[email-masked]>
Date: Tue, 19 Dec 2006 11:20:10 -0800

> likely to become an ION).

I have been meaning to give you a vigorous thank you for the effort to produce 
that document.

I happened to choose the particular example of Discuss simply because it is of 
immediate concern to me, but my point was intended, of course, to be more general.

That's why your raising the 'cost' issue for the current discussion came as such 
a confusing surprise.

d/
-- 

   Dave Crocker

---

From: Henrik Levkowetz <[email-masked]>
Date: Tue, 19 Dec 2006 21:10:04 +0100

Hi Dave, Brian,

Regarding one point regarding the WG status pages on the tools servers:

on 2006-12-19 15:51 Dave Crocker said the following:

[...]
> Good point.  Then I suggest that the Tools folks upgrade the Status page to

Right.  The tools pages also knows about which drafts are official WG drafts.
Non-Wg drafts are listed in a separate section at the bottom. It is however
not plainly stated that one group is WG drafts and the other isn't; I'll fix
that.

Regarding the obsoleted/updated-by annotation for the RFCs, that's a very
good point.  I'll add that information.

	Henrik

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 20 Dec 2006 09:48:42 +0100

> I happened to choose the particular example of Discuss simply because it 
> is of immediate concern to me, but my point was intended, of course, to 
> be more general.
> 
> That's why your raising the 'cost' issue for the current discussion came 
> as such a confusing surprise.

Sorry to confuse: let me be clear that migrating selected tools from
the volunteer community to the secretariat is one of the IAD's objectives,
and you're perfectly correct that a cost/benefit analysis is the right
way to make the selection.

    Brian

---

From: Dave Crocker <[email-masked]>
Date: Wed, 20 Dec 2006 07:19:10 -0800

Brian E Carpenter wrote:
>> I happened to choose the particular example of Discuss simply because 
>> it is of immediate concern to me, but my point was intended, of 
>> course, to be more general.
>>
>> That's why your raising the 'cost' issue for the current discussion 
>> came as such a confusing surprise.
> 

ack.

However, let me again raise the question of whether the migration should be in 
that direction.

I have no idea whether the Tools folks have any interest in on-going maintenance 
of more of the IETF web pages, but I think it is reasonable to ask, particularly 
for content pertaining to the technical work of the IETF.

Since we rely on volunteers for a number of IETF activities, beyond writing 
specs, it is at least worth exploring this additional avenue of saving money 
(and maybe even getting better operations, although I note this not as a 
criticism of the Secretariat, past or present, but of the normal effects of 
being overworked.)

d/
-- 

   Dave Crocker

---

From: Dave Crocker <[email-masked]>
Date: Wed, 20 Dec 2006 07:19:10 -0800

ack.

d/
-- 

   Dave Crocker

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Wed, 20 Dec 2006 16:37:09 -0500

On Wednesday, December 20, 2006 07:19:10 AM -0800 Dave Crocker 
<[email-masked]> wrote:

> Since we rely on volunteers for a number of IETF activities, beyond
> writing specs, it is at least worth exploring this additional avenue of
> saving money (and maybe even getting better operations, although I note
> this not as a criticism of the Secretariat, past or present, but of the
> normal effects of being overworked.)

The tools team has produced a lot of neat stuff.  However, the more they 
maintain on a long-term basis, the less time they will have to produce new 
neat stuff.  I'd rather see the more-stable things maintained by people who 
are being paid to do so, both to reduce the chances of losing them if one 
or more volunteers become unwilling or unable to do so, and to free the 
time of those volunteers for new development that is both useful to the 
IETF and probably more interesting to work on.

That said, I think we should all keep in mind that the tools team are 
volunteers, and the rest of us do _not_ get to decide what they spend their 
time on.

-- Jeff

---

From: Dave Crocker <[email-masked]>
Date: Tue, 26 Dec 2006 09:12:13 -0800

Brian E Carpenter wrote:
>> explanation of the benefit that justifies the considerable cost in delay
>> and wg effort they are imposing...
> 
> Hence the DISCUSS criteria document, which is a work in progress (and likely
> to become an ION).

Brian,

Which document are you referring to?

The one that I thought you meant has an I-D Tracker status of "dead" and is no 
longer an I-D:

   <https://datatracker.ietf.org/public/idindex.cgi?command=id_detail&id=13314>

d/
-- 

   Dave Crocker

---

From: Fred Baker <[email-masked]>
Date: Tue, 26 Dec 2006 12:24:40 -0800

A look on the web came up with:
	http://www.ietf.org/u/ietfchair/discuss-criteria.html
and	http://tools.ietf.org/html/draft-iesg-discuss-criteria

They have the same date.

On Dec 26, 2006, at 9:12 AM, Dave Crocker wrote:

> Brian E Carpenter wrote:
>>> But, Brian, the concern for costs ought to extend much farther,  
>>> such as to the kinds of issues raised by an IESG Discuss so that  
>>> the AD provides an explanation of the benefit that justifies the  
>>> considerable cost in delay
> The one that I thought you meant has an I-D Tracker status of  
> "dead" and is no longer an I-D:
>

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 28 Dec 2006 10:29:09 +0100

Yes, http://www.ietf.org/u/ietfchair/discuss-criteria.html
is the "random web page" version that I anticipate will become
an ION. This is a very good illustration of
a) something that needs to be accessible without an arbitrary
6 month expiry
b) needs to evolve
c) needs some minimal level of approval
d) is only relevant to the IETF's internal operations

and is therefore suitable to be an ION and not an RFC.

    Brian

On 2006-12-26 21:24, Fred Baker wrote:
>>>> But, Brian, the concern for costs ought to extend much farther, such 
>>>> as to the kinds of issues raised by an IESG Discuss so that the AD 
>>>> provides an explanation of the benefit that justifies the 
>>> likely
>> The one that I thought you meant has an I-D Tracker status of "dead" 
>> and is no longer an I-D:
>>
>>

---

From: Dave Crocker <[email-masked]>
Date: Thu, 28 Dec 2006 17:48:31 +0000

Brian,

Your (a) criterion applies to to anything that is formally under development. 
The real requirement is to produce a stable version and get it approved and 
published, rather than have it languish in development. That is, after all, why 
I-Ds were given a time-out.  What is remaining, in order to get the draft 
approved, to apply to Discuss votes?

Your (b) criterion implies some sort of expected high rate of change, if there 
is a legitimate need for a streamlined publication mechanism. Yet the metabolic 
rate for considering and stabilizing and approving the current draft is quite a 
long way from satisfying that requirement. For that matter, do we really want 
internal procedures to be changing that rapidly?

Your (c) criterion implies that a BCP-like mechanism is not acceptable.  Here, 
the difference between BCP -- with its IETF-wide appoval -- versus ION -- with 
is IESG approval -- is at least an interesting distinction.  On the other hand, 
this particular draft is, if anything, likely to demonstrate why IETF-wide 
approval is a particularly good idea, since its goal is to hold IESG members 
accountable to more stringent criteria than are currently in force for blocking 
approval of working group output. In addition the particular criteria listed in 
the -discuss draft contain a number of items that reasonably could be viewed as 
too stringent, too lenient, too vague, too low-level, or the like.  The public 
process of gaining community consensus seems important for the credibility of 
the IETF process.

That leaves your (d) criterion, which was where the suggestion for a new RFC 
label came from, rather than the creation of an entirely new publication 
mechanism.

But the most important issue is probably getting the document reviewed, 
approved, and applied, no matter what publication mechanism is.

d/
-- 

   Dave Crocker

---

From: Fred Baker <[email-masked]>
Date: Thu, 28 Dec 2006 12:19:15 -0800

I agree with your points here, and I think it points up something I  
have been feeling about this and not putting words to.

I really don't think the issue is whether it is a web page, an I-D,  
an RFC, or something else. The point is that we need a document  
process that will allow people to put things in and have them come  
out the other end in a predictable time period, or remain in the  
intermediate state if they are being actively developed. Inventing a  
new document format feels to me like putting a bandaid on the  
problem, rather than dealing with the root cause, the latter being  
that we are becoming so process-bound that we don't actually get  
things done.

We can discuss separately whether this ION thing is worthwhile.  
Personally, I would prefer to get the hard discussions and decisions  
done, which are those that get us out of this legalistic process  
nonsense.

On Dec 28, 2006, at 9:48 AM, Dave Crocker wrote:
> Your (a) criterion applies to to anything that is formally under  
> development. The real requirement is to produce a stable version  
> and get it approved and published, rather than have it languish in  
> development. That is, after all, why I-Ds were given a time-out.   
> What is remaining, in order to get the draft approved, to apply to  
> Discuss votes?
>
> Your (b) criterion implies some sort of expected high rate of  
> change, if there is a legitimate need for a streamlined publication  
> mechanism. Yet the metabolic rate for considering and stabilizing  
> and approving the current draft is quite a long way from satisfying  
> that requirement. For that matter, do we really want internal  
> procedures to be changing that rapidly?
>
> Your (c) criterion implies that a BCP-like mechanism is not  
> acceptable.  Here, the difference between BCP -- with its IETF-wide  
> appoval -- versus ION -- with is IESG approval -- is at least an  
> interesting distinction.  On the other hand, this particular draft  
> is, if anything, likely to demonstrate why IETF-wide approval is a  
> particularly good idea, since its goal is to hold IESG members  
> accountable to more stringent criteria than are currently in force  
> for blocking approval of working group output. In addition the  
> particular criteria listed in the -discuss draft contain a number  
> of items that reasonably could be viewed as too stringent, too  
> lenient, too vague, too low-level, or the like.  The public process  
> of gaining community consensus seems important for the credibility  
> of the IETF process.
>
> That leaves your (d) criterion, which was where the suggestion for  
> a new RFC label came from, rather than the creation of an entirely  
> new publication mechanism.
>
> But the most important issue is probably getting the document  
> reviewed, approved, and applied, no matter what publication  
> mechanism is.
>
> d/
> -- 
>
>   Dave Crocker

---

From: "Spencer Dawkins" <[email-masked]>
Date: Fri, 29 Dec 2006 06:39:28 -0600

> Your (c) criterion implies that a BCP-like mechanism is not acceptable. 
> Here, the difference between BCP -- with its IETF-wide appoval -- versus 
> ION -- with is IESG approval -- is at least an interesting distinction. 
> On the other hand, this particular draft is, if anything, likely to 
> demonstrate why IETF-wide approval is a particularly good idea, since its 
> goal is to hold IESG members accountable to more stringent criteria than 
> are currently in force for blocking approval of working group output. In 
> addition the particular criteria listed in the -discuss draft contain a 
> number of items that reasonably could be viewed as too stringent, too 
> lenient, too vague, too low-level, or the like.  The public process of 
> gaining community consensus seems important for the credibility of the 
> IETF process.

Just a side note here - I can't remember which IETF it was, but this draft 
was presented to the Working Group Chairs continuing education lunch so long 
ago that the presenters were Allison Mankin, Jon Peterson, and Margaret 
Wasserman, so the possibilities are IETF 65 in Dallas or earlier, and at 
that time, the IESG was using the criteria in some version of the draft as 
the definition of an allowable DISCUSS in balloting.

As one of the IESG narrative scribes, I hear ADs referring to these criteria 
fairly often on telechats (probably an average of every six weeks or so, but 
the timing is only my impression).

So, some version of this critera has been "currently in force" for a while 
now.

Dave is probably correct that the specific criteria are of broader interest 
than just ADs, WG chairs, editors, and process wonks, and might become even 
more perfect with broader review, but that's another issue.

And, since the criteria are public, I'm sure the IESG would be interested in 
feedback on the criteria, especially now that WGs and editors have more than 
a year of experience interacting with IESG under these rules of engagement. 
That URL, again, is http://www.ietf.org/u/ietfchair/discuss-criteria.html.

Thanks,

Spencer

---

From: Dave Crocker <[email-masked]>
Date: Fri, 29 Dec 2006 16:44:45 +0000

> feedback on the criteria, especially now that WGs and

Meta-point:

      Something quite basic that is missing from the draft on
      Discuss Criteria is a requirement that any Discuss not only
      explain its precise normative basis, but that it give a clear
      statement of what actions must be taken to clear the Discuss.

 From the draft:

> * The specification is impossible to implement due to technical or clarity 
> issues.

When this assertion is made, is it sufficient to cite existing implementations
based on the current version of the specification?  Is the AD at least required 
to explain the assertion in detail?

> * The protocol has technical flaws that will prevent it from working 
> properly, or the description is unclear in such a way that the reader cannot 
> understand it without ambiguity.

Will demonstrations of interoperability be sufficient to counter this claim?

> * It is unlikely that multiple implementations of the specification would 
> interoperate, usually due to vagueness or incomplete specification.

Will demonstrations of interoperability be sufficient to counter this claim?

> * Widespread deployment would be damaging to the Internet or an enterprise 
> network for reasons of congestion control, scalability, or the like.

Beyond the simple assertion of this claim, what requirements are there for the 
AD to substantiate it?

> * The specification would create serious security holes, or the described 
> protocol has self-defeating security vulnerabilities (e.g. a protocol that 
> cannot fulfill its purpose without security properties it does not provide).

> * It would present serious operational issues in widespread deployment, by 
> for example neglecting network management or configuration entirely.

There is often a failure to distinguish between new and peculiar problems 
created by a particular specification, versus general problems that already exist.

A classic example of this is citing basic DNS problems, for specifications that 
are merely consumers of the DNS and, hence, are not creating any new problems.

> * Failure to conform with IAB architecture (e.g., RFC1958 (Carpenter, B., 
> “Architectural Principles of the Internet,” June 1996.) [2], or UNSAF 
> (Daigle, L., “IAB Considerations for UNilateral Self-Address Fixing (UNSAF) 
> Across Network Address Translation,” November 2002.) [3]) in the absence of 
> any satisfactory text explaining this architectural decision.

This is an interesting item.

1.  At what point in time did publication of IAB preferences take on the force 
of law?

2.  Note that the list gives some examples, but does not supply a complete list. 
How are working groups to know which IAB documents have been declared normative 
standards for all IETF work and which have not?

3.  Why is the IAB allowed to create normative standards that cover all IETF 
work, without requiring that they first gain IETF-wide approval?

> * The specification was not properly vetted against the I-D Checklist. 
> Symptoms include broken ABNF or XML, missing Security Considerations, and so 
> on.

> * The draft omits a normative reference necessary for its implementation, or 
> cites such a reference merely informatively rather than normatively.

> * The document does not meet criteria for advancement in its designated 
> standards track, for example because it is a document going to Full Standard 
> that contains 'down references' to RFCs at a lower position in the standards 
> track, or a Standards Track document that contains only informational 
> guidance.

> * IETF process related to document advancement was not carried out; e.g., 
> there are unresolved and substantive Last Call comments which the document 
> does not address, the document is outside the scope of the charter of the WG 
> which requested its publication, and so on.

> * The IETF as a whole does not have consensus on the technical approach or 
> document. There are cases where individual working groups or areas have 
> forged rough consensus around a technical approach which does not garner IETF
>  consensus. An AD may DISCUSS a document where she or he believes this to be 
> the case. While the Area Director should describe the technical area where 
> consensus is flawed, the focus of the DISCUSS and its resolution should be on
>  how to forge a cross-IETF consensus.

This is perhaps the scariest of the criteria.  It says that a knowledgeable, 
motivated constituency can spend months on solving a problem that it needs to 
have solved, and then others who have not participated in the work can come 
along and sabotage it.

Yes, I know that is not the intend of this criterion, but it is what the effect 
will be -- and in some cases already is.

Once upon a time, the rule in the IETF was that working group consensus was what 
mattered, absent a clear technical basis for saying that a specification "would 
not work".

That is quite different from now saying that after a working group is done we 
somehow must be assured that the specification has acquired an IETF-wide 
politically correct acceptance.

If this last criterion is meant to be taken as it is stated, there is a pretty 
straightforward basis for believing that ever bringing new work to the IETF is a 
very questionable decision.

The criterion presumes that the IETF, as a whole, must agree on the One True 
Solution to a problem.

In the IETF's history, that has been a requirement in some special cases, but 
not others.  The default view has been to let the market decide among choices. 
Requirement for a single choice has been asserted only when there is a strong 
argument that having multiple choices will cause damage.

d/
-- 

   Dave Crocker

---

From: "Joel M. Halpern" <[email-masked]>
Date: Fri, 29 Dec 2006 23:17:43 -0500

Dave asked a series of pointed questions about the discuss criteria document.

First and foremost, I want to point out that this draft, in whatever 
form it is, represents a good thing.  The IESG is making an effort to 
tell the community what it is doing, and to listen to the comments it gets.

As a practical matter, the IESG needs the freedom to change these 
policies if they turn out to have problems.  It would be really bad 
if the policy had a problem, but they could not work around it for 
the months it took to get a new RFC out.

Now, I want to point out a basic a disagreement I have with the 
perspective Dave suggests, below the extract from his comments.

At 11:44 AM 12/29/2006, Dave Crocker wrote:
>>* The specification is impossible to implement due to technical or 
>>clarity issues.
>
>based on the current version of the specification?  Is the AD at 
>least required to explain the assertion in detail?
>
>
>>* The protocol has technical flaws that will prevent it from 
>>working properly, or the description is unclear in such a way that 
>>the reader cannot understand it without ambiguity.
>
>>* It is unlikely that multiple implementations of the specification 
>>would interoperate, usually due to vagueness or incomplete specification.
>

I think in all of these cases, the question is the wrong question, 
and the answer is no, an implementation (or event three 
implementations) does not answer the concern.  We have had several 
documents in different areas where the documents were severely 
under-specified.  There were implementations.  They were from folks 
who were involved in the working group discussion, and so knew all 
the unstated assumptions.  They tested against each other until they 
got it working.
That set of implementations did not prove that the spec was 
implementatable, clear, or particularly good.

We ostensibly choose the IESG members for technical skill, and we 
ostensibly ask them to apply technical judgement.
Refusing to pay attention to such judgement is a recipe for more 
under-specified (or broken in other ways) specifications.

Yes, the IESG members objecting should be as specific as they can be.
However, I will note that I have reviewed documents where all I could 
say at the end was "I can't make head or tail of this."  And it 
wasn't some obscure, complicated technical document.  There have also 
been documents where my personal reaction was that the documents 
needed to be re-written from scratch by someone who spoke English 
(First second, or third language, but English.)  They were simply too 
badly written to tell if they were accurate.

Yours,
Joel M. Halpern

---

From: Jari Arkko <[email-masked]>
Date: Sat, 30 Dec 2006 09:27:10 +0200

Hi Dave,

Joel and Sam already sent responses that I agree with,
but I wanted to add a couple of things. First of all, most
of the things that the IESG looks for in its review are
judgment calls. Its our job to think about whether, for
instance, underspecification problems in the document
are so bad that it calls for a Discuss even when we know
that interoperable implementations exist. Such a
situation should be rare, and I think most of us value
implementation experience very highly in all stages
of the IETF process. But I remember at least one
case from this year where "look, my implementation works"
claims from the proponents of a protocol were countered
with "no, its broken" from others, and it turned out that
the latter people were right. In any case, if the things
that IESG looks for weren't judgment calls, I would be
calling the tools team to work on the Discuss tool
and have the IESG do something else...

Secondly, when one of us makes a Discuss, there
is significant pressure to keep it within the Discuss
criteria guidelines, make it actionable, and explain
it in a believable manner with justifications. The
first test is whether the rest of the IESG believes
in it. Brian, for instance, gives a lot of feedback on
the other AD's Discusses, and this results often
in changes or removal of the Discuss. To survive
a laugh test in the telechat a Discuss has to have
some substance. Of course at the end of the day
we may disagree on whether there is a problem,
but my experience is that Discusses that stay
have well explained justifications.

>      statement of what actions must be taken to clear the Discuss.
The Discusses have to be actionable in the sense
that author must know what specific problem there
is, and he needs to be able to fix it in this document.
But I hope you are not saying that the AD has to find
the fix...

Finally, I think we should just ship the discuss criteria
document. We tend to hold on to documents for too
long in the expectation that we might find an
improvement later. But I think the right documentation
format for this type of documents is IONs.

--Jari

Dave Crocker kirjoitti:
>> Dave is probably correct that the specific criteria are of broader
>> interest than just ADs, WG chairs, editors, and process wonks, and
>> might become even more perfect with broader review, but that's
>> another issue.
>>
>> And, since the criteria are public, I'm sure the IESG would be
>> interested in feedback on the criteria, especially now that WGs and
>
>
>
> Meta-point:
>
> based on the current version of the specification?  Is the AD at least
> required to explain the assertion in detail?
>
>
>> properly, or the description is unclear in such a way that the reader
>> cannot understand it without ambiguity.
>
>> * Widespread deployment would be damaging to the Internet or an
>> enterprise network for reasons of congestion control, scalability, or
>> the like.
>
> Beyond the simple assertion of this claim, what requirements are there
> for the AD to substantiate it?
>
>
>> * The specification would create serious security holes, or the
>> described protocol has self-defeating security vulnerabilities (e.g.
>> a protocol that cannot fulfill its purpose without security
>> properties it does not provide).
>
>
>> * It would present serious operational issues in widespread
>> deployment, by for example neglecting network management or
>> configuration entirely.
>
> There is often a failure to distinguish between new and peculiar
> problems created by a particular specification, versus general
> problems that already exist.
>
> A classic example of this is citing basic DNS problems, for
> specifications that are merely consumers of the DNS and, hence, are
> not creating any new problems.
>
>
>> * Failure to conform with IAB architecture (e.g., RFC1958 (Carpenter,
>> B., “Architectural Principles of the Internet,” June 1996.) [2], or
>> UNSAF (Daigle, L., “IAB Considerations for UNilateral Self-Address
>> Fixing (UNSAF) Across Network Address Translation,” November 2002.)
>> [3]) in the absence of any satisfactory text explaining this
>> architectural decision.
>
> 1.  At what point in time did publication of IAB preferences take on
> the force of law?
>
> 2.  Note that the list gives some examples, but does not supply a
> complete list. How are working groups to know which IAB documents have
> been declared normative standards for all IETF work and which have not?
>
> 3.  Why is the IAB allowed to create normative standards that cover
> all IETF work, without requiring that they first gain IETF-wide approval?
>
>
>> * The specification was not properly vetted against the I-D
>> Checklist. Symptoms include broken ABNF or XML, missing Security
>> Considerations, and so on.
>
>> * The draft omits a normative reference necessary for its
>> implementation, or cites such a reference merely informatively rather
>> than normatively.
>
>> * The document does not meet criteria for advancement in its
>> designated standards track, for example because it is a document
>> going to Full Standard that contains 'down references' to RFCs at a
>> lower position in the standards track, or a Standards Track document
>> that contains only informational guidance.
>
>> * IETF process related to document advancement was not carried out;
>> e.g., there are unresolved and substantive Last Call comments which
>> the document does not address, the document is outside the scope of
>> the charter of the WG which requested its publication, and so on.
>
>> * The IETF as a whole does not have consensus on the technical
>> approach or document. There are cases where individual working groups
>> or areas have forged rough consensus around a technical approach
>> which does not garner IETF
>>  consensus. An AD may DISCUSS a document where she or he believes
>> this to be the case. While the Area Director should describe the
>> technical area where consensus is flawed, the focus of the DISCUSS
>> and its resolution should be on
> This is perhaps the scariest of the criteria.  It says that a
> knowledgeable, motivated constituency can spend months on solving a
> problem that it needs to have solved, and then others who have not
> participated in the work can come along and sabotage it.
>
> Yes, I know that is not the intend of this criterion, but it is what
> the effect will be -- and in some cases already is.
>
> Once upon a time, the rule in the IETF was that working group
> consensus was what mattered, absent a clear technical basis for saying
> that a specification "would not work".
>
> That is quite different from now saying that after a working group is
> done we somehow must be assured that the specification has acquired an
> IETF-wide politically correct acceptance.
>
> If this last criterion is meant to be taken as it is stated, there is
> a pretty straightforward basis for believing that ever bringing new
> work to the IETF is a very questionable decision.
>
> The criterion presumes that the IETF, as a whole, must agree on the
> One True Solution to a problem.
>
> In the IETF's history, that has been a requirement in some special
> cases, but not others.  The default view has been to let the market
> decide among choices. Requirement for a single choice has been
> asserted only when there is a strong argument that having multiple
> choices will cause damage.
>
> d/

---

From: John C Klensin <[email-masked]>
Date: Sat, 30 Dec 2006 08:56:36 -0500

Dave, Scott,

At the risk of repeating what a few others have said in 
different form, a few observations.  Please understand that 
these comments come from someone who has been more consistently 
and loudly critical about even a hint of IESG arrogance and 
assertions of their power than either of you and who has 
formally proposed a significant number of ways of dealing with 
those problems --real or imagined-- than, I think, anyone else 
in the community... none of which proposals have gone anywhere.

I believe that, ultimately, the IETF has to pick IESG members 
who can do the job of evaluating documents and consensus about 
it and then let them to do that job.  And we had better pick 
people to do that job who technical judgment and good sense we 
trust.  If we can't do that, then we are in big-time, serious, 
trouble: trouble from which no set of rules or procedures can 
rescue us.   Much as it makes me anxious, I think we ultimately 
need to let an AD raise a Discuss because he or she has a bad 
feeling in his or her gut... and pick people who will use that 
particular reason with considerable care and who will challenge 
each other and work to understand the objection and either 
better document it or remove it as appropriate.

If that discussion is abused in particular cases, I think it 
means that we need more appeals and, if there is a pattern, more 
recalls.   In a long-term tradition of the IETF that we seem to 
be losing, we may also need more specific, focused, public abuse 
(in plenaries and otherwise) from the community, not just from 
regular complainers and microphone-hogs.   What we don't need is 
more rigid rules that either try to anticipate every 
circumstance or that give too strong a presumption to the wisdom 
of a too-homogeneous WG, especially at a time when fewer and 
fewer documents seem to be getting widespread commFrom [email-masked] Sat Dec 30 09:01:20 2006
Date: Sat, 30 Dec 2006 08:56:36 -0500
From: John C Klensin <[email-masked]>
To: [email-masked], [email-masked]
	<[email-masked]> <[email-masked]>
Cc: [email-masked]
Subject: Re: "Discuss" criteria

Dave, Scott,

At the risk of repeating what a few others have said in 
different form, a few observations.  Please understand that 
these comments come from someone who has been more consistently 
and loudly critical about even a hint of IESG arrogance and 
assertions of their power than either of you and who has 
formally proposed a significant number of ways of dealing with 
those problems --real or imagined-- than, I think, anyone else 
in the community... none of which proposals have gone anywhere.

I believe that, ultimately, the IETF has to pick IESG members 
who can do the job of evaluating documents and consensus about 
it and then let them to do that job.  And we had better pick 
people to do that job who technical judgment and good sense we 
trust.  If we can't do that, then we are in big-time, serious, 
trouble: trouble from which no set of rules or procedures can 
rescue us.   Much as it makes me anxious, I think we ultimately 
need to let an AD raise a Discuss because he or she has a bad 
feeling in his or her gut... and pick people who will use that 
particular reason with considerable care and who will challenge 
each other and work to understand the objection and either 
better document it or remove it as appropriate.

If that discussion is abused in particular cases, I think it 
means that we need more appeals and, if there is a pattern, more 
recalls.   In a long-term tradition of the IETF that we seem to 
be losing, we may also need more specific, focused, public abuse 
(in plenaries and otherwise) from the community, not just from 
regular complainers and microphone-hogs.   What we don't need is 
more rigid rules that either try to anticipate every 
circumstance or that give too strong a presumption to the wisdom 
of a too-homogeneous WG, especially at a time when fewer and 
fewer documents seem to be getting widespread community review 
during Last Call.

In that context, I can only applaud this document, not as a set 
of rules that the IESG has to follow, but as one that informs 
the community about the mechanisms the IESG is using. 
Information is good.  And, if the IESG discovers that it needs 
to update that information every time its membership changes (or 
every time they discover something isn't working and make an 
adjustment according), I'd consider that a sign of good health: 
at least it would show that, at least in this area, historical 
rules and behavior patterns are not constraining current 
thinking to the extent that replacing IESG members doesn't bring 
about change.

At the risk of giving a sales pitch, my other proposals have 
been intended to reinforce the model above: I think it is always 
going to be hard, in our community, to find IESG members who are 
good at doing these kinds of technical evaluations and sensitive 
to the issues involved and who are also outstanding managers, 
cat-herders, bureaucrats, finance experts, and experts on 
organizational behavior.   So I have sought to separate some of 
those roles.  I think that long terms on the IESG tend to breed 
detachment from the community and a tendency to put IESG 
judgment ahead of that of the community and I don't think we can 
solve that with more rules about IESG behavior.  So I have 
sought to give Nomcoms guidance about terms, to change the 
nomination/appointment model, and to make the recall mechanism 
more effective in practice.   And I have sought ways to simplify 
the job and reduce the workload in the hope that we can go back 
to treating a term or two on the IESG as an obligation that the 
right sorts of people owe the community, rather than a position 
to be sought and in the more general hope of broadening the pool 
of people who are willing to serve.

The fact that my proposals for change have not been instituted 
tells me that the community does not see a serious problem and 
doesn't believe that changes are needed.  While I believe that 
the lack of acceptance of changes has been IESG recalcitrance 
and efforts to protect the authority and ways of working with 
they are familiar and comfortable, I don't think that changes 
the conclusion: the community has ways, however unpleasant, for 
imposing changes that have community consensus but that it IESG 
doesn't like and has chosen to not use them.  I disagree, but I 
think the consensus-in-practice is fairly clear and I have to 
accept that.

To me, it is in the areas of adjusting IESG scope, 
responsiveness, and membership that we need to do our tuning, 
not by trying to restrict the IESG to particular ways of doing 
its technical evaluations or the statements ADs can make about 
specifications submitted for approval and especially what 
arguments an AD can use for forcing the rest of the IESG to take 
a harder look and initiate an in-depth discussion (internally 
and, if appropriate, with the community).  More hard rules about 
how the IESG does its technical evaluation work won't, IMO, help 
us in the common, ordinary, cases and, when an exceptional one 
arrives, such rules are likely to force the IESG into making the 
wrong decisions and doing the wrong things and thereby hurt the 
IETF and the Internet.

    john

---

From: Lisa Dusseault <[email-masked]>
Date: Sat, 30 Dec 2006 09:39:57 -0800

>      statement of what actions must be taken to clear the Discuss.

That's great when it's possible and desirable, but it's sometimes  
impossible, and sometimes undesirable.  How about a bit more nuanced  
view: any DISCUSS position should move towards being actionable as  
all the parties come to understand each other and the alternative  
approaches, rather than simply trash the DISCUSS because it's not  
actionable yet.

It's impossible for an AD to give a clear statement of what actions  
to take to clear a DISCUSS when the issue begins as a question.  It  
can sometimes take a few weeks for an issue like "Can somebody  
explain to me whether this draft really suffers from such-and-such a  
problem, or did I miss something which would make it immune to that  
problem?"  to turn from a beginning exploratory DISCUSS into an  
accurate problem description (if there still is one), and from there  
into a clear statement of actions to take.

It's undesirable for an AD to give a clear statement of what actions  
to take to clear a DISCUSS when there are many possible approaches  
and the AD doesn't know enough about previous discussions rejecting  
some of them.  It unnecessarily hardens everybody's positions if an  
AD blithely suggests "To clear my issues, this document should be  
made Experimental instead of Proposed Standard" when the WG can't  
abide that particular solution and there may be half a dozen other  
solutions.  Proposing a solution too early (before an AD understands  
the problem and the local constraints) is a classic engineering  
mistake.  I hope you'll forgive us when we do suggest the wrong  
solution now and then under pressure to make DISCUSSes actionable.

I've been document shepherd on WG documents where other ADs placed a  
DISCUSS and described their issue in terms of the problem.  Once we  
came to a certain amount of agreement on the problem, I've worked  
with WG chairs and authors to figure out the WG's preferred  
solution.  We checked with the DISCUSSing ADs along the way to see if  
our alternatives would actually satisfy their issues, and those ADs  
have been responsive and open-minded in considering various possible  
solutions.  I've done this both as WG chair (pre-AD) and as  
shepherding AD.  Sometimes it takes a bit more work to find the best  
solution rather than the first one.

Lisa

---

From: Lakshminath Dondeti <[email-masked]>
Date: Sun, 31 Dec 2006 14:27:27 -0800

There is perhaps one more aspect to "Can somebody explain ..." that 
is worth considering.  In some cases, the AD simply does not have the 
expertise or simply has incorrect/wrong understanding.  In that case, 
the burden is on the authors to help the AD understand the context of 
the work, provide references to reading material and such.  Until the 
AD understands at his/her own pace, the work seems to languish (sure 
the authors do delay responses etc., but let us work on one problem 
at a time) in the IESG review stage.

There is of course a simple solution to the problem.  The ADs should 
review drafts at the IETF LC stage and should only have time to 
understand the context of the work during that process.  They might 
also involve other experts in the area to review the I-Ds that they 
don't quite understand.  Futhermore, there shouldn't be two sets of 
wider review processes beyond the WG stage: an "open" and a "closed" 
process as is the case currently.

The General area and the SEC area have review teams now which follow 
the work starting at the IETF LC stage.  Some of it (probably 
unintentionally) is closed.  We can figure out how to open all of 
that.  Perhaps all area directors should consider starting review 
process starting at the IETF LC process.  That overlap and some sense 
of a deadline on IESG reviews might help reduce the delays in the 
processes.  It should also result in invalidating the use of DISCUSS 
to get explanations or more bluntly using DISCUSS to say "teach me."

I will close with saying that sometimes people I know ask "how long 
does it take from concept to RFC status?"  More often than not, the 
answer is we don't quite know.  They simply conclude that it takes 
several years and think of tweaking some existing provisions, e.g., 
use Vendor extensions etc., before going through the pain of making 
extensions to protocols truly interoperable.  Fact is, in many cases, 
RFC publication is quite fast.  In other cases, it does take a long 
time.  It is better to introduce some level of predictability into 
IETF processes.

thanks,
Lakshminath

At 09:39 AM 12/30/2006, Lisa Dusseault wrote:

>solution.  We checked with the DISCUSSing ADs along the way to see 
>if our alternatives would actually satisfy their issues, and those 
>ADs have been responsive and open-minded in considering various 
>possible solutions.  I've done this both as WG chair (pre-AD) and as

---

From: Lisa Dusseault <[email-masked]>
Date: Sun, 31 Dec 2006 19:11:33 -0800

On Dec 31, 2006, at 2:27 PM, Lakshminath Dondeti wrote:

> is worth considering.  In some cases, the AD simply does not have  
> the expertise or simply has incorrect/wrong understanding.  In that  
> case, the burden is on the authors to help the AD understand the  
> context of the work, provide references to reading material and  
> such.  Until the AD understands at his/her own pace, the work seems  
> to languish (sure the authors do delay responses etc., but let us  
> work on one problem at a time) in the IESG review stage.

Sure.  That could happen.  But it's not usually the case that an AD  
who knows that they don't understand something, holds a DISCUSS on a  
document for a long time, all the while getting useful responses and  
help from authors.  I sure wouldn't feel comfortable doing that.   It  
could be that a given document languishes while an AD gains  
understanding, or it could be that eventually the AD gives up or  
manages to get some issues resolved in that document. I'm sure there  
are different perspectives on why something was held up and then  
worked out, for just about any document that did suffer from a long  
DISCUSS.

Note that the authors and WGs have a burden to explain their work to  
more than just ADs.

Lisa