Subject: Document review

From: Bruce Lilly <[email-masked]>
Date: Fri, 6 May 2005 01:03:46 -0400

>   Re: "straightforward, reasonable, and fair"
>  Date: 2005-05-05 11:41
>  From: Keith Moore <[email-masked]>

[excerpts]
> No matter how bad the  
> document is, the most an AD can generally manage to do is to insist  
> on small changes to the text.  Yes, in theory the AD could insist  
> that the document undergo significant revision, but the pressure from  
> both inside and outside the IESG to move even bad documents along is  
> considerable.  [*].  Sometimes there's no way to fix a document with  
> small textual changes.  But because of the pressure to either approve  
> a document or to ask for fairly small changes to the document text,  
> ADs sometimes suggest small changes to the text that end up being  
> poor fixes.
> 
> The best cure for both of these problems, I suspect, is not to give  
> ADs less "favor" in reviewing documents, but to identify such  
> problems earlier, at a time when there is less investment in the WG's  
> output and less sense of urgency to get the document out the door.

> [*] Bad documents take more time to review than good documents --  
> both because they tend to be poorly written and because the AD is  
> struggling to think of a simple fix while reading the document -- and  
> nobody on IESG wants to review a bad document over and over.

Bad documents take more time to *thoroughly* review... but a document
needing significant rework can and should be pushed back, otherwise
the AD ends up doing the detail work, and the IESG should be dealing
with the broad perspective rather than a myriad of details.  Thorough
review takes time in collecting details, pointing out specific problems
and how they can be addressed, etc.  Of course, if a document is of
fairly high quality, that's relatively easy.  For documents with
significant numbers of problems, a checklist may help.  So, in a
somewhat more serious vein than the Slashdot Spam Solution checklist;
cut, paste, add comments, edit to suit, and check boxes:

Review of            draft-foobar-subject-matter-00       by A. Nonymous

Internet-Drafts and RFCs are required to meet certain criteria: [R1.ID],
[R2.Instruct].  These can be checked by using the "idnits" tool:
[R3.idnits].  That tool noted the following regarding the document:

  <idnits complaints go here>

The document contains:

   [ ] MIB

   [ ] ABNF [R4.RFC2234]

   [ ] Internet message (or fragment)

   Verification tools are available at [R5.verify] and/or [R6.mparse].

   Verification tools yielded the following results:

     <verification tool complaints go here>

The wording of the document is poor.  [R7.editor62] may be useful.

The formatting of the document is atrocious. See [R8.formatting] and
[R9.formatting]

The document suffers from the following serious defects:

   [ ] missing or inadequate IANA considerations

   [ ] missing or inadequate security considerations

   [ ] missing or inadequate internationalization considerations

   [ ] incompatible with the Internet Architecture

   [ ] incompatible with one or more Internet Standards

   [ ] uses non-standard terminology which may lead to confusion,
       misinterpretation, and loss of interoperability

   [ ] not backwards compatible with widely deployed,
       standards-conforming infrastructure

   [ ] BCP 14 is referenced, but keywords are not used, or are used
       inappropriately

Please carefully review the references and resources indicated:

   [ ] [R1.ID]

   [ ] [R2.Instruct]

   [ ] [R3.idnits]

   [ ] [R4.RFC2234]

   [ ] [R5.verify]

   [ ] [R6.mparse]

   [ ] [R7.editor62]

   [ ] [R8.formatting]

   [ ] [R9.formatting]

   [ ] [R10.FYI18]

   [ ] [R11.FYI36]

   [ ] [R12.STD10], [R13.STD10], [R14.STD10]

   [ ] [R15.STD11]

   [ ] [R16.STD13], [R17.STD13]

   [ ] [R18.STD17]

   [ ] [R19.BCP9]

   [ ] [R20.BCP14]

   [ ] [R21.BCP18]

   [ ] [R22.BCP22]

   [ ] [R23.BCP26]

   [ ] [R24.BCP32]

   [ ] [R25.BCP41]

   [ ] [R26.BCP44]

   [ ] [R27.BCP56]

   [ ] [R28.BCP61]

   [ ] [R29.BCP72]

   [ ] [R30.BCP82]

   [ ] [R31.BCP90]

   [ ] [R32.BCP95]

   [ ] [R33.BCP97]

   [ ] [R34.RFC1958]

The following applies to the document being reviewed:

   [ ] The document seems promising, but needs work.

   [ ] The document needs substantial rework before serious review can
       take place.

   [ ] The document is a disaster.  It should be withdrawn.

References

   [R1.ID]         "Guidelines to Authors of Internet-Drafts", March
                   2005, ftp://ftp.ietf.org/ietf/1id-guidelines.txt

   [R2.Instruct]   Reynolds, J., and R. Braden, "Instructions to Request
                   for Comments (RFC) Authors", Work in progress (August
                   2004).

   [R3.idnits]     http://ietf.levkowetz.com/tools/idnits/

   [R4.RFC2234]    Crocker, D. and P. Overell, "Augmented BNF for Syntax
                   Specifications: ABNF", RFC 2234, November 1997.

   [R5.verify]     TOOLS, "Available Verification Tools",
                   http://tools.ietf.org/verif-tools

   [R6.mparse]     Internet Message Format validating parser,
                   http://users.erols.com/blilly/mparse/index.html

   [R7.editor62]   RFC Editor, "Tutorial Slides", ftp://ftp.rfc-
                   editor.org/in-notes/rfc-editor/tutorial62.pdf

   [R8.formatting] RFC Editor, "Formatting RFCs and Internet Drafts",
                   http://www.rfc-editor.org/formatting.html

   [R9.formatting] Lilly, B., "Writing Internet-Drafts and Requests For
                   Comments using troff and nroff",
                   http://users.erols.com/blilly/formatting/draft-lilly-
                   using-troff-04.html

   [R10.FYI18]     Malkin, G., "Internet Users' Glossary", RFC 1983,
                   August 1996.

   [R11.FYI36]     Shirey, R., "Internet Security Glossary", RFC 2828,
                   May 2000.

   [R12.STD10]     Postel, J., "Simple Mail Transfer Protocol", STD 10,
                   RFC 821, August 1982.

   [R13.STD10]     Klensin, J., Freed, N., Rose, M., Stefferud, E., and
                   D. Crocker, "SMTP Service Extensions", STD 10,
                   RFC 1869, November 1995.

   [R15.STD11]     Crocker, D., "Standard for the format of ARPA
                   Internet text messages", STD 11, RFC 822, August
                   1982.

   [R16.STD13]     Mockapetris, P., "Domain names - concepts and
                   facilities", STD 13, RFC 1034, November 1987.

   [R17.STD13]     Mockapetris, P., "Domain names - implementation and
                   specification", STD 13, RFC 1035, November 1987.

   [R18.STD17]     McCloghrie, K. and M. Rose, "Management Information
                   Base for Network Management of TCP/IP-based
                   internets:MIB-II", STD 17, RFC 1213, March 1991.

   [R19.BCP9]      Bradner, S., "The Internet Standards Process --
                   Revision 3", BCP 9, RFC 2026, October 1996.

   [R20.BCP14]     Bradner, S., "Key words for use in RFCs to Indicate
                   Requirement Levels", BCP 14, RFC 2119, March 1997.

   [R21.BCP18]     Alvestrand, H., "IETF Policy on Character Sets and
                   Languages", BCP 18, RFC 2277, January 1998.

   [R22.BCP22]     Scott, G., "Guide for Internet Standards Writers",
                   BCP 22, RFC 2360, June 1998.

   [R23.BCP26]     Narten, T. and H. Alvestrand, "Guidelines for Writing
                   an IANA Considerations Section in RFCs", BCP 26,
                   RFC 2434, October 1998.

   [R24.BCP32]     Eastlake 3rd, D. and A. Panitz, "Reserved Top Level
                   DNS Names", BCP 32, RFC 2606, June 1999.

   [R25.BCP41]     Floyd, S., "Congestion Control Principles", BCP 41,
                   RFC 2914, September 2000.

   [R26.BCP44]     Moore, K. and N. Freed, "Use of HTTP State
                   Management", BCP 44, RFC 2964, October 2000.

   [R27.BCP56]     Moore, K., "On the use of HTTP as a Substrate",
                   BCP 56, RFC 3205, February 2002.

   [R28.BCP61]     Schiller, J., "Strong Security Requirements for
                   Internet Engineering Task Force Standard Protocols",
                   BCP 61, RFC 3365, August 2002.

   [R29.BCP72]     Rescorla, E. and B. Korver, "Guidelines for Writing
                   RFC Text on Security Considerations", BCP 72,
                   RFC 3552, July 2003.

   [R30.BCP82]     Narten, T., "Assigning Experimental and Testing
                   Numbers Considered Useful", BCP 82, RFC 3692, January
                   2004.

   [R31.BCP90]     Klyne, G., Nottingham, M., and J. Mogul,
                   "Registration Procedures for Message Header Fields",
                   BCP 90, RFC 3864, September 2004.

   [R32.BCP95]     Alvestrand, H., "A Mission Statement for the IETF",
                   BCP 95, RFC 3935, October 2004.

   [R33.BCP97]     Bush, R. and T. Narten, "Clarifying when Standards
                   Track Documents may Refer Normatively to Documents at
                   a Lower Level", BCP 97, RFC 3967, December 2004.

   [R34.RFC1958]   Carpenter, B., "Architectural Principles of the
                   Internet", RFC 1958, June 1996.

Author's Address

   Ann Nonymous
   Acme Widget Company
   123 Main St.
   Anytown, XX 12345-6789

   Telephone: tel no.
   Fax: fax no.
   Email: [email-masked]
   URL: http://www.example.com/foo

_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 09 May 2005 15:48:39 +0200

Bruce,

Do you think it's OK for the IESG to kick a draft right back to
the WG by saying

   "This is a mess and fundamentally wrong, but we don't have
    time to tell you why, so you have to go find a reviewer." ?

This is a serious question... my concern is that this is a
surefire way to annoy the whole WG (quite apart from the fact that
it would indicate an enormous failure at an earlier stage).

    Brian

---

From: "Adrian Farrel" <[email-masked]>
Date: Mon, 9 May 2005 15:42:43 +0100

Brian,

>     time to tell you why, so you have to go find a reviewer." ?

Yes, but...

If an I-D is really bad, it is simply not possible to do anything
approaching a reasonable review, and the IESG don't have the time or
job-skills for this.

However, they should be able to give some pointers even for an I-D they
haven't looked at in detail. Things like: the use of English is very poor;
the document needs complete restructuring; etc.

And this comment needs to come with some helpful advice, such as: ask on
the WG mailing list for someone to help edit the language; have a look at
draft-xyz.txt and see if you can use that structure.

In most cases, however, the I-D has come to the IESG after review by the
'responsible' AD. So why not boot it straight back to that AD (management
lines should be up-and-down)?

One last point. Sometimes a WG runs out of steam for a draft that is
somewhat useful, but Informational. If the draft is readable but really
needs a significant re-write then what? Either publish as it is or
acknowledge that it will probably be discarded (but still return it to the
WG just in case). - This point does not apply to Standards Track (IMHO):
such drafts must be of good quality.

Adrian