Subject: Re: I-D Action: draft-roach-bis-documents-00.txt

From: Brian E Carpenter <[email-masked]>
Date: Wed, 8 May 2019 13:57:18 +1200

Hi,

This seems reasonable. I'm not even sure it needs to be a BCP, since as far as I can see it doesn't introduce any new process rules. I'm also not sure it should be treated as mandatory, because each case is special.

However...

It doesn't help to clarify what the "Updates:" tag means (it only mentions "Obsoletes:") and it doesn't discuss how protocol extensions should be labelled. Should they carry the "Updates:" tag? Or should we ask the RFC Editor for a new "Extends:" tag, and "Extended by:"?

Regards
   Brian

On 08-May-19 12:18, [email-masked] wrote:
> 
> A New Internet-Draft is available from the on-line Internet-Drafts directories.
> 
> 
>         Title           : Process for Handling Non-Major Revisions to Existing RFCs
>         Author          : Adam Roach
> 	Filename        : draft-roach-bis-documents-00.txt
> 	Pages           : 8
> 	Date            : 2019-05-07
> 
> Abstract:
>    This document discusses mechanisms the IETF has historically used for
>    updating RFCs subsequent to their publication, and outlines an
>    updated procedure for publishing newer versions (colloquially known
>    as "bis versions") that is appropriate in certain circumstances.
>    This procedure is expected to be easier for both authors and
>    consumers of RFCs.
> 
> 
> The IETF datatracker status page for this draft is:
> https://datatracker.ietf.org/doc/draft-roach-bis-documents/
> 
> There are also htmlized versions available at:
> https://tools.ietf.org/html/draft-roach-bis-documents-00
> https://datatracker.ietf.org/doc/html/draft-roach-bis-documents-00
> 
> 
> Please note that it may take a couple of minutes from the time of submission
> until the htmlized version and diff are available at tools.ietf.org.
> 
> Internet-Drafts are also available by anonymous FTP at:
> ftp://ftp.ietf.org/internet-drafts/
> 
> _______________________________________________
> I-D-Announce mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/i-d-announce
> Internet-Draft directories: http://www.ietf.org/shadow.html
> or ftp://ftp.ietf.org/ietf/1shadow-sites.txt
>

---

From: Adam Roach <[email-masked]>
Date: Tue, 7 May 2019 21:49:59 -0500

Brian --

Thanks for noticing and providing proactive feedback. This is a separate 
proposal, which isn't directly related to the discussion of the 
disposition of the "Updates" tag. You should see a separate proposal 
from Ted Hardie and/or Warren Kumari on metadata clarifications related 
to "Updates" shortly.

/a

---

From: "Martin Thomson" <[email-masked]>
Date: Wed, 08 May 2019 06:50:23 -0400

BTW, I really like where this is going.  We need more ways to smooth the path toward maintaining protocols (see what I did there?).  

In the spirit of providing feedback on the document as written...

On Wed, May 8, 2019, at 12:50, Adam Roach wrote:
> Thanks for noticing and providing proactive feedback. 

   2.  The changes from the document being obsoleted must not constitute
       the majority of the document.  This is a subjective evaluation,
       not a mathematical one.

I think that there is case for assessing a "substantial portion of the document or a significant technical change" rather than "majority of the document" test here.  That makes the call even more subjective, but I don't believe that it would be appropriate to pass something that changed fundamental concepts without thorough review.  For instance, it might be possible to amend the definition of dialog-forming requests in SIP without hitting enough of RFC 3261 to meet a "majority" test, but publication of a change like that under the proposed process is not likely to be appropriate.

   4.  The document SHOULD contain an appendix detailing the changes
       from the RFC it is replacing.  Any change for which the rationale
       is not abundantly obvious should be explained in this appendix.

I don't see why an appendix is stipulated.  Concise descriptions of changes in introductory sections are often a very welcome way to explain differences.

  3. The "Last-Call notification" MUST also include a pointer to a
       mechanically-generated diff [...]

This is good, but you might make it easier by citing the section that summarizes changes.

Section 3.3 is I think the most difficult part of this.  I agree with you that finding a way to be able to publish a document without touching certain outdated pieces is very important, but I think that your solution contains another problem.

If some things are controversial, we might find that even listing things that don't follow best practice to be difficult.  I would instead suggest that rather than requiring enumeration of all the "icky bits", we instead label what is updated and say that "everything else is left as is and might not reflect current best practice".  That is,

     This document is a revision of a previously published RFC. Those 
     portions of the text that have not been updated might not meet 
     current best practices for documents published by the IETF.

I know that this is weaker than what you are proposing, but it should avoid having people avoid updates where there are these booby traps.  I'm happy if you encourage the listing of things that are especially problematic, but you use the "m" word (in lowercase, which I don't quite follow), which creates a barrier.

Now, I would prefer that people work through those controversial points, but I've seen enough good work stymied by bickering over unimportant or unrelated points that some caution might be wise. 

Finally, you might consider promoting the real content from Section 6 to the body of the document.

---

From: Christer Holmberg <[email-masked]>
Date: Wed, 8 May 2019 11:27:04 +0000

Hi,
    
>    I don't see why an appendix is stipulated.  Concise descriptions of changes in introductory sections are often a very welcome way to explain differences.
  
Whether it's an Appendix or not, I think having a dedicated "Changes from RFC number-of-the-obsoleted-RFC" is good. 

Regards,

Christer

---

From: Christer Holmberg <[email-masked]>
Date: Wed, 8 May 2019 11:37:47 +0000

Hi,

A couple of comments:

Q1:

In general, I like a "lightweight" process for producing a bis, instead of having a number of update specs, or spending much time on a bis the old way for a minor change/update.

But, while it's easier, I still think we need to be rather conservative when it comes to producing bis. For example, external SDOs reference our RFCs, and it is typically easier for them to adopt an update spec to a referenced RFC than to replace the referenced RFC with a bis.

Q2:

Related to Q1, assume that we have two cases what would justify a bis. Those cases are however completely unrelated to each other. Should we make two bis, or should we fix both cases in a single bis?

Q3:

Regarding deprecated technology, isn't removing deprecated technology a good use-case for a bis using the lightweight process? I guess this is related to Q2: if the original reason for the bis is to fix feature A, could we in the same bis remove deprecated technology?

Regards,

Christer

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 8 May 2019 10:52:04 -0400

I like the approach in this document and think it is a useful 
improvement in how we handle the revision problem.

Thinking about it, there is one issue that it does not address, that 
seems relevant, and that I am not sure how to handle.

What should be done about other documents that update the base document. 
  We have RFC A, updated by RFCs B and C.  We are now producing Abis. 
For the specific purpose of fixing a couple of narrow issues that really 
need to be addressed.
What should this narrow Abis say about B and C?

Yours,
Joel

---

From: Adam Roach <[email-masked]>
Date: Wed, 8 May 2019 11:30:10 -0500

> I think that there is case for assessing a "substantial portion of the document or a significant technical change" rather than "majority of the document" test here.  That makes the call even more s…

That's a good point, and captures more of what I really wanted to convey 
here. I've rephrased as:

2. The changes from the document being obsoleted must not constitute
    the majority of the document, nor must they be a fundamental technical
    change to its underlying mechanism. This is a subjective evaluation,
    not a mathematical one.

> I don't see why an appendix is stipulated.  Concise descriptions of changes in introductory sections are often a very welcome way to explain differences.

I'm not married to the notion of an appendix, but I think the 
description of changes benefits greatly from being in a dedicated 
section -- and since it doesn't really form the core of the document, 
having it as an appendix seems like the right kind of thing. To your 
specific suggestion: like every other section, unless the introduction 
otherwise requires revision, I'd like to avoid seeing changes made to it.

Backing up to some of the philosophy of what I'm trying to accomplish: 
I'd like to see bis documents as taking our initial pass(es) at a 
protocol and bringing them closer to the platonic ideal of what that 
document might have been in the first place, with the benefit of 
experience and hindsight [1]. For any useful protocol we've developed so 
far, the future is bigger than the past, and my goal is to give 
developers in the future a better document that doesn't look like some 
kind of patchwork quilt. Wedging additional text into the body like you 
propose doesn't really accomplish this.

> This is good, but you might make it easier by citing the section that summarizes changes.

Good point. I've added that.

> I know that this is weaker than what you are proposing, but it should avoid having people avoid updates where there are these booby traps.  I'm happy if you encourage the listing of things that are especially problematic, but you use the "m" word... which creates a barrier.

I've had several conversations with standing and former members of the 
IESG about the overall proposal I present in this document, and more 
than one person got hung up on the notion of letting 
known-to-be-deprecated approaches through without comment. Largely, 
these involve issues that have already been documented in BCP  or 
Standards Track documents, and which would otherwise constitute DISCUSS 
criteria. Cast in that light, I think the enumeration of what would be 
different in a green-field protocol is actually quite important, 
although I concede that the current text might not capture it as clearly 
as I like.

> (in lowercase, which I don't quite follow)

It's a -00 version, and I haven't gone through to double-check my 2119 
terms. I'll rationalize these in a future version.

> Now, I would prefer that people work through those controversial points, but I've seen enough good work stymied by bickering over unimportant or unrelated points that some caution might be wise.

I would definitely prefer that such points be worked through as well 
(and say as much in the "Security Considerations" section, as many such 
controversial points fall into the security area). I take your point 
that even agreeing on what is a Bad Idea might cause some heartburn, but 
I don't think a blanket "something in here might be wrong, but we won't 
tell you what" statement accomplishes much at all.

> Finally, you might consider promoting the real content from Section 6 to the body of the document.

Good point. Done.

/a

____
[1] There's also a goal to make the documents easy to review and easy to 
approve that drives the desire to eliminate spurious changes, but this 
is done in service to getting an improved document out as quickly as 
possible.

---

From: Adam Roach <[email-masked]>
Date: Wed, 8 May 2019 17:39:50 -0500

> But, while it's easier, I still think we need to be rather conservative when it comes to producing bis. For example, external SDOs reference our RFCs, and it is typically easier for them to adopt a…

I find this a bit surprising; but I've made a note to check with our 
various liaisons about whether they think this kind of process would 
cause a problem, and, if so, what we might be able to do to help other 
SDOs in the face of such changes.

> Related to Q1, assume that we have two cases what would justify a bis. Those cases are however completely unrelated to each other. Should we make two bis, or should we fix both cases in a single bi…

That's the sort of thing I would expect a working group to make a 
decision about. Currently, WGs decide whether to roll updaing documents 
together or whether to keep them separate. I expect it would be a very 
similar decision-making process.

> Regarding deprecated technology, isn't removing deprecated technology a good use-case for a bis using the lightweight process? I guess this is related to Q2: if the original reason for the bis is t…

I would certainly hope so. The point, however, is that we're not going 
to *require* that work to be done before a bis document can be produced.

/a

---

From: Adam Roach <[email-masked]>
Date: Wed, 8 May 2019 17:46:19 -0500

On 5/8/19 9:52 AM, Joel M. Halpern wrote:
> What should be done about other documents that update the base 
> document.  We have RFC A, updated by RFCs B and C.  We are now 
> producing Abis. For the specific purpose of fixing a couple of narrow 
> issues that really need to be addressed.

I suspect this is also something that working groups will want to decide 
on a case-by-case basis. I tried to address this, to some degree, at the 
bottom of section 3.1:

>     Although not a strict qualification, working groups and authors of
>     documents using this process should carefully evaluate all verified
>     errata on the original RFC and all RFCs that formally update the
>     original RFC to determine which, if any, the new document should
>     incorporate.

This might benefit from an expanded treatment, although I don't want to 
be too prescriptive about what working groups might consider when 
determining what updates make sense to incorporate or otherwise 
explicitly reference.

/a

---

From: "Martin Thomson" <[email-masked]>
Date: Thu, 09 May 2019 02:03:10 -0400

On Thu, May 9, 2019, at 02:30, Adam Roach wrote:
> section 

Yep, it was a mere quibble.  I think that a dedicated section is a very good idea, but that the additional condition that it be hidden away is not always as good an idea.  With a small change, Section 1.4 might be better.

Take RFC 8446 as an example.  It's hardly a bis document, but it takes Sections 1.2 and 1.3, over almost two pages, for this purpose.  I don't want to invalidate that approach.  After all, we have a long-standing requirement to include some statement about what has changed in the abstract (which I assume you aren't changing).

I've no problem with a recommendation to use an appendix.  I think that those sections in TLS 1.3 are a bit of a distraction from the main event and might have been left until later in the document, but that (like much of this) is a point upon which reasonable people can disagree.

> although I concede that the current text might not capture it as clearly 

I think that a *strong* encouragement to perform the analysis (i.e., SHOULD) is better than an absolute mandate (MUST).  That gives the right incentive, but leaves groups that get wrapped around the axle of an old issue an option that they can exercise.

Final point, and it's a new one (sorry): Authorship.  This is likely to be a sensitive topic.  But it is probably not appropriate to replace the entire author list for a small change.  Some guidance on that would be welcome.  My suggestion would be to recognize new authors either as an additional author or editor, or even to list them in the contributors.  Like a lot of this, judgment will need to be exercised, but that should take into account the size of the change and the amount of work that was done.

---

From: Christer Holmberg <[email-masked]>
Date: Thu, 9 May 2019 07:17:49 +0000

Hi,

Q1:
    
>> In general, I like a "lightweight" process for producing a bis, instead of having a number of update specs, or spending much 
>> time on a bis the old way for a minor change/update.
>>
>> But, while it's easier, I still think we need to be rather conservative when it comes to producing bis. For example, external 
>> SDOs reference our RFCs, and it is typically easier for them to adopt an update spec to a referenced RFC than to replace the 
>> referenced RFC with a bis.
>     
  
At the end of the day, no matter whether it's an update or a bis, it's the actual changes/updates that matter. Also, when it comes to SDOs, backward compatibility has a big impact.

So, I wonder whether the draft should include some text regarding backward compatibility, and whether the lightweight process should not be used for such changes?
    
---  
    
Q3:

>> Regarding deprecated technology, isn't removing deprecated technology a good use-case for a bis using the lightweight process? 
>> I guess this is related to Q2: if the original reason for the bis is to fix feature A, could we in the same bis remove deprecated technology?
>      

I also think it would be good to include some text saying that, once the scope of a bis has been agreed upon, people should be allowed to add their favorite-thing-to-fix as the work progresses. Or, IF someone wants to add a fix/update A, adding that fix/update should go through the same process as for making the bis in the first place.

This of course apply to any draft we work on, but as we know the scope quite often change in our drafts.

Regards,

Christer

---

From: John C Klensin <[email-masked]>
Date: Thu, 09 May 2019 11:48:26 -0400

Adam,

Rather than quoting specific other responses but having read
them, a few general comments...

Some of the issues addressed in the document and in assorted
comments, including the type of first-time updates/replacements
for a base document that your draft appears to focus on, the
question of how to update or replace a document that has already
been updated by other RFCs, the related question of how to
address such document collections (sometimes known as "what is
really the standard for...?) when documents are not at full
standard, the slightly-related questions of the status of a
replacement for a document that already is at full standard, 

are all issues that seem worth addressing, some of which are,
IMO, demonstrably seriously overdue for some review and action
(see below).

I appreciate your trying to make this change with an I-D and BCP
target rather than an IETF Proclamation^H^H^H^H^H^H^H^H^H^H^H
Statement.  On the other hand, as another aspect of the issue
SM, Stephen, and others have addressed, I question whether it is
appropriate to use this document to specify how the IESG should
vote or count votes.   If that is legitimate procedurally, then
it is legitimate for the community to develop an I-D, bottom-up,
that specifies IESG voting behavior and expect the IESG to
process it. The IESG has, in the past, sent clear signals that
such a specification would not be welcome.   Assuming the lack
of enthusiasm for the community micromanaging the rule and
procedures by which the IESG makes decisions has not changed,
either take that instruction out, turn it into a general
expression of hope that both IETF Last Call and IESG review will
focus on the changes rather than other issues, or be careful
what you wish for.  Similar comments apply to a few of the
bullet items in Section 3.2.

There are also several issues associated with Section 3.1 (3)
("Except as detailed in verified errata...") that I don't
believe that others have addressed.   First, where "verified" or
not, errata have not gone through IETF Last Call and don't
represent community consensus, only an assertion (for IETF
Stream documents) by the authors and and AD or the IESG that
they look right.  There is also considerable history of
classifying an erratum as "hold for future update" when if it
clear that there was an error (as with "verified") but no one
wants to cope with sorting out the right text to fix it.
Because the latter are not part of the exception, the procedure
specified encourages having a replacement document go through
that includes text known to be erroneous.  And because
"verified" doesn't mean "community consensus when the erratum
was submitted", much less "community consensus when the proposed
replacement document goes into Last Call, discouraging comments
that do not address the changes that were the purpose of the
document seems unwise, even if the careful evaluation at the end
of Section 3.1 is performed.

Section 3.3 interacts with the traditional role of Applicability
Statements.  In order to avoid further confusion (or further
accidental deprecation of Applicability Statements), that
interaction should be called out and discussed.

More broadly, while it was long before your time on the IESG
(and, I believe, before the time on the IESG of any current AD),
some of the issues mentioned above and the associated underlying
problems were addressed several years ago by a WG, called
NEWTRK, whose recommendations and drafts (which were, in the
WG's opinion,  ready for IETF Last Call) took a rather different
and more comprehensive approach to the problem your I-D
identifies and addressed the issues of errata, updated documents
as well as replaced ones, and a complex of documents making up
the specification for a standard or practice as well.  While the
IESG at the time declined to initiate an IETF Last Call on those
documents, it seems to me that, if some of the same problems are
going to be addressed, the community should have the opportunity
for consider whether the approach in this I-D or the more
comprehensive NEWTRK approach will better meet its needs.  

Applying what I understand to have been the IESG's reasoning
about draft-moonesamy-recall-rev, I look forward to a BOF
proposal, an in-depth discussion of scope that includes the
NEWTRK approach as well as this one, a mailing list separate
from the IETF discussion one (or I suppose we could use the
NEWTRK one), and a WG.

best,
    john

---

From: Adam Roach <[email-masked]>
Date: Thu, 9 May 2019 11:06:18 -0500

Thanks. On initial read, these all seem like fair points [1], and I'll 
incorporate the ones that bear on the specific proposal I'm making in 
the next version of the document.

/a

____
[1] I may have reason to quibble with one or two of the conclusions, but 
they don't bear on the contents of the document, so I don't think 
engaging on them would be useful for either of us at the moment. Maybe 
later.

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 10 May 2019 08:27:27 +1200

On 09-May-19 18:03, Martin Thomson wrote:
...> Final point, and it's a new one (sorry): Authorship.  This is likely to be a sensitive topic.  But it is probably not appropriate to replace the entire author list for a small change.  Some guidance on that would be welcome.  My suggestion would be to recognize new authors either as an additional author or editor, or even to list them in the contributors.  Like a lot of this, judgment will need to be exercised, but that should take into account the size of the change and the amount of work that was done.

I think this is probably a general point for many RFCs in all streams, not just the ones targetted by the draft, so I think it would be better to leave it to RFC Editor policy.

It is discussed briefly in https://tools.ietf.org/html/draft-carpenter-whats-an-author-02#section-7 but that draft has no status except "Expired".

Regards
    Brian

---

From: John C Klensin <[email-masked]>
Date: Thu, 09 May 2019 17:26:47 -0400

--On Friday, May 10, 2019 08:27 +1200 Brian E Carpenter
<[email-masked]> wrote:

> ....> Final point, and it's a new one (sorry): Authorship.
> This is likely to be a sensitive topic.  But it is probably
> not appropriate to replace the entire author list for a small
> change.  Some guidance on that would be welcome.  My
> suggestion would be to recognize new authors either as an
> additional author or editor, or even to list them in the
> contributors.  Like a lot of this, judgment will need to be
> exercised, but that should take into account the size of the
> change and the amount of work that was done.
> 
> I think this is probably a general point for many RFCs in all
> streams, not just the ones targetted by the draft, so I think
> it would be better to leave it to RFC Editor policy.

Agreed... and I was about to say that this interacts in
significant ways with issues we have traditionally left to the
RFC Editor.   In particular, listing the new authors as
Contributors generally doesn't work because the listed authors
are (unless the RFC Editor significantly changes policy) the
people responsible as contact points, involved in AUTH48, etc.
In addition, the RFC Editor has traditionally pushed back on
documents with a long list of authors (in the early days, Jon
took the position that more  than one or two authors violated
good sense and the ideas of "contact point" and "final
responsible party.  The limit was later expanded to no more than
five unless exceptional circumstances applied.  One can imagine
a document with four or five authors expanding, on revision, to
a very long list that was quite meaningful except, perhaps, for
academic credit or points from employers.  I hope we don't need
to go there, if only because it would bring discredit on the
series.  Much better to let the RFC Editor set policy and decide
on whatever exceptions are needed.

> It is discussed briefly in
> https://tools.ietf.org/html/draft-carpenter-whats-an-author-02
> #section-7 but that draft has no status except "Expired".

Indeed, but, AFAICT, that section just further describes the
problem rather than suggesting any solutions.

best,
   john