Subject: Comments on draft-roach-bis-documents-00

From: S Moonesamy <[email-masked]>
Date: Thu, 09 May 2019 01:34:23 -0700

Hi Adam,

There is the following sentence in Section 3.2 of draft-roach-bis-documents-00:

    "IESG members SHOULD NOT issue DISCUSS or ABSTAIN ballot positions
     based on unchanged text except as described in Section 3.3."

Why is an ABSTAIN an issue?

What about IESG member "comments"?  Can those comments be ignored?

Regards,
S. Moonesamy

---

From: Magnus Westerlund <[email-masked]>
Date: Thu, 9 May 2019 09:27:16 +0000

Hi,

> Why is an ABSTAIN an issue?

Because if 1/3 of the IESG members put in an ABSTAIN position the
document can not be approved. I think the main scenario would be that
one update something but don't fix something like a serious weakness in
security. Then maybe many IESG members want to put in a "I am holding my
nose on this" with the effect that the document anyway become blocked.

> What about IESG member "comments"?  Can those comments be ignored?

Yes, comments are just comments and authors, editors and WG are free to
ignore them. The IESG appreciate them being addressed, but we don't
consider them critical. If that is the case, the issue should be put in
the Discuss part. 

For more information about the IESG ballot positions please read:

https://www.ietf.org/standards/process/iesg-ballots/

And more relevant about the Discsuss versus comment are:

https://www.ietf.org/blog/discuss-criteria-iesg-review/

Cheers

Magnus Westerlund 

----------------------------------------------------------------------
Network Architecture & Protocols, Ericsson Research
----------------------------------------------------------------------
Ericsson AB                 | Phone  +46 10 7148287
Torshamnsgatan 23           | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden | mailto: [email-masked]
----------------------------------------------------------------------

---

From: S Moonesamy <[email-masked]>
Date: Thu, 09 May 2019 03:11:23 -0700

Hi Magnus,
>nose on this" with the effect that the document anyway become blocked.

The above would require five Area Directors to abstain on the grounds 
that the protocol has fundamental flaws which cannot be fixed.

Regards,
S. Moonesamy

---

From: Stephen Farrell <[email-masked]>
Date: Thu, 9 May 2019 11:28:16 +0100

Hiya,

I've read the draft and like it except for the bit
SM quotes below.

>     based on unchanged text except as described in Section 3.3."

I'm fine with the idea that the IESG would mostly just
review the diff, and the IESG do need to respect the
fact that existing RFC text has IETF consensus, but I
don't think it's ok to try to force ADs to ignore security
or privacy issues that the proponents of a bis would
like to ignore. I read the text above as doing that. I'm
not saying that all such things ought always be fixed in
bis drafts as we clearly do not do that, but a SHOULD NOT
DISCUSS seems wrong.

Separately, if enough ADs ABSTAIN then the draft should
have a problem. ABSTAIN ballots weren't that common when
I was on the IESG so unless that's changed a lot I don't
think that clause is useful or advisable.

So if that text stays in, I would hope that ADs would
ignore it and try do what they consider correct. That may
be another argument to not have a SHOULD NOT - just say
that the goal is to keep reviews to the diff.

Lastly, I'd leave out that text because even if it were
what the IESG wanted, it ought be in the discuss-criteria
IESG statement and not in an RFC/BCP that derives from
this draft. (It is good that it's there now, so I can
whine about it though:-)

Cheers,
S.

---

From: Alexey Melnikov <[email-masked]>
Date: Thu, 9 May 2019 11:30:55 +0100

Hi SM,

> that the protocol has fundamental flaws which cannot be fixed.

Yes (about 5 Abstains). This is not a theoretical issue: this has 
happened recently, although the reason for Abstain was a bit different.

Best Regards,

Alexey

---

From: S Moonesamy <[email-masked]>
Date: Thu, 09 May 2019 03:49:51 -0700

Hi Alexey,
>happened recently, although the reason for Abstain was a bit different.

Thanks for the feedback.  Please see the message [1] from Stephen 
about handling that scenario outside the draft.

Regards,
S. Moonesamy

1. https://mailarchive.ietf.org/arch/msg/ietf/QoQuv7XAZj5llZUguUfnlS1eLhQ

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 9 May 2019 07:49:01 -0400

A big part of the problem I see, which I see addressed by this draft, is 
that as currently constituted we can not fix many existing protocols 
without doing a major modification to address security and / or privacy 
issues.

Hence the bit of text being discussed about not having DISCUSS or 
Abstain on the basis of not fixing existing security and privacy issues 
is important.

ADs are free to comment, and every WG I have seen takes such comments 
seriously.  So if the AD thinks there is a simple improvement in 
security and / or privacy that ought to be included, they can ask about 
it.

Yours,
Joel

---

From: Stephen Farrell <[email-masked]>
Date: Thu, 9 May 2019 13:03:03 +0100

Hiya,

> without doing a major modification to address security and / or privacy

I guess we may have different definitions of "fix":-)

I totally agree that it's not feasible to address all
legacy security and privacy issues in all bis documents.
I don't agree that bis documents ought get a free pass
wrt such issues where they exist.

> security and / or privacy that ought to be included, they can ask about it.

In my little head, that is what most security/privacy DISCUSS
ballots on bis drafts tend to be. Adam's draft seems to me to
call for ADs to not ask/chat/discuss such issues, and I don't
agree with that level of non-discussion being the expectation.

Cheers,
S.

>>>  Â Â  "IESG members SHOULD NOT issue DISCUSS or ABSTAIN ballot positions
>>>  Â Â Â  based on unchanged text except as described in Section 3.3."
>>
>>> What about IESG member "comments"?Â  Can those comments be ignored?
>>>
>>> Regards,
>>> S. Moonesamy
>>>
>>>
>

---

From: Adam Roach <[email-masked]>
Date: Thu, 9 May 2019 11:16:45 -0500

On 5/9/19 7:03 AM, Stephen Farrell wrote:
> Adam's draft seems to me to

To be clear, what I'm trying to establish is an expectation that these 
issues won't block publication, which is a rather different thing than 
suggesting that they not be raised. At the same time, Joel Halpern put 
his finger on the heart of the matter, which is that one of the major 
reasons we don't produce bis documents as frequently as would benefit 
implementors is a fear of late surprise during IESG review on the basis 
of unchanged text. Consequently, we really need a pretty strong signal 
that using the process described in this document is unlikely to result 
in such a situation.

That said, I think John Klensin raises an interesting process point 
about normatively constraining ballot positions, so I plan to reword 
this as an expectation rather than a constraint in the next version of 
the document. I'm hoping you'll find the new phrasing more palatable.

/a

---

From: John C Klensin <[email-masked]>
Date: Thu, 09 May 2019 18:14:00 -0400

--On Thursday, May 9, 2019 11:16 -0500 Adam Roach
<[email-masked]> wrote:

> To be clear, what I'm trying to establish is an expectation
> that these issues won't block publication, which is a rather
> different thing than suggesting that they not be raised. At
> the same time, Joel Halpern put his finger on the heart of the
> matter, which is that one of the major reasons we don't
> produce bis documents as frequently as would benefit
> implementors is a fear of late surprise during IESG review on
> the basis of unchanged text. Consequently, we really need a
> pretty strong signal that using the process described in this
> document is unlikely to result in such a situation.

Adam,

That piece of analysis (and Joel's comments to which you refer)
suggests something else and opens up another can of worms
(although neither a new can nor new worms).

Suppose we have a technical specification and it contains three
kinds of sections:

(i) Those (the majority) that are just fine.

(ii) Those in which we've discovered glitches that are in need
of clarifications or adjustments to make implementations of the
protocol interoperate well, at least most of the time, and for
which we have consensus on fixes.

(iii) Some truly nasty issues, probably associated with edge
cases, for which we have rough consensus that there are
problems, but they are problems only in rare circumstances and
we have no consensus at all about how to fix them.

Now, if only cases (i) and (ii) exist in a particular document,
I think creation of a replacement draft and application of a
procedure similar to the one you describe should work well.
However, if some of that third case exists -- whether they are
known when writing starts or uncovered only during IETF Last
Call on the new "bis" document, it is not clear to me how we
should proceed.  If I correctly understand this I-D, the answer
is to leave the original (iii)-type text intact, to not list it
as changed, and to avoid discussing it when the "bis" I-D is
reviewed.  Fine plan, except that it would deceive the reader
about the state of the (new) specification and would violate our
long-standing rule against publishing standards-track
specifications that contain known technical defects.

If the idea is that the "bis" document should either contain a
health warning or incorporate or reference an Applicability
Statement about the problem feature or cases, it seems to me
that the I-D should discuss those options.  

And, of course, none of that addresses the cases in which other
documents exist that normatively reference the original RFC and
whether they are automagically treated as referencing the
replacement document (works for some types of changes that might
be covered by your I-D but not others) or continuing to
reference the older, now-obsoleted one.

    john

---

From: Adam Roach <[email-masked]>
Date: Thu, 9 May 2019 17:17:13 -0500

On 5/9/19 5:14 PM, John C Klensin wrote:
> Fine plan, except that it would deceive the reader

That's why section 3.3 is, in my opinion, critical. Maybe not in its 
current formulation, but (IMHO) something along those lines is necessary 
to make this work even a little bit.

/a

---

From: John C Klensin <[email-masked]>
Date: Thu, 09 May 2019 18:25:19 -0400

--On Thursday, May 9, 2019 17:17 -0500 Adam Roach
<[email-masked]> wrote:

>> about the state of the (new) specification and would violate
>> our long-standing rule against publishing standards-track
> That's why section 3.3 is, in my opinion, critical. Maybe not
> in its current formulation, but (IMHO) something along those
> lines is necessary to make this work even a little bit.

Agreed.  But, if that section is the solution, it almost
certainly needs work.  In particular, there is the well-known
tendency of people to glance through RFCs (and other technical
specs) looking for answers rather than reading them carefully
from beginning to end (especially likely if the current document
replaces, and largely duplicate, a previous RFC on the same
subject).  Given that, and regardless of what the Introduction
and/or Appendices say, the sections that are known to be
problematic should probably have individual, in-line, text that
calls out the problem and/or indicates (by reference if needed)
why they should not be taken seriously as written.

Leaving the reader with "Some [unspecified] portions of the text
have not been updated and do not meet current best practices for
documents published by the IETF", even if combined with
detailing each specific technique... that would not generally be
acceptable", just does not come up to the standard I think we
hope for in IETF technical specification RFCs.

    john

---

From: Adam Roach <[email-masked]>
Date: Thu, 9 May 2019 17:54:50 -0500

> hope for in IETF technical specification RFCs.

I understand and have some sympathy for your position, but I don't 
necessarily agree with it.

Given Martin's related feedback at 
<https://mailarchive.ietf.org/arch/msg/ietf/3FUqzIyiYo82AFgBajy55eBz2kU>:

> I would instead suggest that rather than requiring enumeration of all the "icky bits", we instead label what is updated and say that "everything else is left as is and might not reflect current best practice".
...I would curious what conclusion the two of you might reach if you 
decided to spend significant face-to-face time discussing the topic.

I'll note that I similarly have some sympathy for (but also don't 
necessarily agree with) Martin's position, and offered the current 
proposal as a compromise between the two countervailing concerns that 
you have respectively brought forth.

/a

---

From: "Martin Thomson" <[email-masked]>
Date: Thu, 09 May 2019 22:10:40 -0400

> > hope for in IETF technical specification RFCs.

Let me make a supposition: a big part of why people don't attempt to update documents is that they are scared of having to deal with problems unrelated to the one they are fixing.

And another one: a lot of RFCs contain problems worth fixing, some of which we all agree on, and some that are in John's third category of skeletons in the closet.

Adam proposes that we allow people to fix the fixable problems, but we need to agree on what to do with the skeletons.  We have a range of options along this continuum:

a/ stick a notice on the doc saying that there might be skeletons anywhere other than the updated stuff
b/ signpost all the skeletons we know about
c/ fix all the skeletons

We've been doing c/ for a while and being shackled to that has meant that we get horrible "patch" RFCs or no updates at all.

Adam quoted an earlier mail where I advocated for a/, but I later suggested that the draft not REQUIRE putting little sticky notes on all the skeletons, but to encourage that.  That's for two reasons.  To remove the disincentive associated with having to perform the analysis, and the disincentives arising from discussion about the content of the list.

For larger documents, it's possible that collecting the list of shortcomings could be a fairly significant task.  And as for getting consensus on a list...  I'm currently watching the TLS WG do its usual thing at the moment, and aside from the usual anger and despair that is evoked by that sort of discussion, I am reminded that we don't always have consensus about whether the skeleton even exists.

So that's why I suggested a slightly weaker version of what Adam proposes.  That way, if someone volunteers to do us a service by fixing our protocol, we don't reward them by giving them another, harder task.

---

From: John C Klensin <[email-masked]>
Date: Thu, 09 May 2019 22:14:11 -0400

Adam,

Understood and appreciated without, as you say, necessarily
agreeing.  I'll think about this further, but one observation in
the interim:   I draw a fairly large distinction between "we
have discovered some issues with this and it might not work as
expected and/or not represent current best practice" (which is
where I see Martin's comments focusing, but I may still not
understand his point of view) and "we have come to understand
this feature better and it definitely will not work as expected
in some list of cases we also understand" (whether those cases
are enumerated or not).  The latter is a known "unresolved
design choice" and/or "known technical omission" as those terms
are used in RFC 2026.   2026 allows the IESG to waive the "no
known technical omissions" requirement, but, as I have read that
text in the last 22+ years doing so far a document that is
replacing another one and without addressing that omission would
put the IESG and authors on rather thin ice if there were not a
very clear and explicit justification.

Of course, one could modify/update 2026 to match the intent of
this I-D, but I assume that would be considered a much more
major step.

best,
   john

--On Thursday, May 9, 2019 17:54 -0500 Adam Roach
<[email-masked]> wrote:

>> Leaving the reader with "Some [unspecified] portions of the
>> text have not been updated and do not meet current best
>> practices for documents published by the IETF", even if
>> combined with detailing each specific technique... that would
>> not generally be acceptable", just does not come up to the
>> standard I think we hope for in IETF technical specification
>> RFCs.
> 
> I understand and have some sympathy for your position, but I
> don't necessarily agree with it.
> 
>> I would instead suggest that rather than requiring
>> enumeration of all the "icky bits", we instead label what is
>> updated and say that "everything else is left as is and might
>> not reflect current best practice".

> ...I would curious what conclusion the two of you might reach
> if you decided to spend significant face-to-face time
> discussing the topic.
> 
> I'll note that I similarly have some sympathy for (but also
> don't necessarily agree with) Martin's position, and offered
> the current proposal as a compromise between the two
> countervailing concerns that you have respectively brought
> forth.
> 
> /a
>

---

From: "John Levine" <[email-masked]>
Date: 10 May 2019 11:20:48 -0400

>this I-D, but I assume that would be considered a much more

While not wanting to derail the ongoing conversation, this topic is
making me think that we have a lot of processes that we never change
for little more reason than that we've never changed them.  Here's
two that this discussion makes me think of:

There's the question of what authors to put on a bis document since
the original authors may have lost interest or in some cases died.
Academic papers often identify one or two of a long list of authors as
the corresponding authors, to whom you write if you have questions or
comments.  The five author limit apparently dates from a desire that
all of the listed authors be the corresponding ones, but that was when
everyone was a lot younger.  Even if we keep the limit at five,
putting a star next to one or two of them wouldn't be that hard.

More radically, the theory that RFCs never change dates from when they
were on paper and nobody wanted to find all the copies and apply
whiteout.  While I don't think it would be a good idea to have
different documents floating around all called, say, RFC 3514, we
could call revised versions 3514.1 and 3514.2 like other SDOs do and
the world wouldn't come to an end.

Again I am not suggesting we stop thinking about this draft, but I do
think we should try and recognize places where we are working around
our processes rather than our processes working for us.

R's,
John

---

From: tom petch <[email-masked]>
Date: Fri, 10 May 2019 16:46:18 +0000

----- Original Message -----
From: "John Levine" <[email-masked]>
To: <[email-masked]>
Sent: Friday, May 10, 2019 4:20 PM
Subject: Re: Comments on draft-roach-bis-documents-00

> the original authors may have lost interest or in some cases died.

Give both and their relationship. I would say that this is quite common
within the IETF as in e.g. RFC2579. I do not think that there is a
formal way of doing this but I find the meaning clear.

Tom Petch