Subject: Evolving Documents (nee "Living Documents") side meeting at IETF105.

From: Warren Kumari <[email-masked]>
Date: Wed, 3 Jul 2019 13:04:56 -0700

Hi there all,

TL;DR: Being able to mark a specific version of an *Internet Draft* as
“stable” would often be useful. By encoding information in the name
(stable-foo-bar-00) we can do this.

Heather and I will be holding a side meeting at IETF 105 to discuss
the idea and get feedback.
When: Tue, July 23, 3:00pm – 4:30pm
Where: C2 (21st Floor)

Details:
Back at IETF 103 (Nov 2018, Bangkok) Job presented on “Living
Documents” at the GROW session -
https://datatracker.ietf.org/meeting/103/materials/slides-103-grow-living-documents-growroutingsecurity-00
.

Ever since then I’ve had an action item to write this up, and get more
feedback on this. I’ve had a number of discussions with people about
this, and have had a bit of a tricky time explaining it -- we’ve
realized that this is because there are multiple use cases, and people
have been pigeonholing the idea into their preconceived use-case.

The two *main* use-cases are:
“Evolving” documents (there were originally called “Living documents”,
but that name is already in use in the web community, and so we chose
another name to make it less confusing).
“State of Play” documents.

My main use-case / interest is the “evolving documents” idea, and so
I’ll be focusing on that
(the “State of Play” idea is taking the sorts of things used to plan
interop events in QUIC and HTTPBIS, and creating a common format and
location for them; this will be fleshed out further in the future).

There are a number of topics / drafts where the best current practice
changes over time - a simple example of this is something like routing
security, and so I’ll use it as an example (also, the concept was
first presented in GROW!), but the concept is applicable to all sorts
of cases.

The best practices for routing security (what / how to filter,
route-leak prevention, etc) change over time and it is not really
feasible to document how to e.g build route filters and then release
-bis documents quickly enough to keep up with how the operational
advice changes; this means that much of this sort of information
doesn’t get documented in the IETF, and instead is scattered around in
institutional knowledge, personal blogs, IRC channels, etc.

An obvious solution is to simply document this in an Internet Draft,
and then update the ID as the advice changes; this is indeed a
reasonable solution, but makes it hard to refer to the WG’s *best
current* understanding while still allowing the WG to continue working
on the draft. If you refer to a specific version
(draft-ietf-foo-bar-23) it is hard to let people (especially external
people!) know when the WG thinks that there is a new “stable” version
(draft-ietf-foo-bar-30); if you refer just to the draft name
(draft-ietf-foo-bar) it makes it hard for the WG to make any changes
to the draft without potentially causing confusion.

In order to allow us to refer to “stable” and “development” versions
of drafts I’m proposing that we use the draft naming scheme to
annotate a version as “stable”:

Currently when draft-wkumari-bar-03 gets adopted by the BAR working
group, it gets renamed to draft-ietf-foo-bar-00. I’m proposing that,
when a WG determines that there is consensus that a version of a draft
is “stable”, a copy of the draft would be created called
draft-stable-foo-bar-00. When a new version of draft-ietf-foo-bar is
deemed stable, it would be copied into draft-stable-foo-bar-01. This
is simple for external people to understand, etc. When and if the
draft becomes an RFC, the link for draft-stable-foo-bar would redirect
to RFCxxx (just like draft-ietf-foo-bar).

There are a few open questions / notes (AKA FAQ!):

1: How does the WG decide that a document is “stable”?
One obvious option would be for the chairs to run something that looks
like a very lightweight WGLC. Another would simply be to have the
chairs decide (chairs are already trusted to judge consensus). This is
still open and I’m looking for feedback…

2: What are we calling these?
The original name for this was “Living Documents” - a number of people
have pointed out that this name is already in use in, and so has
“baggage”. I’d love some suggestions, so far the closest I’ve gotten
is “Evolving Documents”. There are only 3 problems in computer
science: off-by-one errors and naming things…

3: Why did it take so long from IETF 103 till this is being suggested?
<Warren hangs head in shame> This has been sitting on my todo list,
and kept getting pushed down. I had some time at an ICANN meeting and
finally wrote it up…

4: We had also considered a new tag in the Datatracker, and allowing
the chairs to assign it to a specific version of a document; this
would have been simple to implement, but much of the purpose of this
idea is to make it easy for external people to reference the latest
version of a document - and expecting them to understand datatracker
tags seems more complex.

5: What happens when an “Evolving Document” dies?
Evolving documents are just copies of drafts, tagged with a prefix
(just like adopted WG documents are tagged with a draft-ietf) - this
means that they behave just like any other draft - it is hoped that,
if a WG decides to abandon an “evolving document” they would replace
it with a tombstone (e.g “The foo WG has decided to abandon this
document because $reasons”).

6: Can an Evolving Document update an RFC?!
Nope - see the previous question; this is simply a way to mark an ID
as “stable”, they don’t have special powers. Just like an ID cannot
update an RFC, neither can an Evolving Document.

7: Can an Evolving Document become an RFC?
No!, see the previous two questions :-) The ID that an evolving
document is a based on can be published an RFC (and it might be
identical to an evolving document), but this doesn’t in any way change
the RFC publication process. It is expected that if / when the ID gets
published, the datatracker link (e.g:
https://datatracker.ietf.org/doc/stable-foo-bar/ ) will point to the
published RFC, just like ID links (
https://datatracker.ietf.org/doc/draft-ietf-foo-bar/ ) currently do.

8: This looks like you are creating something like semantic versioning
for drafts. Why not simply create something like
draft-ietf-foo-bar-17.42.9?!
This does have some similarities, but I think a better analogy is
“development” and “release” branches. I also believe that the current
draft naming and versioning system has been around for a long time, is
well understood, and works well - we have lots of tooling which uses
it, and it has lots of history. I really really *really* don’t want to
mess with that - this proposal allows those who would like to annotate
a version as “stable” to do so without us mucking with the existing
process, etc.

9: This ain’t new, $person proposed something similar at $meeting_or_list….
Indeed. As an example, Phillip Hallam-Baker mentioned a desire for
something similar at the IETF102 plenary; I think that this
demonstrates that there is an interest for this, let’s explore it
further.

10: Does a WG have to use this? I think it’s a [waste of time |
confusing | useless for my WG | <something>]?
Nope. This is only for those WGs that would like to use it - if you
don’t want to use it, feel free to ignore it. If you think it would be
better if only $something, please come tell us. Obviously, if you feel
that this is harmful, please speak up!

Please come to the side meeting and discuss the idea of Evolving
Documents further,
Warren Kumari and Heather Flanagan

P.S: I'm traveling for the next few weeks (and am also part of the NOC
team, and so get to Montreal early), and so my responses might be
delayed - 'pologies in advance.

-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

---

From: Keith Moore <[email-masked]>
Date: Wed, 3 Jul 2019 16:10:13 -0400

> When: Tue, July 23, 3:00pm – 4:30pm

It seems to me that this would defeat the entire purpose of 
Internet-Drafts and serve to circumvent the IETF process.    There 
should be no expectation of stability until a document has reached 
IETF-wide consensus.

---

From: Richard Barnes <[email-masked]>
Date: Wed, 3 Jul 2019 16:15:50 -0400

> should be no expectation of stability until a document has reached

Why is it necessary to conjoin those two things?

--Richard

---

From: Keith Moore <[email-masked]>
Date: Wed, 3 Jul 2019 16:18:20 -0400

On 7/3/19 4:15 PM, Richard Barnes wrote:

>
>
> On Wed, Jul 3, 2019 at 4:10 PM Keith Moore <[email-masked] 
> <mailto:[email-masked]>> wrote:
>

Because a working group does not have the authority to make such 
decisions on its own.   To the extent that it would be desirable to 
invest such authority in some body for some specific purpose, a working 
group is the wrong kind of body to do that.   The norms around IETF WG 
operation aren't the right ones for such a body.

Keith

---

From: Richard Barnes <[email-masked]>
Date: Wed, 3 Jul 2019 16:23:01 -0400

On Wed, Jul 3, 2019 at 4:18 PM Keith Moore <[email-masked]>
wrote:

> Because a working group does not have the authority to make such decisions
> on its own.   To the extent that it would be desirable to invest such
> authority in some body for some specific purpose, a working group is the
> wrong kind of body to do that.   The norms around IETF WG operation aren't
> the right ones for such a body.
>

Doesn't have the authority to publish stable specifications?  Obviously, a
WG can't publish something and claim it has consensus or is an RFC.  But
WGs already have the ability to publish stable docs, by publishing them on
github or on IPFS.  This is just about making them easier to find and
reference.

I think maybe you're over-inflating the significance of this proposal.

--Richard

---

From: Warren Kumari <[email-masked]>
Date: Wed, 3 Jul 2019 13:25:19 -0700

> should be no expectation of stability until a document has reached

Great, please come to the side meeting and we can try better explain the
use case. I thought I’d better described the use case further down in the
mail, but it seems I may have failed...  This specifically allows the WG to
point at a version of a document that external type people can implement
against **while** still allowing the working group to make changes, etc. If
anything this allows the WG to make more changes because it (hopefully)
removes some of the “we cannot change this bit because someone may have
implemented against it” concern.

But, it’s entirely possible that this is just a bad idea, please come to
the side meeting...

W

I don't think the execution is relevant when it was obviously a bad idea in
the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair of
pants.
   ---maf

---

From: Keith Moore <[email-masked]>
Date: Wed, 3 Jul 2019 16:28:58 -0400

On 7/3/19 4:23 PM, Richard Barnes wrote:
> On Wed, Jul 3, 2019 at 4:18 PM Keith Moore <[email-masked] 
>>     On Wed, Jul 3, 2019 at 4:10 PM Keith Moore
>>     <[email-masked] <mailto:[email-masked]>>
>>     wrote:
>>
>     decisions on its own.   To the extent that it would be desirable
>     to invest such authority in some body for some specific purpose, a
>     working group is the wrong kind of body to do that.   The norms
>     around IETF WG operation aren't the right ones for such a body.
>
>
> Doesn't have the authority to publish stable specifications?  
> Obviously, a WG can't publish something and claim it has consensus or 
> is an RFC.  But WGs already have the ability to publish stable docs, 
> by publishing them on github or on IPFS.  This is just about making 
> them easier to find and reference.
>

I think it's inappropriate for a WG to *claim* that any particular 
version of a document is stable prior to IETF-wide consensus.    
Publishing a document on github or similar doesn't make it stable, it 
just means that changes will be visibly tracked.

Keith

---

From: Keith Moore <[email-masked]>
Date: Wed, 3 Jul 2019 16:30:38 -0400

On 7/3/19 4:25 PM, Warren Kumari wrote:

> Great, please come to the side meeting and we can try better explain 
> the use case.

Great, please send funding.

Keith

---

From: Paul Wouters <[email-masked]>
Date: Wed, 3 Jul 2019 16:32:37 -0400 (EDT)

> (stable-foo-bar-00) we can do this.

Cause if its stable, then you shoulda put an RFC number on it?

> advice changes

Perhaps the IETF and the RFC series is not the right place for such a document?
Re-using the draft system seems wrong, and when used widely would just
cause more confusion about what it means to be a draft.

> 1: How does the WG decide that a document is “stable”?

When it gets an RFC number :)

Seriously though, I can point to an RFC and tell a customer or vendor
"You should do that". I cannot do that with a draft, especially if it
will change all the time.

Paul

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 3 Jul 2019 16:34:07 -0400

Let me phrase it differently, with a similar point to Keith's

An IETF working group can say "we think this has the right content, but 
we are not yet handing it to the AD because ..."
That is a form of stability
It is NOT a promise not to change the content before RFC publication.
As an example, I as co-chair thought the NSH spec was very stable, and 
then a technical issue was raised that required an incompatible change. 
It was still a working group document.  We made the change.

Further, a working group can not label a draft in a way that suggests 
that there is IETF consensus in support of the document.  That is not 
its purview.  And is believe the implication that Keith is concerned about.

I do not expect that either Heather or Warren were looking at the later 
interpretation.  But I can see how someone reading the email could 
reasonably be concerned about such over-ambition.
And your response that working groups can publish whatever they want is 
at best misleading.  It is true that the content of I-Ds is up to the 
working group.  The labeling of them is NOT up to the working group.
(We do, consciously, deliberately, and to significant advantage, make 
all our works-in-progress visible to the world.  They are labelled as such.)

Yours,
Joel

PS: I am not sure what the general benefit of marking an I-D as 'stable" 
would be.  We still would not want it normatively cited.  I tried to 
construct the most positive such label in teh example above.  I may or 
may not have time to join the side-meeting.

On 7/3/19 4:23 PM, Richard Barnes wrote:
>     invest such authority in some body for some specific purpose, a
> Doesn't have the authority to publish stable specifications?  Obviously, 
> a WG can't publish something and claim it has consensus or is an RFC.  
> But WGs already have the ability to publish stable docs, by publishing 
> them on github or on IPFS.  This is just about making them easier to 
> find and reference.
>

---

From: "Scott O. Bradner" <[email-masked]>
Date: Wed, 3 Jul 2019 16:37:54 -0400

see “stable snapshot” in https://tools.ietf.org/html/draft-bradner-ietf-stds-trk-01

Scott

> On Jul 3, 2019, at 4:34 PM, Joel M. Halpern <[email-masked]> wrote:
> 
> An IETF working group can say "we think this has the right content, but we are not yet handing it to the AD because ..."
> As an example, I as co-chair thought the NSH spec was very stable, and then a technical issue was raised that required an incompatible change. It was still a working group document.  We made the change.
> 
> Further, a working group can not label a draft in a way that suggests that there is IETF consensus in support of the document.  That is not its purview.  And is believe the implication that Keith is concerned about.
> 
> I do not expect that either Heather or Warren were looking at the later interpretation.  But I can see how someone reading the email could reasonably be concerned about such over-ambition.
> And your response that working groups can publish whatever they want is at best misleading.  It is true that the content of I-Ds is up to the working group.  The labeling of them is NOT up to the working group.
> (We do, consciously, deliberately, and to significant advantage, make all our works-in-progress visible to the world.  They are labelled as such.)
> 
> Yours,
> Joel
> 
> PS: I am not sure what the general benefit of marking an I-D as 'stable" would be.  We still would not want it normatively cited.  I tried to construct the most positive such label in teh example above.  I may or may not have time to join the side-meeting.
> 
>> On Wed, Jul 3, 2019 at 4:18 PM Keith Moore <[email-masked] <mailto:[email-masked]>> wrote:
>> Doesn't have the authority to publish stable specifications?  Obviously, a WG can't publish something and claim it has consensus or is an RFC.  But WGs already have the ability to publish stable docs, by publishing them on github or on IPFS.  This is just about making them easier to find and reference.

---

From: Nico Williams <[email-masked]>
Date: Wed, 3 Jul 2019 15:39:50 -0500

On Wed, Jul 03, 2019 at 04:10:13PM -0400, Keith Moore wrote:
> It seems to me that this would defeat the entire purpose of Internet-Drafts
> and serve to circumvent the IETF process.    There should be no expectation
> of stability until a document has reached IETF-wide consensus.

Implementors regularly ship pre-RFC code...

But this doesn't have to be about shipping prior to RFC publication.  It
can be just about limiting the impact of future edits on existing code
that hopefully has not yet shipped anyways.

Now, this can be done _today_ by convention -- you just write in the I-D
that version starting at version -XX only backwards-compatible changes
will be entertained.  So maybe there's no need for new work in this
area.

Nico
--

---

From: Keith Moore <[email-masked]>
Date: Wed, 3 Jul 2019 16:45:11 -0400

> Implementors regularly ship pre-RFC code...

Yes, and it's quite often inappropriate for them to do so, a disservice 
to their customers and sometimes to the whole Internet.

(there are specific cases when doing so is mostly harmless, especially 
if there's a seamless way to upgrade without disruption.... but there 
are many cases for which this is not possible.)

> will be entertained.

And in general, it's inappropriate for a WG (or author of a 
standards-track document) to do that before IETF consensus has been reached.

Keith

---

From: Warren Kumari <[email-masked]>
Date: Wed, 3 Jul 2019 15:18:10 -0700

> It was still a working group document.  We made the change.

Exactly.

> its purview.  And is believe the implication that Keith is concerned about.

Yes, it is in no way appropriate to claim / imply that the document has
IETF consensus, any more than it is appropriate to claim that an ID does.
If this idea were to go ahead, we could adapt it to have (more prominent,
with asterisks and similar!) boilerplate to clarify that this is only what
the WG currently thinks, and isn’t an IETF consensus, or an rfc or anything
else...

> reasonably be concerned about such over-ambition.

Fair - that wasn’t the idea, but it definitely need clarification...

> would be.

Primary, if you are external to the IETF, and want to get a “snapshot” of
what the WG has agreed to on a draft.
As an example, I was recently working on a draft where people started
implementing bits which were still very much under discussion - this hurt
the draft because it made it hard to change. I made some proposed edits to
this section anyway to get WG feedback... and implementers suddenly changed
to this...
After a few rounds of “hey, we are changing this again” implementers got
annoyed, the WG got annoyed, and I got annoyed.

I would have liked to be able to easily signal “if you want to implement,
this version is mostly sane. It’s obviously still subject to change, but at
least more than the authors think it’s reasonable.” versus “this version
has many bits which we don’t have WG agreement on: we put them in so they
can be reviewed. Please wait till we agree that it isn’t filled with
craziness before implementing, etc.”

We still would not want it normatively cited.

Indeed. As the FAQ section says, ‘tis just a draft, just like any other...

I tried to

---

From: Warren Kumari <[email-masked]>
Date: Wed, 3 Jul 2019 15:32:16 -0700

On Wed, Jul 3, 2019 at 1:38 PM Scott O. Bradner <[email-masked]> wrote:

> see “stable snapshot” in
> https://tools.ietf.org/html/draft-bradner-ietf-stds-trk-01
>

Ah, nice, thanks. There are some definite parallels with this. What we were
proposing is similar, but less “strong” - it doesn’t have anything like
IETF / IESG consensus, rather just “this is what the *WG* currently thinks”.

W

> then a technical issue was raised that required an incompatible change. It
> was still a working group document.  We made the change.
> >
> that there is IETF consensus in support of the document.  That is not its
> purview.  And is believe the implication that Keith is concerned about.
> >
> construct the most positive such label in teh example above.  I may or may
> not have time to join the side-meeting.
> >
> Obviously, a WG can't publish something and claim it has consensus or is an
> RFC.  But WGs already have the ability to publish stable docs, by
> publishing them on github or on IPFS.  This is just about making them
> easier to find and reference.

---

From: heather flanagan <[email-masked]>
Date: Wed, 3 Jul 2019 15:33:09 -0700

> On Jul 3, 2019, at 1:34 PM, Joel M. Halpern <[email-masked]> wrote:
> 

> Further, a working group can not label a draft in a way that suggests that there is IETF consensus in support of the document.  That is not its purview.  And is believe the implication that Keith i…

Entirely agree.

> (We do, consciously, deliberately, and to significant advantage, make all our works-in-progress visible to the world.  They are labelled as such.)

One of the things that attracts me to this idea of marking specific I-Ds as stable is the possibility that the material in the I-D will be more throughly tested BEFORE it gets to the RFC Editor for publication. I see it as, potentially, a way to improve the quality of what’s published in the RFC Series. 

If it means that the technical quality of the specifications published in the RFC Series improves, then that’s an idea I’d like to talk about to see if it has enough benefits to outweigh the risks of trying it out. So, that said, I’m hoping the side meeting makes sure we’ve identified the risks well enough to know what we’re up against.

-Heather

---

From: Warren Kumari <[email-masked]>
Date: Wed, 3 Jul 2019 15:43:52 -0700

[ A top-post followup ]

I’d like to repeat that this is just a *proposal*.

I still think it is a good idea, and that it is has value — but, like with
anything, what matters is what the community wants.

I’d like to continue the discussion / pushing the idea, but I’m happy to
abandon it if the consensus is that it’s a bad idea (aka, this isn’t a hill
I’m willing to die on...)

W

On Wed, Jul 3, 2019 at 1:04 PM Warren Kumari <[email-masked]> wrote:

> “stable” would often be

useful. By encoding information in the name

---

From: Heather Flanagan <[email-masked]>
Date: Wed, 3 Jul 2019 15:44:25 -0700

> On Jul 3, 2019, at 3:32 PM, Warren Kumari <[email-masked]> wrote:
> 
> 
> 
> On Wed, Jul 3, 2019 at 1:38 PM Scott O. Bradner <[email-masked] <mailto:[email-masked]>> wrote:
> see “stable snapshot” in https://tools.ietf.org/html/draft-bradner-ietf-stds-trk-01 <https://tools.ietf.org/html/draft-bradner-ietf-stds-trk-01>
> 
> Ah, nice, thanks.. There are some definite parallels with this. What we were proposing is similar, but less “strong” - it doesn’t have anything like IETF / IESG consensus, rather just “this is what the *WG* currently thinks”.

Ohhh, this is useful, thank you! I tried to do an archive search to find out why that draft stopped at -01. Would you be willing to share why the idea was rejected and/or did not get traction?

-Heather
(Now from the correct email address)

---

From: "Scott O. Bradner" <[email-masked]>
Date: Wed, 3 Jul 2019 19:01:25 -0400

this ID was one of a number of thoughts I offered up shortly after I left the IESG after a decade as general 
proposals to try to get people to think  about our standards process - but, not unlike the newtrk[1] effort of a 
few years later, there was not enough support on the then current management (IAB & IESG) for much progress

but I do not think it profitable to rehash the details of the time at this point

Scott
[1] https://datatracker.ietf.org/wg/newtrk/about/

> On Jul 3, 2019, at 6:44 PM, Heather Flanagan <[email-masked]> wrote:
> 
> 
>

---

From: Michael Richardson <[email-masked]>
Date: Wed, 03 Jul 2019 19:12:17 -0400

As expressed in the thread, the problem I have with formalizing the stable
properly is that it would seem that it becomes very difficult to fix issues.

We run into the rough consensus problems that RFC7282 explains,
with, I think, an extra push against the minority voice.
So, I would like to understand what is needed to remove the stable tag from a
document.

I think it is best to just do this informally, and come back to how can
we speed some things up elsewhere.

--
Michael Richardson <[email-masked]>, Sandelman Software Works
 -= IPv6 IoT consulting =-

---

From: Keith Moore <[email-masked]>
Date: Wed, 3 Jul 2019 19:21:26 -0400

On 7/3/19 6:18 PM, Warren Kumari wrote:
>
>
> On Wed, Jul 3, 2019 at 1:34 PM Joel M. Halpern <[email-masked] 
> <mailto:[email-masked]>> wrote:
>

Sure, that happens.  But we really don't want the public to think that 
the document is stable until it really is.

This is a longstanding problem in IETF - people implement things before 
they're ready, and this leads to incompatibilities and other operational 
difficulties.    It used to be clear that you didn't deploy 
implementations based on Proposed Standard, but people did anyway.   
IESG tried to keep up the quality by imposing more careful scrutiny, and 
people pushed back on that.  Now people want to justify deploying 
implementations based on Internet-Drafts.   Well, it's hard to stop them 
from doing so, but it certainly doesn't serve IETF's goal of promoting 
interoperability.

> Yes, it is in no way appropriate to claim / imply that the document 
> has IETF consensus, any more than it is appropriate to claim that an 
> ID does. If this idea were to go ahead, we could adapt it to have 
> (more prominent, with asterisks and similar!) boilerplate to clarify 
> that this is only what the WG currently thinks, and isn’t an IETF 
> consensus, or an rfc or anything else...

Just to play devil's advocate (see, I can see multiple sides of a 
situation): if I intend to implement a WG's proposal, I would really 
like to have an idea when I could start implementing (NOT deploying) 
with some confidence that an implementation of the final version would 
be at worst a small change from an implementation of the current 
version.   What I'd like to avoid is implementing a proposal only to 
have to rip it up and start over later because some significant changes 
were agreed to.   But I'm not sure how a WG can know when it's at that 
point, especially in the absence of significant review from the wider 
community.   WGs work too much in silos as it is.

I think this is a fundamental problem with our process - we should be 
able to get increasing /community-wide/ confidence in a proposal as it 
matures, and right now we're just not set up to do that.

> Primary, if you are external to the IETF, and want to get a “snapshot” 
> of what the WG has agreed to on a draft.

In practice, the WG doesn't come to agreement until its own Last Call.   
But the status of a draft is already available on the tracker page.   
IMO the LAST thing we need is to convey misleading impressions to those 
external to the IETF, and the potential for misinterpretation and 
deliberate misleading would be huge if such an indication were provided 
in the document ID.

> implementing bits which were still very much under discussion - this 
> hurt the draft because it made it hard to change. I made some proposed 
> edits to this section anyway to get WG feedback... and implementers 
> suddenly changed to this...
> After a few rounds of “hey, we are changing this again” implementers 
> got annoyed, the WG got annoyed, and I got annoyed.

Well, sure, but was there any way to know "this version is finally 
stable" at the time that it happened?   I have often thought I had a 
document that was ready to pass Last Call only to find out otherwise.

> I would have liked to be able to easily signal “if you want to 
> implement, this version is mostly sane. It’s obviously still subject 
> to change, but at least more than the authors think it’s reasonable.” 
> versus “this version has many bits which we don’t have WG agreement 
> on: we put them in so they can be reviewed. Please wait till we agree 
> that it isn’t filled with craziness before implementing, etc.”

Fine.  Put that in the Abstract and/or the Introduction.   Don't try to 
formally codify it.

Keith

---

From: john heasley <[email-masked]>
Date: Wed, 3 Jul 2019 23:54:25 +0000

> “State of Play” documents.

I wish that you would not create new terminology if it is the same idea.
It decreases accessibility, IMO.  Instead label it with IETF; "IETF
Evergreen Document", which seemed the most common term for this when I
looked a few weeks ago, or "IETF Evolving Document".  I do not understand
why it is relevant or how some other group's use of it creates baggage
for IETF.  Seems wholly a NiH problem.  Decide for yourself what the
most common term is and use that.

BTW, I read a page that seemed to suggest that the Canadian Gov't uses
all three terms.

> institutional knowledge, personal blogs, IRC channels, etc.

This I'd like to see - a stable document name (URL) that documents an
ever evolving technology.  For example, RFC 7525 (TLS Recommendations)
attempts to document TLS standads; protocol revisions, cipher suites,
etc.  Tremendously useful RFC for developers and users, that seems already
out of date and probably was shortly after publication.

It would be useful to have a document like RFC7525 that could evolve
with TLS, but more nimbly than an RFC.  One that an RFC of another
discipline could reference, and follow the evolution without itself
needing updating.  The evolution needs to be captured in a public SCM,
so that previous versions can be retrieved.

I do not think that I've addressed any of your points, but this is
perhaps another use case.

https://www.w3.org/wiki/Evergreen_Standards

> something similar at the IETF102 plenary; I think that this

1992: https://datatracker.ietf.org/wg/livdoc/about/

---

From: Ted Lemon <[email-masked]>
Date: Wed, 3 Jul 2019 20:20:09 -0400

Perhaps it would be effective to mark documents not stable and not label stable documents.  IOW, encourage wgs to explicitly say “for the love of Mike, don’t ship this.”

Sent from my iPhone

> On Jul 3, 2019, at 7:54 PM, john heasley <[email-masked]> wrote:
>

---

From: Andrew Sullivan <[email-masked]>
Date: Wed, 3 Jul 2019 21:30:09 -0400

Hi,

I work for ISOC but don't represent it in this discussion.  Also, I
seem to be sending more mail than is my habit today.

On Wed, Jul 03, 2019 at 07:21:26PM -0400, Keith Moore wrote:

> difficulties.    It used to be clear that you didn't deploy implementations
> based on Proposed Standard, but people did anyway.

When was that "clear"?  Just to pick a random example, RFC 2181
includes clarifications to the DNS specification that are absolutely
critical to the way DNS actually works, and that are certainly widely
deployed.  It's arguably something that ought to be considered a
formal part of STD 13.  It was published in 1997 and is a Proposed
Standard.  Given that STD 13 is only 10 years older than that, we've
lived with this Proposed Standard for much longer than we lived with
the unclear bits 2181 was designed to address.

And, of course, when STD 13 was published as RFCs 1034 and 1035 back
in 1987, it _wasn't_ an Internet standard, yet it seemed to get
deployed.  Indeed, the whole idea of "Internet Standard" seems to be
rather fluid leading up to the publication of RFC 1310, which came out
in 1992.  I note that RFC 1011, from 1987, lists the DNS with
"Recommended" status, but does not refer to the protocol documents we
rely on today but instead refers to RFCs 881-883.

I'm not trying to be querulous.  But some of my recent experiences
make me leery of assertions about how things got deployed in the past,
because I don't think that things were nearly as tidy as many people
today seem to be suggesting they were.  I was not among the people
connected to the Internet in the 1980s, but I know when I connected in
the 1990s I was both surprised how well things worked and dismayed at
how poorly they worked, often at the same time.  That experience, at
least, has not changed!

Best regards,

A

-- 
Andrew Sullivan
[email-masked]

---

From: Keith Moore <[email-masked]>
Date: Wed, 3 Jul 2019 21:52:03 -0400

On 7/3/19 9:30 PM, Andrew Sullivan wrote:

> When was that "clear"?

Probably I was thinking of RFC2026 section 4.1.1, last paragraph:

    Implementors should treat Proposed Standards as immature
    specifications.  It is desirable to implement them in order to gain
    experience and to validate, test, and clarify the specification.
    However, since the content of Proposed Standards may be changed if
    problems are found or better solutions are identified,/deploying implementations of such standards into a disruption-sensitive 
environment is not recommended./

But of course that's not stating it as strongly as I remembered, and the 
problem of deploying implementations based on Proposed Standard existed 
even before that.   I remember a flap about telnet implementations circa 
1992 in which implementations of a certain option didn't interoperate - 
one vendor followed the PS text and all of the others implemented it in 
the opposite way, and I heard a lot of people saying "they shouldn't 
have deployed at Proposed".

Keith

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 4 Jul 2019 06:55:21 +0200

> One of the things that attracts me to this idea of marking specific I-Ds as stable is the possibility that the material in the I-D will be more throughly tested BEFORE it gets to the RFC Editor for…

Most of the confusion here is about the term “stable”, because that means different things to different people.

Clearly, it’s not “stable” if it can be toppled next week.

What we occasionally do is label some draft versions as “implementation drafts” (certainly not my idea).  That designation means “it is worth implementing this now; this is no longer quicksand”.  It may simply be motivated by the need to get interop experience, so it does not mean “stable” in a more serious way; I’d rather have a different label for “we are not going to break this any more unless we really have to” label (e.g., the CDDL I-D was in that state since 2015 before it became RFC 8610; it did gain additional features during this time).

So I think providing a way for a WG to slap a useful label on a WG I-D is a good thing; standardizing on a small set of such labels is useful, but not a prerequisite.

The whole discussion is a bit of an expression of the too-high-threshold we now have for publishing things as an IETF document; there really should be a “proposed standard” level.

(Ceterum censeo: Our whole nomenclature of publications, where only some Internet standards are “Internet Standards” while the more important group is “Proposed Standards”, and valid documents are “obsoleted” by documents that are just revised editions of them, etc., is incomprehensible to casual observers and can be used and is being used against the IETF –– we really need to work on that terminology.)

Grüße, Carsten

---

From: Nico Williams <[email-masked]>
Date: Thu, 4 Jul 2019 00:23:36 -0500

On Wed, Jul 03, 2019 at 09:52:03PM -0400, Keith Moore wrote:
>    problems are found or better solutions are identified,/deploying
> implementations of such standards into a disruption-sensitive environment is
> not recommended./
> 
> problem of deploying implementations based on Proposed Standard existed even
> before that.   I remember a flap about telnet implementations circa 1992 in
> which implementations of a certain option didn't interoperate - one vendor
> followed the PS text and all of the others implemented it in the opposite
> way, and I heard a lot of people saying "they shouldn't have deployed at
> Proposed".

In the security area just about all major Internet protocols are at
Proposed Standard.  PKIX?  Proposed Standard.  Kerberos?  Ditto.  TLS?
Yup.  SSHv2?  Indeed.  IKEv2?  No, IKEv2 and CMS are among the
exceptions, though what good IKEv2 might do anyone w/o ESP, or CMS w/o
PKIX, I don't know.

Whatever the intention originally might have been, it's certainly long
not been the case that one should not deploy protocols that are at
Proposed Standard.

And it's very difficult to stop vendors from shipping pre-RFC protocols.
We don't have a protocol police, and we move too slowly.  If we don't
adapt, other SDOs will do more of our work.  A big selling point of the
IETF is its review processes -- the adults in the room to keep authors
from doing dreadful things.  But we need to speed up the cycle somewhat,
and one way to do it might be to have a way to indicate expected
stability in I-Ds, and probably only in WG work items only, and at some
cost (e.g., early directorate reviews?).  I don't quite know -- maybe
after reflection we might conclude we shouldn't do this, but we should
certainly discuss it, and be able to discuss it.

Nico
--

---

From: Nico Williams <[email-masked]>
Date: Thu, 4 Jul 2019 00:28:37 -0500

On Wed, Jul 03, 2019 at 03:33:09PM -0700, heather flanagan wrote:
> One of the things that attracts me to this idea of marking specific
> I-Ds as stable is the possibility that the material in the I-D will be
> more throughly tested BEFORE it gets to the RFC Editor for
> publication. I see it as, potentially, a way to improve the quality of
> what’s published in the RFC Series. 

An alternative would be to have interop test events.  The IETF does not
hold any such.  Sun used to host an annual interop test event (named
"Connectathon").  Perhaps the IETF/ISOC should host one.

That said, for some RFCs we've had many I-D versions prior to
publication, with a certain degree of stability being understood at some
point.  Formalizing that wouldn't be a bad idea, provided that it is
understood that IETF and IESG review can force backwards-incompatible
changes.

> If it means that the technical quality of the specifications published
> in the RFC Series improves, then that’s an idea I’d like to talk about
> to see if it has enough benefits to outweigh the risks of trying it
> out. So, that said, I’m hoping the side meeting makes sure we’ve
> identified the risks well enough to know what we’re up against.

+1

Nico
--

---

From: Keith Moore <[email-masked]>
Date: Thu, 4 Jul 2019 01:38:00 -0400

> exceptions, though what good IKEv2 might do anyone w/o ESP, or CMS w/o
Yah, I know.  It's hard to get the energy required to move up from PS.
> not been the case that one should not deploy protocols that are at
Not sure I agree with that :)  I still think it's unwise to promote 
deployment before there's been interoperability tests.   But clearly 
we're not getting that done with our current process.
> adapt, other SDOs will do more of our work.
yup, it's a race to the bottom :(
> A big selling point of the

So the way we get more review is to encourage deployment even earlier in 
the draft cycle?  Seems like an odd way to do it.

But maybe something like this:  What if WGs labeled drafts with 
"preliminary" (not ready for implementation), "ready for outside review" 
(after WG thinks the overall shape of the proposal is good, inviting 
explicit review/feedback from IETF in general and others), "ready for 
test implementation" (after favorable review and IESG approval), "WG 
last call candidate" (after favorable implementation and interop tests), 
and finally "IETF last call candidate"?   Probably not in the doc name 
itself, but in the tracker, and in the document text when appropriate.

Keith

---

From: Nico Williams <[email-masked]>
Date: Thu, 4 Jul 2019 01:18:33 -0500

> Not sure I agree with that :) I still think it's unwise to promote

I'm not convinced that we were never supposed to ship pre-Draft-Standard
RFC implementations.

> we're not getting that done with our current process.

And it's a safe bet we never will.

> yup, it's a race to the bottom :(

That's not a good argument for not attempting to improve the rate at
which we get things done.

> So the way we get more review is to encourage deployment even earlier in the
> draft cycle?  Seems like an odd way to do it.

It's the opposite.  In order to get to "deploy earlier" (see below),
first get more review than you would have had we not had that option.

Mind you, a stability attribute needn't be about deployment, but a) a
guide to participants about what changes are considered acceptable going
forward, b) an indication to implementors that the protocol is mature
enough to implement (not necessarily deploy).

> explicit review/feedback from IETF in general and others), "ready for test
> implementation" (after favorable review and IESG approval), "WG last call

Eh??  Can't test until you have IESG approval?  Did you really mean
that?

> candidate" (after favorable implementation and interop tests), and finally
> "IETF last call candidate"?   Probably not in the doc name itself, but in
> the tracker, and in the document text when appropriate.

I wouldn't mind some such designations, no, but these should probably be
Data-Tracker metadata, and probably set by the WG chairs, not the
authors.  Any designation about stability should probably also be
Data-Tracker metadata, and set by the responsible AD and/or an assigned
expert reviewer whose responsibility is to ascertain the likelihood that
future reviews will require backwards-incompatible changes.

I rather like the idea that any such designations should be Data-Tracker
metadata that can only be set by appropriate authorities.

Nico
--

---

From: John C Klensin <[email-masked]>
Date: Thu, 04 Jul 2019 03:39:36 -0400

--On Wednesday, July 3, 2019 21:52 -0400 Keith Moore
<[email-masked]> wrote:

>...
> But of course that's not stating it as strongly as I
> remembered, and the problem of deploying implementations based
> on Proposed Standard existed even before that.   I remember
> a flap about telnet implementations circa 1992 in which
> implementations of a certain option didn't interoperate - one
> vendor followed the PS text and all of the others implemented
> it in the opposite way, and I heard a lot of people saying
> "they shouldn't have deployed at Proposed".

Keith,

I don't know if it was the same incident or a different one, but
there was an incident around then that I remember vividly (for
reasons that will become clear).  In that case, an implementer
implemented and deployed to a PS (or maybe an I-D) and then the
IETF changed its mind and revised the specification.   That
first vendor complained loudly and bitterly.  I had just come
onto the IESG and Phill Gross told me that my first job was to
go explain to that vendor that they could complain all they
liked but they weren't going to get much sympathy for jumping
the gun and deploying something that wasn't ready and that
turned out to be a bad idea.  Not my idea of fun, then or now.

For better or worse, the three-level idea in 2026 and elsewhere
has always been aspirational  -- a good idea in theory but rare
in practice.  You will recall that the aspiration was high
enough at one stage that documents were expected to be
automatically obsoleted if not advanced.  That didn't work.
The October 2011 decision to drop to two levels (RFC 6410) was
expected to improve the fraction of documents that advanced and
our image be dropping things to two levels.  AFAICT, that didn't
do much either: my entirely subjective impression is that, after
a fairly brief burst of activity, the fraction of documents
advanced out of Proposed Standard has remained rather close to
constant and very low: by my very rough count, we have published
around a dozen documents as Internet Standards since late 2011,
plus a few more documents that were advanced in place.

The problem is that there has rarely been energy to go back and
do the cleanup work.  In some cases, there have been specific
reasons to not do it.  

We also have a mechanism for saying "you really need to
implement this", "that is optional", and "that other is is not
ready for prime time except under unusual circumstances".  We
don't use that either.

So I agree that things were once clear had we followed our own
rules.  However, we have never consistently done that.

And there are multiple problems with more types of labels in
addition to the question of who gets to assign them.  If a WG
does it, we face a particularly difficult version of a problem
that probably should be challenged more frequently: approval of
documents after IETF Last Call on the say-so of WGs because
there are few if any substantive reviews during Last Call and
hence no real evidence of informed community consensus.   We
would also certainly discover that neat categories don't work
and we need textual descriptions in at least some cases, a
requirement that brings us back in the direction of the NEWTRK
work, the core of which the IESG disliked enough to refuse to
put out for Last Call. 

So maybe we are ready for some new ideas.  But I suggest that,
the more subtle our categories and distinctions, the easier it
will get for implementing organizations to advertise their
implementations as conforming to IETF-approved standards and the
more incentive they will have to do so.  

best,
    john

---

From: Keith Moore <[email-masked]>
Date: Thu, 4 Jul 2019 03:54:32 -0400

Without responding point-by-point, I'll just say that I'm all for 
figuring out how to get earlier and wider implementation, and also wider 
early review, and also for minimizing late process glitches.  (I used to 
say a WG was in FIN-WAIT status when it was all but done but waiting for 
some bit of process to finish).    A lot of calendar time that a WG 
spends prior to RFC publication isn't adding value to the document 
because of pipeline stalls.

> which we get things done.
Again, all for improving the rate but only if we're doing quality 
work.    If we're not doing quality work, everything else is 
pointless.   And quality isn't measured by the number of RFCs we 
produce, perhaps closer to the opposite.
> first get more review than you would have had we not had that option.
Well, that certainly sounds a lot better than just having a WG tag its 
documents in a way that will effectively encourage early deployment 
without review.
> enough to implement (not necessarily deploy).
Making a clear distinction between "ready for test implementation" and 
"ready for deployment" sounds like a step up from what we're doing now.
> Eh??  Can't test until you have IESG approval?  Did you really mean
To clarify, people can certainly test at any time, but I think having 
cross-area review and IESG approval is about right before the WG can 
publicly declare it's stuff ready for testing.   The point is that if 
people are going to invest significant implementation effort there 
should be some reason to believe that the WG really is going in that 
direction and the broader community will back it.   I'm not saying it 
should be the same degree of IESG approval that happens prior to RFC 
publication.  It's more like "ok, now that we think we know what this 
protocol is going to look like, let's get community buyin on this 
direction".   Part of the idea would be to get early feedback about how 
this protocol might, say, fail to adequately deal with security, or 
conflict with some other protocol, or have scaling problems, or be 
abusive of lower layers, or be difficult to manage.   Much better to get 
that feedback early than at IETF-wide Last Call when it's generally too 
late to fix anything.   And IESG's role here might usually not be saying 
yes or no (though sometimes no would probably be the right answer) so 
much as providing direction (e.g. "you really do need a 
mandatory-to-implement codec/ciphersuite/whatever" to ensure 
interoperability)

Also that was just off the top of my head, and I'm sure that it could be 
improved by some tweaking.
> future reviews will require backwards-incompatible changes.
Sounds about right.   I'm thinking most such state changes should be up 
to the WG chair, perhaps after getting a hum or show of hands (or the 
email equivalent) from the WG.

Keith

---

From: Lars Eggert <[email-masked]>
Date: Thu, 4 Jul 2019 15:37:20 +0300

> When: Tue, July 23, 3:00pm – 4:30pm

Overlapping with sessions is not great. Any reason this isn't in one of the morning slots (before 10am)?

Lars

---

From: Ted Lemon <[email-masked]>
Date: Thu, 4 Jul 2019 09:24:45 -0400

Keith, why would you put “ready for test implementation” after “ready for outside review?”  We want test implementations. These are a great way of finding bugs in the spec. Arguably, there is no point in spending IETF cycles on “outside review” until someone has validated that what is written down is at least implementable by someone who’s been following the work. 

Sent from my iPhone

> On Jul 4, 2019, at 1:38 AM, Keith Moore <[email-masked]> wrote:
> 
> Not sure I agree with that :)  I still think it's unwise to promote deployment before there's been interoperability tests.   But clearly we're not getting that done with our current process.
> So the way we get more review is to encourage deployment even earlier in the draft cycle?  Seems like an odd way to do it.
> 
> But maybe something like this:  What if WGs labeled drafts with "preliminary" (not ready for implementation), "ready for outside review" (after WG thinks the overall shape of the proposal is good, inviting explicit review/feedback from IETF in general and others), "ready for test implementation" (after favorable review and IESG approval), "WG last call candidate" (after favorable implementation and interop tests), and finally "IETF last call candidate"?   Probably not in the doc name itself, but in the tracker, and in the document text when appropriate.
> 
> Keith
> 
>

---

From: Job Snijders <[email-masked]>
Date: Thu, 4 Jul 2019 16:05:52 +0200

Dear all,

This discussion focusses on an potential aspect of evolving documents
that I didn't forsee, perhaps it helps if I clarify what I am looking
for.

1) EDs do *not* carry the label of IETF-wide consensus, afterall, such
   consensus currently is not part not part of the potential publication
   process for EDs.

2) EDs are probably unsuitable for protocol specifications or protocol
   extensions, because of (1)

3) Where EDs can really shine is that they provide stable URLs with a
   short identifier (like the RFCs and BCPs) that can be referenced.

I'm interested to see if we can develop a document type that can
reflects a working group's understanding of a certain problem spaces
(like tls settings, cyphers, bgp filters), has received review from
subject matter experts (the working group participants), the WG chairs
and perhaps the AD. If these people agree that something is worthwhile
publishing on the stable URL; why not?

Especially in context of elaborating on security aspects I believe there
can be value in a more expedient publishing process.

I'm looking for something that is a step up from a wiki page anyone can
edit without supervision, something slightly different than a draft as
drafts are what they are (maybe drafts are what a WG can promote to
ED?).

In any case, I'd like to have an open discussion whether we can device a
mechanism that addresses some things that aren't exactly the best fit
for the currently available options. I'm not interested to create
something that could update RFCs. I suspect EDs can exist to provide
more context around RFCs, such as use case documents, implementation
reports, security-related recommendation.

Kind regards,

Job

---

From: "Salz, Rich" <[email-masked]>
Date: Thu, 4 Jul 2019 14:35:12 +0000

>    An alternative would be to have interop test events.  The IETF does not
    hold any such.

The hackathon has aspects of this.  TLS and QUIC did good ongoing interop, by including a "which draft is this" identifier in the initial handshake messages and it also worked well.

---

From: Heather Flanagan <[email-masked]>
Date: Thu, 4 Jul 2019 08:04:43 -0700

> On Jul 3, 2019, at 9:55 PM, Carsten Bormann <[email-masked]> wrote:
> 

Makes sense. I’m not married to any particular terminology for this idea. I would like the terminology to be clear and limited; as you mentioned below, we already use words that really only make sense if you’re embedded in the IETF. We don’t need to make matters worse in that regard. 

> What we occasionally do is label some draft versions as “implementation drafts” (certainly not my idea).  That designation means “it is worth implementing this now; this is no longer quicksand”.  I…

One of my early suggestions, which was not well received, was to leave the draft name alone and instead use a CSS+language in the document itself to indicate the document could be considered an implementation draft. But the concern over bike-shedding the CSS was, I think, death to this idea. A shame; seams like a reasonable tool for the effort.

+1

-Heather

---

From: John C Klensin <[email-masked]>
Date: Thu, 04 Jul 2019 11:28:22 -0400

Hi.

I don't know if anyone else is having the same problem, but I'm
having tremendous difficulty following all of the messages in
this thread, much less the thread about the happenings
surrounding the RFC Editor function.  That includes the
side-thread about, e.g., whether one can describe, with facts
and careful reasoning, an idea or action as stupid without
insulting the intelligence of whomever was advocating it.  Many,
many messages.   If only to reduce the odds of re-inventing and
repeating what has been said before, I'm trying to read
everything in a thread before I start a response but it means I
am running hours, sometimes days, behind.  

It is especially problematic because I'm also trying to find
time to get some technical work done that much of the IETF seems
to think it important as long as _someone else_ puts in the time
to understand the issues and do the work [1].  I've has it
suggested to me off list that maybe most of those participating
actively in the discussions either don't have technical work to
do or have too much time on their hands.  I'd rather think that
priorities (and levels of support for IETF work) are different,
but I do suggest, as Randy Bush has pointed out a couple of
times, maybe it would be better if people took a deep breath,
let some of these discussions sink in, and actual get some
technical work done, including trying to finish I-Ds before
Monday's posting deadline and then actually reading enough of
those drafts to make IETF 106 more productive.  

Or perhaps, if these topics are really important enough, we
should call off the IETF 105 agenda and devote the meeting to
them, without conflicts and one day per topic.   I understand
the logistical problems with trying to do that in one large room
but perhaps smaller group parallel sessions and discussions on
the same subject (with people allocated at random or when rooms
fill up) and then summaries of conclusions would work even
better.

In any event, it is probably worth noting and remembering that
these very long threads on the IETF list get participation from
a very small fraction of the IETF community.  Whether others
tune out after looking at one message or a dozen, we aren't
hearing from them.  Because these topics have effects on the
entire IETF, how it works, and how it is perceived, real IETF
consensus --across the entire spectrum of IETF contributors and
other materially affected parties-- is important if decisions
are to be made.  It is difficult and perhaps impossible to draw
inferences about consensus across the IETF when only a handful
(or two) of people are participating.  It is even closer to
impossible if those people participating in a way that
discourages others from getting involved, however unintentional
(and respectful) the mechanisms that are being used.

A more substantive note or two follow, but I believe the above
is important and it saddens me that it didn't come from someone
in "the leadership".   

    john

[1] I'm not trying to be cryptic, just to keep the above short.
The very high-level summary of the topic area might be best
described as a set of questions.   Do any of you, or your
organizations, colleagues, or customers, use a language  that
cannot be written properly in ASCII characters and even
occasionally want to write it properly while using the Internet?
(Note that list of languages includes English.)  Would you, or
your organizations, colleagues, or customers find it helpful to
be able to use identifiers, user names, or other descriptors or
mnemonics that are not tied to English?  Do you think it is
important that everyone have confidence that strings that should
compare equal do so and that ones that shouldn't, don't?  Do you
understand that the problems the other questions imply are hard
and require either significant digging in or suggest that the
IETF should publicly give up and accept the consequences?   If
your answer to any of those questions is "yes", why are you not
digging in personally and why are you allowing the IESG to
essentially ignore the topics, not answer questions about how to
progress documents, and to set up mechanisms and then neglect
them sufficiently to ensure failure?

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 4 Jul 2019 08:31:47 -0700

I largely agree with Nico's take here.

Ignoring labelling for a moment, in a number of WGs (HTTP, TLS, and
QUIC) we have found it necessary to have full implementations and
large-scale deployments quite early in the design process, long before
anyone thinks that the document is done.

Common practice has become something like this:

1. Once the draft has become complete enough that it's possible to do
   so, start doing test implementations based on labelled versions
   of the draft and try to interop those.

2. Once the draft has become complete enough that it's believed to be
   safe, do coordinated field trials based on identified versions of
   the draft. This may include, for instance, deploying to a large
   fraction of the user base (e.g., all Firefox Beta users or a
   fraction of the Firefox release population) and/or to a big server
   farm (e.g., Cloudflare's users). We use version identifiers that
   are associated with the draft number to avoid interop problems.

3. Once the document is published at PS, switch over to the final
   version (which is usually nearly identical to the last draft version).

This does not really fit into the PS/DS model: We absolutely need
to deploy early versions to find potential issues (this was critical
to TLS 1.3) but we all know that those documents don't meet the PS
bar as they may have known defects or at least open issues. There's
also no real use for Full Standard. The market doesn't care about it
and the people who are doing the work either are (1) fixing/extending
the protocol (using the process above) or (2) have moved onto some
other protocol.

It's not clear to me how well this Evolving Documents proposal would
fit into such a model [0], but I thought a field report on what is becoming
common practice might be useful.

-Ekr

[0] The real need I find is to be able to make minor fixes to the
documents (mostly editorial errata or clarification of points on which
there was consensus) without re-spinning the RFC, which people don't
have the energy for.

On Wed, Jul 3, 2019 at 10:24 PM Nico Williams <[email-masked]> wrote:

> implementations
> even

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 4 Jul 2019 08:38:30 -0700

> is important and it saddens me that it didn't come from someone

It's worth noting that Alissa said something that reads to me as fairly
close to this last Wednesday.

https://mailarchive.ietf.org/arch/msg/ietf/5Q3KQOBY5FQTxh_-abI4M_fepQU

-Ekr

    john

---

From: Joe Abley <[email-masked]>
Date: Thu, 4 Jul 2019 11:51:15 -0400

Hi,

> anyone thinks that the document is done.

I wondered whether there were two scenarios that were worth considering, separately:

1. Instances where design and implementation are both happening in communities that are engaged in the IETF. The examples above seem like handy examples. The people who are likely to implement are involved with or aware of the design discussions; the precise disposition of individual draft specifications is likely fairly easy to socialise within that group of people.

[In this case it's not entirely obvious to me that there is a problem to solve, but perhaps I am over-stating the degree of overlap in people involved in implementation and design (the WGs you mentioned are not ones that I have ever really followed).]

2. Instances where design and implementation are separate. The easy example that springs to my mind is the implementation of EPP before the original publication of RFC 3730 by various domain registries who more engaged in other communities than in regext.

[In the regext/EPP example I think it's clear that there were annoying interop problems resulting from different registries implementing different revisions of an internet-draft rather than the same, stable specification. It's not clear to me that in that case having notices of the form "this has been in the oven long enough that you probably won't get sick" would have helped, though; such notices could have been attached to precisely the same set of drafts that were implemented, and the result would still have been that a single EPP client would work with one registry but not another.]

However, having typed all of that I seem to have convinced myself that neither case would benefit tremendously from a new disposition marker for the document, and I wonder again what the original suggestion is trying to achieve.

If what is actually required is better coordination between people preparing early implementations, regardless of what communities those people might be engaged in, carrying an applicability statement in a section that is preserved and extended through revisions of an internet draft might be better than a short label. Such a section could be left in-place (or perhaps moved to an Appendix) during final publication in the RFC series as documentation of early implementation experience. It is surely possible today for any wg to decide to require such a section to exist and to be maintained as part of the workflow they adopt in carrying out design work.

Joe

---

From: John C Klensin <[email-masked]>
Date: Thu, 04 Jul 2019 12:39:58 -0400

--On Thursday, July 4, 2019 08:38 -0700 Eric Rescorla
<[email-masked]> wrote:

>> is important and it saddens me that it didn't come from
>> someone in "the leadership".
>> 
> 
> It's worth noting that Alissa said something that reads to me
> as fairly close to this last Wednesday.
> 

I read her note a little bit differently and the scope as much
less broad, but yes, there is some overlap.   Her note and
Randy's comments that were ultimately along some of the same
lines didn't seem to have had much effect.  Perhaps the
additional perspective in my note will.  Probably we won't be
that lucky, but I (and you, she, and Randy) can hope.

   john

---

From: Keith Moore <[email-masked]>
Date: Thu, 4 Jul 2019 13:04:15 -0400

That's just what seemed right to me - don't encourage people outside the 
working group (which is the point of such notices after all) to invest 
time/effort in implementing the protocol until there's been some 
cross-area review to verify that the community is okay with the shape of 
the proposed protocol and the direction.   It might be that the label 
"ready for test implementation" should be slightly different, e.g. 
"ready for outside implementation".   Or maybe that the nature of 
"outside review" should be clarified.

(also don't let that choice kill the idea - this was something off the 
top of my head, and I'd expect there to be discussion with lots of folks 
about the proper order of things before trying this out with a group or 
two)

Keith

On 7/4/19 9:24 AM, Ted Lemon wrote:
> Keith, why would you put “ready for test implementation” /after/ 
> “ready for outside review?”  We /want/ test implementations. These are 
> a great way of finding bugs in the spec. Arguably, there is no point 
> in spending IETF cycles on “outside review” until someone has 
> validated that what is written down is at least implementable by 
> someone who’s been following the work.
>
> Sent from my iPhone
>
> On Jul 4, 2019, at 1:38 AM, Keith Moore <[email-masked] 
>> So the way we get more review is to encourage deployment even earlier 
>> in the draft cycle?  Seems like an odd way to do it.
>>
>> "preliminary" (not ready for implementation), "ready for outside 
>> review" (after WG thinks the overall shape of the proposal is good, 
>> inviting explicit review/feedback from IETF in general and others), 
>> "ready for test implementation" (after favorable review and IESG 
>> approval), "WG last call candidate" (after favorable implementation 
>> and interop tests), and finally "IETF last call candidate"?   
>> Probably not in the doc name itself, but in the tracker, and in the 
>> document text when appropriate.
>>
>> Keith
>>
>>

---

From: Keith Moore <[email-masked]>
Date: Thu, 4 Jul 2019 13:07:48 -0400

> publishing on the stable URL; why not?

Because it could easily have the extremely undesirable effect of 
promoting premature deployment.

Also, in a sense we already have a "stable URL" - the I-D name without 
the version number.   I was of the impression that the tracker would 
already respond to that, but if not, it's a simple tweak.   In other 
words, don't point people to the document, point them to the tracker 
page that shows more context.

Keith

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 4 Jul 2019 10:24:58 -0700

> promoting premature deployment.

It seems to me that this is the point of contention, namely that in certain
parts of the industry, there is a strong feeling that there is a lot of
value in widespread deployment of pre-standard protocols as long as the
versioning is done correctly, and so we do in fact want to promote
deployment.

-Ekr

---

From: Randy Bush <[email-masked]>
Date: Thu, 04 Jul 2019 11:08:04 -0700

< stick in the mud >

> versioning is done correctly, and so we do in fact want to promote

and your description of how tls did this with 1.3, 'marked' versions of
internet-drafts, seemed to work well.  and those with colder feet could
wait for the rfc.  but i note that the triel implementations seemed not
to be deployed in production until the ietf sausage was made.  this
seems prudent.

i have seen some, shall we say, insufficiently well thought out ideas
pushed in wgs.  for me, the key here is review and consensus before it
has an ietf label.  some wgs put a higher bar for becoming adopted than
ohters.  my sympathy for that is increasing.

a few of us are working on some drafts which have trial implementations,
one already published and at least one in the cooker.  the drafts and
the implementations are being evolved.  git repositories seem to work
pretty well; though we tend to gogs over github when we want a gooey
over git.  we hope to have some interop, and expect to go through the wg
and ietf last call sausage machines if we want to put the ietf
imprimatur on them.

randy

---

From: Nico Williams <[email-masked]>
Date: Thu, 4 Jul 2019 14:20:59 -0500

> anyone thinks that the document is done.

I had that experience in mind.

Except for QUIC (whose implementors and deployers understood and
expected to have to make backwards-incompatible changes / move to HTTP/2
and /3), HTTP/2 and TLS 1.3 didn't get widespread deployment during this
process.  But they did get some, and that "some deployment" was
absolutely critical to their success.

> [...]

Not only that, but when a WG has the energy for this, it's the only way
to move at a pace that doesn't kill that energy.

    I-D -> PS -> DS -> STD  ==  fizzle

> there was consensus) without re-spinning the RFC, which people don't

And if we're going to continue to have STDs, then we'll want to have
tooling for submission of implementation, deployment, and interop
reports.

Also, if we want to stick to the idea of removing parts of PSes that are
not interop tested, then we might want to have a more formal way of
marking every "feature" that needs interop testing.  Heck, even if we do
drop that old idea, we should do this.

Perhaps we could take a page from specs like x.680 and x.690 and
friends, which have lots of very short sub-sections that match closely
to features that one could then list by their section numbers.

Nico
--

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 4 Jul 2019 13:11:21 -0700

On Thu, Jul 4, 2019 at 11:08 AM Randy Bush <[email-masked]> wrote:

> to be deployed in production until the ietf sausage was made.

I don't want to debate the precise meaning of production, but what I will
say is that at in the case of TLS 1.3, we had pre-draft versions on in
Release
versions of Chrome and Firefox and serving live domains at Cloudflare
(and I think Facebook and Google, but I'd have to check my notes). This
was actually essential to finding problems because there are environments
which will not run prerelease software.

> has an ietf label.

Can you be a little more precise about what you mean by "an IETF label".
We pretty routinely have I-Ds that have the draft-ietf prefix without having
consensus on a number of points. Or do you just mean PS, etc.?

-Ekr

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 4 Jul 2019 13:13:11 -0700

> absolutely critical to their success.

I don't want to nitpick about widespread, but we were seeing single digit
fractions
of Firefox connections with TLS 1.3 during this period. I would anticipate
that
QUIC will be similar.

-Ekr

---

From: Michael Richardson <[email-masked]>
Date: Thu, 04 Jul 2019 16:54:02 -0400

> which will not run prerelease software.

I agree how valuable it was.

But, the parties involved all understood (in a deep way, up to product
managers, I expect) that there might be incompatible changes.

I have been through protocols with obvious flaws waiting for a zero-day bugs
where I could not get a fix in ID-07 because "it's been deployed".

---

From: Leif Johansson <[email-masked]>
Date: Thu, 4 Jul 2019 23:20:15 +0200

On 2019-07-04 22:13, Eric Rescorla wrote:
> 
> 
> On Thu, Jul 4, 2019 at 12:21 PM Nico Williams <[email-masked]
> <mailto:[email-masked]>> wrote:
> 
> I don't want to nitpick about widespread, but we were seeing single
> digit fractions
> of Firefox connections with TLS 1.3 during this period. I would
> anticipate that
> QUIC will be similar.
> 
> -Ekr
> 

These are all success stories (TLS, QUIC...) that seem to follow the
pattern of a fairly tight knit community of comitted actors who are
willing to progress at roughly the same pace.

I was here when MSFT suddenly announced that a particular draft of
pkinit was going to get shipped in Active Directory krb and that was
that. Would we have gotten further down the road of practical interop
for asymmetric key authn in kerberos had we use the same pattern of
work as TLS 1.3 did? Maybe we're just better at this now? I doubt it.

I do know that for every success story there is a case of a WGs killed
by a big actor who decides it is no longer going to play ball... some
very recently.

You can lead the horse to water but you can't force it to drink
but to what extent is flexibility wrt the publication process enabling
this behaviour?

	Cheers Leif

---

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Thu, 4 Jul 2019 22:02:16 +0000

I've been wondering for days whether I should comment on these threads, and whether any benefit can be gained by my commenting.  [John, this response isn't directly in response to your email, it was just a convenient point to reply.]

Probably in contrast to most other participants in this thread, I still regard myself as a relatively IETF newcomer (first meeting was IETF 93, and I've attended the majority since).  So, I am going to give my perceptions of this thread as a relative newcomer in the hope that it might be useful input to the wider community.  My comments are not intended to offend anyone, and I apologise in advance in any offence is taken. [But perhaps it also worth noting that nobody is forced to read an email, anyone can stop reading at time, delete the email, suppress the thread, etc.]

I have not had time to read all the messages, perhaps not even the majority of them. I, and I expect many others, do not have the time to do this, or perhaps instead wish to spend their limited time on other things.

My principle view is that technical criticism is vitally important when engineering the complex systems that we all work on.  My perception is that the majority of folks actively participating in IETF do so because they care deeply about the technology we work on, and are trying to reach the best solution (for some definition of best - I suspect that differences in this definition may be part of the reason why discussions can become so heated at times).

But in my personal experience, I would say that I am more receptive to someone's feedback if it is given in a polite and constructive way.  Given that I prefer a constructive critique of my work rather than comments that are unduly terse or aggressively negative, then I strive (but don't necessary always succeed) to also give feedback in a positive way.  I would go further and say that the people that I find most inspirational at IETF are those experienced folk who are knowledgeable, welcoming and supportive of newcomers, and receptive to new ideas.

Conversely, I acknowledge that some individuals tend to give very direct and terse feedback.  When I was a newcomer I initially found this feedback to be combative and abrasive, which seemed to slow down progress, so I question whether this really is a more efficient way to get to a good technical solution.  Admittedly, over time, I gained a level of respect for some of those individuals, and probably think that their style of feedback is more a personality trait than intended as a personal attack, so I tolerate the direct and terse feedback from these individuals to the extent that I would actively seek/request their feedback.

I do question whether airing these grievances on the main IETF mailing list is the best place for this.  My impression is that they don't paint IETF in a particularly positive light.  Note, I'm not advocating that these issues shouldn't be discussed in public, but perhaps there could be an "ietf-constructive-feedback" mail alias that these sorts of discussions could be directed to.  Everyone still has the same opportunity to comment openly and publicly, but it prevents these discussions from dominating over the other important discussions on the main ietf mailing list.

In a similar vein, I also ponder the motives of the questions asked at the "Open Mic" sessions in the plenary.  Sometimes the questions seem to be raising a valid point that is important to be discussed openly, but other times they come across as older/experienced folk having a public dig at the IESG or IAB for something that hasn't gone quite as it should.  Again, I question whether the most effective way of giving this type of feedback is in the plenary, or whether a private email (and perhaps short meeting) with some members of the appropriate group might be a more effective way of getting the message across in a slightly less combative way, at least as a first effort.

On a positive note, I look forward to seeing everyone in Montreal, and for those attending the social event, I would be happy to debate the merits of direct vs polite criticism over a beer or two at the social event.

Kind regards,
Rob

-----Original Message-----
From: ietf <[email-masked]> On Behalf Of John C Klensin
Sent: 04 July 2019 16:28
To: [email-masked]
Subject: Clarity, evolving documents, living documents, the RSE, IETF Management Styles, and traffic volume on this list.

Hi.

I don't know if anyone else is having the same problem, but I'm having tremendous difficulty following all of the messages in this thread, much less the thread about the happenings surrounding the RFC Editor function.  That includes the side-thread about, e.g., whether one can describe, with facts and careful reasoning, an idea or action as stupid without insulting the intelligence of whomever was advocating it.  Many,
many messages.   If only to reduce the odds of re-inventing and
repeating what has been said before, I'm trying to read everything in a thread before I start a response but it means I am running hours, sometimes days, behind.  

It is especially problematic because I'm also trying to find time to get some technical work done that much of the IETF seems to think it important as long as _someone else_ puts in the time to understand the issues and do the work [1].  I've has it suggested to me off list that maybe most of those participating actively in the discussions either don't have technical work to do or have too much time on their hands.  I'd rather think that priorities (and levels of support for IETF work) are different, but I do suggest, as Randy Bush has pointed out a couple of times, maybe it would be better if people took a deep breath, let some of these discussions sink in, and actual get some technical work done, including trying to finish I-Ds before Monday's posting deadline and then actually reading enough of those drafts to make IETF 106 more productive.  

Or perhaps, if these topics are really important enough, we should call off the IETF 105 agenda and devote the meeting to
them, without conflicts and one day per topic.   I understand
the logistical problems with trying to do that in one large room but perhaps smaller group parallel sessions and discussions on the same subject (with people allocated at random or when rooms fill up) and then summaries of conclusions would work even better.

In any event, it is probably worth noting and remembering that these very long threads on the IETF list get participation from a very small fraction of the IETF community.  Whether others tune out after looking at one message or a dozen, we aren't hearing from them.  Because these topics have effects on the entire IETF, how it works, and how it is perceived, real IETF consensus --across the entire spectrum of IETF contributors and other materially affected parties-- is important if decisions are to be made.  It is difficult and perhaps impossible to draw inferences about consensus across the IETF when only a handful (or two) of people are participating.  It is even closer to impossible if those people participating in a way that discourages others from getting involved, however unintentional (and respectful) the mechanisms that are being used.

A more substantive note or two follow, but I believe the above is important and it saddens me that it didn't come from someone
in "the leadership".   

    john

[1] I'm not trying to be cryptic, just to keep the above short.
The very high-level summary of the topic area might be best
described as a set of questions.   Do any of you, or your
organizations, colleagues, or customers, use a language  that cannot be written properly in ASCII characters and even occasionally want to write it properly while using the Internet?
(Note that list of languages includes English.)  Would you, or your organizations, colleagues, or customers find it helpful to be able to use identifiers, user names, or other descriptors or mnemonics that are not tied to English?  Do you think it is important that everyone have confidence that strings that should compare equal do so and that ones that shouldn't, don't?  Do you understand that the problems the other questions imply are hard and require either significant digging in or suggest that the
IETF should publicly give up and accept the consequences?   If
your answer to any of those questions is "yes", why are you not digging in personally and why are you allowing the IESG to essentially ignore the topics, not answer questions about how to progress documents, and to set up mechanisms and then neglect them sufficiently to ensure failure?

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 4 Jul 2019 15:37:30 -0700

> managers, I expect) that there might be incompatible changes.

Yes, I think this is a fair point. It's also notable that these were also
protocols that
were "optional", either in the sense that you could downwardly negotiate to
other
versions (TLS, HTTP) or fall back to another protocol (QUIC). So that meant
that
tight coordination was less relevant.

> where I could not get a fix in ID-07 because "it's been deployed".

I agree that this is bad. Everybody involved in these protocols understood
that
"it's deployed" wasn't going to be a very strong argument.

-Ekr

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 4 Jul 2019 15:40:59 -0700

> work as TLS 1.3 did? Maybe we're just better at this now? I doubt it.

I actually do think we are a bit better. In particular, the idea of having
"interop versions" that we all converge on and using protocol versions
on the wire that corresponds to the draft version seems to be key.
This may have been done in other protocols but at least from my
perspective, it took a while to get those idioms right.

> but to what extent is flexibility wrt the publication process enabling

Yeah, I don't think enormously. This worked fine with the ID system.
As I said, what would be helpful for big protocols like TLS, QUIC, etc.
seems to me to be the ability to make "editorial" changes to the document
post-publication. I scare-quote editorial because it would also include
clarifying points that basically everyone agreed on but that could be
misinterpreted and would impede interop if there were multiple
interpretations.

-Ekr

---

From: Michael Richardson <[email-masked]>
Date: Fri, 05 Jul 2019 11:15:30 -0400

> by a big actor who decides it is no longer going to play ball... some

Thank you for bringing up additional places where we have had pain.

    > but to what extent is flexibility wrt the publication process enabling

My preference is the current (unfortunately slow) behaviour.

Among a tightly knit group (like QUIC), I don't think we need any formal
marker.  In the ANIMA WG, we have the GRASP protocol, in RFC-editor queue for
500+ days, because of MISREFs.  That's a pretty strong formal marker.

--
]               Never tell me the odds!                 | ipv6 mesh networks [
]   Michael Richardson, Sandelman Software Works        |    IoT architect   [
]     [email-masked]  http://www.sandelman.ca/        |   ruby on rails    [

---

From: Michael Richardson <[email-masked]>
Date: Fri, 05 Jul 2019 11:19:29 -0400

> misinterpreted and would impede interop if there were multiple

I agree with your goal.

Maybe being able to have RFC5288.02 or RFC8446.02 would make it easier to get
through the rather long post-WGLC.

While we are pretty good at getting cross-area review now, and getting
security review early, I feel that there is still too much ("last minute")
IESG meddling^Wpush back.  It's not that I don't appreciate much of what
gets fixed, I just wish it got fixed earlier in the process.

---

From: Nico Williams <[email-masked]>
Date: Fri, 5 Jul 2019 11:31:03 -0500

> gets fixed, I just wish it got fixed earlier in the process.

At Sun we had inception reviews, interim reviews, and commitment
reviews.  (I'm referring to architecture reviews.)

At IETF we have commitment reviews, naturally, but we only have
inception reviews for WGs and their initial work items, but no inception
reviews for subsequent work items, and no interim reviews.

The only way you'll get currently-late reviews to happen earlier is to
provide a mechanism for that.  That could be as simple as having a WG
chair request an interim review from an appropriate directorate.

Of course, we may not have the directorate cycles for interim reviews,
in which case we might need to improve the directorates.

Nico
--

---

From: Ted Lemon <[email-masked]>
Date: Fri, 5 Jul 2019 13:02:18 -0400

> reviews for subsequent work items, and no interim reviews.

Isn’t a call for adoption an inception review?   If not, it should be.

We actually kind of suck at commitment reviews, in a lot of cases: the work is done, after all, and everybody who thought it was a good idea at adoption time is tired of looking at it.   So it’s not uncommon (and I’ve seen it on documents I’ve finished) that the document is done and polished, and then there’s no response at all to the WGLC.

From a process perspective, I actually consider “adopted -> completed + no objection” to mean that there is WG consensus to publish, because otherwise this phenomenon can easily kill a completed document.   But this is also a process failure in the sense that commitment reviews are important.

The problem the IETF has is that unlike at your former job, there is no way to enforce this process other than by killing documents, and that often only impacts the people who are working on them and the people who would benefit from publication, and not all the people who thought it was a good idea at adoption time, but have wandered off during the course of the work.

---

From: Nico Williams <[email-masked]>
Date: Fri, 5 Jul 2019 12:28:35 -0500

> Isn’t a call for adoption an inception review?   If not, it should be.

I was referring to review from *outside* the WG.  Calls for adoption are
typically intra-WG only.

Reviews by a shepherd/AD, directorates, the IESG, and IETF (IETF LC) are
commitment reviews.

The WG spinup process is an inception review for the WG and its initial
work item slate.

We have no formal external-to-WGs interim review process.

I have seen ADs ask for interim reviews by experts outside a WG, so we
do have an informal interim review process.  And maybe that should be
good enough provided we engaged it *often* -- currently we don't.

> We actually kind of suck at commitment reviews, in a lot of cases: the
> work is done, after all, and everybody who thought it was a good idea
> at adoption time is tired of looking at it.   So it’s not uncommon
> (and I’ve seen it on documents I’ve finished) that the document is
> done and polished, and then there’s no response at all to the WGLC.

I'd say this is actually us being bad at interim reviews.  Our
commitment review process is very good, but very heavy-duty and thus
slow and painful.

By reserving all that pain for the very end of the process -precisely
when the WG participants are out of steam- the end result sucks.  The
problem isn't that we're bad at commitment reviews, rather the opposite
combined with the lack of interim reviews.  All the _surprises_ come
at the end, when authors/WGs least want them.

> [...]
>
> The problem the IETF has is that unlike at your former job, there is
> no way to enforce this process other than by killing documents, and
> that often only impacts the people who are working on them and the
> people who would benefit from publication, and not all the people who
> thought it was a good idea at adoption time, but have wandered off
> during the course of the work.

Ok, so let's fix that.

My take is that a) directorates and shepherds/ADs should be expected to
provide interim reviews as requested, b) WG chairs should request them
when the ether the chairs believe, or the WG has consensus that, "now is
a good time for interim review.

Interim reviews should also be announced on the [email-masked] list, but
review feedback should be on WG list.  This invites interim reviews from
the wider community as well.

Now, I don't know how to incentivize the wider community to provide
interim reviews.  Reviewing I-Ds is really time-consuming and energy
sapping for me personally -- the biggest problem is making the time and
finding a way to get it funded when it's a lot of time.

I suspect that the directorates are short on cycles too.  I get that
what I'm proposing may not be feasible at this time.

Nico
--

---

From: Ted Lemon <[email-masked]>
Date: Fri, 5 Jul 2019 13:45:36 -0400

On Jul 5, 2019, at 1:28 PM, Nico Williams <[email-masked]> wrote:
> Interim reviews should also be announced on the [email-masked] <mailto:[email-masked]> list, but

This is the essence of the problem, and you’ve proposed a solution that’s incompatible with it.   If it were possible to do your solution, I would agree with it.

I’m not sure there is a solution, but it would be interesting to see if there is something left once we throw out the “should be expected” idea.   Is there a way to do more interim reviews?   What is actually stopping us from doing this stuff?

A lot of times what stops me is that the document is badly written and difficult to read, and so it’s so much work to read it that I put it off until it’s too late, or give up.  It may be that it’s possible for us to get better at writing understandable and readable documents, and that this would have a greater positive impact on the process than shoulding at directorates.

What’s nice about getting better at writing understandable documents is that it’s the authors who would have to do this, and they are motivated to do what it takes to get more review (or if they aren’t, maybe it’s okay that the document died).

Do you see this as a useful thing to attempt, and if so, do you think it’s possible to do it?

---

From: Nico Williams <[email-masked]>
Date: Fri, 5 Jul 2019 13:00:37 -0500

On Fri, Jul 05, 2019 at 01:45:36PM -0400, Ted Lemon wrote:
> This is the essence of the problem, and you’ve proposed a solution
> that’s incompatible with it.   If it were possible to do your
> solution, I would agree with it.

So would I :(

> I’m not sure there is a solution, but it would be interesting to see
> if there is something left once we throw out the “should be expected”
> idea.   Is there a way to do more interim reviews?   What is actually
> stopping us from doing this stuff?

To do more interim reviews all we have to do is request them.  (And
teach WG chairs to do so.)

Lack of cycles is almost certainly what will be stopping us once we try.
A perception that we lack the cycles is almost certainly what is
implicitly stopping us from even thinking about interim reviews.

> A lot of times what stops me is that the document is badly written and
> difficult to read, and so it’s so much work to read it that I put it
> off until it’s too late, or give up.  It may be that it’s possible for
> us to get better at writing understandable and readable documents, and
> that this would have a greater positive impact on the process than
> shoulding at directorates.

"Too difficult to read your document" is a useful interim review!

"Don't come back until you've made this easier to review.  Here are some
example RFCs; look at the I-Ds that led to their publication."  -- also
useful.

> What’s nice about getting better at writing understandable documents
> is that it’s the authors who would have to do this, and they are
> motivated to do what it takes to get more review (or if they aren’t,
> maybe it’s okay that the document died).

We could use having classes about how to write I-Ds.

I'll note that the ITU-T writes beautiful specs.  I find the x.68x
(ASN.1) and x.69x (ASN.1 encoding rules) series to be brilliantly
written.  But that's an SDO that costs $$$$ to participate in, and uses
that money to produce quality results.

If ISOC spent more on the RSE function, perhaps the RSE could provide
early intervention services.  But any time we think of spending a lot
more $$$ on such functions we need to think about how we keep the IETF a
volunteer organization -- whence the funding??

> Do you see this as a useful thing to attempt, and if so, do you think

I'm guessing the biggest problem is that interim reviews == a bit of a
step function in reviewer cycles demand.

But we could do a pilot.  Identify a WG whose work could really use
interim reviews.  Arrange for its chairs to request them.  Run this
experiment for a while, then survey the authors, WG participants,
shepherds, interim reviews, IESG, and IETF -- was this better?  will it
scale?  Then we can think about how to scale it further than one WG.

Nico
--

---

From: Michael Richardson <[email-masked]>
Date: Fri, 05 Jul 2019 15:10:34 -0400

> in which case we might need to improve the directorates.

Actually we are pretty good about getting early reviews from, for instance,
sec-dir.  Chairs ask, and they happen, and I think that this has been good.
We also get food feedback from IANA, and we can get it early as well.

The problem is getting the majority of the IESG to put credence in these
reviews is part where I think we lose time.

---

From: Michael Richardson <[email-masked]>
Date: Fri, 05 Jul 2019 15:12:42 -0400

> at the end, when authors/WGs least want them.

Bingo.

Maybe we could always have the responsible AD do his/her review during WGLC
call time, and WG chairs would actually need to coordinate with the AD to
make sure that they have the time.

---

From: Michael Richardson <[email-masked]>
Date: Fri, 05 Jul 2019 15:13:58 -0400

> finding a way to get it funded when it's a lot of time.

This is why I wanted to have the list of structured reviews encoded in the
XML so that the datatracker and rfc-editor pages could at least provide a
kudos here.

---

From: Michael Richardson <[email-masked]>
Date: Fri, 05 Jul 2019 15:16:33 -0400

> shoulding at directorates.

Me too.
So I wind up trying to do grammar edits while reading the document, and I
really want it to turn into Google-Doc-like github pull requests.

I want to see the review process/software the EKR described last year.
I mean to bug him over beer to see it.
It seems rather involved, but maybe would work for me too. I don't know yet.

---

From: Nico Williams <[email-masked]>
Date: Fri, 5 Jul 2019 14:24:05 -0500

> XML so that the datatracker and rfc-editor pages could at least provide a

Yes, certainly better tooling for review formalism will help.

So if I am asked to review draft-ietf-foowg-xyz-11 and I can see that
there was an interim review at -07, I can start by reading that review,
then reading -11, then examining the differences between -07 and -11.

Moreover, recording the review history, complete with links to mailing
list archives (even if review were to happen in github or similar,
provided we have notifications go to an IETF list, then we can use that
as the record), in the datatracker, will greatly help anyone doing RFC
archeology later on.

Nico
--

---

From: Nico Williams <[email-masked]>
Date: Fri, 5 Jul 2019 14:26:57 -0500

> We also get food feedback from IANA, and we can get it early as well.

I didn't know IANA could be asked to review early...

I know we sometimes do early reviews in sec-dir, but not necessarily
often enough?

> reviews is part where I think we lose time.

Interesting.  Well, maybe that should change.  How much de-novo review
does IESG do?  Do most ADs fully read and review every I-D that comes
up, or do they mostly rely on the review record from the shepherd and
directorates?

Nico
--

---

From: Michael Richardson <[email-masked]>
Date: Fri, 05 Jul 2019 16:12:58 -0400

> I know we sometimes do early reviews in sec-dir, but not necessarily

It varies greatly, but I think that the sec-dir reviews are occuring much
earlier on average.  Maybe Tero has some stats, but my guess is that more
than 50% of documents get a sec-dir review before WGLC.

    > up, or do they mostly rely on the review record from the shepherd and

Sponsoring ADs seem to fully read.  I'd just like them to do so during WGLC,
I think.  Many times sponsoring ADs will know what other cross-area review
would be useful because they know what's going on elsewhere.

I think that other ADs are relying more and more on shepherd write-ups, and
are more and more splitting the load among the ADs (trusting each other).
But, we do see many documents that still get pretty heavy editing during IESG
Discuss.  This is often seriously dis-heartening to document authors, who
often have done a marathon of edits for WGLC two months earlier.
The time between beginning WGLC and exiting IESG is around 6 months is my
guess.

I think it was Bill Fenner that had a way to get stats?
I don't know if they are continuously captured, and if so, where.

---

From: Michael Richardson <[email-masked]>
Date: Fri, 05 Jul 2019 16:14:51 -0400

> then reading -11, then examining the differences between -07 and -11.

Gosh, I hadn't thought about recording a link to the review.
I just wanted to give kudos.  Recording a link to the review is a great idea,
and deals with some of the question of who gets a acknowledgement and who
doesn't.

+1

---

From: john heasley <[email-masked]>
Date: Fri, 5 Jul 2019 20:57:23 +0000

Thu, Jul 04, 2019 at 01:07:48PM -0400, Keith Moore:
> Because it could easily have the extremely undesirable effect of promoting
> premature deployment.
> 

Please re-read Snjider's email, to which you replied, without this
tangental notion of a "stable draft mark".

I feel that you and others have mis-read it.  Maybe I have, but I feel that
this stable draft idea is *not* what he is promoting; that was someone
else's idea and the two are being, but should not be, confused.

[thanks to the person who changed the subject but did not create a new thread]

---

From: "Theodore Ts'o" <[email-masked]>
Date: Sat, 6 Jul 2019 14:54:15 -0400

> else's idea and the two are being, but should not be, confused.

I don't think Snjider's problem statement is terribly difficult to
solve, and we have an answer already, which is an Informational RFC.
Note his problem goals/requirements statement:

  * Protocol specifications or extensions are a non-goal
  * Provide more context around RFCs, such as use case documents,
    implementation reports, security-related recommendation, etc.
  * With a short identifier

We have a large number of use case documents as Informational RFC's
already.  For example, RFC 7744, RFC 8151, RFC 6394, etc.  We also
have requirements documents that define the problem --- for example,
RFC 2820.  We have security-related recommendations as well, such as
RFC 1750, "Randomness Requirements for Security" (although that's now
a BCP, not just an Informational RFC).

I suspect people have been jumping off to something which is harder,
and perhaps for them, more interesting, which is signalling that a
particular I-D version is one that is worthy of being implemented, and
perhaps, deployed in a world where new implementations can be reliably
rolled out to a large percentage of the installed base in 2-3 months.
One answer is of course the experimental RFC, but the problem is that
a lot of people see RFC and immediately assume, it's a stable,
IETF-blessed standard documentation, regardless of the "Experimental"
tag on the top of every single page of said document.

But for Snjider's use case, where the document in question isn't a
specification in any way, but instead is a use case or requirement
document, implementation hints, etc., we don't have to worry about it
being it mistaken for a protocol specification.

Regards,

						- Ted

---

From: Eric Rescorla <[email-masked]>
Date: Sat, 6 Jul 2019 12:44:14 -0700

> tag on the top of every single page of said document.

An experimental RFC would not address the need I am talking about: we're
spinning one of these every 1-4 months, and doing WGLC, IETF-LC, and RFC
processing would cause far too much delay.

-Ekr

---

From: Warren Kumari <[email-masked]>
Date: Sun, 7 Jul 2019 05:04:17 -0700

> Overlapping with sessions is not great. Any reason this isn't in one of the morning slots (before 10am)?

Actually, I lied -- it is at:
When: Tue, July 23, 6:00pm – 7:30pm
Where: C2 (21st Floor) - Tuesday (map)
https://www.ietf.org/how/meetings/105/side-meetings/

I had just blindly copied it from the Side Meetings calendar (
https://www.ietf.org/how/meetings/105/side-meetings/ ), not realizing
that the calendar was in the wrong timezone (this is since been
fixed).
Whatever the case, this time is also not ideal, I'll try see if I can
find an available morning slot...

W

---

From: john heasley <[email-masked]>
Date: Mon, 8 Jul 2019 20:26:13 +0000

> processing would cause far too much delay.

exactly; neither experimental nor informational address the desire completely.

---

From: Keith Moore <[email-masked]>
Date: Mon, 8 Jul 2019 16:54:06 -0400

> exactly; neither experimental nor informational address the desire completely.

So what it sounds like you need is a link to an internet-draft but 
without the version number at the end, that always points to the current 
version of that Internet-draft.  And IMO the link should actually point 
to active content that allows the reader to easily query the revision 
history and diffs between changes, and recommendation status of the 
draft, instead of (merely) the plain text of the draft.   Perhaps the 
header portion of the active content should also include that link for 
easy referencing: "to obtain the current version of this internet-draft, 
visit 
https://tools.ietf.org/active-internet-drafts/draft-ietf-xxx-yyy.html".

Use the normal internet-draft submission mechanism to update such documents.

If we don't already have a page that does this, it doesn't seem like it 
would be terribly difficult to add.  If you really want to get fancy, 
splice the current status and links to revision history and diffs into 
the XML before rendering the XML.   But that might be overkill.

Seems like it should work just fine up until at least revision -99.

Of course one could always ask for more features.  But if it's worth 
doing, it seems like it's worth doing simply first.

Keith

---

From: Eric Rescorla <[email-masked]>
Date: Mon, 8 Jul 2019 14:06:05 -0700

On Mon, Jul 8, 2019 at 1:54 PM Keith Moore <[email-masked]>
wrote:

> version of that Internet-draft.

Hmm... Not quite [0]. As I said, the current I-D system works adequately
for the
purposes of test deployments. The point I was trying to make here was that
we regularly deploy on I-Ds, so a rule about how you shouldn't be deploying
on anything but an RFC wouldn't work for us.

>From my perspective, what is lacking is the ability to make minor updates
to the RFC to correct grammar errors, fix obvious errors, clarify
ambiguities,
etc. without re-spinning the RFC. Something like TLS quickly accumulates
a bunch of trivial errata of this kind that it's not worth re-spinning the
RFC
for, but would be useful to update the for readers. But as I said, that's
a different problem.

-Ekr

[0] Though this exists, e.g.,
https://tools.ietf.org/html/draft-ietf-quic-transport

---

From: Keith Moore <[email-masked]>
Date: Mon, 8 Jul 2019 17:22:34 -0400

On 7/8/19 5:06 PM, Eric Rescorla wrote:

> a bunch of trivial errata of this kind that it's not worth re-spinning 
> the RFC

I guess I could see having a page to an annotated version of an RFC with 
change bars, strikethroughs, etc. to indicate fixes to identified 
errata.   In other words, more or less the existing errata mechanism but 
with a better presentation.   But I'd want that page to show the 
provenance of each change (submitted date, submitted by, approval date, 
approved by, review comments) as footnotes or side-notes or whatever.   
My concern is that if it becomes too easy to update an RFC, this 
mechanism might be used to cause substantive changes that bypass the 
IETF Consensus process.   This seems especially likely to happen for 
specifications that endure for decades and people who lack memory of why 
certain decisions were made in the first place, exhibit action bias 
toward making changes.

I'd also be okay with having RFCs be updated this way in perpetuity, but 
only if nontrivial updates required the IETF Consensus process (or 
something like it) to approve the updates. I actually think that would 
probably be better in many cases than re-spinning the RFC.  Anytime an 
RFC is re-spun there's a temptation to rewrite old language, which can 
create unintended incompatibilities.   If changes are approved as just 
deltas, maybe this temptation could be minimized.

There would still be a need to unambiguously refer to a particular 
revision of an RFC.

Keith

---

From: Warren Kumari <[email-masked]>
Date: Mon, 8 Jul 2019 17:52:13 -0400

> version of that Internet-draft.

We already have that --
https://datatracker.ietf.org/doc/draft-ietf-mboned-ieee802-mcast-problems/

> And IMO the link should actually point
> history and diffs between changes,

We already have that --
https://datatracker.ietf.org/doc/draft-ietf-mboned-ieee802-mcast-problems/history/

> and recommendation status of the
> draft, instead of (merely) the plain text of the draft.

We had considered having a datatracker tag which could be attached to
a draft to do exactly this -- and decided that it would be unclear to
external people.

>  Perhaps the

Something like:
"The list of current Internet-
 Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress.""
?

> doing, it seems like it's worth doing simply first.

I'm really not sure if you were subtly trying to point out that we
already have all of this and so why are we proposing anything at
all...

What we were looking for is something a *little* bit more formal than
"just an ID", and *much less formal* (and also easier to update!) than
an RFC - basically an intermediate step to signal that a version has
more agreement than just what the authors believe.
If I'm an author of a WG document (draft-ietf-foo-bar-23) I publish a
new version with whatever *I* think the WG wants - but often it's
really hard to determine what exactly that is (conversations in email
quicky run off-topic, getting the *exact* wording that makes everyone
happy is tricky, working groups are fickle and change their minds,
determining consensus is difficult, etc).
What I was trying to do is be able to signal that version A of a draft
contains what the WG has (currently!) agreed to, while version B add
what the author(s) are proposing the next version should look like;
basically reasonable analogies are that version B is the "development
branch" or something similar to a Pull Request.

I had tried making an analogy to semantic versioning, with the MAJOR
version being fixed at 0 and A being something like Version 0.5.0 and
B is Version 0.5.4. Having the major being 0 means that anything can
change at any time (https://semver.org/#spec-item-4) -- but this
analogy implies more stability than I intend.
I'd point out that WGs can already do this -- for example, the foo WG
could use draft-ietf-foo-stable-bar, and draft-ietf-foo-devel-bar, or
make it well known that draft-ietf-foo-bar will always contain the WGs
agreed to changes, and that the "development" branch is at
www.github.com/<something>/draft-ietf-foo-bar, or ...

The (proposed) stable-ietf-foo-bar proposal is / was intended to be a
clear signal that the WG doesn't think that text is sane, not that it
is carved in stone.

W

---

From: Nico Williams <[email-masked]>
Date: Mon, 8 Jul 2019 17:33:52 -0500

On Mon, Jul 08, 2019 at 02:06:05PM -0700, Eric Rescorla wrote:
> Hmm... Not quite [0]. As I said, the current I-D system works
> adequately for the purposes of test deployments. The point I was
> trying to make here was that we regularly deploy on I-Ds, so a rule
> about how you shouldn't be deploying on anything but an RFC wouldn't
> work for us.
> 
> From my perspective, what is lacking is the ability to make minor
> updates to the RFC to correct grammar errors, fix obvious errors,
> clarify ambiguities, etc. without re-spinning the RFC. Something like
> TLS quickly accumulates a bunch of trivial errata of this kind that
> it's not worth re-spinning the RFC for, but would be useful to update
> the for readers. But as I said, that's a different problem.

It'd be nice to have something like a wikified "living RFC" where we can
keep track of proposed, accepted, and rejected edits kinda like we do
errata, but with a much better UI, and where we can keep track of
consensus status for each proposed edit, all such that at some point we
can cut a new RFC from that by pressing a button.

xml2rfc is great, but it lacks wiki-ness, though we could probably
develop HTML+JS tooling to give xml2rfc that missing wiki-ness.

This really does feel a bit like a tooling problem.  That if we had the
right tools, something like a cross between GitHub, Markdown, wiki,
xml2rfc, the datatracker, and the errata tracker, we'd have a much much
easier time re-spinning RFCs.

Things like Google docs are great at concurrent editing (which we
probably don't quite need for this, provided there is always someone
willing to be editor for a living RFC), but are no good for editing
RFCs.  I'd much rather have something like LyX[0] as a UI for editing
RFCs, but that's not (and never will be) webby so forget that.

Another thing is that I'm starting to have trouble keeping track of all
the RFC numbers in my head for all the RFCs I need to be aware of.  It'd
be very nice if we really did have RFC XXXX-YY where -YY is subsequent
versions.  E.g., instead of RFC 3280 and 5280 we should have RFCs 2459,
2459-01, 2459-02.  This would be especially useful for WGs that have
closed down, and also for RFCs like the PKIX ones that have lots of
extensibility and room for extensions to be supplied in other RFCs.

Also, think of the channel binding type IANA registry, which doesn't
require an RFC for each type, just a specification somewhere.  A lot of
what we do in the IETF doesn't really need a publication process as
heavy-duty as the RFC publication process.

None of the above addresses the need for I-D stability markers.  We're
identifying a lot of related issues and thinking up possible solutions.
Let's not lose track of the specific needs/problems we identify.

The TLS WG, besides needing something much better than errata and RFC
respins, also needs better I-D metadata for the protocol and
implementation development processes.

And it's not TLS WG alone either.

For example, we don't have a lot of energy in KITTEN WG, so we don't
really need better I-D metadata (at least not at this time), but better
RFC maintenance/respin processes and tooling would be a godsend.

Nico

[0] Surprise! after all, I wrote the now-unmaintained lyx2rfc for a
    reason...

---

From: Warren Kumari <[email-masked]>
Date: Mon, 8 Jul 2019 18:40:08 -0400

On Mon, Jul 8, 2019 at 5:52 PM Warren Kumari <[email-masked]> wrote:
[ SNIP]

> clear signal that the WG doesn't think that text is sane, not that it

<blush>
"clear signal that the WG doesn't think that text is *insane*".

Thanks to the kind soul who pointed that out off-list instead of on...: -)
W

---

From: Keith Moore <[email-masked]>
Date: Mon, 8 Jul 2019 19:03:00 -0400

> https://datatracker.ietf.org/doc/draft-ietf-mboned-ieee802-mcast-problems/
why, yes.
> https://datatracker.ietf.org/doc/draft-ietf-mboned-ieee802-mcast-problems/history/

Yes, though the "history" link could be made explicit on the former page.

> a draft to do exactly this -- and decided that it would be unclear to

To some degree that might depend on how the status is presented.

>     material or to cite them other than as "work in progress.""
Well, not quite.   I'm thinking if you have a living document that 
sometimes also gets printed out, it would be useful for that document to 
make obvious where to go to find the latest version _of that 
document_.   You could certainly get there via the list of 
internet-drafts, but it's a long list and it's not hard to provide 
something better.
> already have all of this and so why are we proposing anything at

More like, if we can describe what is wanted in terms of simple deltas 
to what we have already, it would be easier for us to understand what 
various people want.  That and if desirable, we could probably implement 
those deltas on an experimental basis fairly easily.

> more agreement than just what the authors believe.

I think I've seen at least three different things that people are 
looking for.   I think my concerns in most cases pretty much boil down 
to making sure that in each case appropriate process is followed and 
documented and transparent to readers, and that document versions aren't 
misleadingly labeled.   And if we're talking about changing existing 
processes, that those changes be appropriately reviewed and meet IETF 
Consensus.

> determining consensus is difficult, etc).

I think that's true for every version of an I-D.  It's what I think the 
WG wants or hope the WG will accept.   If the WG accepts it, it goes to 
IESG; if not, I iterate again.

If WGs want to say "we approve this document but we don't want to send 
it to IESG just yet" (or maybe not ever?), I think that's a process that 
doesn't exist yet, and perhaps should not exist. The danger is that WG 
approval will be taken as "good enough" without broader community 
review.   There are probably corner cases in which this would be ok, 
though - for which IESG could allow specific WGs to do this for specific 
documents that meet pre-established criteria.

> clear signal that the WG doesn't think that text is sane, not that it

I agree this could be done now for WGs using git* to maintain documents, 
as long as a WG doesn't claim that the master branch of the document has 
somehow been approved.   But it's a slippery slope for a WG to label a 
document as having been approved, since that would look like new process 
and bypassing wider review.

Anyway, it may be that what you want is more a matter of 
defining/approving process changes than developing new tools.

Keith

---

From: Keith Moore <[email-masked]>
Date: Mon, 8 Jul 2019 19:22:20 -0400

> develop HTML+JS tooling to give xml2rfc that missing wiki-ness.

Actually I'd love it if xml2rfc were phased out in favor of something 
better.   IMO it imposes a significant barrier to contributions, 
especially from newer IETF participants, but really from everybody.   
But I realize that it's hard for IETF to build and maintain real 
document editing tools that run on everybody's platforms.   It's hard 
enough to maintain xml2rfc.  And I could certainly imagine worse tools, 
like (gasp!) Word.

(I'm not exactly fond of wikis' UIs either.)

> heavy-duty as the RFC publication process.
Well, IANA exists for the case you're citing already.   (such things 
used to be published in RFCs)   What other cases do you have in mind?
> Let's not lose track of the specific needs/problems we identify.

Keeping explicit track of those things in one place seems like a good 
next step.

Keith

---

From: Benjamin Kaduk <[email-masked]>
Date: Mon, 8 Jul 2019 18:24:42 -0500

> with a better presentation.   But I'd want that page to show the 

Note that there's funding and, IIRC, a statement of work/RFP out for
something roughly like this -- an annotated version of RFCs with
(verified?) errata applied on top in a prominent way.

https://www.ietf.org/media/documents/RFC_Errata_Merge_Tool_RFP_04-08-19_Appended.pdf

-Ben

---

From: Benjamin Kaduk <[email-masked]>
Date: Mon, 8 Jul 2019 18:27:00 -0500

> and deals with some of the question of who gets a acknowledgement and who

The datatracker has been slowly getting better about these things; e.g., if
I look at https://datatracker.ietf.org/doc/draft-ietf-tram-turnbis/ there's
a "reviews" section with links to the requested directorate reviews, which
in turn have links to the email threads.  But of course this does not
integrate to non-directorate reviews (yet?), and requires the reviewer (or
directorate secretary) to make the review in the datatracker.

-Ben

---

From: Nico Williams <[email-masked]>
Date: Mon, 8 Jul 2019 18:34:39 -0500

On Mon, Jul 08, 2019 at 07:22:20PM -0400, Keith Moore wrote:
> better.   IMO it imposes a significant barrier to contributions, especially
> from newer IETF participants, but really from everybody.   But I realize
> that it's hard for IETF to build and maintain real document editing tools
> that run on everybody's platforms.   It's hard enough to maintain xml2rfc. 
> And I could certainly imagine worse tools, like (gasp!) Word.
> 

Well, I don't really care what it is, as long as a) we get the
typesetting right, b) we get the UI right.

Markdown seems incomplete.

XML with webby $EDITOR tooling would do.

> Well, IANA exists for the case you're citing already.   (such things used to
> be published in RFCs)   What other cases do you have in mind?

PKIX extensions (e.g., SANs).  Kerberos extensions.  TLS extensions.
SSHv2 extensions.  And so on and on.

> Keeping explicit track of those things in one place seems like a good next
> step.

Yes.  And I think we have enough material to justify a BoF.

---

From: Michael Richardson <[email-masked]>
Date: Mon, 08 Jul 2019 22:08:38 -0400

> https://www.ietf.org/media/documents/RFC_Errata_Merge_Tool_RFP_04-08-19_Appended.pdf

A two week submission time, and the work started May 13?
Did it get awarded?  I guess I can find the LLC minutes.

---

From: Alissa Cooper <[email-masked]>
Date: Tue, 9 Jul 2019 06:20:41 -0400

> On Jul 8, 2019, at 10:08 PM, Michael Richardson <[email-masked]> wrote:
> 
> 

Yes. https://github.com/ietf-tools/Rfc-Errata

I believe coding and testing are done and deployment is awaiting the RFC format transition next month.

Alissa

---

From: Christian Huitema <[email-masked]>
Date: Tue, 9 Jul 2019 21:01:11 -0700

> XML with webby $EDITOR tooling would do.

Tooling is just one of the problems with XML2RFC. The real issue is that
XML2RFC is completely specific to the IETF. This translate into training
requirements for people who need to actually use that markup language,
absence of easy to use tools because the user pool is too small to
sustain development, and then a reliance on translators between an
easy-to-edit format and the publication. For example, a team of authors
would be using markdown and Github, and using a tool chain to produce
XML2RFC. But if a copy editor suggests updates to the XML text, these
updates cannot easily imported to the original markdown document, or to
the markdown starter for the "bis" project.

I understand why we adopted an XML format 20 years ago. That was better
than NROFF, and there was a hope that the whole publishing industry
would standardize on XML. It did not, and now the IETF has its very own
markup language.

-- Christian Huitema

---

From: Nico Williams <[email-masked]>
Date: Tue, 9 Jul 2019 23:52:03 -0500

On Tue, Jul 09, 2019 at 09:01:11PM -0700, Christian Huitema wrote:
> Tooling is just one of the problems with XML2RFC. The real issue is
> that XML2RFC is completely specific to the IETF. This translate into
> training requirements for people who need to actually use that markup
> language, absence of easy to use tools because the user pool is too
> small to sustain development, and then a reliance on translators
> between an easy-to-edit format and the publication. For example, a
> team of authors would be using markdown and Github, and using a tool
> chain to produce XML2RFC. But if a copy editor suggests updates to the
> XML text, these updates cannot easily imported to the original
> markdown document, or to the markdown starter for the "bis" project.

Office and LibreOffice use XML too, but users don't see it.  That's what
I meant by "webby $EDITOR tooling" above: a bloody real UI, a browser UI.

> I understand why we adopted an XML format 20 years ago. That was
> better than NROFF, and there was a hope that the whole publishing
> industry would standardize on XML. It did not, and now the IETF has
> its very own markup language.

If I really understood TeX and LaTeX and LyX, I'd have produced a
template for LyX and use it directly, no XML in sight.  That's because
LyX is the best GUI editor I know for this sort of document.

As it is I still don't have XML in sight with lyx2rfc anyways: the XML
just a hidden step in the tooling[*].  Just like Word and friends.

Because I knew (and still know) very little about TeX, I wrote lyx2rfc
instead.  lyx2rfc exports LyX docs to XHTML and then applies an XSL to
convert to xml2rfc schema, then xml2rfc to render.  When I use lyx2rfc I
do not see raw XML ever, nor raw TeX for that matter.  It just works
(though it's been a while since I touched it, and it really needs an
update to the latest xml2rfc schema).  Which, along with *Office, proves
that we can have our XML cake and eat it too.

So why didn't I go with Office or LibreOffice?  LyX was very accessible,
and the learning curve for its XHTML export format much lower than for
the others.

If we'd gone with roff... then writing any tooling like lyx2rfc would
be much harder.  I shudder at the thought, and instead have to think
about developing an editor, or thank my lucky stars if someone else does
it first (think NroffEdit).  Extracting structure and semantics from
roff is much harder than XML, where no extraction is necessary.

The whole point of XML is that it's quite good for _documents_ (as
opposed to JSON, which is decidedly not) and has XSLT, so you can
convert between schemas.  lyx2rfc's XSLs, and Julian Reschke's XSLs,
prove that XSLT is a powerful tool, and that XML was the right choice
because of it.

But anyways, XML or whatever -- that should just be a detail most users
never need be aware of.  xml2rfc is just incomplete, that's all.

I had hoped that others would like lyx2rfc enough that I could get help
developing and maintaining it.  It's doubtful that that would happen
now, but I still think that LyX a viable approach to having a decent GUI
for authoring I-Ds and typesetting RFCs.  LyX is open source, after all,
and much smaller and more accessible than LibreOffice, and it is more
semantic too.  The only reason not to recommend LyX is that we really do
need a browser UI, and LyX ain't that.

No, IMO XML was by far the best choice.  XML was the right choice.

Nico
--

---

From: Nico Williams <[email-masked]>
Date: Wed, 10 Jul 2019 01:44:52 -0500

Off-list I'm told that there is at least one online, collaborative, BUI
LaTeX editors, and asked what it might take to use one.

My response: to use an online LaTeX editor we'd have to write LaTeX
templates for I-Ds and RFCs.

That would be *very* nice, actually, especially if there are good tools
for converting to XML even if we then no longer need xml2rfc.

Now, XML is super convenient because of XSLT, so all in all I'd rather
not abandon XML altogether.  I can see that there are TeX->XML tools,
but I don't know how good they might be -- if they're good enough,
switching to LaTeX might actually be a very good idea.

Nico
--

---

From: Ted Lemon <[email-masked]>
Date: Wed, 10 Jul 2019 02:50:39 -0400

Given that the RFC input format is XML, any editor that doesn’t operate on that XML is always going to put you at a disadvantage.  It would be nice if there were a standard format we could use, but I don’t know of one that would work.  Using markdown, or Latex, or whatever, just adds one more layer of translation you have to deal with later.   I occasionally try to use markdown, but keep reverting back to XML because it’s just one less step.

---

From: Julian Reschke <[email-masked]>
Date: Wed, 10 Jul 2019 10:14:47 +0200

> XML2RFC is completely specific to the IETF. This translate into training

It also has been optimized for the production of RFCs. Also note that
many changes in the v3 vocab just align the language with HTML (lists
and tables come to mind).

> the markdown starter for the "bis" project.

And the solution for that of course is to let the copy editor suggest
updates to the actual document source.

Best regards, Julian

---

From: Keith Moore <[email-masked]>
Date: Wed, 10 Jul 2019 06:09:00 -0400

> I meant by "webby $EDITOR tooling" above: a bloody real UI, a browser UI.

While that's probably better than editing raw XML, I'm unfavorably 
impressed with UIs for editing XML (and that includes UIs that edit HTML 
and variants thereof).   A possibly familiar example of what I'm talking 
about: you're editing a document that is internally represented by HTML 
or XML and trying to delete white space between two chunks of text that 
are at different levels of hierarchy.   All of a sudden you've "deleted 
too much" - the visual difference between those two chunks, that 
reflects the difference in the XML hierarchy, disappears.   You weren't 
trying to collapse the hierarchy, you were just trying to get rid of 
distracting and meaningless white space.   Or a similar problem - you 
/want/ extra white space, say, between items in a bulleted list, and the 
editor keeps trying to optimize out that white space because it sees it 
as superfluous.

At first glance one might assume that the problem is the editor 
implementation.   But you really can't fix it in a WYSIWYG editor 
specifically because it hides the underlying representation from the 
user.    And that means that there are circumstances in which "delete 
text at the cursor" is ambiguous, or potentially means multiple things, 
some of which are invisible.   The designer of the editor has a choice - 
does each delete remove something in the underlying representation (some 
of which may be invisible, so it looks to the user like the delete key 
is unreliable), or does each delete remove all of the differences in 
markup between the preceding and following chunk of text, or something 
in between? There's no good answer, especially because the order in 
which those invisible things get deleted from the underlying 
representation really matters and the user can't see the order. (of 
course it's not only XML-ish representations that have this problem, but 
XML-ish representations exacerbate it).

The fundamental problem is that XML is really a poor representation of 
text.  This is especially true for editing, but not just for editing.   
Text is not hierarchical.   How do you represent in XML a comment on a 
particular block of text that, say, overlaps multiple XML elements but 
doesn't completely contain all of them?   In a document which has been 
edited by multiple users, how do you represent in XML the changes made 
by each user?   I'm not saying that it absolutely cannot be done, but 
it's either going to be ugly or it's going to abandon many of the 
properties that made XML appear to be attractive in the first place.

>> its very own markup language.

In some sense, nroff really was better.   Probably not better overall, 
but at least nroff usually wouldn't throw up its hands and completely 
refuse to render a document (within an hour of a deadline) because you 
left out or misspelled some directive.   And in nroff you didn't have 
the UI problem.    I'm not arguing for a return to nroff.   XML is 
definitely more powerful in some ways, and XSLT is nice.  (I've written 
tools to convert nroff to other representations and it wasn't either 
easy or fun.)  But we went from one obscure and specialized text 
representation to another, and the newer representation is in some ways 
a poorer reflection of the text than the older one.

Anyway, if we're really going to try to improve our tools, we shouldn't 
naively assume that XML is the right direction for underlying 
representation.   Again, it can probably be made to work, but I suspect 
only by realizing that we simply can't force everything into a 
hierarchy.   So the XML would not be a natural representation for the 
text, it would only be a contrived representation that had to be 
converted back-and-forth between a better internal representation.  (And 
if you don't define that internal representation and the algorithm for 
conversion, each editing tool is going to do it differently, which 
creates another problem - importing the text into a tool, making any 
edit at all, and saving it will change the representation and likely how 
the text is displayed).

Keith

---

From: Randy Bush <[email-masked]>
Date: Wed, 10 Jul 2019 06:46:37 -0700

>> training ...

from an xml non-lover:

xml2rfc rocks!  it produces the baroqe, designed by committee, internet
draft format from input which i can easily produce in my text editor.

and the support, maintenance, and responsiveness of the tools team is
simply stunning.

[ and for the poster who wished for a gooey, there is one ]

thank you!

randy

---

From: Richard Barnes <[email-masked]>
Date: Wed, 10 Jul 2019 09:56:56 -0400

I'm glad it works for you.  The non-IETF-habituated new authors I've worked
with have found it mystifying.  Including everyone from junior engineers to
Fellows of major engineering organizations.

As Christian says, our continued attachment to bespoke tools is a barrier
to getting new work in the IETF, and thus detrimental to the long-term
health of the organization.

--RLB

---

From: Randy Bush <[email-masked]>
Date: Wed, 10 Jul 2019 07:01:16 -0700

> I'm glad it works for you.  The non-IETF-habituated new authors I've
> worked with have found it mystifying.  Including everyone from junior
> engineers to Fellows of major engineering organizations.
> 
> As Christian says, our continued attachment to bespoke tools is a
> barrier to getting new work in the IETF, and thus detrimental to the
> long-term health of the organization.

i have faith that the iab and ietf chair can fix this problem just as
they have fixed the rse problem.

randy

---

From: Eric Rescorla <[email-masked]>
Date: Wed, 10 Jul 2019 07:09:12 -0700

> engineers to Fellows of major engineering organizations.

Indeed. And even a large number of habituated authors have gravitated
towards markdown, despite the rather klunky workflow of MD -> XML -> TXT.

-Ekr

---

From: "Adrian Farrel" <[email-masked]>
Date: Wed, 10 Jul 2019 15:11:58 +0100

Is it really that hard?

Even crusty old idiots like me have worked out how to use an editor to make XML that is acceptable to XML2RFC. I doubt that I am clever or more talented than Fellows of major engineering organizations.

All tools (even github) require to be learned.

Replace tools with better tools, by all means.

But don’t make changes for personal preference: that way lies unending debates about whose preference is best.

Thanks,

Adrian

From: ietf <[email-masked]> On Behalf Of Richard Barnes
Sent: 10 July 2019 14:57
To: Randy Bush <[email-masked]>
Cc: Julian Reschke <[email-masked]>; IETF Rinse Repeat <[email-masked]>
Subject: Re: Things that used to be clear (was Re: Evolving Documents (nee "Living Documents") side meeting at IETF105.)

I'm glad it works for you.  The non-IETF-habituated new authors I've worked with have found it mystifying.  Including everyone from junior engineers to Fellows of major engineering organizations.

As Christian says, our continued attachment to bespoke tools is a barrier to getting new work in the IETF, and thus detrimental to the long-term health of the organization.

--RLB

from an xml non-lover:

thank you!

randy

---

From: Kathleen Moriarty <[email-masked]>
Date: Wed, 10 Jul 2019 10:17:07 -0400

On Wed, Jul 10, 2019 at 10:12 AM Adrian Farrel <[email-masked]> wrote:

> Is it really that hard?
>
> Even crusty old idiots like me have worked out how to use an editor to
> make XML that is acceptable to XML2RFC. I doubt that I am clever or more
> talented than Fellows of major engineering organizations.
>

Agreed, the language of the day will continue to evolve.  As soon as you
standardize on something, we are onto the next thing.  JSON was cool for it
seemed a year or two.  How long will Markdown and CBOR be preferred?

Since there's a way to use markdown and convert, the problem is solved.
Maybe tooling to make it more seamless if there is still an issue is a
better answer.

Thanks,
Kathleen

>
>
> Thanks,
>
> Adrian
>
>
>
> *From:* ietf <[email-masked]> *On Behalf Of *Richard Barnes
> *Sent:* 10 July 2019 14:57
> *To:* Randy Bush <[email-masked]>
> *Cc:* Julian Reschke <[email-masked]>; IETF Rinse Repeat <
> [email-masked]>
> *Subject:* Re: Things that used to be clear (was Re: Evolving Documents
> (nee "Living Documents") side meeting at IETF105.)
>
>
>

-- 

Best regards,
Kathleen

---

From: Stewart Bryant <[email-masked]>
Date: Wed, 10 Jul 2019 15:47:30 +0100

If you don't like XML, use Markdown.

There is not a lot more to remember than a # to start a section level
and four spaces for a figure.

The header template is a bit strange, but you crib that from an existing 
draft, then you need a few minor bits of # and {} for references and you 
know everything you need to know for 99% of the work.

- Stewart

On 10/07/2019 15:11, Adrian Farrel wrote:
> *From:*ietf <[email-masked]> *On Behalf Of *Richard Barnes
> *Cc:* Julian Reschke <[email-masked]>; IETF Rinse Repeat 
> <[email-masked]>
> On Wed, Jul 10, 2019 at 9:47 AM Randy Bush <[email-masked] 
> <mailto:[email-masked]>> wrote:
>

---

From: Leif Johansson <[email-masked]>
Date: Wed, 10 Jul 2019 16:53:14 +0200

> standardize on something, we are onto the next thing.  JSON was cool for
> it seemed a year or two.  How long will Markdown and CBOR be preferred?

Life is ultimately about revolting against your moms's [1] data
description language.

	Cheers Leif

[1] I am not related to Eve tho

---

From: Ted Lemon <[email-masked]>
Date: Wed, 10 Jul 2019 10:56:54 -0400

> I'm glad it works for you.  The non-IETF-habituated new authors I've worked with have found it mystifying.  Including everyone from junior engineers to Fellows of major engineering organizations.

What do they use instead?

---

From: Sarah B <[email-masked]>
Date: Wed, 10 Jul 2019 08:01:31 -0700

If we want to make the IETF approachable for all, then for me this statement from Stewart is true, and part of the problem. I'm not an engineer. I don't spend my days in code. Github is not part of my workflow. More over, as a newcomer, i think the idea of cribbing someone else's draft and using it as a basis is probably a non starter. I share Kathleen's musing, in that I wonder too what will be popular in <x> years from now. I'll take the unpopular stand that being able to write in (gasp) Word docs works perfectly for me - in fact, it's highly desirable. :)

</rathole>
Sarah

> On Jul 10, 2019, at 7:47 AM, Stewart Bryant <[email-masked]> wrote:
> 
> 
> The header template is a bit strange, but you crib that from an existing draft, then you need a few minor bits of # and {} for references and you know everything you need to know for 99% of the work.
> 
> - Stewart
> 
>> *Cc:* Julian Reschke <[email-masked]>; IETF Rinse Repeat <[email-masked]>
>> *Subject:* Re: Things that used to be clear (was Re: Evolving Documents (nee "Living Documents") side meeting at IETF105.)
>> On Wed, Jul 10, 2019 at 9:47 AM Randy Bush <[email-masked] <mailto:[email-masked]>> wrote:

---

From: Randy Bush <[email-masked]>
Date: Wed, 10 Jul 2019 08:04:28 -0700

> towards markdown, despite the rather klunky workflow of MD -> XML -> TXT.

i do latex, md, xml2rfc, and even text.  tool use is what differentiates
us from revived 40,000 year old nematodes.  i would be happier with the
md -> xml path if it did the boilerplate.  and there are lot of features
in xml2rfc to which i am attached, e.g. intra-doc xrefs.

when we submit a paper to some conference/journal, there is a latex
template, serious formatting rules, ...  we all have our rituals and
flavors of ice cream.

i, for one, am happy if good work gets done well elsewhere.  i care that
good work gets done.  and, to be tactless, a fair bit of what we produce
may not fit well in that set.  and the problem does not lie with he
tooling or document format.

i actually try to help (when asked) folk from other cultures get work
done in the ietf.  i think the barriers we place to entry are not so
much xml, which pretty much anyone who has a *technical* contribution
can deal with in short.  it is arrogance, arcane rituals, artificial
arguments, and alliteration.

randy

---

From: Richard Barnes <[email-masked]>
Date: Wed, 10 Jul 2019 11:05:36 -0400

> What do they use instead?

The web-based Markdown tool I wrote a little while ago [1] has a few
users.  The more motivated ones clone Martin Thomson's template repo [2]
and work from there in Markdown.

I'm guessing many more just don't contribute.

--Richard

[1] https://ipv.sx/draftr
[2] https://github.com/martinthomson/i-d-template

---

From: Ted Lemon <[email-masked]>
Date: Wed, 10 Jul 2019 11:06:42 -0400

> I'm guessing many more just don't contribute.

No, I mean what do they use that makes them think what we use is hard?

---

From: Richard Barnes <[email-masked]>
Date: Wed, 10 Jul 2019 11:11:45 -0400

> No, I mean what do they use that makes them think what we use is hard?

Well, by definition, they do not use the IETF's bespoke tools.

Think about what you use in your non-IETF life.  Word, Google Docs, maybe
LaTeX, ... A lot of engineers these days have already encountered Markdown
via things like GitHub's support for it in READMEs and comments [1],
ReadTheDocs [2], or Hugo [3].

--Richard

[1] https://help.github.com/en/articles/basic-writing-and-formatting-syntax
[2]
https://docs.readthedocs.io/en/stable/intro/getting-started-with-sphinx.html#using-markdown-with-sphinx
[3] https://gohugo.io/content-management/formats/

---

From: Randy Bush <[email-masked]>
Date: Wed, 10 Jul 2019 08:13:26 -0700

> I'm guessing many more just don't contribute.

we love to speak for the great unrepresnted

---

From: Richard Barnes <[email-masked]>
Date: Wed, 10 Jul 2019 11:17:02 -0400

You're absolutely correct that any tool comes with a learning curve.  The
trick is "bringing the mountain to Mohammed" as it were; using tools for
which many people have already climbed the curve.  By definition, anything
that is IETF-custom is brand-new learning curve for a newcomer.

--Richard

---

From: Leif Johansson <[email-masked]>
Date: Wed, 10 Jul 2019 17:20:46 +0200

On 2019-07-10 17:17, Richard Barnes wrote:
> You're absolutely correct that any tool comes with a learning curve. 
> The trick is "bringing the mountain to Mohammed" as it were; using tools
> for which many people have already climbed the curve.  By definition,
> anything that is IETF-custom is brand-new learning curve for a newcomer.

That will have us running up and down hills all the time.

---

From: "Andrew G. Malis" <[email-masked]>
Date: Wed, 10 Jul 2019 11:24:08 -0400

> towards markdown, despite the rather klunky workflow of MD -> XML -> TXT.

 Like Stewart says, learning Kramdown is very easy. And
https://xml2rfc.tools.ietf.org/experimental.html really simplifies the
toolchain as well - just use the topmost blue box to choose your input
file, and then select "kramdown" rather than "xm2rfc". This will directly
convert your Kramdown source to text, HTML, or PDF without an XML
intermediate step. You can have the result in a frame (for debugging your
file) or saved to a file.

Cheers,
Andy

---

From: Donald Eastlake <[email-masked]>
Date: Wed, 10 Jul 2019 12:26:03 -0400

>From what I see, lots of people use MS Word for their Internet Drafts.
Except for very rare cases where I'm making a small tweak to a draft
already in MS Word or XML, I have always used and continue to use
nroff. It's not that I think other tools are particularly hard, it's
just that about the time I think it might be worthwhile to switch, the
tool of the day changes.

Thanks,
Donald
=============================
 Donald E. Eastlake 3rd   +1-508-333-2270 (cell)
 1424 Pro Shop Court, Davenport, FL 33896 USA
 [email-masked]

---

From: Nico Williams <[email-masked]>
Date: Wed, 10 Jul 2019 11:31:31 -0500

On Wed, Jul 10, 2019 at 02:50:39AM -0400, Ted Lemon wrote:
> Given that the RFC input format is XML, any editor that doesn’t
> operate on that XML is always going to put you at a disadvantage.  It

lyx2rfc is proof to the contrary!  :)

Really, it produces XML in xml2rfc format.  If all you want is .unpg,
.txt, or .html format, you'll never see the XML.  But you get to submit
it (since you have to).

> would be nice if there were a standard format we could use, but I

XML is standard; I assume you meant "some other standard format".

I don't propose doing away with XML entirely, and perhaps not at all.
As I've said, I think XML is absolutely the right tool for this job.
What sucks is that XML is not a UI.

Even if a LaTeX collaborative editor could be used, I'd still expect to
use LaTeX->XML + XSLs to be able to programmatically extract metadata
and other tasks, and at least for some time to convert to xml2rfc.

> don’t know of one that would work.  Using markdown, or Latex, or
> whatever, just adds one more layer of translation you have to deal
> with later.   I occasionally try to use markdown, but keep reverting
> back to XML because it’s just one less step.

I agree as to markdown.   Markdown we might never be able to
programmatically extract metadata we need, and we wouldn't have a good
migration path for RFC production.  It's XML, or LaTeX that can be
converted to XML, or something similar.

Nico
--

---

From: Keith Moore <[email-masked]>
Date: Wed, 10 Jul 2019 12:52:16 -0400

On 7/10/19 12:31 PM, Nico Williams wrote:

> XML is standard;

Ah.  That explains why every time I try to write a new RFC, the required 
format has changed in an incompatible way.

Presuming that XML is good for something, I'm not actually sure what it 
is.   It's not good for user input.    It's not a good final-form 
document format.  It's not good as an intermediate format for WYSIWYG 
editors.   Apparently it's not good at being a long-term stable 
intermediate format either (though perhaps it could be if people just 
accepted that it's going to be ugly and resisted the temptation to make 
it prettier.)

I suppose it's good at being able to be translated into other formats 
that are equally bad or worse.  Progress?

Keith

---

From: Nico Williams <[email-masked]>
Date: Wed, 10 Jul 2019 11:55:03 -0500

On Wed, Jul 10, 2019 at 06:09:00AM -0400, Keith Moore wrote:
> > Office and LibreOffice use XML too, but users don't see it.  That's
> > what I meant by "webby $EDITOR tooling" above: a bloody real UI, a
> > browser UI.
> 
> impressed with UIs for editing XML (and that includes UIs that edit
> HTML and variants thereof).  [...]

When I use LyX to write I-Ds, I never see either the raw .lyx or the raw
.xml that is produced by lyx2rfc.  I do submit the XML lyx2rfc produces,
naturally, but I don't look at it.  Instead I review the rendered
output.

That's what a good $EDITOR for I-Ds and RFCs should be like to use.

>                      [...].   A possibly familiar example of what I'm
> talking about: you're editing a document that is internally
> represented by HTML or XML and trying to delete white space between
> two chunks of text that are at different levels of hierarchy.   All of
> a sudden you've "deleted too much" - the visual difference between
> those two chunks, that reflects the difference in the XML hierarchy,
> disappears.   You weren't trying to collapse the hierarchy, you were
> just trying to get rid of distracting and meaningless white space.  

I've never had this problem with LyX.  That's because it's NOT a WYSIWYG
editor by a WYSIWYM editor, and among other things does not let you add
arbitrary amounts of whitespace accidentally.

Maybe I should make it a point to attend a meeting and hackathon just to
showcase LyX and lyx2rfc live, so y'all believe me...  There's
screenshots in https://github.com/nicowilliams/lyx2rfc and you can find
videos of people using LyX with searches like: https://www.youtube.com/results?search_query=LyX

> Or a similar problem - you /want/ extra white space, say, between
> items in a bulleted list, and the editor keeps trying to optimize out
> that white space because it sees it as superfluous.

LyX keeps you from adding unnecessary whitespace accidentally, but will
let you add it if you really want to (IIRC it's <ctrl><space>).

> implementation.   But you really can't fix it in a WYSIWYG editor

That's right, you cannot fix this in a WYSIWYG editor.  WYSIWYG is not
the right tool for this job.

Good thing that LyX is a WYSIWYM editor.  Take a look at screenshots or
videos, or https://www.lyx.org/ and https://en.wikipedia.org/wiki/WYSIWYM...

>               [...]. (of course it's not only XML-ish representations
> that have this problem, but XML-ish representations exacerbate it).
> 
>   Text is not hierarchical.   How do you represent in XML a comment on

LyX does a pretty good job of exposing hierarchy without making the user
deal with TeX/LaTeX/XML/any other internal representation.  It can be
done, because it has been done.

> a particular block of text that, say, overlaps multiple XML elements
> but doesn't completely contain all of them?   [...]

Not sure how to do that in *any* document representation format.  In XML
you could encode the relations in element attributes, but while text can
be hierarchical, it's not _relational_ in a semantic way, so you can't
keep users from screwing up any such relations you choose to try to
represent by moving text around.  Still, I also don't understand why you
couldn't make a comment contain the text it comments on for this
particular use case, or why this particular use case is too important.

>                                      [...]?   In a document which has
> been edited by multiple users, how do you represent in XML the changes
> made by each user?   I'm not saying that it absolutely cannot be done,
> but it's either going to be ugly or it's going to abandon many of the

I'm not sure that I care about who made what changes in a collaborative
editor.  If I do care, then I'll prefer a merge-based workflow.  LyX
does have a mode for diff/merge, where you get to review every proposed
edit, and accept (then possibly further edit) or reject it, and LyX does
have version control support.

> > > industry would standardize on XML. It did not, and now the IETF
> > > has its very own markup language.
> 

Roff does not represent metadata in any way such that it can be
programmatically extracted, except by convention -- there's no "roff
schema".

> left out or misspelled some directive.   And in nroff you didn't have

roff can error out, or throw warnings and misrender in subtle ways that
you might not see if you miss the warnings.

roff is also a real pain on the eyes.

> definitely more powerful in some ways, and XSLT is nice.  (I've
> written tools to convert nroff to other representations and it wasn't
> either easy or fun.)  But we went from one obscure and specialized
> text representation to another, and the newer representation is in
> some ways a poorer reflection of the text than the older one.
> 
> Anyway, if we're really going to try to improve our tools, we
> shouldn't naively assume that XML is the right direction for
> underlying representation.   [...]

Perhaps, but any alternative needs to be possible to use in such a way
that we can convert to XML and preserve semantics, or else the
alternative has to be as powerful as XML (which is roughly saying the
same thing).

I would only consider LaTeX as mentioned earlier because a) there are
very good collaborative and non-collaborative web and non-web $EDITORs
for it, and b) it can be used in such a way that one can convert it to
XML while preserving semantics.

Nico
--

---

From: Michael Richardson <[email-masked]>
Date: Wed, 10 Jul 2019 12:55:16 -0400

+1

I expect engineers who want to contribute to how the Internet works to know
how to make a HTML page that can validate using the most basic of editor.
The web is now older than many university students.

Our XML format is no more arcane than HTML4, and there are easier ways
(markdown, word template, etc.), and people could also contribute to existing
documents and get help from existing authors. It's not like someone should
be doing base jumping on their first day here.

---

From: Nico Williams <[email-masked]>
Date: Wed, 10 Jul 2019 11:57:15 -0500

> [ and for the poster who wished for a gooey, there is one ]

I want to echo this.

Even I don't need a GUI editor for this.  I strongly prefer TUIs in
general.  But I'm fond of LyX, and since others have wanted a GUI...
thus lyx2rfc.

Nico
--

---

From: Nico Williams <[email-masked]>
Date: Wed, 10 Jul 2019 12:00:43 -0500

> format has changed in an incompatible way.

Schema and encoding are different things.

The xml2rfc schema is a standards work in progress.

> is.   It's not good for user input.    It's not a good final-form document
> format.  It's not good as an intermediate format for WYSIWYG editors.  

Can I get you to try LyX?  :)

Nico
--

---

From: Keith Moore <[email-masked]>
Date: Wed, 10 Jul 2019 13:04:19 -0400

> be doing base jumping on their first day here.

Clearly people who are capable of making useful technical contributions 
to Internet protocols are also capable of figuring out how to use 
xml2rfc, given enough extra time to sort through the varying and 
sometimes conflicting layers of documentation, tools, and error messages 
(or lack thereof).   My question is, is it really necessary that we 
require people to spend time on those things when they could instead be 
doing anything else?   Like more technical work, relaxing, or flossing 
their cats?

Or perhaps something like xml2rfc is a necessary evil, and it's within 
epsilon of an optimal format for our purposes.   I'd actually rather 
have a stable language and toolset for this than keep changing it.

Keith

---

From: Keith Moore <[email-masked]>
Date: Wed, 10 Jul 2019 13:09:44 -0400

On 7/10/19 1:00 PM, Nico Williams wrote:

> Can I get you to try LyX?:)

So you want me to learn a new obscure tool in order to avoid using an 
obscure tool that I'm somewhat familiar with?  :)

Convince me that I won't have to still know about xml2rfc because 
there's some important case that LyX doesn't handle, or doesn't 
translate properly.   That's the usual problem with adding another layer 
- not that the new layer isn't useful (sometimes it is) but the new 
layer typically doesn't remove the need to be knowledgeable about the 
underlying layers and deal with bugs that result from stacking the new 
layer on top.

(Translation: sure, I'm probably willing to try it at some point.   I'm 
still looking for the Holy Grail too.)

Keith

---

From: Eric Gray <[email-masked]>
Date: Wed, 10 Jul 2019 17:18:37 +0000

Richard,

A few things that people who suggest a complete replacement of tools should think about:

  1.  frequently introducing yet another learning curve doesn’t shorten the overall learning curve for anyone.  If there is any benefit in it for anyone, it is that it tends to level the playing field. I doubt there is any way to determine definitively whether that is a good thing, or a bad thing.  YMMV.
  2.  wholesale replacement of tools has a direct analog in many other areas.  Imagine for instance if, in trades (wood working, metal working, etc.), someone suggested obsoleting the tools used by experienced craftsmen (ignore any seeming gender specificity) in order to (theoretically) simplify the learning curve for apprentices.  The theory behind any such move would be that this would allow apprentices to more quickly become qualified tradesmen – but that theory would require time to validate and would likely severely impact skilled production in the meantime.
  3.  in this particular work (i.e. – standardization) – as with a number of other areas – “tools” is actually the area with the least significant learning curve.  A far more significant learning curve is associated with learning the many things that have been tried, and determined to be unworkable, over a time period far greater than the tools learning curve for even the most casual tools user.

I suspect that a prolonged discussion of the benefits of introducing new tools is misplaced effort.

--
Eric

From: ietf <[email-masked]> On Behalf Of Richard Barnes
Sent: Wednesday, July 10, 2019 11:17 AM
To: [email-masked]
Cc: Julian Reschke <[email-masked]>; IETF Rinse Repeat <[email-masked]>
Subject: Re: Things that used to be clear (was Re: Evolving Documents (nee "Living Documents") side meeting at IETF105.)

You're absolutely correct that any tool comes with a learning curve.  The trick is "bringing the mountain to Mohammed" as it were; using tools for which many people have already climbed the curve.  By definition, anything that is IETF-custom is brand-new learning curve for a newcomer.

--Richard

On Wed, Jul 10, 2019 at 10:12 AM Adrian Farrel <[email-masked]<mailto:[email-masked]>> wrote:
Is it really that hard?
Even crusty old idiots like me have worked out how to use an editor to make XML that is acceptable to XML2RFC. I doubt that I am clever or more talented than Fellows of major engineering organizations.
All tools (even github) require to be learned.

Thanks,
Adrian

From: ietf <[email-masked]<mailto:[email-masked]>> On Behalf Of Richard Barnes
Sent: 10 July 2019 14:57
To: Randy Bush <[email-masked]<mailto:[email-masked]>>
Cc: Julian Reschke <[email-masked]<mailto:[email-masked]>>; IETF Rinse Repeat <[email-masked]<mailto:[email-masked]>>
Subject: Re: Things that used to be clear (was Re: Evolving Documents (nee "Living Documents") side meeting at IETF105.)

--RLB

from an xml non-lover:

thank you!

randy

---

From: Michael StJohns <[email-masked]>
Date: Wed, 10 Jul 2019 13:24:40 -0400

On 7/10/2019 1:18 PM, Eric Gray wrote:
>  1. frequently introducing yet another learning curve doesn’t shorten
>     the overall learning curve for anyone.  If there is any benefit in
>     it for anyone, it is that it tends to level the playing field. I
>     doubt there is any way to determine definitively whether that is a
>     good thing, or a bad thing.  YMMV.
>  2. wholesale replacement of tools has a direct analog in many other
>     areas.  Imagine for instance if, in trades (wood working, metal
>     working, etc.), someone suggested obsoleting the tools used by
>     experienced craftsmen (ignore any seeming gender specificity) in
>     order to (theoretically) simplify the learning curve for
>     apprentices.  The theory behind any such move would be that this
>     would allow apprentices to more quickly become qualified tradesmen
>     – but that theory would require time to validate and would likely
>     severely impact skilled production in the meantime.
>  3. in this particular work (i.e. – standardization) – as with a
>     number of other areas – “tools” is actually the area with the
>     least significant learning curve.  A far more significant learning
>     curve is associated with learning the many things that have been
>     tried, and determined to be unworkable, over a time period far
>     greater than the tools learning curve for even the most casual
>     tools user.
>

*applause from a long time Emacs user*

Mike

> --
>
> Eric
>
> *From:* ietf <[email-masked]> *On Behalf Of * Richard Barnes
> *Sent:* Wednesday, July 10, 2019 11:17 AM
> *To:* [email-masked]
> *Subject:* Re: Things that used to be clear (was Re: Evolving 
> Documents (nee "Living Documents") side meeting at IETF105.)
>
> The trick is "bringing the mountain to Mohammed" as it were; using 
> tools for which many people have already climbed the curve.  By 
> definition, anything that is IETF-custom is brand-new learning curve 
> for a newcomer.
>
> --Richard
>
>     Even crusty old idiots like me have worked out how to use an
>     editor to make XML that is acceptable to XML2RFC. I doubt that I
>     am clever or more talented than Fellows of major engineering
>     organizations.
>
>     But don’t make changes for personal preference: that way lies
>     unending debates about whose preference is best.
>
>     Thanks,
>
>     Adrian
>
>     *From:* ietf <[email-masked]
>     <mailto:[email-masked]>> *On Behalf Of *Richard Barnes
>     *To:* Randy Bush <[email-masked] <mailto:[email-masked]>>
>     *Cc:* Julian Reschke <[email-masked]
>     <mailto:[email-masked]>>; IETF Rinse Repeat <[email-masked]
>     <mailto:[email-masked]>>
>     *Subject:* Re: Things that used to be clear (was Re: Evolving
>     Documents (nee "Living Documents") side meeting at IETF105.)
>
>     I'm glad it works for you.  The non-IETF-habituated new authors
>     I've worked with have found it mystifying.  Including everyone
>     from junior engineers to Fellows of major engineering organizations.
>
>     barrier to getting new work in the IETF, and thus detrimental to
>     the long-term health of the organization.
>
>     --RLB
>

---

From: Melinda Shore <[email-masked]>
Date: Wed, 10 Jul 2019 10:09:28 -0800

I'm really not at all convinced that our tooling is, by itself,
presenting any kind of meaningful barrier to participation.  People
learn how to use new tools all the time, and at any rate if we
have any goals here I think it's to find ways to see that people
have the tools they need and want, not to take away options.

That said, I do think that our tooling tends to reinforce the
view that we're ossified, slow, inflexible, and behind the times,
and I suspect that contributes to us being somewhat unattractive
to potential new participants.  I.e. it's cultural rather than
technical.

Melinda

-- 
Melinda Shore
[email-masked]

Software longa, hardware brevis

---

From: Nico Williams <[email-masked]>
Date: Wed, 10 Jul 2019 13:14:15 -0500

On Wed, Jul 10, 2019 at 01:09:44PM -0400, Keith Moore wrote:
> Convince me that I won't have to still know about xml2rfc because there's
> [...]

Just LyX.  I've no idea if lyx2rfc still works -- it's been unmaintained
for some years, but if you like the GUI and would use lyx2rfc, then I
might maintain it again.

> still looking for the Holy Grail too.)

WYIWYM is that, for me anyways.

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Wed, 10 Jul 2019 14:32:30 -0400

I agree with much of this discussion but I have come to a very different
perspective:

* Internet Standard status means nothing more than the fact that the legacy
deployment is sufficiently large that further development of the
specification is no longer feasible.

* Internet Standard status is not necessarily a desirable condition.

* Internet Standard status will be reached regardless of whether the
documents are good or not.

One of the sad parts of the development of the Web is that HTTP became a
standard in early 1993. By the time we had a team able to start getting the
protocols into shape, legacy deployment was already constraining
development. If people recall, Simon Spero proposed an HTTP-NG that is
remarkably close to QUIC given that over twenty years separate them. We
couldn't deploy because there was too much legacy.

We need to stop being so dogmatic about IETF process. It is not like it is
working for us now or has worked well in the past. I propose a different
approach

1) Keep the two standards process levels but drop the notion that Internet
Standard means the work is finished and will never change. Specifications
that are being used by real people will always need some level of
maintenance. Maybe not a great deal but some. Add an additional status
LEGACY to describe the specs that are in widespread use, have been
superseded but are not going to go away for decades if ever. IPv4 would be
a good candidate for LEGACY as it will always be with us just as the 6502
microprocessor will always be with us in standard cell. Even if that is
after 0.0.0.0/0 has been declared (reserved for private use).

2) Recognize the fact that HTTP/1.1, TLS/1.2, PKIX etc are now Internet
Standards. They may not be perfect, the implementations may not be fully
interoperable but anyone who wants to make a Web browser (for example) is
going to have to support those specs for a considerable time to come.
Declaring TLS/1.2 an Internet Standard does not mean ending development of
TLS/1.3, it is merely recognizing the situation.

3) Establish a maintenance WG in each area that is the place for continuing
incremental development of standards. The Security area has LAMPS. I think
that instead of the 'do the work, shut the WG' model applying to every WG
in IETF, the maintenance groups should act as standing committees for very
limited updates.

I fully sympathize with John Klensin's point about making changes to a spec
after deployment. One of the reasons I have not been looking to get people
using the Mesh until now is that I knew that the minute I started
attracting users, that would eliminate my scope to make fundamental changes.

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Wed, 10 Jul 2019 14:40:32 -0400

If people want document editing tools, may I recommend my RFCTool?

It is written in C#, runs on Linux, OSX and Windows and is MIT License.

It accepts as input formats: xml2rfc, markdown, Word, HTML
It generates as output formats: xml2rfc, markdown, Word, HTML, TXT

It converts citations of the form <norm="RFC322"> to the proper reference
automatically and builds tables of figures, contents and normative language.

The main limitation is that it is designed to target the new formats and so
I haven't made it a priority of late till the tooling for those is ready. I
wanted to be able to make use of diagrams and math notation in my docs:

http://mathmesh.com/Documents/draft-hallambaker-mesh-architecture.html

If it was to be taken further, the main change I would make is to change
the markdown format to use the same tags that GitHub uses as this has
emerged as the standard.

The code is here:

https://github.com/hallambaker/PHB-Build-Tools

On Mon, Jul 8, 2019 at 7:40 PM Nico Williams <[email-masked]> wrote:

> especially
> used to
> next

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Wed, 10 Jul 2019 14:43:30 -0400

My tool takes in Markdown and spits out XML2RFC.

It also takes in Microsoft Word .docx format which is supported by almost
every editor as an output format.

Most of my docs are a combination of text written in Word so that the spell
and grammar checker can be used and examples that are generated in
markdown.

---

From: Kathleen Moriarty <[email-masked]>
Date: Wed, 10 Jul 2019 14:44:30 -0400

Building on Melinda's comments, I recently helped a couple of new authors.
I pointed them to the XML template and the plug-in to XMLmind.  They
produced a draft rather quickly with very few questions.  They had not
worked much with XML, but had enough experience with other format
languages.  They made use of the plug-in features and used it differently
than I do.  I switch back to see the XML and fix things.  They were just
excited to generate a draft in which they felt they were contributing.

-- 

Best regards,
Kathleen

---

From: Keith Moore <[email-masked]>
Date: Wed, 10 Jul 2019 14:50:18 -0400

On 7/10/19 2:32 PM, Phillip Hallam-Baker wrote:

> I agree with much of this discussion but I have come to a very 
> different perspective:
>
> * Internet Standard status means nothing more than the fact that the 
> legacy deployment is sufficiently large that further development of 
> the specification is no longer feasible.

Without arguing about the above, I would be okay with a single level + 
periodic review (say, once every 2 years) resulting in an applicability 
note on the RFC Editor's page for that RFC.   The applicability note 
would note interoperability issues, deployment level, identify known 
significant bugs and technical omissions, and make deployment 
recommendations.   (whether suitable for widespread deployment or not, 
perhaps only in specific corner cases, perhaps legacy use only, perhaps 
a recommendation to discontinue use or transition to a different protocol).

If we did things that way then maybe also task the WG that produced the 
RFC with doing interop tests (or arranging for them) within 2 years of 
publication.   We desperately need the feedback loop.

> One of the sad parts of the development of the Web is that HTTP became 
> a standard in early 1993. By the time we had a team able to start 
> getting the protocols into shape, legacy deployment was already 
> constraining development. If people recall, Simon Spero proposed an 
> HTTP-NG that is remarkably close to QUIC given that over twenty years 
> separate them. We couldn't deploy because there was too much legacy.

He wasn't the only one who saw the need.   What I've come to realize is 
that conditions change over time.   Sometimes those conditions 
facilitate improving or transitioning away from legacy protocols, 
sometimes they don't.   The trick is to recognize when conditions 
facilitate useful change and take advantage of that.

> We need to stop being so dogmatic about IETF process. It is not like 
> it is working for us now or has worked well in the past. I propose a 
> different approach
>
> 1) Keep the two standards process levels but drop the notion that 
> Internet Standard means the work is finished and will never change. 
> Specifications that are being used by real people will always need 
> some level of maintenance. Maybe not a great deal but some. Add an 
> additional status LEGACY to describe the specs that are in widespread 
> use, have been superseded but are not going to go away for decades if 
> ever. IPv4 would be a good candidate for LEGACY as it will always be 
> with us just as the 6502 microprocessor will always be with us in 
> standard cell. Even if that is after 0.0.0.0/0 <http://0.0.0.0/0> has 
> been declared (reserved for private use).
>
> 2) Recognize the fact that HTTP/1.1, TLS/1.2, PKIX etc are now 
> Internet Standards. They may not be perfect, the implementations may 
> not be fully interoperable but anyone who wants to make a Web browser 
> (for example) is going to have to support those specs for a 
> considerable time to come. Declaring TLS/1.2 an Internet Standard does 
> not mean ending development of TLS/1.3, it is merely recognizing the 
> situation.

I would not support that, but I would support writing an applicability 
statement about each of them saying essentially what you've said above - 
they have these known problems but implementing them in browsers is for 
the time being a practical necessity.

> 3) Establish a maintenance WG in each area that is the place for 
> continuing incremental development of standards. The Security area has 
> LAMPS. I think that instead of the 'do the work, shut the WG' model 
> applying to every WG in IETF, the maintenance groups should act as 
> standing committees for very limited updates.

That strikes me as an invitation to disaster unless somehow groups can 
be made to avoid constantly doing new work of dubious value, and also 
avoid mission creep.    We are producing too many RFCs of dubious value 
as it is, and IESG is already taxed reviewing the drafts.

Keith

---

From: Nico Williams <[email-masked]>
Date: Wed, 10 Jul 2019 13:52:49 -0500

> specification is no longer feasible.

s/further development/further backwards-incompatible development/

> * Internet Standard status is not necessarily a desirable condition.

PS is enough for the vast majority of participants.

The thought of "when shall we move RFX xxxx to Standard?" never keeps me
awake at night.  It probably never keeps anyone else up at night.

That fact however might keep some of us awake at night :)

> documents are good or not.

Indeed.  Respinning an RFC is hard work.  Respinning _and_ rewriting an
RFC is prohibitively expensive.

> Standard means the work is finished and will never change. [...]

That notion never existed!  The thing that is unchanging is the contents
of an RFC.  STD designations can (and they do) move to new RFCs that
obsolete the old ones.

> in IETF, the maintenance groups should act as standing committees for very

We are able to maintain RFCs from concluded WGs (e.g., SECSH).  But I
agree that having a WG per-area specifically for mainenance would be a
better way to do that.

> attracting users, that would eliminate my scope to make fundamental changes.

s/making changes/making backwards-incompatible changes/

---

From: Nico Williams <[email-masked]>
Date: Wed, 10 Jul 2019 14:05:08 -0500

> excited to generate a draft in which they felt they were contributing.

Yeah, it does seem lik the XML hate is mostly just that, but that our
use of XML is not really a blocker.  But these are feelings we're
talking about -- all subjective.

Better than talking about the RSE situation though.

Nico
--

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Wed, 10 Jul 2019 15:18:25 -0400

> Better than talking about the RSE situation though.

It is not the fact that XML is used, it is the way XML is used.

SGML is a horrible specification with a lot of stuff that is just terrible.
But it does have some good parts and there is a logic to some (not all) of
the baroque complexity.

The XML2RFC DTD was written by someone who didn't have much familiarity
with HTML or XML. It puts text in places where text should not go and it
uses a completely different set of tags to achieve what was already
established.

It doesn't much matter because XML2RFC is not an editing format, it is
merely the input format to the editing tools. It is easy to convert HTML,
markdown or word to that format.

I have absolutely no problem using XML as a document format. It is what it
is good for. I have a LOT of problems trying to use XML as a data format
and not for want of trying. I wrote some of the first XML protocol specs.
The problem using XML for data is that it gives you six ways to encode a
piece of data when you really want there to be one data model and limit the
encoding choices to the absolute bare minimum.

---

From: "[email-masked]" <[email-masked]>
Date: Thu, 11 Jul 2019 02:45:14 +0000 (UTC)

Sarah

See RFC 5385, J. Touch 
Version 2.0 Microsoft Word Template for Creating Internet Drafts and RFCs

I've used the Word template for writing what became an RFC. I later found XML easier to collaborate on with better group revision control, for diagramming, and better for complex documents.

Most technical documents (and code) start by editing down another document to get a framework to work in and expand...

L.

at least it's not LaTeX....

Lloyd Wood [email-masked] http://about.me/lloydwood

On Thursday, 11 July 2019, 01:02:45 GMT+10, Sarah B <[email-masked]> wrote: 

</rathole>
Sarah

>>    >> Tooling is just one of the problems with XML2RFC. The real issue is
>>    >> that XML2RFC is completely specific to the IETF. This translate into
>>    >> training ...
>>    > It also has been optimized for the production of RFCs. Also note that
>>    > many changes in the v3 vocab just align the language with HTML (lists
>>    > and tables come to mind).
>>    from an xml non-lover:

---

From: Julian Reschke <[email-masked]>
Date: Thu, 11 Jul 2019 10:32:53 +0200

> format has changed in an incompatible way.

Example?

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 11 Jul 2019 10:37:02 +0200

On 10.07.2019 21:18, Phillip Hallam-Baker wrote:
>
>
> On Wed, Jul 10, 2019 at 3:05 PM Nico Williams <[email-masked]

Right. That, and how we're introducing V3 without having a definition
for it.

> SGML is a horrible specification with a lot of stuff that is just
> terrible. But it does have some good parts and there is a logic to some
> (not all) of the baroque complexity.
>

Yes, it has some of these problems. V3 improves things.

Best regards, Julian

---

From: Keith Moore <[email-masked]>
Date: Thu, 11 Jul 2019 05:24:17 -0400

>> format has changed in an incompatible way.

It has been long enough that I don't remember the details.   Of course 
it wasn't XML itself that had changed, it was the DTD and/or the 
requirements for boilerplate and/or the magic incantations required to 
get references to work.

It's a bit like saying that SMTP is a standard, but if you actually want 
to be able to send outgoing mail without it getting blocked by random 
spam filters there is a set of magic and arbitrary things that you have 
to do to appease the mail gods.   I suspect part of it involves parts of 
a walrus and must be done at midnight on a full moon.

Keith

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 11 Jul 2019 11:33:27 +0200

> converted to XML, or something similar.

Of course, having a common submission format after the authoring step is a requirement, and XML does that job reasonably well.  It is also the format from which I do my extractions.  XML has well-understood tools for data modeling (e.g., Relax-NG) that come in handy for defining that submission format.

Using a more human-centric authoring language (markdown, LaTex if that rocks your boat) and then translating to XML does introduce the occasional error condition where you actually have to look at the XML to figure it out.  Still, the majority of the work can be done in a significantly more humane environment.  That is a net win for most people, and it begins to really shine for complex documents and complex collaborations.

On the comments about stability:  For a decade, RFCXMLv2 has had the advantage that it was relatively stable (defined by a single implementation that for most of that period wasn’t hacked much).  That advantage is not available for RFCXMLv3.  Authoring formats such as markdown might be instrumental in bridging the period of instability.

Grüße, Carsten

---

From: Julian Reschke <[email-masked]>
Date: Thu, 11 Jul 2019 11:30:19 +0200

> It has been long enough that I don't remember the details.   Of course

So I guess it's not as dramatic as it sounded...

> requirements for boilerplate and/or the magic incantations required to

As far as I recall, there have been zero incompatible DTD changes (up to
v3 which indeed will change things).

Boilerplate changes are caused by IAB requirements, not the tool.
Looking at
<https://greenbytes.de/tech/webdav/rfc7749.html#attribute-ipr>, the last
author-visible changes (that affected the XML source) happened 10 years
ago...

Best regards, Julian

---

From: Keith Moore <[email-masked]>
Date: Thu, 11 Jul 2019 05:46:11 -0400

> So I guess it's not as dramatic as it sounded...

I suppose next time I need to record in detail the problems that crop up 
as I'm frantically trying to get a draft in before the deadline... 
though this might cause me to miss the deadline, so I probably won't.

But it's definitely been more recent than in the past 10 years. (of 
course it's possible that before that I had been using some sort of 
compatibility hack.)

Something I've observed my entire career is that people don't like 
changing tools that already work for them.   Even if the new tool is 
much better, though usually it isn't.

Keith

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 11 Jul 2019 12:07:28 +0200

> v3 which indeed will change things).

Which by itself is great, but there are many compatibility aspects not captured by the DTD.
Some instabilities came with the transition from the v1 TCL tool to the v2 Python tool (which also actually started to enforce more of that DTD).
Also, the bibxml infrastructure that we use to support reference generation has had some surprises (I know, you prefer to do that work manually, but the server-side tool is arguably part of the format).

But I agree, the RFCXMLv1/v2 format has been managed quite well over the decades, even through a complete change of tooling and ownership.

Grüße, Carsten

---

From: Julian Reschke <[email-masked]>
Date: Thu, 11 Jul 2019 12:11:47 +0200

> Some instabilities came with the transition from the v1 TCL tool to the v2 Python tool (which also actually started to enforce more of that DTD).

Ah, that is true. The TCL code even accepted non-wellformed XML. That
never affected me as author, because my documents always passed through
a true XML parser before getting passed to the formatter. But I can see
how this caused breakage back then.

> Also, the bibxml infrastructure that we use to support reference generation has had some surprises (I know, you prefer to do that work manually, but the server-side tool is arguably part of the for…

Yes.

+1

Best regards, Julian

---

From: Stewart Bryant <[email-masked]>
Date: Thu, 11 Jul 2019 11:52:29 +0100

On 10/07/2019 19:44, Kathleen Moriarty wrote:
> and the plug-in to XMLmind.

Is that still available licence free?

- Stewart

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Thu, 11 Jul 2019 09:41:09 -0400

On Wed, Jul 10, 2019 at 2:51 AM Ted Lemon <[email-masked]> wrote:

> Given that the RFC input format is XML, any editor that doesn’t operate on
> that XML is always going to put you at a disadvantage.  It would be nice if
> there were a standard format we could use, but I don’t know of one that
> would work.  Using markdown, or Latex, or whatever, just adds one more
> layer of translation you have to deal with later.   I occasionally try to
> use markdown, but keep reverting back to XML because it’s just one less
> step.
>

The XML2RFC markup requires me to make edits in three different places to
do references. Using Word as my source editor, I only put the citation
where I want it to go and the converter does all the rest.

Since I have a form of dyslexia, I rely on Word to check spelling. It is
also useful to have the grammar checking where Microsoft doesn't get it
wrong. Their team seems to be under the misapprehension that Skunk and
Blight isn't stupid and ignorant. At any rate, passive voice is often what
is wanted in technical documentation.

But the bigger point here surely, is that this thread is now an editor war.
People have different preferences in their editors. That is only a problem
when folk who insist on using 40 year old or 30 year old tooling because
they refuse to learn better try to foist their stuff on other people.

I agree with Keith that XML editors suck. I have used tons of them and they
all seem to be suckky. Generating *a* form of XML is one thing. Generating
XML to arbitrary schemas quite another. I suspect it is because there are
actual constraints in text formatting and XML is more expressive.

I haven't found a really good HTML editor either.

---

From: Keith Moore <[email-masked]>
Date: Thu, 11 Jul 2019 09:44:13 -0400

> But the bigger point here surely, is that this thread is now an editor 

oops, fair point.   Guess I'm going back to editing I-Ds with Gosling 
Emacs.  For VMS.  Because that's what I was using in 1980.

:)

Keith

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 11 Jul 2019 15:46:37 +0200

On Jul 11, 2019, at 15:41, Phillip Hallam-Baker <[email-masked]> wrote:
> 
> At any rate, passive voice is often what is wanted in technical documentation.

WHO WANTS THAT?

(SCNR.)

Grüße, Carsten

---

From: Mary B <[email-masked]>
Date: Thu, 11 Jul 2019 09:13:50 -0500

I agree on the XML. And, there are tools that allow you to edit WYSIWG.  I
personally like working with the XML.  It takes me back to the IBM green
screen days 30 years ago and editing docs in GML.

Mary.

---

From: Julian Reschke <[email-masked]>
Date: Thu, 11 Jul 2019 16:25:32 +0200

On 11.07.2019 15:41, Phillip Hallam-Baker wrote:
>
>
> On Wed, Jul 10, 2019 at 2:51 AM Ted Lemon <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>     operate on that XML is always going to put you at a disadvantage.
>     It would be nice if there were a standard format we could use, but I
> The XML2RFC markup requires me to make edits in three different places
> to do references. Using Word as my source editor, I only put the

No, it does not. Two places.

The insertion of the second one could indeed be automated, but then you
would have to give a hint about normative vs informative.

> citation where I want it to go and the converter does all the rest.
>
> Blight isn't stupid and ignorant. At any rate, passive voice is often
> what is wanted in technical documentation.
> ...

I occasionally use Word for that (generate HTML, copy, paste into Word).

Best regards, Julian

---

From: Warren Kumari <[email-masked]>
Date: Thu, 11 Jul 2019 10:34:58 -0400

> find an available morning slot...

... and I found a morning slot -- the original time was squished
between the end of sessions and the start of the social; I had wildly
underestimated how much interest and discussion this would generate,
and that slot was only suitable if we were expecting a small number of
people...

It is now:
When: Thu, July 25, 8:30am – 9:45am
Where: Duluth - Thursday (map)
Description: Evolving Documents

Sorry for the inconvenience...

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Thu, 11 Jul 2019 10:55:47 -0400

Thinking through comments made by John Klensin on the other thread, I came
to a similar conclusion. Warren may not have the right mechanism but I
think that it is in principle right that a Working Group should decide the
degree of stability of their spec as they work.

There is precedent for this, if you look at the QUIC group they are doing
effectively that. It is always good to work from actual practice.

When writing a spec, there are two separate sets of milestones

1) Is the technology fixed?
2) Is the description of the technology fixed?

The first has to precede the second of course. And there are degrees of
done-ness. But in most cases there are clear waterfalls that the WG goes
over where it is understood that a decision has been taken and won't be
un-made unless there is a really really good reason.

Sometimes those decisions are made before a spec gets to IETF. If there are
10,000 users of a spec and it is using XML, it is going to be a slog
persuading people to switch to JSON because it is the flavor of the month.
If there are a million legacy users, the change probably ain't going to
happen.

Questions of this type should (and often are) negotiated during WG
formation.

If the spec has more than a certain complexity, there is usually a point
where the WG decides that the functionality and technology are essentially
decided and from this point on the work will be basically word smithing.
There can be a lot of that left to do of course. But the point is that
there is usually a decision that from that point on, the WG is not going to
make any breaking changes without considering the interim deployments.

Again, these are shades of gray, not bright lines.

It might well be the case that we could use some more formal notice of when
these points are crossed. It probably isn't a good idea to make this
require IESG action but it might be something that is recognized as a
milestone.

There are exceptions where this doesn't work of course but those are
usually when you are faced with trying to extend a legacy platform that
isn't designed to extend in the way you want and so there are inevitable
conflicts in the spec. CAA presented a choice between two different
approaches to handling CNAME records that met different sets of use cases
and you have to choose one or the other.

On Wed, Jul 3, 2019 at 4:26 PM Warren Kumari <[email-masked]> wrote:

> I don't think the execution is relevant when it was obviously a bad idea
> in the first place.

---

From: "Andrew G. Malis" <[email-masked]>
Date: Thu, 11 Jul 2019 11:07:22 -0400

Kramdown makes it very easy. All you need is to put, for example,
{{!RFC2119}} for a normative reference or {{?RFC6241}} for an informative
reference in the right place in the text. Everything else is automatic.
I-Ds are also just as easy, for
example {{!I-D.ietf-detnet-data-plane-framework}}.

Cheers,
Andy

---

From: Ted Lemon <[email-masked]>
Date: Thu, 11 Jul 2019 11:09:15 -0400

> would have to give a hint about normative vs informative.

That would actually be a really nice enhancement.   It could be the rule that if no reference says, it’s normative, if all references either don’t say or say normative, it’s normative, if all don’t say or say informative, it’s informative, and if some say normative and some say informative, it’s informative.   The idea being that some references to the same document may be normative and some informative, and it makes sense to be able to signal that in the XML.  Maybe every reference should say, but I’m sure that would be a pain.

It would also be nice if the publication date could default to “today” and if the draft number could be determined automatically based on what is most recent…

---

From: Julian Reschke <[email-masked]>
Date: Thu, 11 Jul 2019 17:20:23 +0200

On 11.07.2019 17:09, Ted Lemon wrote:
> On Jul 11, 2019, at 10:25 AM, Julian Reschke <[email-masked]
> <mailto:[email-masked]>> wrote:
> That would actually be a really nice enhancement.   It could be the rule
> that if no reference says, it’s normative, if all references either
> don’t say or say normative, it’s normative, if all don’t say or say
> informative, it’s informative, and if some say normative and some say
> informative, it’s informative.   The idea being that some references to

If one says "normative", it's normative, no?

> the same document may be normative and some informative, and it makes
> sense to be able to signal that in the XML.  Maybe every reference
> should say, but I’m sure that would be a pain.

Lots can be done with more metadata :-).

> It would also be nice if the publication date could default to “today”
> and if the draft number could be determined automatically based on what
> is most recent…

It already does default to "today", unless I'm missing something.

Calculating the draft number *could* be done at submission time; I
wouldn't want to do it always, because then, if the last was "01", the
generated one will say "02" - even if it never gets submitted.

Best regards, Julian

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 11 Jul 2019 17:27:07 +0200

> On Jul 11, 2019, at 17:09, Ted Lemon <[email-masked]> wrote:
> 

    <date />

or in kramdown-rfc, just leave off the date: member, and kramdown-rfc will insert the date of conversion into the XML.

> and if the draft number could be determined automatically based on what is most recent…

I think, more generally, there should be some support for the -latest convention (e.g., in idnits, or in xml2rfc that is currently spewing a warning).  I think it is a good thing to have an explicit action for actually spending a number.
(But the number of times I got that specific number wrong, only to be reminded by idnits, is amazing.)

Grüße, Carsten

---

From: Ted Lemon <[email-masked]>
Date: Thu, 11 Jul 2019 11:30:25 -0400

> If one says "normative", it's normative, no?

In terms of process, yes.   But I might refer to it informatively and normatively in the same document, and that could be marked properly in the XML if we cared.   I don’t think it’s very important, but if you add the ability to mark a reference normative, you either have to allow this or forbid it, and if you allow it, I think this is the right way to treat it.   I’d prefer to allow it—I think it’s useful.  The downside is that it requires more care, and that would be a good reason to forbid it. :)

> It already does default to "today", unless I'm missing something.

Oh cool.   Either I didn’t think to just leave the date tag empty, as Carsten suggests, or this was added at some point after I started using xml2rfc.   Of course, if the XML is normative, isn’t it the case that the published XML has to have a date tag that contains the publication date?

> generated one will say "02" - even if it never gets submitted.

Sure, but that’s effectively what happens anyway, isn’t it?   The difference is that I might generate several -01s after -01 is published, or I might generate several -02s before -02 is published, depending on my habit.   This would eliminate that variability.   It’s always possible to generate a -02 that’s not the published -02, so I think making this automatic doesn’ t make the situation any worse, and does increase usability.

---

From: Julian Reschke <[email-masked]>
Date: Thu, 11 Jul 2019 17:34:42 +0200

Yes.

"When one or more of "year", "month", or "day" are left out, the
processor will attempt to use the current system date if the attributes
that are present are consistent with that date." --
<https://greenbytes.de/tech/webdav/rfc7749.html#rfc.section.2.13.p.3>

And in V3 it is actually optional, see
<https://greenbytes.de/tech/webdav/rfc7991.html#element.front>.

> I think, more generally, there should be some support for the -latest convention (e.g., in idnits, or in xml2rfc that is currently spewing a warning). (...)

+1000

FWIW:

"Furthermore, it is good practice to disambiguate current editor copies
from submitted drafts (for instance, by replacing the draft number with
the string "latest")." --
<https://greenbytes.de/tech/webdav/rfc7749.html#rfc.section.2.33.3.p.4>

It's very annoying that xml2rfc complains about something the spec calls
"good practice".

 > ..

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 11 Jul 2019 17:57:23 +0200

> Oh cool.   Either I didn’t think to just leave the date tag empty, as Carsten suggests, or this was added at some point after I started using xml2rfc.   Of course, if the XML is normative, isn’t it…

In RFCs, yes. Elsewhere? Would need to be decided.

> Sure, but that’s effectively what happens anyway, isn’t it?   The difference is that I might generate several -01s after -01 is published, or I might generate several -02s before -02 is published,…

I do not disagree; it's just a question when this should happen.

Best regards, Julian

---

From: Randy Bush <[email-masked]>
Date: Thu, 11 Jul 2019 09:27:25 -0700

i wish we had a mailing list for discussing xml2rfc arcana. :)

fwiw, i am trying a few other tools.  the problem is that a bespoke
toolchain, which is well maintained by competent engineers who are also
users, is gonna do the job better than a repurposed general purpose
tool.

when the xml haters' replacement tools actually get significant
mindshare, maybe they can move to the high road.  though attack
marketing seems to be popular these years, at least in the states.

the apocrypha is that sendmail.cf was not meant for human consumption or
editing.  it was to be a compiler intermediate.

randy, who klutzes through xml2rfc with emacs 26

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 11 Jul 2019 09:59:41 -0700

> marketing seems to be popular these years, at least in the states.

What would you consider "significant mindshare"? I just synced the ID
directory. Out of 1177 drafts that had XML uploaded, 203 seem to have been
generated by kramdown-2629

-Ekr

---

From: Nico Williams <[email-masked]>
Date: Thu, 11 Jul 2019 12:07:39 -0500

On Thu, Jul 11, 2019 at 11:33:27AM +0200, Carsten Bormann wrote:
> Using a more human-centric authoring language (markdown, LaTex if that
> rocks your boat) and then translating to XML does introduce the
> occasional error condition where you actually have to look at the XML
> [...]

Just to clarify: TeX very much does not please me, except for the fact
that there are good BUI collaborative editors for it.  My preference is
very much for XML, but with a UI that hides it as well as LyX hides TeX
from the user.

> On the comments about stability:  For a decade, RFCXMLv2 has had the
> advantage that it was relatively stable (defined by a single
> implementation that for most of that period wasn’t hacked much).  That
> advantage is not available for RFCXMLv3.  Authoring formats such as
> markdown might be instrumental in bridging the period of instability.

v3 will have that advantage in a few years :)

Nico
--

---

From: Nico Williams <[email-masked]>
Date: Thu, 11 Jul 2019 12:12:43 -0500

> would have to give a hint about normative vs informative.

That would be nice, actually, though if you have to xrefs to the same
document, which two xrefs diagree as to normativity you'd have a
problem, but whatever.

> I occasionally use Word for that (generate HTML, copy, paste into Word).

Ah, yes, spell checking is one reason I like to use LyX: it has spell
checking functionality (including continuous spell-checking).

But I don't have to copy/paste around -- I just use LyX then lyx2rfc
(directly from File->Export->rfc).

Nico
--

---

From: Nico Williams <[email-masked]>
Date: Thu, 11 Jul 2019 12:14:49 -0500

On Thu, Jul 11, 2019 at 11:09:15AM -0400, Ted Lemon wrote:
> That would actually be a really nice enhancement.   It could be the
> rule that if no reference says, it’s normative, if all references
> either don’t say or say normative, it’s normative, if all don’t say or
> say informative, it’s informative, and if some say normative and some
> say informative, it’s informative.   The idea being that some
> references to the same document may be normative and some informative,
> and it makes sense to be able to signal that in the XML.  Maybe every
> reference should say, but I’m sure that would be a pain.
> 

+1 to all that.

s/"today"/"time of rendering"/ (and if you only submit XML, then "time
of rendering" will be closely related to "time of submission", which is
very much what I usually want).

---

From: Nico Williams <[email-masked]>
Date: Thu, 11 Jul 2019 12:18:44 -0500

On Thu, Jul 11, 2019 at 05:27:07PM +0200, Carsten Bormann wrote:
> I think, more generally, there should be some support for the -latest
> convention (e.g., in idnits, or in xml2rfc that is currently spewing a
> warning).  I think it is a good thing to have an explicit action for
> actually spending a number.  (But the number of times I got that
> specific number wrong, only to be reminded by idnits, is amazing.)

-next would be nice -- all we need is a bibxml reference for -current
and then the renderer can lookup what the previous -XY was and just + 1.

---

From: Nico Williams <[email-masked]>
Date: Thu, 11 Jul 2019 12:24:42 -0500

> But the bigger point here surely, is that this thread is now an editor war.

Sigh.  That was not my intention, but the opposite.  My point was/is
that the choice of $EDITOR shouldn't matter, and that our choice of XML
as the canonical or submission format should not -and DO NOT- preclude
$EDITORs that hide the XML from the user.

---

From: Keith Moore <[email-masked]>
Date: Thu, 11 Jul 2019 13:32:07 -0400

Sent from my iPhone

> On Jul 11, 2019, at 1:24 PM, Nico Williams <[email-masked]> wrote:
> 

Actually the one thing that xml seems like it might be a decent choice for, is a stable back end interface that could accept input from a variety of editors and text processors and allow RFCs and I-Ds to be handled by a common toolset.  Except that it’s not stable.  

(It’s not a particularly good interface for that purpose, but it doesn’t have to be - the main requirements are that it be adequate and stable.)

Keith

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Thu, 11 Jul 2019 13:59:25 -0400

> example {{!I-D.ietf-detnet-data-plane-framework}}.

That is exactly how I used to do it in Markdown. I changed to a more
XML-ish markup because I was editing a lot of JSON and you really want your
editing markup to use a different set of braces to your code.

Double braces could be the answer though because it is not legal C, JSON or
any other encoding that comes to mind.

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Thu, 11 Jul 2019 14:16:30 -0400

On Thu, Jul 11, 2019 at 11:09 AM Ted Lemon <[email-masked]> wrote:

> that if no reference says, it’s normative, if all references either don’t
> say or say normative, it’s normative, if all don’t say or say informative,
> it’s informative, and if some say normative and some say informative, it’s
> informative.   The idea being that some references to the same document may
> be normative and some informative, and it makes sense to be able to signal
> that in the XML.  Maybe every reference should say, but I’m sure that would
> be a pain.
>
> It would also be nice if the publication date could default to “today” and
> if the draft number could be determined automatically based on what is most
> recent…
>

RFCTool does the following when converting documents.

1) References of the form <info="RFC822"> and <norm="RFC32"> are converted
to the appropriate references. Same for drafts.

2) Defaults to today's date

3) works out the correct revision number so as soon as
draft-hallambaker-mesh-architecture-09 appeared in the registry, my tools
started generating architecture-10

4) MUST, MAY, SHOULD are recognized as normative language in the text and
can be used to build an optional table of normative language.
Alternatively, these could be rendered in bold.

5) Underlined text is rendered as fixed spaced font. This is because very
few documents need underlining but most have references to identifiers.

What I don't bother to do is to implement all the curlicues on lists like
starting at a particular item. This could be added but I have never found a
need for it. And references to Word section headings seem to have broken
recently... ooops.

The thing I would most like to add is the ability to identify words as the
point at which defined terms are defined and used and use that to build an
index. Easy enough to define the code to build the index, less easy to work
out a convenient way to enter the information in a Word document.

I think it would be very helpful to agree on a set of markup tags. What I
use currently is essentially re-purposing the xml2rfc tags. So this is the
pre-amble in one of my files:

<series>draft-hallambaker-mesh-cryptography
<status>informational
<stream>independent
<ipr>trust200902
<author>Phillip Hallam-Baker
    <surname>Hallam-Baker
    <initials>P. M.
    <firstname>Phillip
    <email>[email-masked]
<also>http://mathmesh.com/Documents/draft-hallambaker-mesh-cryptography.html

The reason I need also is that my documents depend on features in the HTML
output like superscripts and subscripts.

The only other XML style markup I use most of the time is to include
figures and examples:

<figuresvg="../Images/Cogen.svg">Two party key pair generation.
<include=..\Examples\ExamplesAdvancedCoGeneration.md>
And yes, this isn't legit XML because legit xml is rather a bore. why type
<a href="http..."> when the tag always takes at least one attribute and it
is usually the only attribute?

I could happily agree to move to {{a="http..."}}

There is an open source MIT license library for interacting with Word docx
format and it works pretty well. I have not tried using OpenOffice but I
strip out 90% of the markup as irrelevant in any case.

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Thu, 11 Jul 2019 14:33:25 -0400

The interface to the IETF document process is exactly what I use xml2rfc
for.

If I am producing a -bis document, I pull the .xml from wherever I can find
it and convert it to word. Then I convert it back to upload it.

But since we are discussing all this. How about we specify the rest of the
interface. I would really like it if I could make the document submission
from RFCtool.

So normally, I type
rfctool %DocSource%\hallambaker-mesh-5-protocol.docx  /auto /cache=bib.xml

It would be nice to be able to submit a changed document with something like
rfctool %DocSource%\hallambaker-mesh-5-protocol.docx  /auto /cache=bib.xml
/submit

There is a small corner case which is that if you have multiple documents
and you are submitting updates to three out of five, you want the new docs
to reference the new versions of the updates documents but the old versions
of the ones that are not being submitted.

So it is probably something a bit more git-like and there is a /stage
operation to mark the draft as one of the ones to be updated and a /submit
operation to ship them out.

[I would also prefer to be able to authenticate the submission by signing
it with a digital signature rather than the email callback scheme. Which
might sound like feature creep but could actually simplify providing the
capability as it would allow a stateless submission. The XML would have to
be expanded to specify the signature key(s) allowed.]

On Thu, Jul 11, 2019 at 1:32 PM Keith Moore <[email-masked]>
wrote:

> Actually the one thing that xml seems like it might be a decent choice
> for, is a stable back end interface that could accept input from a variety
> of editors and text processors and allow RFCs and I-Ds to be handled by a
> common toolset.  Except that it’s not stable.
>
> (It’s not a particularly good interface for that purpose, but it doesn’t
> have to be - the main requirements are that it be adequate and stable.)
>
> Keith
>
>
>

---

From: Joe Touch <[email-masked]>
Date: Thu, 11 Jul 2019 13:18:57 -0700

> On Jul 11, 2019, at 11:33 AM, Phillip Hallam-Baker <[email-masked]> wrote:
> 
> If I am producing a -bis document, I pull the .xml from wherever I can find it and convert it to word. Then I convert it back to upload it.

It’s 2019.  It’d be nice if it wasn’t just me supporting Word for RFCs. The output could still print/dump to XML with IETF-specific tags. 

Joe

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Thu, 11 Jul 2019 20:10:21 -0400

On Thu, Jul 11, 2019 at 4:19 PM Joe Touch <[email-masked]> wrote:

> > If I am producing a -bis document, I pull the .xml from wherever I can
> find it and convert it to word. Then I convert it back to upload it.
>
> It’s 2019.  It’d be nice if it wasn’t just me supporting Word for RFCs.
> The output could still print/dump to XML with IETF-specific tags.
>
> Joe
>

All my code is on GitHub and MIT License. It supports use of Word to
generate RFCs in TXT or HTML.

The only downside right now is that it has been on hiatus for 18 months
while I wait for a final set of IETF tooling to interface to.

I plan to set it up so that it can read HTML and recover the structure so
documents should round trip.

The basic concept is that if you have four people editing a document they
will want at least four editing tools to be supported. So it should be
possible for Alice to write in word, hand it to Bob who converts it to
markdown and so on.

---

From: "[email-masked]" <[email-masked]>
Date: Fri, 12 Jul 2019 06:49:56 +0000 (UTC)

Joe,

"It’d be nice if it wasn’t just me supporting Word for RFCs."?

As you know, I recently submitted an erratum on the Word template RFC
that the IETF itself should provide more support for Word for RFCs by
hosting and maintaining the template etc., rather than referencing
a dead url. As a reminder, see:
https://www.rfc-editor.org/errata/eid5778

You said, in advocating rejecting that erratum, that

"Appendix A provides the detailed information and procedure needed to create the .dot file from the Normal.dot included with Word. 

The version provided online [now at your personal website] was as a convenience and is not critical to the use of this RFC"

i.e. in providing a non-essential non-critical convenience at a non-referenced url, you're not actually supporting Word for RFCs.

So, you're rejecting an erratum that points out the need for more IETF support for the Word template,
while also wishing for more support for the Word template but saying that any Word
user can just build their own template from scratch in an RFC they'll never think to find
without needing any other support whatsoever?

That's... quite something. Probably not very inclusive, though.

L.

Joe

---

From: Joe Touch <[email-masked]>
Date: Fri, 12 Jul 2019 00:31:51 -0700

> On Jul 11, 2019, at 11:49 PM, "[email-masked]" <[email-masked]> wrote:
> 

I’m more than happy to have the template and the pst-processing file copied to the IETF tools page rather than just linked there. I offered that when the RFC was published and would still be glad to do so.

My issue with the errata is that it presumes that URLs in RFCs ever need to be updated or corrected. They do not and should not. Search engines are more than sufficient to address the issue of URL ephemerality - which, FWIW, happens as much (if not more) at large public sites (such as ietf.org and Isi.edu) as privately run sites (such as mine at strayalpha.com).

I apologize if the two issues were not clear in my previous response.

Joe

---

From: Ted Lemon <[email-masked]>
Date: Fri, 12 Jul 2019 09:17:52 -0400

On Jul 12, 2019, at 3:31 AM, Joe Touch <[email-masked]> wrote:
> My issue with the errata is that it presumes that URLs in RFCs ever need to be updated or corrected. They do not and should not. Search engines are more than sufficient to address the issue of URL ephemerality - which, FWIW, happens as much (if not more) at large public sites (such as ietf.org <http://ietf.org/> and Isi.edu <http://isi.edu/>) as privately run sites (such as mine at strayalpha.com <http://strayalpha.com/>).

This isn’t entirely true: a dangling pointer in an RFC actually becomes an attack surface if it’s important enough to search for the missing thing it once pointed to.  Now I can publish an alternative piece of software that you can download that not only converts word documents to RFCs, but also sends me your personal information, etc.

The real problem here is that the RFC ever pointed to a reference link that wasn’t on an IETF web site.   The only time that I think this makes sense is as an informative reference.

Of course, in this case in theory they are just Word templates, but can’t those contain Visual Basic?

---

From: Joe Touch <[email-masked]>
Date: Fri, 12 Jul 2019 06:48:50 -0700

> On Jul 12, 2019, at 6:17 AM, Ted Lemon <[email-masked]> wrote:
> 

Microsoft disabled macros by default since Office 2016, so this won’t happen unless the user explicitly disables that protection (it’s a pop up on each Word doc with macros). I never mention macros in the RFC, so that’s not advisable.

But there are many RFCs with dangling pointers. Every RFC that pointed to ftp.isi.edu, for example. And as I noted, the IETF tools site URL has changed - in fact, it even has a dangling pointer ON ITS OWN page (to http://www.ietf.org/tools, which is even more insecure).

> The real problem here is that the RFC ever pointed to a reference link that wasn’t on an IETF web site.   The only time that I think this makes sense is as an informative reference.

That’s what this was, essentially. 

> Of course, in this case in theory they are just Word templates, but can’t those contain Visual Basic?

Yes, but see above about that being disabled by default when opening new docs unless the user overrides them.

Joe

---

From: "John Levine" <[email-masked]>
Date: 12 Jul 2019 21:46:52 -0400

>talking about -- all subjective.

I think the practical difference is between a toolset that is unique
to the IETF and one that is more widely used.

It is utterly unclear to me whether better bridge tools would help.  I
am pretty sure that with enough effort I could write a two-way
converter between xml2rfc and a Microsoft Word file with suitable
stylesheets and macros.  (It'd be a lot harder than what Joe has
done.)  So you could edit stuff in Word, and revise by importing
xml2rfc into Word and then exporting when you're done.  But would
anyone use it?

R's,
John

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 12 Jul 2019 22:09:49 -0400

> xml2rfc into Word and then exporting when you're done.  But would

Certainly not those of us using operating systems for which Word is not 
relevant.

Scott K

---

From: Joe Touch <[email-masked]>
Date: Fri, 12 Jul 2019 19:51:27 -0700

> On Jul 12, 2019, at 7:09 PM, Scott Kitterman <[email-masked]> wrote:
> 

Which OSes are those?  (LibreOffice works on Linux, MacOS, and Windows, e.g.)

Joe

---

From: Scott Kitterman <[email-masked]>
Date: Sat, 13 Jul 2019 03:07:22 +0000

>Which OSes are those?  (LibreOffice works on Linux, MacOS, and Windows,

That's true, but may or may not be relevant.  There are plenty of ways to make such things that only work in Word.

Scott K

---

From: John C Klensin <[email-masked]>
Date: Fri, 12 Jul 2019 23:25:50 -0400

--On Friday, July 12, 2019 21:46 -0400 John Levine
<[email-masked]> wrote:

>> Yeah, it does seem lik the XML hate is mostly just that, but
>> that our use of XML is not really a blocker.  But these are
>> feelings we're talking about -- all subjective.
> 
> I think the practical difference is between a toolset that is
> unique to the IETF and one that is more widely used.
> 
> It is utterly unclear to me whether better bridge tools would
> help.  I am pretty sure that with enough effort I could write
> a two-way converter between xml2rfc and a Microsoft Word file
> with suitable stylesheets and macros.  (It'd be a lot harder
> than what Joe has done.)  So you could edit stuff in Word, and
> revise by importing xml2rfc into Word and then exporting when
> you're done.  But would anyone use it?

The answer to the specific question is probably a subset, maybe
a large subset, of those who are using Joe's tool.  I don't have
a clue as to how many people it would attract from markdown.

But it seems to me that there is a generic problem in much of
this discussion.  The original motivation for xml2rfc (see the
introduction to RFC 2629) was to allow enough descriptive markup
to facilitate RFC production and permit element identification
for other purposes.  If one generalizes a bit from that, one
gets to the motivation for SGML itself: allowing authors to
describe the elements of a document well enough that someone or
something else can deal with formatting, page layout, and a
variety of stylistic issues in a way that produces reasonable
results. As an aside, at least some SGML purists argued that
giving authors control at the formatting level was not only a
poor use of their time but that it encouraged evil behavior.
That particular approach is particularly important if one wants
to produce a variety of different output formats from a single
source document, something that was definitely not part of the
program for RFC 2629 but is critical for the new format setup
and xmlrfc v3.

Perhaps unfortunately, most of us at one time or another want to
mess with formatting and layout. Carl and Marshall were quite
careful about that, introducing some elements that were clearly
about low-level formatting but being rather clear that most of
them were kludges and they knew it.  Whatever the other
strengths and weaknesses of v3, elements like <strong> and <sub>
are format markup.  Go very far in that direction (and I'm
trying to remain agnostic about whether the right choices were
made) and one ends up back to nroff.

Where this relates to the above is that formatting engines, even
those with style sheets, don't provide, other than
heuristically, anything other than format-level behavior and
markup.  Relative to the expectations of identifying material by
its nature and application of generic markup, there isn't a lot
of difference among Word, nroff, TeX and LaTex, etc.  Getting
from generic markup to one of them as an intermediate step to
layout and formatting is fairly straightforward.  The historical
RFC Editor path of xml2rfc input -> nroff -> final documents is
a rather good example.   But getting from final documents for
format/layout material back to generic markup is a dicey
business that is going to fall into the "sometimes it works and
sometimes it doesn't" category.

How important that is depends on what one wants and expects.

   john

---

From: Nico Williams <[email-masked]>
Date: Sat, 13 Jul 2019 02:16:49 -0500

> to the IETF and one that is more widely used.

The toolchain is unique to the IETF.  That the IETF has its own metadata
and typesetting needs is not.  Also, just as other SDOs have adopted our
RFC2119, they might adopt our document production system if it was
flexible enough.

Before XML we had roff, so we weren't in too terrible a place, but it
wasn't very good either.  Roff's is not a trivial learning curve either,
and it sure doesn't look pretty, and very few people know roff.  Not
everyone will agree, but I find xml2rfc to be a step up from roff[*].

We've seen people in this thread say Word is not acceptable to them.
At this time there simple are no choices other than XML now that are
universally available and manageable to all IETF participants,
reluctantly or otherwise.

> It is utterly unclear to me whether better bridge tools would help.  I

We know we can build bridges.  Some of us have.

Do note that in order for co-authors to collaborate while using
different $EDITORs they would need not just conversion to XML/xml2rfc,
but from it as well.  That's two bridges per-$EDITOR.

> stylesheets and macros.  (It'd be a lot harder than what Joe has

That's essentially what I did in lyx2rfc, though if I was TeX-able I'd
probably have built a proper template and not used "macros".  That would
have allowed me to not only render via xml2rfc, but also natively via
TeX tooling (which LyX's UI takes care of), with all the PostScript and
PDF typesetting prowess of the TeX ecosystem.

> xml2rfc into Word and then exporting when you're done.  But would

I hoped others would use LyX...  I think only two or three people other
than myself did.  You might be right about Word.  I would not use Word
to edit I-Ds, and I would find it difficult to co-author with people who
might insist me using Word.

Nico

[*] I do edit roff on a regular basis -- man pages, basically -- and
    mostly I do by copying the patterns I see, occasionally deep-diving
    to pick up some unwelcome arcane roff knowledge, ready to jettison
    it at the first opportunity.

---

From: "Stan Kalisch" <[email-masked]>
Date: Sat, 13 Jul 2019 04:06:45 -0400

> That's true, but may or may not be relevant. There are plenty of ways to make such things that only work in Word.

It's also worth noting that Word has a history of contemporaneous versions for different OSes sometimes rendering documents differently.

Stan

---

From: Joe Touch <[email-masked]>
Date: Sat, 13 Jul 2019 07:34:13 -0700

> On Jul 13, 2019, at 1:06 AM, Stan Kalisch <[email-masked]> wrote:
> 

Perhaps, but it’s been more stable for the core functions over the past 30 years than nearly anything else I’ve used. I can still open docs from 1988.

Besides, I’m only suggesting that use of .docx files be supported as authoring tools; the output can still be XML.

Joe

---

From: Carsten Bormann <[email-masked]>
Date: Sat, 13 Jul 2019 16:41:44 +0200

> Besides, I’m only suggesting that use of .docx files be supported as authoring tools; the output can still be XML.

But that’s easy — use MS-Word to edit markdown and then process that.

Maybe you do have something different in mind…

Grüße, Carsten

---

From: Joe Touch <[email-masked]>
Date: Sat, 13 Jul 2019 07:45:37 -0700

> On Jul 13, 2019, at 7:41 AM, Carsten Bormann <[email-masked]> wrote:
> 

Yeah - something where the headings in Word turn into XML headings, etc. It looks like Phillip’s system might achieve the output conversion I’m looking for, but we may benefit from a .dotx template that shows better how to use it...

FWIW, I haven’t edited source code for documents since 1987 (Scribe, when at the SEI), and I don’t really understand why anyone does.

Joe

---

From: Julian Reschke <[email-masked]>
Date: Sat, 13 Jul 2019 16:50:35 +0200

> FWIW, I haven’t edited source code for documents since 1987 (Scribe, when at the SEI), and I don’t really understand why anyone does.

Because treating technical documents just like source code (ASCII,
source control, reviewing diffs) feels very natural to me as a programmer.

Best regards, Julian

---

From: "John R Levine" <[email-masked]>
Date: 13 Jul 2019 11:19:26 -0400

> everyone will agree, but I find xml2rfc to be a step up from roff[*].

I love troff/nroff and use it for all sorts of stuff, but it is too low 
level.  To do the kind of semantic tagging that is routine in xml, you 
have to use complex macros in an extremely disciplined way, and not many 
people can do that.  We sure didn't.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail. https://jl.ly

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Sat, 13 Jul 2019 11:59:05 -0400

A converter between Word and Xml2RFC is exactly what I have. It goes in
both directions. I am the only user I know of.

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Sat, 13 Jul 2019 12:05:19 -0400

On Sat, Jul 13, 2019 at 10:46 AM Joe Touch <[email-masked]> wrote:

> >> Besides, I’m only suggesting that use of .docx files be supported as
> authoring tools; the output can still be XML.
> >>
> >
> Yeah - something where the headings in Word turn into XML headings, etc.
> It looks like Phillip’s system might achieve the output conversion I’m
> looking for, but we may benefit from a .dotx template that shows better how
> to use it...
>
> FWIW, I haven’t edited source code for documents since 1987 (Scribe, when
> at the SEI), and I don’t really understand why anyone does.
>
>
If people have feature requests, I can put some time in. I have been
waiting for the HTML tooling to arrive at IETF.

My tool will also generate a dummy rfc template with placeholders for the
stuff that needs to be filled in. It does that for Markdown, Word or HTML.

What it doesn't do is to generate comments to say what the range of choices
are.

I did think about doing more with the google docs format. Then people could
collaboratively edit the same source.

---

From: Carsten Bormann <[email-masked]>
Date: Sat, 13 Jul 2019 18:51:00 +0200

On Jul 13, 2019, at 18:05, Phillip Hallam-Baker <[email-masked]> wrote:
> 
> google docs format. Then people could collaboratively edit the same source

Our students tend to do that with markdown files via hackmd.io (or actually the local open-source instance of that we have at the university).  But then they are CS people, who live and breathe that documentation is code.

And that may actually be the reason why we will need two different toolchains for the foreseeable future:  people who come with a developer mindset tend to use developer workflows and developer tools (git, github, automation, CI, …).   Some other people view document generation as word processing, and therefore gravitate to word processing tools.  Some organizations are relatively homogeneous with respect to this split (e.g., ITU doesn’t have a lot of developer-minded people), so they can standardize on one of the sides (e.g., ITU on MS-Word).  

The IETF mixes both developer-minded people and, for lack of a better word for the other category, engineers, so we’ll need both sides.  If a development team for specific document is mixed between the two categories, there will be some mismatch, and that is the place where markdown can shine compared to, say, W3C’s respec.

Grüße, Carsten

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Sat, 13 Jul 2019 17:32:53 -0400

+1

Most of the docs I have describing the Mesh have three separate sources:

Word
Visio
Visual Studio projects generating markdown files.

Generating documentation from the schema is obviously the thing to do. In
fact it is the main reason I use schemas to parse JSON. Generating Word
document format and converting to XML would be painful. Generating examples
and reference sections in markdown is straightforward.

I use Word to edit most of the text and generate SVG figures using Visio.
My mechanism for including SVG is not currently compliant. I actually
encode the SVG as a DATA uri. This is because I am only going to write a
tool to conform my SVG with the requirements for IETF when there is
something to test against.

Getting the SVG workflow working is going to be interesting for folk
because the IETF has very restrictive requirements and very few tools
actually comply with them out of the box. My plan is to dump out the Viso
files into SVGs using a tool that does the necessary processing to strip
out all the stuff that the IETF tools don't grok. The problem here being
that SVGs are used on the Web as separate files. When you embed them in
another file as XML data, the rules change.

One point that I do think important here is that we should agree to
standardize on the Github flavor of Markdown. That is not the one I am
currently using but will move to it.

On Sat, Jul 13, 2019 at 12:51 PM Carsten Bormann <[email-masked]> wrote:

> Our students tend to do that with markdown files via hackmd.io (or
> actually the local open-source instance of that we have at the
> university).  But then they are CS people, who live and breathe that
> documentation is code.
>
> And that may actually be the reason why we will need two different
> toolchains for the foreseeable future:  people who come with a developer
> mindset tend to use developer workflows and developer tools (git, github,
> automation, CI, …).   Some other people view document generation as word
> processing, and therefore gravitate to word processing tools.  Some
> organizations are relatively homogeneous with respect to this split (e.g.,
> ITU doesn’t have a lot of developer-minded people), so they can standardize
> on one of the sides (e.g., ITU on MS-Word).
>
> The IETF mixes both developer-minded people and, for lack of a better word
> for the other category, engineers, so we’ll need both sides.  If a
> development team for specific document is mixed between the two categories,
> there will be some mismatch, and that is the place where markdown can shine
> compared to, say, W3C’s respec.
>
> Grüße, Carsten
>
>

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Mon, 15 Jul 2019 12:49:34 -0500

Top posting ...

It seemed to me that we spilled a large number of words in NewTrk trying to
describe when working groups could declare a Stable Snap Shot (SSS), or in
a parallel proposal, a Working Group Snapshot (WGS), and who would need to
approve that (choices included the IESG, ADs, and WG chairs), and after
tripping over a number of corner cases, what seemed appropriate to me was
to let working group chairs declare that a draft was stable for a specific
purpose - as we do now with QUIC implementation drafts, the chairs say "at
the next interim, we'll be testing -22, whether there are later revisions
or not", and see what happens after that.

Writing process BCPs that work for all IETF working groups is hard, and
trying to nail down every detail and anticipate every possible problem with
every possible usage makes it harder.

But it seems important to realize that we're not talking about something
the IETF isn't doing now - we're talking about how obvious what the IETF is
doing now will be to people who aren't at the epicenter of specific work.

Best wishes, of course.

Spencer

On Thu, Jul 11, 2019 at 9:56 AM Phillip Hallam-Baker <[email-masked]>
wrote:

> Sometimes those decisions are made before a spec gets to IETF. If there
> are 10,000 users of a spec and it is using XML, it is going to be a slog
> It might well be the case that we could use some more formal notice of
> when these points are crossed. It probably isn't a good idea to make this
>> On Wed, Jul 3, 2019 at 1:10 PM Keith Moore <moore@network-heretics..com
>> <[email-masked]>> wrote:
>>

---

From: john heasley <[email-masked]>
Date: Wed, 17 Jul 2019 00:46:59 +0000

> version of that Internet-draft.

eh, not exactly; there are baggage and expectation associated with that
name that does not seem appropriate/necessary.

I want RFC 7525 to not be a RFC nor a draft, instead a Living Document (LD)
that the WG publishes on github (or pick another SCM) with WG consensus,
and does not take a year+ to publish or update.

It may reference RFCs, drafts, CVEs, free/commerical tools, other LDs,
etc etc to form recommendations; such as ciphers required, what to avoid,
how to deploy, etc.  if there is a burning urge/need in someone's pants to
chisel some part of it into a rfc as a separate effort, do that separately
and maybe it will be published before it is a relic.

I'd accept not permitting the reference of drafts, but there could be value
to pointing the direction the WG is heading.  A draft, a WG item or not,
already has different perceived value from a RFC.

it is NOT for spec of new protocols, altering protocols, ...

Other drafts and RFCs, eg: BGP over TLS, should be able to point to this
and simply say follow these WG recommendations and make no or very very
few TLS-specific statements/recommendations of its own.

Thus that BGP over TLS RFC does not need to be updated as TLS evolves or
TLS RFCs come/go, nor does it need to make recommendations about TLS
itself - something that the authors may not be qualified to comment about.

Another example would be Benoit's periodic state of netmod/netconf emails,
while he was AD.  Very useful and much of it would be useful in an LD.

It could even include a map of documents for a technology; which RFCs
apply to BGP, what are the dependencies for BGP LS, etc.

I expect the audience to be IETF participants/authors, implementers, and
users.  And, the authors to be the relevent WG and published with/by WG
consensus; without specific solicitation of IESG/IAB/etc, though not
forbidden.

I believe this is close to what Snijders is pitching, perhaps equivalent,
but dissimilar from Kumari's pitch.  And, I wish the two be separated.
This is not about experimentation or early development; that is and
should be a separate discussion.  it is about easing mapping of
dependencies or a given technology, communication of recommenations, and
application/deployment recommendations that evolve more quickly.

---

From: Keith Moore <[email-masked]>
Date: Tue, 16 Jul 2019 21:04:17 -0400

> and does not take a year+ to publish or update.

I could see some utility in having some documents being able to be 
updated in place.  But I would have serious concerns with document 
content like RFC7525 (i.e. technical recommendations for implementation 
and/or operation of protocols) approved without IETF consensus.   It is 
essentially part of a protocol specification.   So a WG should not be 
able to "publish" such a document, nor approve it based entirely on its 
own consensus. That would not only bypass IETF consensus (and cross-area 
review), it would effectively bypass appeals and other safeguards we 
have in place.   Even if the effect were "mostly harmless" in most 
cases, it would be a really bad precedent for others.

I would also object to IETF consensus documents making normative 
reference to such a document.

I also think trying to define IETF consensus for a moving target could 
be challenging.   Not impossible, perhaps, just challenging.

But I'd be supportive of trying to streamline the IETF consensus process 
for such documents, on the theory that review of such documents 
(including analysis of likely effects) should be easier than review of 
full protocol specifications.

Keith

---

From: Stephen Farrell <[email-masked]>
Date: Wed, 17 Jul 2019 11:40:42 +0100

Hiya,

> itself - something that the authors may not be qualified to comment about.

That is already true. Simply refer to BCP195 instead of
RFC7525 and you get that bit.

I understand the desire for more quickly moving things
but IMO changing too quickly also has downsides so I'm
ok with RFC7525 being the latest instance of BCP195 and
for that RFC needing to be updated or the BCP having
another RFC added to it for TLS1.3 (hopefully in the not
too distant).

I'd be against github-hosted evolving, living or dying
documents. Github is ok for drafts as we do now, but even
today leads to too much off-list discussion and decision
making in my experience.

That said, I do like the idea of a WG nominating a draft
(or set of drafts with additional commentary) as the
current interop draft, but I think I'd be against such an
artefact being a long-lived thing to which e.g. RFCs may
normatively refer.

Cheers,
S.

---

From: john heasley <[email-masked]>
Date: Thu, 18 Jul 2019 21:00:06 +0000

> RFC7525 and you get that bit.

BCP numbers are static?  An RFC that supersedes 7525 and becomes BCP
will also be 195?  I see no indication of this in 2026: thus I assume
it will receive a new number.  What have I missed?

> another RFC added to it for TLS1.3 (hopefully in the not

I am not; recommendations, esp. for security topics, need to move near the
speed of blackops, not at RFC speed - or worse, BCP speed, which I perceive
to be even slower and carry more weight.  I feel the same for other
technologies, security just has more inherent urgency.

The RFC itself says
" Community knowledge about the strength of various algorithms and
   feasible attacks can change quickly, and experience shows that a Best
   Current Practice (BCP) document about security is a point-in-time
   statement.  Readers are advised to seek out any errata or updates
   that apply to this document."

> today leads to too much off-list discussion and decision

An SCM that is publicly available, of which I do not expect debate.  Which
and where, I expect will be debated ad nauseam.

> artefact being a long-lived thing to which e.g. RFCs may

Think of it as a library or compatibility library.

---

From: john heasley <[email-masked]>
Date: Thu, 18 Jul 2019 23:19:11 +0000

Tue, Jul 16, 2019 at 09:04:17PM -0400, Keith Moore:
> have in place.

Since no one is excluded from participating in any WG, anyone can comment or
object to any such document.  Appeals also can be submitted by anyone, though
the process for such a document would likely need to be different.

> full protocol specifications.

How?  Coca leaves?

I do not find any protocol changes in 7525 and it does not have an "Updates"
header.  That appears to have taken a year to be published.  Could that be
reduced to 1 month or less, esp. for updates, as you noted?

Admittedly, I do not see the value in IAB/IESG review, in the sense that
their construction has no particular value to me over other's review.  A
larger pool of potential (and equally competent) reviewers has greater
value.

Again, this is also about operations; 8212 and 8327 could have been part
of a larger LD about BGP operations.

---

From: Keith Moore <[email-masked]>
Date: Thu, 18 Jul 2019 20:06:11 -0400

On 7/18/19 7:19 PM, john heasley wrote:

> object to any such document.

WG-only review has been demonstrated over and over again, for literally 
decades, to be woefully insufficient.   There's far too little wide 
review with the current process, and you're proposing to make it worse.

> Appeals also can be submitted by anyone, though

The appeals process is far too cumbersome and taxing to IESG et al, to 
use for the number of issues that are likely to result from lack of 
cross-area review.

>> full protocol specifications.
I was thinking more in terms of shotgun duels at ten paces, but coca 
leaves would be an interesting idea.  :)

I hadn't thought in depth about how to streamline such decisions, I just 
said I'd be supportive of it.   For instance, if one can somehow analyze 
the delta between version 1 and version 2, and version 1 has already 
been analyzed, it may not be necessary to analyze version 2 in its 
entirety.

One benefit of living documents is that it could be easier for IESG and 
its directorates to look at just the changes and avoid re-reviewing the 
entire document each time.   And the writeup for such a document could 
be expected to do exactly that.   Offhand I don't see a good way to 
bypass steps in the current process, but perhaps delta revisions could 
be given priority (by default) since they should take up much less 
review time than new documents. But it's still possible to miss things, 
of course, and occasionally (say every N revisions) a full review should 
be done.

(Thinking about how time consuming it has been to clean up some old 
standards, I'm intrigued by the possibility of a process that would 
encourage making small deltas over major rewrites.)

> reduced to 1 month or less, esp. for updates, as you noted?

I'd think 6-8 weeks is doable, depending on when the clock is started.  
Something like:

(presumably this is after WG last call.   if WG can argue that a 
per-feature last call is sufficient and no per-document last call is 
needed, IESG might accept that.)
0. WG sends request to IESG with pre-written analysis of changes and 
arguments as to why they are beneficial or benign.
1. responsible AD reviews WG's request and analysis (or has a 
directorate member do it) and makes a recommendation to IESG
2. if recommendation is positive, announce 2 week last call
3. last call comments are evaluated and (if found valid) triaged: (a) 
problems with changes requested -> document goes back to WG; (b) 
problems other than with the changes requested -> comments forwarded to 
WG for evaluation and possible incorporation to a future revision but 
the WG doesn't necessarily have to fix it immediately - WG gets to decide
4. Assuming no category (a) bugs and neither WG nor IESG thinks any 
category (b) bugs have to be fixed immediately, IESG approves and issues 
public notice.
5. Somebody (RFC Editor?) updates the current version pointer in 
response to IESG's announcement.

IESG generally does things on 2-week cycles so a lot is paced by the 
timing of IESG meetings.   But things can happen faster if, for example, 
directorates return reviews more quickly.

> larger pool of potential (and equally competent) reviewers has greater

IESG review can also consider process issues, and sometimes it's the 
only external review a document gets, and sometimes IESG are the only 
ones watching out for cross-area conflicts.   So I don't think it can be 
dispensed with from a process perspective, even though it's more useful 
for some kinds of WGs and documents than others.

> of a larger LD about BGP operations.

I'm always going to be thinking in terms of how much sense a process 
change makes for IETF as a whole.   If there's a specific corner case 
for which an expedited review process makes sense, that can be approved 
by writing up a BCP describing it and getting IETF Consensus behind that 
process.   But I fear it's a slippery slope - if we make an exception 
for one kind of group, other groups will want their own exceptions.   So 
it makes sense to think in terms of more broadly applicable mechanisms.

Keith

---

From: Job Snijders <[email-masked]>
Date: Thu, 18 Jul 2019 21:16:42 -0400

> review with the current process, and you're proposing to make it worse.

I think you just now offended basically every working group participant.

Regards,

Job

---

From: Keith Moore <[email-masked]>
Date: Thu, 18 Jul 2019 21:23:44 -0400

Sent from my iPhone

> On Jul 18, 2019, at 9:16 PM, Job Snijders <[email-masked]> wrote:
> 

Not deliberately, of course.  But in general WGs can sometimes exhibit tunnel vision.  And ietf has a bad habit of trying to bypass tussles by creating WGs with overly narrow scope.., which doesn’t solve the tussle, just exacerbates it.  So yeah, in general, WG output needs external review.

Keith

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 19 Jul 2019 13:46:57 +1200

On 19-Jul-19 13:23, Keith Moore wrote:
> 
> 
> Sent from my iPhone
> 
> On Jul 18, 2019, at 9:16 PM, Job Snijders <[email-masked] <mailto:[email-masked]>> wrote:
>>

I think it varies enormously between WGs. There are some that are small, expert, highly specialised and really do know more than anyone else about their topic, which is relatively self-contained. There are others that don't meet any of those criteria. Job's right about the first kind. Keith's right about the second kind, which really do need third party review. As a GenART reviewer, I've seen all sorts.

   Brian

---

From: Keith Moore <[email-masked]>
Date: Thu, 18 Jul 2019 21:50:59 -0400

Sent from my iPhone

> On Jul 18, 2019, at 9:46 PM, Brian E Carpenter <[email-masked]> wrote:
> 

Yes, and I’ve repeatedly said I could see optimizing in corner cases.  But I think it’s a rare WG that doesn’t have any potential to adversely affect other interests.

Keith

---

From: Ted Lemon <[email-masked]>
Date: Thu, 18 Jul 2019 22:00:38 -0400

On Jul 18, 2019, at 9:50 PM, Keith Moore <[email-masked]> wrote:
> Yes, and I’ve repeatedly said I could see optimizing in corner cases..  But I think it’s a rare WG that doesn’t have any potential to adversely affect other interests.

Another way to look at this is a well-known cognitive bias: “I am right.”   If you look at what a working group is doing and don’t understand it, there is a tendency to think they don’t know what they are doing, and that you know what they should have done.   This bias is frequently wrong, and I’ve seen it turned against good work numerous times.

---

From: Keith Moore <[email-masked]>
Date: Thu, 18 Jul 2019 22:10:43 -0400

Sent from my iPhone

> On Jul 18, 2019, at 10:00 PM, Ted Lemon <[email-masked]> wrote:
> 

That argument applies equally well to itself.

This is silly.  I’ve lost count of the number of WGs I’ve seen for which I did understand what they were doing, and did understand how they could harm other interests.  And in general Last Call is too late to fix those problems.  I agree with Brian that that’s not a description of _every_ WG, for the reasons he stated.  But as long as we’re talking about process in general, the discussion needs to consider the potential for tussles and how to manage that.

---

From: Ted Lemon <[email-masked]>
Date: Thu, 18 Jul 2019 22:31:14 -0400

On Jul 18, 2019, at 10:10 PM, Keith Moore <[email-masked]> wrote:
>  I’ve lost count of the number of WGs I’ve seen for which I did understand what they were doing, and did understand how they could harm other interests.

This is the cognitive bias to which I was referring.   You think you know better than they do; indeed, you are sure of it.  Quite possibly some of the time you are correct, but I’ve known you for a long time, and I’ve seen you be utterly sure of yourself when you were clearly wrong.   I’m sure you’ve seen me do it too.

Yes, we definitely need to know how to manage this, and yes, we currently do it badly.

---

From: Nico Williams <[email-masked]>
Date: Thu, 18 Jul 2019 22:15:14 -0500

> I think you just now offended basically every working group participant.

Ah, yes, you jest.

In the off chance you meant that, rest assured I was not offended, and
fail to see why anyone should be :)

Nico
--

---

From: Nico Williams <[email-masked]>
Date: Thu, 18 Jul 2019 22:25:59 -0500

On Thu, Jul 18, 2019 at 10:31:14PM -0400, Ted Lemon wrote:
> >  I’ve lost count of the number of WGs I’ve seen for which I did
> >  understand what they were doing, and did understand how they could
> >  harm other interests.
> 
> This is the cognitive bias to which I was referring.   You think you
> know better than they do; indeed, you are sure of it.  Quite possibly
> some of the time you are correct, but I’ve known you for a long time,
> and I’ve seen you be utterly sure of yourself when you were clearly
> wrong.   I’m sure you’ve seen me do it too.

Certainly Keith's phrasing is a bit of an argument from authority.

But! the proposition that WGs should not be allowed to publish without
IESG and IETF review is hardly controversial.  The proposition that WGs
should be allowed to get portions of an Internet-Draft to be considered
"stable" prior to IESG publication approval is.

There was no need to make that argument about how many times he's seen
WGs be unaware of the harm they could do.

> Yes, we definitely need to know how to manage this, and yes, we

Well, for starters we can call these things out when we see them (as you
just did), then, as we get used to that, each of us will learn that it's
not always good to risk getting called out.

OTOH, I don't really want discourse here to get too dry.  Participants'
personalities are part of the attraction.

Nico
--

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 18 Jul 2019 23:58:06 -0400

(Supporting Keith on this.)

One of the key benefits of IETF meetings is cross-area review.  One of 
the key reasons for having WG last call is the observed need for review 
outside the working group.  One of the observation from many such 
reviews is that it is amazing how much a working group can miss while 
getting its core stuff right.
Yes, this also means that periodically folks raise objections that are 
spurious, miss the point, or have been addressed already.  But the cost 
of not having the review is VErY high.

Yes, folks have suggested that the review should be lightened or 
eliminated.  So far, the community has refused to do that.  And I for 
one am very glad that is so.  In spite of having had to deal with some 
frustrating objections in many cases.

Yours,
Joel

On 7/18/2019 10:10 PM, Keith Moore wrote:
> 
> 
> Sent from my iPhone
> 
> On Jul 18, 2019, at 10:00 PM, Ted Lemon <[email-masked] 
>> On Jul 18, 2019, at 9:50 PM, Keith Moore <[email-masked] 
>>> Yes, and I’ve repeatedly said I could see optimizing in corner 
>>> cases..  But I think it’s a rare WG that doesn’t have any potential 
>>> to adversely affect other interests.
>>
>> Another way to look at this is a well-known cognitive bias: “I am 
>> right.”   If you look at what a working group is doing and don’t 
>> understand it, there is a tendency to think they don’t know what they 
>> are doing, and that you know what they should have done.   This bias 
>> is frequently wrong, and I’ve seen it turned against good work 
>> numerous times.
> 
> This is silly.  I’ve lost count of the number of WGs I’ve seen for which 
> I did understand what they were doing, and did understand how they could 
> harm other interests.  And in general Last Call is too late to fix those 
> problems.  I agree with Brian that that’s not a description of _every_ 
> WG, for the reasons he stated.  But as long as we’re talking about 
> process in general, the discussion needs to consider the potential for 
> tussles and how to manage that.

---

From: Christopher Morrow <[email-masked]>
Date: Fri, 19 Jul 2019 00:05:49 -0400

> of not having the review is VErY high.

Is this cost really when there is a 'published standard which vendors
are expected to write code to support'
or also for: "Hey we operate things like this, wait not like that
anymore... now we do it this way!"

My experience is that often the operations focused work items get
strung along and nibbled to death by tiny ducks... to the point that
by the time the document sees the light of day it's OBE :( This is
disheartening to the authors... They're goal wasn't really 'publish an
rfc!' but to make a best practice more accessible to others.

> eliminated.  So far, the community has refused to do that.  And I for

err, but this is what the LD path is actually doing. There's no reason
that the document can't get reviewed by folk outside the WG
responsible for taking care of it... in fact it can get published and
reviewed (and revised!) as often as is necessary. That seems like a
great step forward for a bunch of ops related work.

---

From: Job Snijders <[email-masked]>
Date: Fri, 19 Jul 2019 04:14:56 +0000

On Thu, Jul 18, 2019 at 11:58:06PM -0400, Joel M. Halpern wrote:
> the key reasons for having WG last call is the observed need for
> review outside the working group.  One of the observation from many
> such reviews is that it is amazing how much a working group can miss
> while getting its core stuff right.  Yes, this also means that
> periodically folks raise objections that are spurious, miss the point,
> or have been addressed already.  But the cost of not having the review
> is VErY high.
> 

Joel,

My take on it is that the context of this conversation is not protocol
specifications or extensions, but operational guidance. What some in
this thread are advocating for is a pathway to publish an equivalent of
BCPs in a shorter timeframe than 12 to 36 months.

Kind regards,

Job

---

From: "Joel M. Halpern" <[email-masked]>
Date: Fri, 19 Jul 2019 00:56:19 -0400

Responding here both to Job and to Chris Morrow.

There is indeed an argument that operational guidance has the dual 
properties of
1) needing to be out promptly
2) changing over time as the operational environment changes.

I do realize that Job's initial motivation for this was specifically 
operational.  But most of the discussion has not seemed to be restricted 
to that.  I do know that various people have asked for much more dynamic 
protocol specs.  And some of the examples cited have been protocol 
specs.  That is what makes me nervous.

If the focus is operational documents, there ought to be a way to do 
something, and it ought to be worth a try.  Finding ways for the IETF to 
be more useful to operators, and for operators to be able to participate 
in a fashion taht is more eff3ective for what they need, does seem 
valuable.  And with the restriction, many of my concerns do not apply. 
(We do, for example, allow the contents of a BCP to change even though 
the underlying individual RFCs are immutable.  While this is aimed to be 
much faster, it seems related.)

Yours,
Joel

---

From: Randy Bush <[email-masked]>
Date: Thu, 18 Jul 2019 22:15:08 -0700

> rfc!' but to make a best practice more accessible to others.

why does an ops organisation such as ripe not create a reviewed document
repository?  oh, wait.  how about one i helped on,
https://www.ripe.net/publications/docs/ripe-580

why don't ops, not satisfied with the ripe doc repo create yet another
repo, something such as Best Current Ops Practices?  oh, wait.  ...

what you describe sounds like wanting to get the ietf imprimatur on an
incompletely reviewed document.  drafts in the routing area require two
implementations to progress; is that silly?

implement a marked draft.  does it interop?  is it clear?  has it taken
security into account?  ...  then we have a process to publish it with
the ietf imprimatur.

randy

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 19 Jul 2019 17:20:39 +1200

> 2) changing over time as the operational environment changes.

How long does RIPE take to produce a BCOP?
See https://www.ripe.net/publications/docs/ripe-690#2--what-is-a-bcop-

   Brian

---

From: Job Snijders <[email-masked]>
Date: Fri, 19 Jul 2019 05:25:11 +0000

Both RIPE and BCOP efforts seemed centered around the concept of
geography-specific documents; I think in a construct like the Internet
that doesn't really know borders or regions this isn't always helpful.
It may be in some cases, not in all cases.

Also, the IETF has excellent publication facilities and a well-defined
process to help define process; something the BCOP efforts seem to
have lacked in the past.

IETF produces protocol specifications, it seems logical to me to also
ensure we can provide after-care.

Kind regards,

Job

---

From: Jared Mauch <[email-masked]>
Date: Fri, 19 Jul 2019 01:27:16 -0400

> On Jul 19, 2019, at 1:20 AM, Brian E Carpenter <[email-masked]> wrote:
> 
> 

Some IETF WGs may take several years to produce a BCP.

Sometimes advice needs to change quickly as new things are learned, the IETF process may not reflect that.  Iterative drafts reflect the best collective knowledge at that point in time.  Capturing that historical view may not feel interesting today, but over time understanding the iterations and why things that changed quickly (eg: ip directed-broadcast) is useful.  Those recommendations often shift faster than the IETF could move even in the best of scenarios.

- Jared

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 19 Jul 2019 19:15:34 +1200

Yes but I want a standard for comparison: how long does RIPE take? I have never been involved in that side of RIPE, so I have no idea whether they manage it in weeks, months or years. What is a realistic target?

If we want to fast-track Rapid Operational Advice Documents*, I don't believe it will be done via the RFC process.

*Oh look, I made you an acronym, although we have used it before.

    Brian

---

From: Stephen Farrell <[email-masked]>
Date: Fri, 19 Jul 2019 10:06:32 +0100

Hiya,

Just on this bit, as it seems less widely known...

On 18/07/2019 22:00, john heasley wrote:
> it will receive a new number.  

Nope, it'll still be BCP195, if that's what the community want
at that time, which'd I guess be the default assumption for a
7525bis.

BCP9 [1] says: "A specification, or group of specifications,
that has, or have been approved as a BCP is assigned a number
in the BCP series while retaining its RFC number(s)."

BCP9 itself currently consists of about 8 RFCs. So we can add
RFCs to a BCP. We can also subtract RFCs from a BCP too. So you
could think of the change you mention above as adding 7525bis
to BCP195 while at the same time subtracting 7525:-)

I'd guess BCP10 [2] is a good example of one that has undergone
such changes a few times, for example RFC6859 used to be, but is
no longer, part of BCP10. [3]

Bottom line is that yes, the BCP series already has some, but
by no meals all, of the flexibility we're after in this thread.

Another perhaps also less widely known consequence is that it's
a better idea to reference BCP195 in other RFCs rather than
RFC7525.

Cheers,
S.

[1] https://tools.ietf.org/html/bcp9
[2] https://tools.ietf.org/html/bcp10
[3] https://tools.ietf.org/html/rfc6859

---

From: Keith Moore <[email-masked]>
Date: Fri, 19 Jul 2019 05:49:33 -0400

Sent from my iPhone

> On Jul 18, 2019, at 10:31 PM, Ted Lemon <[email-masked]> wrote:
> 
> This is the cognitive bias to which I was referring.   You think you know better than they do; indeed, you are sure of it. 

Non sequitur.   Just because I saw a way that a WG’s output could harm other interests does not mean I thought I knew better than the WG, especially in the absence of evidence that the WG considered the potential for that harm at all.  There may also be a writing convention at work here: anytime one is expressing an opinion, the words “I think...” are implied.  Leaving them out doesn’t mean the writer is certain, and some consider it poor writing style to include words like that.

But the model that most often seems to apply, as far as I can tell, is the one of the elephant in the dark room, where nobody can see the whole elephant.  But each of several people has a hand on a different part of the elephant and each one insists that the part he or she is touching is representative of the whole elephant.  Even for someone who realizes that the elephant is somehow like a tree, spear, rope, hose, etc., all at the same time, it’s a challenge to imagine the whole elephant.  But the person who says “there’s an important part of the elephant you’re not seeing” isn’t wrong to point that out.

To apply the analogy to ietf, it seems that ietf WGs often focus entirely on one or two parts of an elephant and either fail to consider the others or consider them out of scope.  And sometimes this is baked into their charters.
  
> Quite possibly some of the time you are correct, but I’ve known you for a long time, and I’ve seen you be utterly sure of yourself when you were clearly wrong.   I’m sure you’ve seen me do it too.

You seem quite sure of yourself in declaring me wrong.  :). And in insisting that you know what I’m thinking. 

But if someone keeps telling me I’m wrong by simple nay saying, without showing me that they understand the argument (or when it’s clear that they don’t), or if they insist they know what I’m thinking when they clearly don’t, or if (for example) they insist that a particular proposal is demonstrated to be harmless by significant deployment when the deployed protocol is significantly different than the one proposed - I’m not likely to be (favorably) impressed by any of those lines of argument.   It doesn’t mean I’m certain of my own argument, but I’m probably all the more certain that the other party is mistaken in some way.  (Of course just because a proponent ‘s argument is unpersuasive doesn’t mean the proposal is inherently bad or can’t be fixed.)

Keith

---

From: Ted Lemon <[email-masked]>
Date: Fri, 19 Jul 2019 06:10:37 -0400

> (Supporting Keith on this.)

It might be good to not make this about supporting Keith or supporting whomever.  I think Keith made some good points, as do you.

However, I don’t think we’re talking about getting rid of cross-area review, so Keith’s point is a bit of a non-sequitur.  If this side meeting were about eliminating cross-area review in the IETF, I would agree with you that there was a problem.   But it’s not.   So it might be salutary to discuss what the side meeting is actually about.

> On Jul 19, 2019, at 5:49 AM, Keith Moore <[email-masked]> wrote:

>> Non sequitur.   Just because I saw a way that a WG’s output could harm other interests does not mean I thought I knew better than the WG, especially in the absence of evidence that the WG consider…

I don’t think that’s a good convention.  I think it’s good to express uncertainty when you are uncertain.

That said, my purpose here was not to say that you’re wrong, but to use what you were saying to make an additional point.  Cross-area review is a good thing, and I don’t want to get rid of it.  But there is an extreme that it can reach that causes harm.  It is not, shall we say, guaranteed to always be done right.

---

From: Keith Moore <[email-masked]>
Date: Fri, 19 Jul 2019 06:17:48 -0400

On 7/19/19 6:10 AM, Ted Lemon wrote:

>  Cross-area review is a good thing, and I don’t want to get rid of it. 
>  But there is an extreme that it can reach that causes harm.  It is 
> not, shall we say, guaranteed to always be done right.

No, of course not.  Anybody can be wrong.   Both the WG and the reviewer 
can be wrong, or (more likely) both partially right and wrong in 
conflicting ways.   Ideally the differing views result in an improved 
understanding for everybody, though of course this doesn't always happen.

I also see no reason to have a presumption, for example, that a WG is 
"right" in a matter that is outside of its subject area.

Keith

---

From: Keith Moore <[email-masked]>
Date: Fri, 19 Jul 2019 06:23:23 -0400

> anymore... now we do it this way!"

I expect it's more for "here's a new way that we operate things. Sure it 
breaks things that people are using, but never mind because we've 
decided what is best without their input."

Keith

---

From: Ted Lemon <[email-masked]>
Date: Fri, 19 Jul 2019 06:55:33 -0400

On Jul 19, 2019, at 6:17 AM, Keith Moore <[email-masked]> wrote:
> I also see no reason to have a presumption, for example, that a WG is "right" in a matter that is outside of its subject area.

I agree.   However, we’ve been trying to improve our cross-area review diversity since I was on the IESG, and probably since you were, and it’s not clear to me that there’s a lot more win to be had here.  Where we do succeed, I think it’s because we’re able to rope experts in to the working group.   The IETF last call process doesn’t work well.   The directorates are good, but not always sufficient, and review quality is widely variable, although frequently very good.

That’s why I think this is a bit of a non-sequitur to the “living documents” discussion.  Yes, this is a problem, and we should be careful not to make it worse.  But I really don’t think it’s the case that the people who have organized this meeting are unaware of the issue.

---

From: Jared Mauch <[email-masked]>
Date: Fri, 19 Jul 2019 08:31:07 -0400

> On Jul 19, 2019, at 3:15 AM, Brian E Carpenter <[email-masked]> wrote:
> 

I think one of the issues here is venue shopping.  IETF actually has decent tools to look at the history of a document and to publish a new one regularly as advice changes.

If I wanted to make draft-mauch-ops-recommendations I could do that today, put source in GitHub and hit version 99 before the end of the day.  That’s either good or bad depending on your view of things, but if it makes sense it can go to one or more tracks and be available for the coming years to anyone who wants or needs it.

On the operator side of things there are so many conferences be them network operator (RIPE, NANOG, NLNOG, AUSNOG, BOSNOG), dns operator (DNS-OARC), etc.. it really depends on who you are trying to reach and in what language the messages are in as well.  I’m lucky in that I was born in an English speaking country and the document languages are in English as well.

- Jared

---

From: Keith Moore <[email-masked]>
Date: Fri, 19 Jul 2019 09:10:59 -0400

On 7/19/19 6:55 AM, Ted Lemon wrote:

> On Jul 19, 2019, at 6:17 AM, Keith Moore <[email-masked] 
> I agree.   However, we’ve been trying to improve our cross-area review 
> diversity since I was on the IESG, and probably since you were, and 
> it’s not clear to me that there’s a lot more win to be had here. 
>  Where we do succeed, I think it’s because we’re able to rope experts 
> in to the working group.

I've noticed the effort to improve cross-area review.  I can't tell yet 
how effective it is, but I'm pleased to see the effort being made.

> The IETF last call process doesn’t work well.

I agree.   Last Call is mostly useful for problems that are easy to fix, 
and uncontroversial.   That seems to be a tiny fraction of the problems 
that still exist by that time.

It's long seemed to me that the points at which IETF manages to get some 
broad review on an idea or proposal are at the BOFs that precede WG 
formation, and Last Call.   These also seem like the times at which 
broad review is least likely to be effective (unless, as is sometimes 
the case, the WG starts with a document that it believes is already a 
mature proposal).

The best time period for broad review seems to be after a fairly 
complete proposal exists, but before there's a lot of investment in it.  
The WG is more receptive (less exhausted) at that point, and changes 
made as a result of such feedback are less likely to be disruptive.

> That’s why I think this is a bit of a non-sequitur to the “living 
> documents” discussion.  Yes, this is a problem, and we should be 
> careful not to make it worse.  But I really don’t think it’s the case 
> that the people who have organized this meeting are unaware of the issue.

One reason I'm interested is that I'm intrigued by the potential for 
"living documents" to facilitate earlier broad review of WG proposals.   
I think use of LDs has the potential to get feedback to the WG sooner, 
and also track how the WG responds to such feedback.   I can imagine a 
day when the most external reviews come in response to "First Call" or 
"Second Call", rather than Last Call, and the IESG final review consists 
largely of looking at how the WG and authors responded to the earlier 
calls for feedback, and whether the document has changed in such a way 
as to present new issues not previously identified.    Ideally the 
responsible AD will have been tracking such feedback and responses all 
along.

Another reason I'm interested in LDs is that some people seem to want to 
use them as a way to bypass community-wide review.   There may be cases 
for which bypassing such review is Mostly Harmless, but offhand I don't 
see a way to draw a clear line between Mostly Harmless and Potentially 
Harmful.

So I see both a potential for positive change, and a danger of negative 
change.   I don't know how aware the people who have organized the 
meeting are of these potentials, though I expect that they're at least 
somewhat aware.   Mostly I've been responding to opinions expressed on 
the list.

Keith

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Fri, 19 Jul 2019 10:23:13 -0400

On Thu, Jul 18, 2019 at 10:02 PM Ted Lemon <[email-masked]> wrote:

> Yes, and I’ve repeatedly said I could see optimizing in corner cases..
> But I think it’s a rare WG that doesn’t have any potential to adversely
> affect other interests.
>
>
> Another way to look at this is a well-known cognitive bias: “I am right.”
>   If you look at what a working group is doing and don’t understand it,
> there is a tendency to think they don’t know what they are doing, and that
> you know what they should have done.   This bias is frequently wrong, and
> I’ve seen it turned against good work numerous times.
>

Thing is that sometimes the criticism is correct but irrelevant. If you are
fixing a 30 year old protocol to respond to a need that wasn't originally
anticipated, there are going to be times when you have to break things. And
very often the pushback that comes is of the form 'there are still people
using fidonet who will be inconvenienced', to which my response is
generally of the form, 'this is not a historical re-enactiment society, we
do not support retro-computing.'

There is a bias in favor of supporting the oldest established communities
rather than the needs of the largest communities of users. The Internet now
has 4 billion people using it and there is a tendency for some folk to
prioritize the perceived needs of legacy users that may not even exist.

The other pathological behavior I see very often is people insisting that
the real problem is deployment who also insist on technical approaches that
make deployment impossible.

I spend more time thinking about deployment of my protocol designs than the
protocol itself. I look at the business models of each of the parties that
I need for adoption. For the Web, I was running simulations of different
deployment scenarios.

Which is why I find statements of the form 'we have to move fast on this so
we will require the deployment of a new TCP feature requiring kernel level
support' farcical.

---

From: John C Klensin <[email-masked]>
Date: Fri, 19 Jul 2019 10:48:41 -0400

Hi.
I'm not going to repeat my earlier (and much longer) note, which
I think contains useful information about this issue, however...

--On Friday, 19 July, 2019 08:31 -0400 Jared Mauch
<[email-masked]> wrote:

>...
>> If we want to fast-track Rapid Operational Advice Documents*,
>> I don't believe it will be done via the RFC process.
>> 

Indeed.
 
> I think one of the issues here is venue shopping.  IETF
> actually has decent tools to look at the history of a document
> and to publish a new one regularly as advice changes.

Indeed.  And that history, stability, and quality and breadth of
review are, at least in the world up to now, absolutely critical
for standards, especially standards that might find their way
into parts of procurement and regulatory processes.  Venue
shopping almost always benefits those who have ideas and want a
stamp of approval that adds credibility to those ideas.  It
hurts the credibility of standards bodies who are willing to
plan and it quite often hurts the public.

There are important roles for living documents, identification
of some of those documents as more stable (or appropriate for
trial implementation, etc.) than others, and review only by
self-selected specialist groups.  But, if careful, structured,
cross-area review (and the formal mechanisms need to support
that) are not seen as needed, I don't see what value doing the
work in the IETF provides other than an apparent endorsement
that can be presented as involving more than a working group.

> If I wanted to make draft-mauch-ops-recommendations I could do
> that today, put source in GitHub and hit version 99 before the
> end of the day.  That's either good or bad depending on your
> view of things, but if it makes sense it can go to one or more
> tracks and be available for the coming years to anyone who
> wants or needs it.

Exactly.  And if I wanted to push parts of my i18n work forward,
I could put the documents themselves in Github and identify them
as "John and friends".  But, because I'm trying to tune IETF
Standards Track documents, the only reasonable mechanism (unless
the IETF is not interested) is to work within the IETF and push
documents through through the IETF  process.  I cannot, and
should not be able to, round up a dozen interested people,
persuade the IESG to authorize a WG, and then start putting WG
work, especially work that essentially modifies, clarifies, or
supplements Standards Track documents without at least a serious
attempt at IETF review.

Back to lurking.

best,
   john

---

From: john heasley <[email-masked]>
Date: Fri, 19 Jul 2019 18:04:24 +0000

Thu, Jul 18, 2019 at 10:25:59PM -0500, Nico Williams:
> the proposition that WGs should not be allowed to publish without
> IESG and IETF review is hardly controversial.

Maybe; publish what?  This can not be argued nor affirmed without knowing
the what.  Ignoring keith's cross review argument for the moment.

---

From: john heasley <[email-masked]>
Date: Fri, 19 Jul 2019 18:37:22 +0000

> operational.  But most of the discussion has not seemed to be restricted 

My motivation is operation, deployment, and development.  Eg; 7525 is all
three, mpov.

> specs.  That is what makes me nervous.

7525 is not protocol spec, if that is what you are referring to; it even
specifically says that it alters no RFC.

If you are referring to something emitted from Kumari's pitch of "stable
rfc version" (or whatever variation of that); I again ask that that
discussion be separate.

> valuable.

So too would be the additional operator involvement, in general.

---

From: Nico Williams <[email-masked]>
Date: Fri, 19 Jul 2019 14:00:08 -0500

> the what.  Ignoring keith's cross review argument for the moment.

At the IETF we _publish_ RFCs, and we _submit_ Internet-Drafts.  So my
use of "publish" was short-hand "publish RFCs", but I left "RFCs" out
because part of the point of this thread was to find a way to "publish"
something more "stable" than Internet-Drafts with less process than RFC
publication requires.

I used "publish", then, as short-hand for "more stable than an I-D, less
heavy-duty than an RFC" since we don't have a name for what that thing
would be.

Nico
--

---

From: john heasley <[email-masked]>
Date: Fri, 19 Jul 2019 19:05:34 +0000

Fri, Jul 19, 2019 at 10:48:41AM -0400, John C Klensin:
> and review only by
> self-selected specialist groups.

No one has said/suggested 'self-selected'.  anyone can review any document.

> I don't see what value doing the

Because it is related to products of the ietf and that is where,
presumably, the expertise for the given product lies.  If you want to
fix the pluggable optic MSA, I presume you would go to the IEEE - not
my area of expertise, so don't bother correcting which group mangled it.

> that can be presented as involving more than a working group.

This seems like a claim that a LD with the aforementioned review process
would affect the reputation of RFCs, which I reject.  it is presented as
involving the process and those whom the document type claims.  RFC has
a process, BCP has one, LD would have one, and PRETZEL would have one.
BCP, mpov, carries far more weight than RFC.

> Exactly.  And if I wanted to push parts of my i18n work forward,

which is what?  What is your i18n work?  I do not know.  If it is
definition of the wire formats, etc - no, that is not the goal here.
Not mine, nor Job's.  A 7525-like document about i18n would be, again
mpov.

---

From: Keith Moore <[email-masked]>
Date: Fri, 19 Jul 2019 15:24:03 -0400

It seems like we've reached a point of diminishing returns with this 
discussion since AFAIK we don't actually have specifics to evaluate or 
argue about.    Maybe the next thing to do for those who want to propose 
this for ops is to go off and write an I-D proposing mechanisms and 
processes?    One that could address the expressed concerns?   Any 
process change has to earn IETF consensus after all.

(or if I missed the I-D that proposes mechanisms and process, please 
point me to it)

Keith

---

From: john heasley <[email-masked]>
Date: Fri, 19 Jul 2019 19:39:11 +0000

> heavy-duty than an RFC" since we don't have a name for what that thing

drafts, (iab/irtf/individual) rfcs, bcps, stds, and i suspect more that I
am forgetting or am unaware of.  All related, but not equal.

The stable idea is Kumari's.

Except for Keith's suggestion of accelerating the bcp/rfc processes, we
are not suggesting changes to those processes.  If a way to accelerate
were agreed upon, such as review of changes only, it may still deserve
a new name to make this difference clear.  And it MAY not have the same
review requirements - again mpov.

---

From: john heasley <[email-masked]>
Date: Fri, 19 Jul 2019 19:45:25 +0000

> (or if I missed the I-D that proposes mechanisms and process, please 

there is a side meeting at 105 that will be hosted by Kumari.  I am not
attending and do not believe there is remote participation support.  I
am hoping that folks will report to the maillist.

Thrs 8:30-9:45  Side Meetings / Open Time

---

From: Christopher Morrow <[email-masked]>
Date: Fri, 19 Jul 2019 15:50:14 -0400

> heavy-duty than an RFC" since we don't have a name for what that thing

This, 'publish between an rfc and ID' is, I think, what the
living-documents/evolving-documents work was supposed to enable.
I was going to dither with John's suggestion of 7525 as an example,
mostly because TLS tends to get people's hackles up :) but.. actually,
because of things like:
  https://tools.ietf.org/html/rfc7525#page-11

this is a great example actually. The advice to set cipher suites and
other options for TLS has been changing quite a bit of late, having a
consistent location that's not 'rando website cipherl.st ?' seems like
a great plan, to me. If that location is able to keep up in close to
real time with all the various 'heartbleed' type problems so  much the
better. We should be making this simpler and easier to locate and
digest, right? and up-to-date to the best of our ability?

---

From: john heasley <[email-masked]>
Date: Fri, 19 Jul 2019 19:57:35 +0000

> digest, right? and up-to-date to the best of our ability?

Please also see 7540 appendix A - 11 pages - that should not exist.  it
belongs in a LD.  the appendix is incomplete now, it is not specific to
7540, ....

---

From: Nico Williams <[email-masked]>
Date: Fri, 19 Jul 2019 15:12:50 -0500

> digest, right? and up-to-date to the best of our ability?

Good point.  Not only that, but the notion of "cryptographic strength"
of various cipher/digest/MAC algorithms and ciphersuites, public key
algorithms and protocols, PRFs, and other functions, vary over time as
cryptanalysis advances.  A registry of current perceived cryptographic
strength, subject to significantly less strenuous review than Standards
Action, would be useful.  Perhaps we could make this an IANA registry,
though that seems a bit of a stretch.

Nico
--

---

From: Christopher Morrow <[email-masked]>
Date: Fri, 19 Jul 2019 16:38:12 -0400

> though that seems a bit of a stretch.

I think a registry doesn't really get to the point though.. I can
imagine the example here is really a few documents:
  o  cipher/algorithm/key-size/etc advice (which is mostly 7525's aim)
  o documents for common IETF server/client focused things:
        webserver config advice
        mailserver advice
        ssh client/server advice
        ....
  o advice on how often to rotate keys used in securing basic plumbing
infrastructure (dns, bgp, tacacs+ etc)

Having all of these have to spin up multi-month long processes in the
IETF to get errata + rfc + ... is going to be less useful.
Having these managed at a faster pace based on advice from experts
(from the ietf even!  :) ) seems like  good compromise...

-chris

---

From: Christopher Morrow <[email-masked]>
Date: Fri, 19 Jul 2019 16:40:14 -0400

> belongs in a LD.  the appendix is incomplete now, it is not specific to

oh good ${diety} wtf :( why would this have been encoded here?
CERTAINLY this is good advice, CERTAINLY it is going to be wrong by
the time this document was published and read :( ugh... Yes, I agree
that this(the appendix) belongs in a document which we can revise
quickly when problems arise in the decision process which arrived at
the 'do not use' list :(

thanks for making me more sad.
-chris

---

From: Nico Williams <[email-masked]>
Date: Fri, 19 Jul 2019 17:30:48 -0500

> oh good ${diety} wtf :( why would this have been encoded here?

Probably because it was a convenient vehicle for issuing that advice.

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 20 Jul 2019 11:17:23 +1200

That's exactly why I asked about RIPE's solution. Before proposing
a solution, don't we want to study one that apparently works?

Regards
   Brian Carpenter

---

From: Job Snijders <[email-masked]>
Date: Fri, 19 Jul 2019 19:19:54 -0400

> a solution, don't we want to study one that apparently works?

Works by what metric?

IETF has take some responsibility for after-care here.

Kind regards,

Job

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 20 Jul 2019 11:23:35 +1200

On 20-Jul-19 11:19, Job Snijders wrote:
> On Fri, Jul 19, 2019 at 19:17 Brian E Carpenter <[email-masked] <mailto:[email-masked]>> wrote:
> 

I'd rather someone active in the RIPE region answered that.
 
> IETF has take some responsibility for after-care here. 

I don't disagree.

   Brian

---

From: Matthew Kerwin <[email-masked]>
Date: Sat, 20 Jul 2019 11:16:03 +1000

> Probably because it was a convenient vehicle for issuing that advice.

For what it's worth, I want to undo a similar thing with RFC 8089 -- split
out the appendices to non-standards-track, informational, easy to update
guidance documents. They're the parts people really want standardisation
on, and having *something* to point at, even if it's not rigorous or with
consensus, is better than everyone going their own way.

Cheers
-- 
Matthew Kerwin

---

From: John C Klensin <[email-masked]>
Date: Sat, 20 Jul 2019 00:08:31 -0400

> No one has said/suggested 'self-selected'.  anyone can review

But "anyone can review any document" is completely consistent
with "self-selected".  If we had a mechanism whereby people were
told which documents to review and were then required to do so,
that would still be consistent with "anyone can review any
document".  But we don't do that.  Even members of area review
teams who, in some cases, may draw assignments for documents for
review on a rotary basis, have volunteered for those teams and
are hence self-selected to some extent.  So, yes, anyone can
review any document but for them to actually do so, they have to
decide that is worthwhile and then go do so.  And that is the
very essence of "self-selected".

Participation in WGs is much the same situation.  In principle,
anyone can participate in any WG.  The people who do so select
themselves -- we don't cast lots in the general IETF participant
population and then tell people which WGs they are required to
participate in.   And we have a considerable history of a
relatively small group of people with common interests and
background forming a WG that draws on those interests and
background.  Nothing prevents anyone else from participating in
the WG, but, if they are neither expert in the WG's subject
matter nor particularly interested in it, they typically don't.

Or am I misunderstanding the point you are trying to make?

> Because it is related to products of the ietf and that is
> where, presumably, the expertise for the given product lies.
> If you want to fix the pluggable optic MSA, I presume you
> would go to the IEEE - not my area of expertise, so don't
> bother correcting which group mangled it.

But that is exactly what confused me about this.  With the
understanding that I don't even know what a pluggable optic MSA
is, much less what is wrong with it, if it is an IEEE standard
and one wants to fix it, one takes it to the IEEE.  Then one
goes through the IEEE's process for revising standards, a
process that requires several levels of review.   Unless things
have changed a lot since I had significant contact with IEEE
standards, WG-level are publishing documents on their own and
indicating that those documents supercede the standards in
practice.  

But, as I understand it, that is precisely what these proposals
are about: allowing a WG to declare a piece of its work as ready
to go without review and validation by the rest of the IETF.  I
do believe that it may beentirely appropriate for some portions
of a WG's work to be handled that way, but I think that those
cases (the cases, not the details) should get consensus in the
IETF on a case by case basis.  That is, of course, exactly what
we do by appointing Designated Experts or a Designated Expert
team review process for IANA registry modifications.  In
general, the decision to create the registry and to use that
review model requires IETF consensus but individual entries do
not.  Are there other things, such as operational advice that
stops short of Applicability Statements in particular WGs?  I
don't know, but I assume the answer is probably yes.  And, like
several others, I'd like to see a very specific proposal in I-Ds
form, not more of this somewhat wandering conversation.

> This seems like a claim that a LD with the aforementioned
> review process would affect the reputation of RFCs, which I
> reject.  it is presented as involving the process and those
> whom the document type claims.  RFC has a process, BCP has
> one, LD would have one, and PRETZEL would have one. BCP, mpov,
> carries far more weight than RFC.

We probably need to agree to disagree about some or most of
that, including BCPs carrying more weight than "RFCs".  I assume
when you say that, you means Standards Track RFCs.  If not, I
probably agree, but, I think it takes us into the "not all RFCs
are standards" discussion.   In any event, while I note your
opinion, many people who worry about procurement or conformance
would disagree with you.

best,
   john

> which is what?  What is your i18n work?  I do not know.  If it
> is definition of the wire formats, etc - no, that is not the
> goal here. Not mine, nor Job's.  A 7525-like document about
> i18n would be, again mpov.

---

From: Warren Kumari <[email-masked]>
Date: Sat, 20 Jul 2019 08:11:17 -0400

On Sat, Jul 20, 2019 at 12:08 AM John C Klensin <[email-masked]> wrote:
> to go without review and validation by the rest of the IETF.

Whoa, no no no - that's not the intent, but I think I now finally
understand much more of the disconnect in this thread.

When Job initially presented his idea it seemed to me (and still seems
to me) that this would solve a bunch of very real needs, and I started
playing with some ideas on how we could implement something like this.
Unsurprisingly I was viewing this though the lens of my own
experiences, and I really didn't intend marking a draft in this way as
carrying as much weight as people seems to think it would.

I suspect that much of this comes down to poor terminology, and people
(me!) not reevaluating what the term means both to themselves others -
I've been following the thread, but it was only when I chatted with
some people here in Montreal that I finally realized that the term
"stable" means much more "stable" to people than I was intending it to
convey - I suspect that much of the disconnect here stems from this
terminology issue -- the term "checkpoint" was suggested, and this is
probably much closer to what I was intending.

I wasn't intending marking a draft to allow "a WG to declare a piece
of its work as ready to go without review and validation by the rest
of the IETF." - as I'd initially said, these would still be drafts,
and the "It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."" type
disclaimer should definitely still apply -- a marked draft (at least
in my mind, obviously I communicated this all poorly) would have much
much *much* less standing than a draft which had passed WGLC - which
is still not "ready to go without review and validation by the rest of
the IETF." but rather "ready to go *for* review and validation by the
rest of the IETF. A "marked" draft would mainly allow external people
to see a *snapshot* of what the WG has currently agreed to in a
document -- from the "Operators and the IETF" survey
(https://tools.ietf.org/html/draft-opsawg-operators-ietf-00#section-2.2.2)
lots of external people want to follow what the IETF is doing, but
don't have the time or stomach to follow all the discussions

The way in which we might "mark" a draft evolved over time:

The first idea (chosen to be trivial to implement) would be a webpage
with pointers to (regular) drafts. The (rough) implementation was that
WG chairs could suggest to their ADs that a document should be a
"Living Document", the AD would almost definitely) agree[0]. The
chairs could then move this pointer to future versions of the draft if
and when the WG agrees that a version has settled.

The second iteration of the idea was "Hey, we could just use a tag in
the datatracker" -- we currently have tags like "New", "Expires soon",
etc -- we could have a new tag which could be applied by the chairs
saying that this particular version reflects what the WG *currently*
thinks... as someone who uses the datatracker almost every day, this
seemed like a fine idea -- until it was pointed out to me that people
outside the IETF have no idea what this datatracker is, nor how it
would be used.

The third (current) iteration was "Ooooh, we change a document name
from draft-bob-foo-bar to draft-ietf-foo-bar - this indicates that a
WG has decided it is worth working on. What if we use
draft-ietf-foo-bar -> draft-stable-foo-bar (or stable-foo-bar or
similar)? This follows an understood paradigm, etc..." -- I liked the
term "stable", but I was viewing it much more as a stable development
tree, not a release tree, and the term stable was clearly the wrong
one.... draft-checkpoint-foo-bar would have been better (but still
imperfect) representation of what I was intending it to mean.

I was also thinking that a "draft-<some_sort_of_marker>-foo-bar" type
document could / should include some sort of strong / visible
boilerplate along the lines of:
***********************************************************************************************************
* This document reflects the **current** views of the foo working
group. It has not received
* IETF review, it is not an RFC, it is not set in stone. It is a
snapshot in time of the current
* views, and is, like any ID, subject to change, etc.
* It is inappropriate to use / refer to this as reference material or
to cite this as anything other
* than as "work in progress.".  If you build a test / demo / PoC
implementation from this,
* understand that it is all subject to change, etc.
***********************************************************************************************************

I really should have communicated what *I* thought a marking would be
intending to convey; I did want others to be able to use this proposal
to other use cases, but I really really don't think that it should
ever imply that "a piece of its work as ready to go without review and
validation by the rest of the IETF."

I still think that there are real problems to solve, but I didn't
expect or want this proposal to carry as much weight as people seem to
think it may, nor did I want it to take up this much of the ietf@
lists time and energy.
It's just an idea / proposal -- it is entirely possible that it  is a
bad one / simply won't work for us / needs major refinement -- I'd
like to still discuss it, but this really isn't worth any animosity,
etc.

W

> I

---

From: Keith Moore <[email-masked]>
Date: Sat, 20 Jul 2019 08:50:47 -0400

On 7/20/19 8:11 AM, Warren Kumari wrote:

> I really should have communicated what*I*  thought a marking would be

I believe your statement of intent.  But I think it's fair to realize 
that there is already tremendous pressure to deploy implementations 
before they've been formally approved, and there's a danger that any 
kind of distinguished "mark" will have the (unintended) effect of 
promoting deployment of the marked version of a protocol.

It also appears that there are people in IETF who want WGs to be able to 
approve work without community-wide review.   For some isolated 
instances such power might even be appropriate, but knowing where to 
draw the line might be tricky.

So this effort might carry with it more risks than were initially 
apparent, and a great deal of caution seems warranted.

Keith

---

From: Christopher Morrow <[email-masked]>
Date: Sat, 20 Jul 2019 08:59:59 -0400

> promoting deployment of the marked version of a protocol.

regardless of 'published or not' folk will always push for the early
implementation of FOO before it has been 'ratified'.
it's pretty clear that this happens, and that nothing about this
discussion is going to change that.

---

From: Keith Moore <[email-masked]>
Date: Sat, 20 Jul 2019 09:08:37 -0400

> discussion is going to change that.

The last part does not follow.

Keith

---

From: "Martin Thomson" <[email-masked]>
Date: Sat, 20 Jul 2019 09:56:04 -0400

> belongs in a LD.  the appendix is incomplete now, it is not specific to

As editor of that RFC and responsible for that appendix, this is not a suitable candidate for any sort of change.  A moving target would not be good for implementations as it has direct interoperability consequences, so the idea that this is "incomplete" doesn't really apply.  If this were advice, I'd agree, but this is a requirement.

---

From: Warren Kumari <[email-masked]>
Date: Sat, 20 Jul 2019 10:17:32 -0400

> discussion is going to change that.

*I* think that we really want early implementations of FOO before it
has been ratified - without this we don't have the "running code" part
of "rough consensus and running code.
I think the trouble comes in when there are *deployments* of FOO
before it has been ratified....

W

---

From: John C Klensin <[email-masked]>
Date: Sat, 20 Jul 2019 11:01:39 -0400

Warren,

This is very helpful.  See inline below.

--On Saturday, 20 July, 2019 08:11 -0400 Warren Kumari
<[email-masked]> wrote:

>> But, as I understand it, that is precisely what these
>> proposals are about: allowing a WG to declare a piece of its
>> work as ready to go without review and validation by the rest
>> of the IETF.
> 
> Whoa, no no no - that's not the intent, but I think I now
> finally understand much more of the disconnect in this thread.
> 
> When Job initially presented his idea it seemed to me (and
> still seems to me) that this would solve a bunch of very real
> needs, and I started playing with some ideas on how we could
> implement something like this. Unsurprisingly I was viewing
> this though the lens of my own experiences, and I really
> didn't intend marking a draft in this way as carrying as much
> weight as people seems to think it would.

Another distinction may be important here. Let's assume that the
IETF reached consensus that a particular type of document should
be called a "Kitten".   We all understand what "kitten" means,
just as we understand that the LAMPS WG isn't about lighting
fixtures.  No problem.  Now, whether out of marketing needs or
simple misunderstanding (or even malice) someone outside reads
or is told "kitten document from the IETF" and hears (or wants
to hear) either "some sort of feline from the IETF" or just
"IETF document".  And then, if they are interested in promoting
the document, and are in an environment where lions are highly
respected, it is presented as "IETF Lion document" or, if they
are opposed to the specification or the IETF itself, "IETF
person-eating tiger".

Now, again, _we_ would never make that mistake.   But substitute
"Informational RFC" for "kitten" and then replay in your mind
every heated discussion we have had (at least since you started
participating) about the problems caused when people
accidentally or deliberately confuse "RFC" with "[IETF] Internet
Standard".  Think about the proposed remedies for that
preventing Informational or Experimental documents from being
numbered and designated as RFCs.  Or, only a year ago, the
proposal to stop identifying documents from non-IETF Streams as
RFCs and giving them RFC numbers.  

Some important philosophical issues about what we are actually
about (see my long note about comparisons to other standards
bodies and how they handle documents aside, the problem with
having multiple type of documents and finer categories is that,
to an outsider or with an incentive to misconstrue, none of it
makes any difference.  That should have been clear to us the
first time marketing materials showed up that said that an
Internet-Draft was an "IETF document" and implied it was an
Internet Standard.    If the IETF produces it, or any IETF
subgroup produces it, or it is posted using IETF facilities, we
have decades of experience that suggest it will be misconstrued.

The marketers make an easy target but, for example, note that
everyone who has put an I-D on a resume without clearly
identified as a draft, a work in progress, or an expired draft
is violating our official rules too.  Different incentives, same
outcome.  And the same risk to them that the marketers have of
being found out and never believed again... but that doesn't
seem to be enough of a deterrent.

It is likely that the more categories of documents "we" have,
the easier it is for someone who has incentives to do so to
create confusion.  

And _that_ is what at least some of us are worried about.  

> I suspect that much of this comes down to poor terminology,
> and people (me!) not reevaluating what the term means both to
> themselves others - I've been following the thread, but it was
> only when I chatted with some people here in Montreal that I
> finally realized that the term "stable" means much more
> "stable" to people than I was intending it to convey - I
> suspect that much of the disconnect here stems from this
> terminology issue -- the term "checkpoint" was suggested, and
> this is probably much closer to what I was intending.

While "checkpoint" or "snapshot" would be far better than
"stable", the problem above remains.  We would figure out what
it meant and, at least most of the time, would get that right.
People who suffered from either deep ignorance, laziness about
reading definitions or boilerplate, or some incentive to
mislead, would do their thing no matter what we called it.
> 
> I wasn't intending marking a draft to allow "a WG to declare a
> piece of its work as ready to go without review and validation
> by the rest of the IETF." - as I'd initially said, these would
> still be drafts, and the "It is inappropriate to use
> Internet-Drafts as reference material or to cite them other
> than as "work in progress."" type disclaimer should definitely
> still apply -- a marked draft (at least in my mind, obviously
> I communicated this all poorly) would have much much *much*
> less standing than a draft which had passed WGLC - which is
> still not "ready to go without review and validation by the
> rest of the IETF." but rather "ready to go *for* review and
> validation by the rest of the IETF. A "marked" draft would
> mainly allow external people to see a *snapshot* of what the
> WG has currently agreed to in a document -- from the
> "Operators and the IETF" survey
> (https://tools.ietf.org/html/draft-opsawg-operators-ietf-00#se
> ction-2.2.2) lots of external people want to follow what the
> IETF is doing, but don't have the time or stomach to follow
> all the discussions

Yep.  Got that (and appreciate it).  Some of the others arguing
for this have been less clear.  But, if we have ever had reason
to worry about whether a survey that is published as an
Informational RFC is construed as a formal recommendation from
the IETF about what should be done rather than what some think
should be done, then lowering that bar doesn't help.

But, taking that document as an example with the understanding
that I just glanced at it for the first time, it appears to be
an ISOC study.  If posting a summary for the information of a WG
is helpful then do it - that doesn't require or benefit from
special labels.  If ISOC wants to publish a summary, they are
more than capable of doing that -- they don't need Internet
Drafts of any subspecies.  If you put it out for IETF LC
tomorrow, some jerk might stand up and point out that the
international statistical community (including the other ISI,
which predates the one more familiar to most IETF participants
by about a century) has clear recommendations for information
that should be included when anything is portrayed as a "survey'
or "survey results" and this document doesn't meet them.  That
is not an IETF technical topic, but it is a place where a little
cross-discipline review could reduce the risk of IETF WGs, at
least in the ops area, getting a reputation for unprofessional
and sloppy work.   As long as it is just an internal working
draft, you get to say "we'll get to that later, even if the need
to get to it hadn't occurred to you.  But, as soon as you hang a
flat on it that says "others, outside the IETF and its document
development process, might want to pay attention to this",
you've launched yourself down a slippery slope.

Eliding the rest of your note to save space, one other
observation:  There are existing, fairly well-defined, rules for
naming of I-Ds. The IESG apparently eliminated any notion of
enforcing those rules long before you joined it.  Your example,
draft-opsawg-operators-ietf-00 violates them and no one seems to
care.  If I were to post a draft on Monday called
draft-satan-future-of-IETF, someone might get me for bad taste,
identity theft, or blasphemy. but there is no mechanism to take
it down.  If someone tried, I'm confident someone who defend my
right to express myself and compare it to other postings with
names that don't follow the rules.

So, independent of the details of the proposal and whether it is
a good idea, if you ever want I-D names to contain important
indicative information, I suggest that you, as an AD, do two
things:

(1) To the extent possible, clean up your act and the act in
your Area.  A draft from the OPSAWG, even one that is not being
considered for some sort of special status, should be posted as
draft-ietf-opsawg-...   Any documents named draft-opsawg-...
(whether add insult to injury by including "IETF" in the name)
such be superceded and taken down unless they posted by Joe (or
Mary) Oppsawg and are _not_ WG documents.  Start setting an
example.  Even then, start watching for that important future
IETF participant Alice Stable.  Perhaps she will be a newcomer
in Montreal.

(2) Work with the IESG to start enforcing the naming rules.  Fix
it so that anything that is not
  draft-ietf-<activeWG>-...
  draft-<recognized-body>-... (where <recognized body> is a very
short list that presumably included IAB, IESG, IRTF, and
probably not much more)
  draft-<author>-...  (where <author) is the surname of one of
the listed authors)

can be taken down at the request of anyone in the community
(saving setting an I-D Police role comparable to SAA ones,
although that is another possibility).  If the IESG has to be
involved in case-by-case decisions, it will fail and we will be
back to uncontrolled names.  This is just a quick idea, but I'd
suggest a complaint to the secretariat, notice by them to the
authors, 24 hours for the authors to explain why the name is
really legitimate, and then the I-D goes away or, if tooling and
time permit, is renamed more appropriately.   Modifying the
posting tool so that drafts with non-conforming names are harder
to even post would be another improvement.  

Not only would that improve the understandability of I-D names
within the collection, but it would make any idea of conveying
more information through those names at least a bit more
plausible.

best,
   john

---

From: John C Klensin <[email-masked]>
Date: Sat, 20 Jul 2019 12:01:26 -0400

--On Saturday, 20 July, 2019 10:17 -0400 Warren Kumari
<[email-masked]> wrote:

>> > I believe your statement of intent.  But I think it's fair
>> > to realize that there is already tremendous pressure to
>> > deploy implementations before they've been formally
>> > approved, and there's a danger that any kind of
>> > distinguished "mark" will have the (unintended) effect of
>> regardless of 'published or not' folk will always push for
>> the early implementation of FOO before it has been 'ratified'.
>> it's pretty clear that this happens, and that nothing about
>> this discussion is going to change that.
>> 
> 
> *I* think that we really want early implementations of FOO
> before it has been ratified - without this we don't have the
> "running code" part of "rough consensus and running code.
> I think the trouble comes in when there are *deployments* of
> FOO before it has been ratified....

Exactly.  And, at the risk of singing an old song, that was
exactly what Proposed Standard was supposed to be about.

So, with the understanding that I'm not optimistic about this
for the reasons I gave earlier and a quarter-century of history,
consider an alternate proposal: Return Proposed Standard to its
original intended status.  Try to make it clear that anyone
deploying a new specification at Proposed Standard is a fool and
deliberately misleading customers about stability and the
future.  Make it lots easier and faster to get one through the
system, e.g., by forbidding or ignoring editorial nit-picking
during IETF LC and, as BCP9 clearly allowed, simply documenting
known omissions and areas of uncertainty and moving on.  Let
Proposed Standards be close enough to preliminary engineering
specifications that there are real incentives to advance (and
carefully finish end edit) documents after implementation and
implementability have been proven.   The main review should be
"is this clear and precise enough to implement for test
purposes, with or without judicious application of the
robustness principle" (one thing we should learn from
implementation, interoperability testing, and running code more
generally is a list of things that need to be documented more
precisely but that, again, was part of the original idea).
Increase RFC Production Center staffing and resources for
editorial cleanups to make documents clear, but use different
criteria for deciding whether a document is "good enough" to
publication to Proposed Standards and Internet Standards.
Narrow AUTH48 for Proposed Standards to a "speak now; explain to
the RFC Editor, IESG, and the community why you need more time;
or just hold your piece"... and, if you are the one author who
has not signed off where others have, note that you will be
dropped to a Contributor by editorial action if you don't
respond within that window.    Identify ADs who are holding
things up and where to the community and future nomcoms, even
explicitly considering it grounds for recall if any one AD is
consistently out of line.  

Would it work?  I obviously have my doubts.  Would it have
chances of working at least as good as trying to devise an
entirely new system and convincing people to understand exactly
what it is about?  IMO, probably yes or at least sooner.   Would
it be equally fast as a WG simply attaching a stamp of approval
to one of its documents?  Probably not quite, but keep in mind
that, once a WG decides a document is ready for publication,
nothing in BCP 9 requires shepherd reports, extensive IESG )or
even AD) review and evaluation, more than two weeks of IETF LC
for WG documents, and then an extensive IESG review and debate
afterwards.  If a document takes more than a month between when
the WG decides it is ready for community review before it is the
hands of the RFC Editor, there is either something substantively
wrong with it, the WG didn't do its job (reflecting badly on WG
leadership and the responsible AD), or we are dealing with
impediments that the community and/or the IESG set up,
presumably in the name of closer attempts at perfection.   If we
think that is a problem, let's fix it, rather than looking for
new terminology that just kicks assorted cans down the road or
that avoids timely review outside the developing WG (again, IETF
LC on WG documents is nominally only two weeks -- that is not
where the problem lies).   And, if all of that means we have too
many WGs and two many work efforts going to be consistent with
quality, let's do some serious prioritizing and change that.

best,
   john

---

From: Keith Moore <[email-masked]>
Date: Sat, 20 Jul 2019 12:16:21 -0400

On 7/20/19 12:01 PM, John C Klensin wrote:
> original intended status.

I'm fairly certain that there's far too much baggage and inertia 
associated with the current notion of Proposed Standard to do that.   
Part of that baggage, of course, includes IESG effectively raising the 
bar for Proposed Standard in order to try to make it actually worth 
deploying.  But part of the baggage is also the well-entrenched practice 
of deploying at or before Proposed.  And a lot of that is due to market 
pressure and "move fast and break things" industry practice that we're 
unlikely to have much influence over.

So if we were going to make a significant change to the process, it 
seems likely that we either keep Proposed Standard similar to what it 
actually is now, and make necessary changes elsewhere in the process 
(most likely prior to WGLC, though changes after PS might also be 
useful), or we rethink the whole thing and assign new names to the 
resulting stages.   I have a lot more confidence in the former approach, 
because of the tremendous hazard of second-system effect with a clean 
sheet approach.

Keith

---

From: "Salz, Rich" <[email-masked]>
Date: Sat, 20 Jul 2019 17:25:07 +0000

>    As editor of that RFC and responsible for that appendix, this is not a suitable candidate for any sort of change.  A moving target would not be good for implementations as it has direct interope…

Martin is right.

---

From: Randy Bush <[email-masked]>
Date: Sat, 20 Jul 2019 14:12:40 -0400

>> a solution, don't we want to study one that apparently works?

worked for us in publishing an op doc which was eventually followed by
an rfc

> IETF has take some responsibility for after-care here.

for ripe's doc repo?

randy

---

From: Jared Mauch <[email-masked]>
Date: Sat, 20 Jul 2019 14:16:19 -0400

> On Jul 20, 2019, at 2:12 PM, Randy Bush <[email-masked]> wrote:
> 

I think an example of a RFC of this sort is 2182/BCP-16.

>> IETF has take some responsibility for after-care here.

Heh :-)

- Jared

---

From: Randy Bush <[email-masked]>
Date: Sat, 20 Jul 2019 14:39:36 -0400

> I think an example of a RFC of this sort is 2182/BCP-16.

uh, non sequitur.  the subject was ripe docs working.  2182 was never a
ripe doc, and the ietf pub cycle was just fine for it.

but maybe folk could get the point here if (the metaphorical) you could
point to an example op doc which is needed asap, is in good shape, and
ietf rfc processing would cause ops issues.  a real example, might help.

randy

---

From: Jared Mauch <[email-masked]>
Date: Sat, 20 Jul 2019 14:57:02 -0400

> On Jul 20, 2019, at 2:39 PM, Randy Bush <[email-masked]> wrote:
> 

I don’t think as much as you do here.  That document has aged well and is good ops advice (for DNS operators).

> ietf rfc processing would cause ops issues.  a real example, might help.

As I said previously, the tools at the IETF are decent, and I can publish something before the day is out if I really think it’s necessary.  Maybe given recent events I should write up things like AS_PATH filtering plus prefix list filtering is a good thing so my ops alerting system will stop telling me of weird routing issues.  It fires all the time and periodically with something big going on (eg: recent 701 event).

- Jared

---

From: Hans Petter Holen <[email-masked]>
Date: Sat, 20 Jul 2019 16:13:04 -0400

On Fri, 19 Jul 2019 at 03:15, Brian E Carpenter <[email-masked]>
wrote:

> Yes but I want a standard for comparison: how long does RIPE take? I have
> never been involved in that side of RIPE, so I have no idea whether they
> manage it in weeks, months or years. What is a realistic target?

For _policy proposals_ there is a well defined process described in RIPE 710
https://www.ripe.net/publications/docs/ripe-710

Which would take you a minmimum of 20ish weeks.

BCPs are not bound by this process, so it is hard to give an accurate
answer.

We just had a discussion last week that we really should write down the
process for developing and publising RIPE documents in general.

Hans Petter Holen
RIPE Chair

-- 
-- Hans Petter Holen Mobile +47 45 06 60 54 | [email-masked] |
http://hph.oslo.net

---

From: Miles Fidelman <[email-masked]>
Date: Sat, 20 Jul 2019 16:54:01 -0400

I've been watching these go by, and I don't believe anyone has mentioned 
the HTML5 Living Standard as a model 
(https://html.spec.whatwg.org/multipage/). It has the advantage of 
working pretty well, over an extended period.

Miles Fidelman

-- 
In theory, there is no difference between theory and practice.
In practice, there is.  .... Yogi Berra

Theory is when you know everything but nothing works.
Practice is when everything works but no one knows why.
In our lab, theory and practice are combined:
nothing works and no one knows why.  ... unknown