Subject: Re: RFC Editor model

From: Henrik Levkowetz <[email-masked]>
Date: Tue, 2 Jul 2019 17:19:50 +0200

Hi Adam,

I posed a simple question to Sara earlier, one I'd really like to have
an answer to.  Your points about fabricated assumptions below brings
this to mind again, because fabricated assumptions are prone to appear
in the absence of fact.  So I'll ask you what I asked Sara, with added
emphasis:

Did the RSE indicate that she did not intend to renew _before_ you informed
her that you were going to put the contract out for bid in 2021, or
_after_, (possibly as a result of you informing her about the early re-bid)?

Regards,

	Henrik

On 2019-06-26 21:29, Adam Roach wrote:
> By way of disclosure, I'll be the first to point out that I'm on both 
> the IESG and the RSOC, and so I'm going to have a certain perspective on 
> the events underway. I hope that my statements below stand on their own, 
> independently of whatever interests my position may imply.
> 
> On 6/26/19 10:20 AM, Michael Richardson wrote:
>> I am claiming that some think that this situation has just occured, and it
>> resulted in the RSE deciding to do something else rather than attempt to
>> continue fighting against some bad thing happening.
>> (I am not saying that I even understand what the "thing" was, or agree that
>> it was "bad")
> 
> 
> You or anyone else for that matter. What happened is:
> 
>  1. We, the community, liked Heather personally
>  2. Heather is leaving
>  3. So we're sad [1]
> 
> 
> You're kind of pointing sideways at some conspiracy theories that people 
> have come up with to explain why #2 happened, but they're not really 
> supported by facts in evidence. This is natural: because of #3, it's 
> understandable to try to find someone to blame. But this is why you're 
> having a hard time understanding what the "thing" is: it's whatever 
> boogeyman the conspiracy theorists have chosen to invent for that 
> moment. And so it's definitely "bad", but it isn't actually "real".
> 
> I'm not saying that all of the critical posts on this topic are wrong. 
> There are some valid points being made about the overall RFC Editor 
> model, its history, and where its future may lie; and some of these are 
> necessarily being couched as criticism.
> 
> But there is also some poorly motivated rage being expressed based on 
> wholly fabricated assumptions, much of which seems to be impervious to 
> facts and unable to cite sources. Again, this is an understandable and 
> natural reaction to being sad, although it is far from helpful. Even 
> worse, it may harm our ability to find a suitable replacement for 
> Heather: who wants to walk into a community full of rage?
> 
> And so I strongly encourage you -- and others -- to be wary of arguments 
> based on supposition. Share what you know and think, but please don't 
> amplify untested theories.
> 
> /a
> 
> ____
> [1] I'm using "sad" here as a proxy for a complicated maelstrom of 
> negative emotions that people seem to be undergoing at the moment. 
> There's probably an entire doctoral thesis's worth of explanation that 
> could be used to describe these emotions more accurately, but I don't 
> have the tools to do so.
> 
>

---

From: S Moonesamy <[email-masked]>
Date: Wed, 03 Jul 2019 12:38:39 -0700

Hi Adam,

I sent an email on 28 June to request some information [1].  May I 
enquire about the status of the request?

By the way, the email published on the Internet Architecture Board 
web site is incorrect [2].

Regards,
S. Moonesamy

1. https://mailarchive.ietf.org/arch/msg/ietf/38dbedALiIwvf80ReoBjvZWRYTA
2. https://www.iab.org/activities/programs/rfc-editor-program/

---

From: Adam Roach <[email-masked]>
Date: Wed, 3 Jul 2019 14:51:17 -0500

Henrik --

Sorry for the delayed response here. I was on holiday through yesterday, 
and am just now digging out from under the email backlog.

I want to be clear that I'm responding on behalf of myself, and not on 
behalf of the RSOC as a whole.

Ted's email ("RSE bid Process") accurately captures the timeline: 
Heather's resignation came after the RSOC's recommendation to the IAB, 
but before the IAB deliberated on said recommendation.

I would take care with suppositions in this space ("possibly as a result 
of..."). Without direct input from Heather, it's difficult to estimate 
how much of a factor this recommendation may have played in her 
decision. I also want to emphasize that the singular goal of the 
recommendation was an attempt to de-risk the process of finding a 
replacement for Heather in the eventuality that she were to step down, 
which is why it was carefully couched in language that explained that 
the recommendation was in no way a reflection of Heather's performance. 
Clearly, things didn't play out the way we hoped, and the irony of the 
actual outcome is not lost on me.

On a personal note, I do very much wish that Heather had chosen to stay 
on as RSE. I'm currently having to deal with the dual hit of losing 
Heather as the RSE as well as the weight of the community's reaction to 
an outcome that I believe no one on the RSOC were steering towards, 
including allegations that we have been less than straightforward in 
explaining our motivations. So I apologize if this mail and others on 
the topic seem a bit defensive.

/a

---

From: Henrik Levkowetz <[email-masked]>
Date: Wed, 3 Jul 2019 22:19:04 +0200

Hi Adam,

> and am just now digging out from under the email backlog.

Ah.  Ok, understood.  Certainly you should be able to have a holiday :-)

> but before the IAB deliberated on said recommendation.

Umm.  Yes, I saw that, but I actually asked if Heather's resignation came
in after _she_ was informed, or before.  I have no information telling me
at which point Heather was informed, relative to RSOC's recommendation to
the IAB; whether she was copied on that, informed before, or informed after.

It seems curious to me that it's so hard to get a clear answer to this point.

Sarah has been silent, Ted didn't mention it, and yet it seems to me that
it must be relevant how and when Heather was informed, even more than when
the IAB was informed of the recommendation.  I'm now at the point when I
am unwilling to back off from this question exactly because there seems to
be such a resistance to answer it clearly.

> actual outcome is not lost on me.

Understood.  And indeed.

> the topic seem a bit defensive.

Ok.  I understand.  And I feel like many others that this has gone very
wrong.  If I was on the RSOC, I think I would first clean house internally,
and then go cap in hand to Heather and ask her to bid in the upcoming RFP,
pretty please.  But the cleanup has to happen first, because I'm very
convinced that there are good reasons beyond just the message I asked
about above (even if I think it had a part) why Heather chose not to
renew.

Best regards,

	Henrik

---

From: Ted Hardie <[email-masked]>
Date: Wed, 3 Jul 2019 13:37:02 -0700

Hi Henrik,

On Wed, Jul 3, 2019 at 1:19 PM Henrik Levkowetz <[email-masked]>
wrote:

> The mail to Heather with the recommendation went before the mail to the
IAB, but only by a few minutes.  Since Heather is on the IAB list which
received the recommendation, she would have gotten the same information
from that.  Unless she happened to be reading her email when the messages
were received, it's probably best to treat them as arriving simultaneously.

regards,

Ted

---

From: Henrik Levkowetz <[email-masked]>
Date: Wed, 3 Jul 2019 22:40:36 +0200

Hi Ted,

> were received, it's probably best to treat them as arriving simultaneously.

Thank you.  I'm glad to have a clear answer to that.

Best regards,

	Henrik

---

From: "Scott O. Bradner" <[email-masked]>
Date: Wed, 3 Jul 2019 16:42:47 -0400

> On Jul 3, 2019, at 4:37 PM, Ted Hardie <[email-masked]> wrote:
> 
> Hi Henrik,
> 
> On Wed, Jul 3, 2019 at 1:19 PM Henrik Levkowetz <[email-masked]> wrote:
> Hi Adam,
> 
> the IAB; whether she was copied on that, informed before, or informed after..
> 
> The mail to Heather with the recommendation went before the mail to the IAB, but only by a few minutes.  Since Heather is on the IAB list which received the recommendation, she would have gotten the same information from that.  Unless she happened to be reading her email when the messages were received, it's probably best to treat them as arriving simultaneously.

so Heather resigned after receiving the message (through one or the other path)?  (which I think was Henrik’s
question)

Scott

---

From: Ted Hardie <[email-masked]>
Date: Wed, 3 Jul 2019 13:53:47 -0700

On Wed, Jul 3, 2019 at 1:42 PM Scott O. Bradner <[email-masked]> wrote:

> She notified the RSOC, IAB, and IETF LLC on June 7th, a day after the
messages noted above were sent.

regards,

Ted

---

From: Adam Roach <[email-masked]>
Date: Wed, 3 Jul 2019 16:05:32 -0500

On 7/3/19 3:19 PM, Henrik Levkowetz wrote:
> in after_she_  was informed, or before.  I have no information telling me

Sorry! I was not intending to be evasive -- I just had incorrect 
assumptions in my head when I wrote this: in the back of my mind, I 
thought that Sarah's initial email indicated that Heather received the 
recommendation email to the IAB, since it was copied to the RSOC mailing 
list, to which she belongs. I see that this hasn't been mentioned yet 
(or, at least, not prominently), so I can see why this looks like we're 
trying to hide information. That wasn't my intention, and I apologize 
for the confusion.

/a

---

From: Adam Roach <[email-masked]>
Date: Wed, 3 Jul 2019 16:18:53 -0500

> enquire about the status of the request?

Sorry, it's easy for that kind of thing to get lost with the current 
volume of discussion currently underway. The query you're pointing to 
is, I believe, one about what impact YANG module documents have on the 
SLA. This is the kind of question that I would generally defer to the 
RSE and RPC, as it hasn't been specifically brought to our attention as 
a factor in meeting the SLAs one way or another.

Having spent significant time reviewing documents containing YANG 
modules, I'm happy to hazard a guess, but that's all it is. The contents 
of the YANG modules themselves have two relevant characteristics. First, 
they are written in a formal language for which we have a suite of 
tools, meaning they almost certainly require less proofreading and 
editing than normal English prose. At the same time, the contents of the 
modules are more sparse than page-filling paragraphs. So if I had to 
guess, these combined factors mean that they *probably* increase the 
page count of documents in which they appear while increasing the 
required editing work at a proportionally smaller rate as compared to 
prose specification. Since the SLA tiers are determined by total pages 
gone to edit, this would probably have a marginal -- although I would 
guess insignificant -- effect of making the SLA easier to meet.

If you want something better than my somewhat-educated guess on the 
topic, I encourage you to reach out to Heather directly at 
<[email-masked]>.

/a

---

From: Henrik Levkowetz <[email-masked]>
Date: Wed, 3 Jul 2019 23:33:55 +0200

Hi Adam,

> trying to hide information. That wasn't my intention, and I apologize 

Ok, no worries.  Thanks for clarifying!

	Henrik

---

From: S Moonesamy <[email-masked]>
Date: Wed, 03 Jul 2019 17:35:28 -0700

Hi Adam,
At 02:18 PM 03-07-2019, Adam Roach wrote:
>volume of discussion currently underway. The query you're pointing 
>to is, I believe, one about what impact YANG module documents have 
>on the SLA. This is the kind of question that I would generally 
>defer to the RSE and RPC, as it hasn't been specifically brought to 
>our attention as a factor in meeting the SLAs one way or another.

Thank you for the explanation.

>topic, I encourage you to reach out to Heather directly at 

Ok.

Regards,
S. Moonesamy

---

From: Heather Flanagan <[email-masked]>
Date: Mon, 8 Jul 2019 09:26:22 -0700

> On Jul 3, 2019, at 5:35 PM, S Moonesamy <[email-masked]> wrote:
> 
> Hi Adam,
>> Sorry, it's easy for that kind of thing to get lost with the current volume of discussion currently underway. The query you're pointing to is, I believe, one about what impact YANG module documents have on the SLA. This is the kind of question that I would generally defer to the RSE and RPC, as it hasn't been specifically brought to our attention as a factor in meeting the SLAs one way or another.
> 
>> If you want something better than my somewhat-educated guess on the topic, I encourage you to reach out to Heather directly at <[email-masked]>.
> 
> Ok.
> 

Hola a todos, and sorry it took long for me to find this in the thread! I can provide some information, but not hard numbers - editors don’t use a stopwatch to determine exactly how long each step takes, as that would both take an unfortunate amount of time and drive us insane for very little benefit.

Checking YANG modules became quite challenging in 2018, because it was no longer about ensuring only that the module parsed, but that it was well formatted (per pyang) as well.  The formatting part of the process took a substantial amount of effort, as the tools to check the YANG modules were still in development during that time, and we found out much later that the structure of the YANG modules as generated by the tools was not something that had full buy-in from the community.  So, the RPC found itself working closely with the tool maintainer and the authors to get well-formatted YANG modules, experimenting with the updated toolset and updating procedures, training staff on the new features of the tool and procedures, and sometimes manually editing the automated output because the author(s) disagreed with parts of the formatting.  

While the RPC will continue to check the format of YANG modules, we have suggested updating the YANG-related documentation for authors and we have asked the IESG to consider how the authors can be involved earlier in the process, as AUTH48 does not seem to be the correct place to initiate this check (especially if the authors are unaware that this is step is happening).

I hope that helps answer your question; let me know if you’d like to discuss further.

-Heather

---

From: Kent Watsen <[email-masked]>
Date: Wed, 10 Jul 2019 02:52:56 +0000

Ignas, Martin, and I met with Sandy (all CC-ed) during IETF 104 to the YANG module formatting issue described by Heather below.

The plan was to quickly add a process step (to test formatting) into the shepherd writeup, followed later integration into Datatracker submission checks.  The goal was/is to remove this burden from the copyeditors entirely.

Unfortunately, I don't see the process step added to https://www.ietf.org/chairs/document-writeups <https://www.ietf.org/chairs/document-writeups> yet...

To be clear, the shepherd steps are roughly:

  1)  pyang -f yang --keep-comments --max-line-length 69 <infile> > <outfile>
 2) diff <infile> <outfile>
 3) explain why any diffs are acceptable.

Kent // NETMOD co-chair

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 10 Jul 2019 15:15:34 +0200

Kent Watsen <[email-masked]> writes:

> Ignas, Martin, and I met with Sandy (all CC-ed) during IETF 104 
> to the YANG module formatting issue described by Heather below. 
> 
> The plan was to quickly add a process step (to test formatting) 
> into the shepherd writeup, followed later integration into 
> Datatracker submission checks.  The goal was/is to remove this 
> burden from the copyeditors entirely.

It would IMO be much better to remove most of this burden from 
*everybody* and do the visual formatting of YANG modules 
automatically along with the rest of the RFC or I-D document. In 
my own experience, it is entirely possible if YANG modules are 
written in the XML syntax (YIN) using additional logical markup in 
descriptions and some (scarce) processing instructions as 
formatting hints. Here is an example:

https://github.com/CZ-NIC/iana-dns-zone-types/blob/master/iana-dns-class-rr-type.yinx

I have been preparing all my YANG modules this way. As a module 
author, I never need, e.g., to reformat a description after a 
minor edit.

In practical terms, the xml2rfc source with embedded YANG modules 
could be a compound XML document in which the YANG parts live in 
their own XML namespace. Another huge advantage of this approach 
would be that HTML-formatted RFCs could also include 
HTML-formatted YANG modules with hyperlinks, prettyprinting etc.

Lada

PS. Here is the toolchain that I use:

https://github.com/llhotka/YANG-I-D
 
> 
> Unfortunately, I don't see the process step added to 
> https://www.ietf.org/chairs/document-writeups 
> <https://www.ietf.org/chairs/document-writeups> yet... 
> 
>  2) diff <infile> <outfile> 3) explain why any diffs are 
>  acceptable. 
> 
> Kent // NETMOD co-chair 
>   
>> Hola a todos, and sorry it took long for me to find this in the 
>> thread! I can provide some information, but not hard numbers - 
>> editors don’t use a stopwatch to determine exactly how long 
>> each step takes, as that would both take an unfortunate amount 
>> of time and drive us insane for very little benefit.   Checking 
>> YANG modules became quite challenging in 2018, because it was 
>> no longer about ensuring only that the module parsed, but that 
>> it was well formatted (per pyang) as well.  The formatting part 
>> of the process took a substantial amount of effort, as the 
>> tools to check the YANG modules were still in development 
>> during that time, and we found out much later that the 
>> structure of the YANG modules as generated by the tools was not 
>> something that had full buy-in from the community.  So, the RPC 
>> found itself working closely with the tool maintainer and the 
>> authors to get well-formatted YANG modules, experimenting with 
>> the updated toolset and updating procedures, training staff on 
>> the new features of the tool and procedures, and sometimes 
>> manually editing the automated output because the author(s) 
>> disagreed with parts of the formatting.     While the RPC will 
>> continue to check the format of YANG modules, we have suggested 
>> updating the YANG-related documentation for authors and we have 
>> asked the IESG to consider how the authors can be involved 
>> earlier in the process, as AUTH48 does not seem to be the 
>> correct place to initiate this check (especially if the authors 
>> are unaware that this is step is happening).   I hope that 
>> helps answer your question; let me know if you’d like to 
>> discuss further.   -Heather 
> 

-- 
Ladislav Lhotka Head, CZ.NIC Labs PGP Key ID: 0xB8F92B08A9F76C67

---

From: S Moonesamy <[email-masked]>
Date: Wed, 10 Jul 2019 10:37:39 -0700

Hi Heather,
At 09:26 AM 08-07-2019, Heather Flanagan wrote:
>editors don't use a stopwatch to determine exactly how long each 
>step takes, as that would both take an unfortunate amount of time 
>and drive us insane for very little benefit.

Thanks for providing some information.

>Checking YANG modules became quite challenging in 2018, because it 
>was no longer about ensuring only that the module parsed, but that 
>it was well formatted (per pyang) as well.  The formatting part of 
>the process took a substantial amount of effort, as the tools to 
>check the YANG modules were still in development during that time, 
>and we found out much later that the structure of the YANG modules 
>as generated by the tools was not something that had full buy-in 
>from the community.  So, the RPC found itself working closely with 
>the tool maintainer and the authors to get well-formatted YANG 
>modules, experimenting with the updated toolset and updating 
>procedures, training staff on the new features of the tool and 
>procedures, and sometimes manually editing the automated output 
>because the author(s) disagreed with parts of the formatting.

The above would probably have an impact on the SLA for documents 
published.  I understand that it may be a step to far to use a 
stopwatch to do an assessment.  However, it would be useful to 
consider the impact on the other parts of the RFC Editor puzzle.

>While the RPC will continue to check the format of YANG modules, we 
>have suggested updating the YANG-related documentation for authors 
>and we have asked the IESG to consider how the authors can be 
>involved earlier in the process, as AUTH48 does not seem to be the 
>correct place to initiate this check (especially if the authors are 
>unaware that this is step is happening).

Was there an Area Director who is responsible for considering the 
upstream changes?

Regards,
S. Moonesamy