Subject: BCP97bis

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 15 Oct 2021 10:12:53 -0700

Colleagues,

I've got a draft that seeks to update BCP 97, which is the guidance around
how we handle normative downward references.  It's currently made up of
three separate RFCs and an erratum, so this will consolidate those into a
single document.  The main mission here, though, is to update the guidance
around normative references to external documents, especially those behind
paywalls.

The draft is being sponsored by Erik Kline and can be found in the
datatracker here:
https://datatracker.ietf.org/doc/draft-kucherawy-bcp97bis/

Feedback is welcome, either on this thread or to me or Erik directly.  If
people are generally happy with it as-is, we can initiate Last Call before
IETF 112 begins next month.

Thanks,

-MSK, ART AD

---

From: Russ Housley <[email-masked]>
Date: Fri, 15 Oct 2021 15:27:20 -0400

Hi Murray.

I have on concern and a few editorial suggestions.

CONCERN

Section 4.1 says:

   o  A note is included in the reference text that indicates that the
      reference is to a target document of a lower maturity level, that
      some caution should be used since it may be less stable than the
      document from which it is being referenced, and, optionally,
      explaining why the downref is appropriate.

There are many cases where cryptographic algorithms are specified in Informational RFC, and then a Standards-Track document is used to specify protocol conventions for using that algorithm.  the algorithm specification is not unstable, and requiring a note like this sends the wrong message to the reader.

EDITORIAL

Section 1 says:

   It should also be noted that Best Current Practice documents
   [RFC1818] have generally been considered similar to Standards Track
   documents in terms of what they can reference.  For example, a
   normative reference to an Experimental RFC has been considered an
   improper reference per [RFC2026].

These two sentences are not really making the same point.  With the second sentence starting with "For example", I expected it to be related to the first sentence.

Section 1.1 uses the term "new RFC".  However, the "new" is not needed.  The defintion ofr a normative reference apply to very old RFCs too.

Russ

> On Oct 15, 2021, at 1:12 PM, Murray S. Kucherawy <[email-masked]> wrote:
> 
> Colleagues,
> 
> I've got a draft that seeks to update BCP 97, which is the guidance around how we handle normative downward references.  It's currently made up of three separate RFCs and an erratum, so this will consolidate those into a single document.  The main mission here, though, is to update the guidance around normative references to external documents, especially those behind paywalls.
> 
> The draft is being sponsored by Erik Kline and can be found in the datatracker here:
> https://datatracker.ietf.org/doc/draft-kucherawy-bcp97bis/ <https://datatracker.ietf.org/doc/draft-kucherawy-bcp97bis/>
> 
> Feedback is welcome, either on this thread or to me or Erik directly.  If people are generally happy with it as-is, we can initiate Last Call before IETF 112 begins next month.
> 
> Thanks,
> 
> -MSK, ART AD

---

From: Scott Bradner <[email-masked]>
Date: Fri, 15 Oct 2021 15:38:32 -0400

RFC 1818 is not a good reference for BCPs - RFC 2026 is the better one

Scott

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 15 Oct 2021 21:47:22 -0700

Hi Scott,

> RFC 1818 is not a good reference for BCPs - RFC 2026 is the better one

That reference is preserved from RFC 3967.  Also, RFC 1818 is not obsolete,
so presumably what it says is still valid.

Should that spot refer to both 2026 and 1818, perhaps?

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 15 Oct 2021 21:56:08 -0700

Hi Russ,

On Fri, Oct 15, 2021 at 12:27 PM Russ Housley <[email-masked]> wrote:

> There are many cases where cryptographic algorithms are specified in
> Informational RFC, and then a Standards-Track document is used to specify
> protocol conventions for using that algorithm.  the algorithm specification
> is not unstable, and requiring a note like this sends the wrong message to
> the reader.
>

Interesting.  This text is preserved from RFC 4897.  But now that you
mention it, I don't think this particular bit of process has ever been used
for as long as I've been observing.

Also, the paragraph immediately after that one gives the IESG discretion
about what such a note should include.  The case you cite seems to me to be
an ideal one in which to use that discretion, perhaps with the advice of
the authors/editors/chairs.

EDITORIAL
> These two sentences are not really making the same point.  With the second
> sentence starting with "For example", I expected it to be related to the
> first sentence.
>

And that one is copied right out of RFC 3967, which got all this started.
But I agree.  Perhaps "For example, a normative reference from a BCP to an
Experimental ..." ?

Section 1.1 uses the term "new RFC".  However, the "new" is not needed.
> The defintion ofr a normative reference apply to very old RFCs too.
>

Also copied from RFC 3967, but I agree.  Will edit accordingly.

-MSK

---

From: David Farmer <[email-masked]>
Date: Sat, 16 Oct 2021 00:50:03 -0500

On Fri, Oct 15, 2021 at 23:47 Murray S. Kucherawy <[email-masked]>
wrote:

> That reference is preserved from RFC 3967.  Also, RFC 1818 is not
> obsolete, so presumably what it says is still valid.
>

Actually, RFC 1818 is marked Historic and RFC 2026 Section 5 has a much
more complete discussion of the purpose of a BCP and the processes
associated with them, at least in my opinion.

https://datatracker.ietf.org/doc/rfc1818/

Thanks

-- 
===============================================
David Farmer               Email:[email-masked]
Networking & Telecommunication Services
Office of Information Technology
University of Minnesota
2218 University Ave SE        Phone: 612-626-0815
Minneapolis, MN 55414-3029   Cell: 612-812-9952
===============================================

---

From: Brian Carpenter <[email-masked]>
Date: Sat, 16 Oct 2021 19:25:32 +1300

The RFC Editor site also states that 1818 was downgraded from BCP. It seems
to be yet another error in RFC2026 that it does not formally obsolete 1818.

Regards,
    Brian Carpenter
    (via tiny screen & keyboard)

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 15 Oct 2021 23:51:30 -0700

On Fri, Oct 15, 2021 at 10:50 PM David Farmer <[email-masked]> wrote:

> associated with them, at least in my opinion

Ah, you're right.  I overlooked that annotation.  Thanks.

-MSK

---

From: Scott Bradner <[email-masked]>
Date: Sat, 16 Oct 2021 06:45:53 -0400

RFC 1818 is Historic - 

1818 Best Current Practices. J. Postel, T. Li, Y. Rekhter. August 1995.
     (Format: TXT, HTML) (Status: HISTORIC) (DOI: 10.17487/RFC1818)

and thus should not be a normative reference

Scott

> On Oct 16, 2021, at 12:47 AM, Murray S. Kucherawy <[email-masked]> wrote:
> 
> Hi Scott,
> 
> That reference is preserved from RFC 3967.  Also, RFC 1818 is not obsolete, so presumably what it says is still valid.
>

---

From: Russ Housley <[email-masked]>
Date: Sat, 16 Oct 2021 11:45:30 -0400

> On Oct 16, 2021, at 12:56 AM, Murray S. Kucherawy <[email-masked]> wrote:
> 
> Hi Russ,
> 
> On Fri, Oct 15, 2021 at 12:27 PM Russ Housley <[email-masked] <mailto:[email-masked]>> wrote:
> Interesting.  This text is preserved from RFC 4897.  But now that you mention it, I don't think this particular bit of process has ever been used for as long as I've been observing.
> 
> Also, the paragraph immediately after that one gives the IESG discretion about what such a note should include.  The case you cite seems to me to be an ideal one in which to use that discretion, perhaps with the advice of the authors/editors/chairs.

I think that the text should be included ONLY when the downref is less stable than the document making the reference.

> And that one is copied right out of RFC 3967, which got all this started.  But I agree.  Perhaps "For example, a normative reference from a BCP to an Experimental ..." ?

That is an improvement.

> Also copied from RFC 3967, but I agree.  Will edit accordingly.

Thanks.

Russ

---

From: "Salz, Rich" <[email-masked]>
Date: Sat, 16 Oct 2021 20:25:32 +0000

*   I think that the text should be included ONLY when the downref is less stable than the document making the reference.

I agree.  That requires some judgement, perhaps, but that’s okay.

---

From: Michael Richardson <[email-masked]>
Date: Sat, 16 Oct 2021 16:29:12 -0400

{reading the document before I read the thread of replies}

Thank you for doing this revision: this kind of work is often thankless.

A question as I read section 4.2 (The IESG).
I think that idnits presently finds and notes downrefs.
As a document shepherd, I've never had to research them.
(Maybe that's a fault on my part, maybe I should have been)
I'm mostly thinking that downrefs should ideally be noted and resolved during
WGLC.  I think that this happens, but maybe I have too small a sample.
Are we having a problem getting downrefs identified early enough, or are
things okay?

I also wonder if we shouldn't have a process to attempt to clean the downref
registry out, but reclassifying the documents.  The repeated MD5 and HMAC
document listed as examples seem like good datapoints. (MD5 might be
Historic, but HMAC is still awesome)

} At a minimum, authors/editors of source documents need to secure freely
} available copies of the target documents for use by all anticipated reviewers
} during the source document's life cycle, which includes working group
} participants, any member of the community that chooses to participate in Last
} Call discussions, area review teams, IANA expert reviewers, and members of
} the IESG.

It seems to me, if this requirement is to be taken seriously, that either the
DT or some other resource needs to contain notes on how to obtain the
references.
My opinion is that the IETF has allowed the IEEE to embrace and suffocate the
open-stand.org effort.  During RFC8995 review, authors were told by many
about about 802.1AR had a new revision and shoudln't we update to it.
But, the new revision was unobtainable because the IEEE had 500 errors on
their web site for more than a year, which is why we did not cite it.
I feel that it shouldn't be just my problem to solve this problem on my own.

I have very strong opinions about what happens when a standard becomes part
of a lawful regulation.   It then becomes part of a law, and the public
should have full access to know the law. (Because, ignorance is not a defense)
This is particularly important for regulations relating to safety.
(I could insert "cyber" before safety, but the word cyber is almost always
redundant).  In particular,  it shouldn't cost money to find out if a product
one is being subjected to is safe.

As background to a talk I was asked to do, I actually have been doing some
reading on requirements to publish laws, and I thought it would be a
no-brainer in the UN Charter, but I was surprised.  Surely, Canada's 1982
Constitution and Charter of Rights would say something.... not obvious
actually.  Just that anything published federally has to be in both official
languages, and a bunch of provincial statements.  Some papers on how
governments have used copyright to generate revenue from publishing the laws
and legal decisions.  Only the Yukon territorial constiution has an actual
mandate.

A reason I choose to do work at the IETF is because I could read the
specifications (using RFC/FTP by email...) without hassle.

So I am reacting the word "At a minium", and I don't think we are even
getting to that minimum.   There are a bunch of specifications that aren't
just transitioning from some legacy specification to an RFC, but which we are
allocating numbers/etc. for entities which make it very difficult to read their
documents.  I think that we should be doing more than the minimum here.

} Appendix A. Changes Since RFC8067
} The following are the changes in this document relative to the prior state of BCP 97:
}
} o Apply erratum #2793.

https://www.rfc-editor.org/info/rfc8067
doesn't mention any errata.
https://www.rfc-editor.org/errata_search.php?eid=2793  indicates that it is
errata against RFC4897.

--
Michael Richardson <[email-masked]>   . o O ( IPv6 IøT consulting )
           Sandelman Software Works Inc, Ottawa and Worldwide

---

From: Michael Richardson <[email-masked]>
Date: Sat, 16 Oct 2021 16:38:35 -0400

> Section 4.1 says:

    } o  A note is included in the reference text that indicates that the
    } reference is to a target document of a lower maturity level, that
    } some caution should be used since it may be less stable than the
    } document from which it is being referenced, and, optionally,
    } explaining why the downref is appropriate.

    > Informational RFC, and then a Standards-Track document is used to
    > specify protocol conventions for using that algorithm.  the algorithm
    > specification is not unstable, and requiring a note like this sends the
    > wrong message to the reader.

I think that this is really a process/RFC2026 process bug that maybe we could
find a way to slightly amend RFC2026 about.  (How many gendispatches ago was
the 2026 revision discussion...?)

I think that we document cryptographic algorithms as Informational because it
is not the IETF's goal to take change control over the algorithm.
It's not that we don't think of them as Standards-Track, it's that our
standards track rules say that can't unless we have change control.
The IETF RFC, because it's accessibility, often become the defacto industry
reference.   It is cited way beyond being cited in some other RFC.

Why doesn't the industry (and the IETF?) cite the original NIST,
etc. document?  Well, historically, because the documents predate the WWW, we
had RFC-by-FTP-email, and NIST didn't.

My thoughts are:

a) create some new category which is pseudo-authoritative standards track,
   which the IESG can annoit a document with.
b) cite the original documents/papers/NIST-etc. publications more often.

Perhaps CFRG has already struggled with this question before.
At this point, what I see is that there is significant original work being
brought to the CFRG *first*, that a CFRG RFC is becoming authoritative for
some algorithms. (No, I don't have an example.)

---

From: Larry Masinter <[email-masked]>
Date: Sat, 16 Oct 2021 15:24:19 -0700

If there were a system that made copies of anything linked in a document, how many other kinds of stability would you need?

I thought doing so could be the default. 

To make a stable reference in an unstable world, add a date. 

draft-masinter-dated-uri-10 - The 'tdb' and 'duri' URI schemes, based on dated URIs (ietf.org) <https://datatracker.ietf.org/doc/draft-masinter-dated-uri/> 

--

 <https://LarryMasinter.net> https://LarryMasinter.net  <https://interlisp.org> https://interlisp.org

From: ietf <[email-masked]> On Behalf Of Salz, Rich
Sent: Saturday, October 16, 2021 1:26 PM
To: Russ Housley <[email-masked]>; Murray S. Kucherawy <[email-masked]>
Cc: IETF <[email-masked]>
Subject: Re: BCP97bis

---

From: "Joel M. Halpern" <[email-masked]>
Date: Sat, 16 Oct 2021 19:10:59 -0400

Just make a copy might work if we have the legal right to do so. 
(There are other complications, but they pale beside this one.  In the 
case of IEEE specs that Michael Richardson has been talking about, we do 
NOT have a right to make a copy and give it away.)

Yours,
Joel

On 10/16/2021 6:24 PM, Larry Masinter wrote:
> If there were a system that made copies of anything linked in a 
> document, how many other kinds of stability would you need?
> 
> draft-masinter-dated-uri-10 - The 'tdb' and 'duri' URI schemes, based on 
> dated URIs (ietf.org) 
> <https://datatracker.ietf.org/doc/draft-masinter-dated-uri/>
> 
> --
> 
> https://LarryMasinter.net <https://LarryMasinter.net> 
> https://interlisp.org <https://interlisp.org>
> 
> *From:* ietf <[email-masked]> *On Behalf Of *Salz, Rich
> *Sent:* Saturday, October 16, 2021 1:26 PM
> *To:* Russ Housley <[email-masked]>; Murray S. Kucherawy 
> <[email-masked]>
> *Cc:* IETF <[email-masked]>
> *Subject:* Re: BCP97bis
> 
>   * I think that the text should be included ONLY when the downref is
>     less stable than the document making the reference.
>

---

From: Michael Richardson <[email-masked]>
Date: Sat, 16 Oct 2021 19:49:37 -0400

Joel M. Halpern <[email-masked]> wrote:
    > Just make a copy might work if we have the legal right to do so. (There are
    > other complications, but they pale beside this one.  In the case of IEEE
    > specs that Michael Richardson has been talking about, we do NOT have a right
    > to make a copy and give it away.)

For anything that has an archive.org copy, we could use that.

IEEE makes ure that archive.org can't archive their stuff.

---

From: Brian E Carpenter <[email-masked]>
Date: Sun, 17 Oct 2021 13:13:04 +1300

> IEEE makes ure that archive.org can't archive their stuff.

If I have this right (and Scott Bradner will probably correct me if
I'm wrong), the basic rules in RFC2026 section 7.1.1 recognize the reality
that some open standards are essential and unavoidable references which
are not available free of charge to the general public.

To be clear, the phrase "open standard" in that section doesn't
mean "free of charge". (Long essay on what it *does* mean elided.)

(IEEE 802 and numerous CCITT Recommendations were the original
problem cases, I think, and that was about hard copies, of course.)

Wherever possible we try to avoid dependency on such references, but
when you can't, there really isn't any choice.

  Brian

---

From: Larry Masinter <[email-masked]>
Date: Sat, 16 Oct 2021 18:20:51 -0700

This goes into the same mix as "SDO's rethinking business model".
Surely IEEE would benefit from working citations from IETF to IEEE
documents, in a form that IEEE commits to stability.

https://www.w3.org/blog/news/archives/2875

References to resources that can't be read by IETF reviewers
should not be accepted -- perhaps there were some cases in the
past but things have changed.

References to resources that are available now but you're worried they will be moved or exhobitantly paywalled or lost -- keep "if needed" copies.
Just to take "stability of reference" off the table. I read of legal documents and judements being subject to this for similar archival goals.

--
https://LarryMasinter.net https://interlisp.org

---

From: "John Levine" <[email-masked]>
Date: 16 Oct 2021 22:31:48 -0400

>mean "free of charge". (Long essay on what it *does* mean elided.)

As I am sure you are aware, our model of giving away copies of our
standards for free used to be extremely unusual, and is still far from
universal. Historically, SDOs like ANSI and ISO got a lot of their
funding from selling paper copies of their standards. I have a bunch
of paper ANSI standards that I bought a long time ago, at their office
in Times Square and paid with a paper check.

R's.
John

PS: It probably would be best not to inquire too closely about the status of RFC 20.

---

From: Scott Bradner <[email-masked]>
Date: Sun, 17 Oct 2021 07:25:53 -0400

no correction needed - remember that when 2026 was approved most SDOs (e.g., the ITU-T) did not make their
standards available for free so that had to be part of the world the IETF lived in

fwiw - I think that ANSI X3.4-1986 (the standard used as an example in RC 2026)  was not available for free at that point

Scott

> On Oct 16, 2021, at 8:13 PM, Brian E Carpenter <[email-masked]> wrote:
>

---

From: John C Klensin <[email-masked]>
Date: Sun, 17 Oct 2021 08:47:10 -0400

--On Sunday, October 17, 2021 07:25 -0400 Scott Bradner
<[email-masked]> wrote:

> no correction needed - remember that when 2026 was approved
> most SDOs (e.g., the ITU-T) did not make their standards
> available for free so that had to be part of the world the
> IETF lived in
> 
> fwiw - I think that ANSI X3.4-1986 (the standard used as an
> example in RC 2026)  was not available for free at that point

That is correct.  It was easier to get permission from some ANSI
accredited SDOs to reprint parts of standards than it was with
many other bodies, but the standard itself was generally not
available for free.  It is probably also worth remembering that
they generally tried to minimize or avoid registration fees for
meeting in the interest of openness and diversity of opinions.
Partially as a result, revenue from sales of those standards was
a significant fraction of the income that kept those bodies
going, i.e., they were not being restrictive just to be
difficult.  For some SDOs, their way of thinking about that
tradeoff has not changed very much.

FWIW, I have no idea whether permission was obtained to extract
and reproduce material from X3.4-1968 to act as the basis for
RFC 20 or whether the conclusion at the time was that it was not
necessary.  Categories like "Information" and "Internet
Standard" did not exist at the time, so those choices did not
need to be made at the time.

    john

   john

---

From: Carsten Bormann <[email-masked]>
Date: Sun, 17 Oct 2021 14:54:23 +0200

On 2021-10-17, at 14:47, John C Klensin <[email-masked]> wrote:
> necessary.  

(Random fuzzy recollection: I seem to remember that what made me start campaigning for RFC 20 to be a STD was that I wanted to stop people from referencing X3.4 just in order to avoid a downref.)

We should do this more often.

Grüße, Carsten

---

From: "Salz, Rich" <[email-masked]>
Date: Sun, 17 Oct 2021 17:13:52 +0000

*   To make a stable reference in an unstable world, add a date.

Or add text “retrieved on xxx xxx xxx” like Wikipedia often does.

---

From: John C Klensin <[email-masked]>
Date: Sun, 17 Oct 2021 14:04:14 -0400

--On Sunday, October 17, 2021 17:13 +0000 "Salz, Rich"
<rsalz=[email-masked]> wrote:

> Or add text "retrieved on xxx xxx xxx" like Wikipedia
> often does.

That makes it clear what is being referenced, which is good.
Absent other provisions/ requirements, it does nothing to
guarantee that the content will be there when someone goes
looking for it years (or decades) later.

   john

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 18 Oct 2021 08:50:34 +1300

> looking for it years (or decades) later.

Indeed. I've used that formulation in academic publications,
and in my mind it's an abbreviation for "I apologise if this
citation is no longer valid when you happen to look for it,
and I would have preferred a properly archived reference but
I was too lazy to find one."

Which isn't really good enough for a normative reference in
a standard, of course.

    Brian

---

From: Russ Housley <[email-masked]>
Date: Sun, 17 Oct 2021 15:52:51 -0400

> On Oct 17, 2021, at 8:54 AM, Carsten Bormann <[email-masked]> wrote:
> 

Wouldn't it have been trivial to add RFC 20 to the downref registry?  Of course, there is no need to do it now that it is a standard.

0020 ASCII format for network interchange. V.G. Cerf. October 1969.
     (Format: TXT, PDF, HTML) (Also STD0080) (Status: INTERNET STANDARD)
     (DOI: 10.17487/RFC0020) 

Russ

---

From: Carsten Bormann <[email-masked]>
Date: Sun, 17 Oct 2021 23:19:44 +0200

> On 2021-10-17, at 21:52, Russ Housley <[email-masked]> wrote:
> 
> 
> 
> Wouldn't it have been trivial to add RFC 20 to the downref registry?  

To those that know about that concept, yes.
To those that knee-jerk to an idnits throw-away comment, not so sure.

> Of course, there is no need to do it now that it is a standard.
> 

Indeed, and I think the outcome was a well-deserved advancement.
The only RFC that survived from the 1960s or 1970s in std-index…

             .oOo.

But back to the subject: maybe we need a “behind-the-wall-ref” registry…

E.g., in IoT we sometimes need to exchange data about temperatures.
The quantities and units for that are defined in ISO/IEC 80000, which is paywalled.
(Fortunately, the actual definitions in that tome are in the “Definitions” section, most of which are freely available via “OBP” [e.g., [1]] — but that is just distracting from my point.)
Of course, we can point to secondary sources such as BIPM’s SI brochure [2], but we wouldn’t accept a document referencing TLS via Wikipedia, would we?

So once we have decided we can reference ISO/IEC 80000 with impunity (*), it should be put into a behind-the-wall-ref registry.

Grüße, Carsten

(*) which seems to have happened with RFC 8428 and RFC 8798
[1] https://www.iso.org/obp/ui/#iso:std:iso:80000:-3:ed-2:v1:en 
[2] https://www.bipm.org/documents/20126/41483022/SI-Brochure-9.pdf/fcf090b2-04e6-88cc-1149-c3e029ad8232

---

From: John C Klensin <[email-masked]>
Date: Sun, 17 Oct 2021 18:45:17 -0400

--On Sunday, October 17, 2021 15:52 -0400 Russ Housley
<[email-masked]> wrote:

>>> FWIW, I have no idea whether permission was obtained to
>>> extract and reproduce material from X3.4-1968 to act as the
>>> basis for RFC 20 or whether the conclusion at the time was
>>> that it was not necessary.  
>> 
>> (Random fuzzy recollection: I seem to remember that what made
>> me start campaigning for RFC 20 to be a STD was that I wanted
>> to stop people from referencing X3.4 just in order to avoid a
>> downref.)
>> 
> Wouldn't it have been trivial to add RFC 20 to the downref
> registry?  Of course, there is no need to do it now that it is
> a standard.
> 
> 0020 ASCII format for network interchange. V.G. Cerf. October
> 1969.      (Format: TXT, PDF, HTML) (Also STD0080) (Status:
> INTERNET STANDARD)      (DOI: 10.17487/RFC0020) 

Carsten's memory is almost certainly better than mine, but I
think we may have moved it to full standard, probably given its
age from "UNKNOWN" long before the downref registry was an
established practice.  Also, from my perspective, we shouldn't
be making decisions like that on the basis of what might or
might not be trivial.   Or, from a different perspective,
reclassifying it to Internet Standard more accurately reflects
the status of ASCII on the Internet and should have been equally
trivial: I can't imagine anyone being about to make a serious
claim either that it has not been implemented, that it is not
widely deployed, or that there have been any issues with
interoperability (except with things that don't claim to be
ASCII).

I actually think that suggests something that should probably be
considered for BCP97bis: that the downref procedure and registry
should be used only when there are substantive reasons why the
relevant document cannot be upgraded or that doing so would
require an unreasonable amount of effort.  That would strengthen
the text that now appears in the last paragraphs of Murray's
Sections 4.2 and 5 but even the current text suggests to me that
"trivial" is not a good enough reason for the use of that
registry.

I also just noticed that the draft does not appear to describe
the contents and format of that registry, what entity is
responsible for keeping it, and where.  Especially if we take
the position that, once something is in that registry, no
special procedures (or different procedures) need be followed to
use the reference in another document, the registry should
record why downref permission was granted, in which the
document's categories the reference falls, and any additional
explanation that seems necessary -- that information should not
just be in the Last Call.  My instinct tells me that the RFC
Editor Function should be responsible for the registry itself,
but that might raise issues I have not thought of yet.

best,
   john

    john

---

From: John C Klensin <[email-masked]>
Date: Sun, 17 Oct 2021 18:49:58 -0400

> I was too lazy to find one."

And I have been a reviewer and on an editorial board for
academic publications where references like that have been
spotted and interpreted exactly as the abbreviation you suggest.
The question is then raised as to whether the reference is
important enough to the article that people will actually want
to look it up and study it.  If the answer to that question is
"yes", the draft goes back to the author with "try harder"
instructions.

> Which isn't really good enough for a normative reference in

Exactly.

    john

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 18 Oct 2021 01:13:55 +0200

On 2021-10-18, at 00:45, John C Klensin <[email-masked]> wrote:
> established practice. 

Since you invoked my memory (which at this point is entirely embodied in my laptop :-):

<http://mailarchive.ietf.org/arch/msg/ietf-announce/rlIsY8yvvKhbkbkuk2mcQ-ylSNw>

That was 2015-01-18.

draft-ymbk-downref-00.txt was published 2003-10-21; draft-ymbk-downref-03.txt was approved as BCP on 2004-07-23 (FAST!) and published as RFC 3967 in December 2004.
I find the fist reference to [1] on 2006-09-06; that apparently was moved to [2] on 2017-04-14.

Grüße, Carsten

[1]: http://www1.tools.ietf.org/group/iesg/trac/wiki/DownrefRegistry
[2]: https://datatracker.ietf.org/doc/downref/

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sun, 17 Oct 2021 18:35:06 -0700

> Are we having a problem getting downrefs identified early enough, or are

My impression is that idnits and the datatracker are catching these a high
percentage of the time.  In my time on the IESG (a measly 1.5 years) I
haven't managed to catch a document that failed to call out a downref.

> Historic, but HMAC is still awesome)

That's an interesting idea, but I think it's perhaps an effort separate
from this one.

> DT or some other resource needs to contain notes on how to obtain the

That'll be different for every source of the references, right?  I'm not
sure how we could do that programmatically for all cases.

Really how you go about getting it is somewhat orthogonal to the point
we're trying to address with this update, which is: If you're going to make
a normative reference to something access to which is restricted, that has
to be resolved somehow before it goes to the IESG; we (and any other
reviewers, e.g., the review teams) need the referenced material to be able
to do our jobs.

> documents.  I think that we should be doing more than the minimum here.

What do you suggest in terms of text changes here to address what you're
talking about?

> doesn't mention any errata.

RFC 8067 was the last document added to the cluster that currently makes up
BCP 97, so this is a list of changes since that milestone.  I didn't mean
to imply that this erratum applies to that RFC.

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sun, 17 Oct 2021 18:53:17 -0700

> but that might raise issues I have not thought of yet.

Describing the registry's current structure and maintenance is easy
enough.  I'll add that.

It would probably be a nightmare to add that retroactively for the entries
already present, but the publication of this revision to BCP 97 could
stipulate that, going forward, the reason for downref permission needs to
be recorded and made visible for future entries, and who the responsible AD
was.

What did you mean by "in which document's categories the reference falls"?

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sun, 17 Oct 2021 19:04:01 -0700

> "trivial" is not a good enough reason for the use of that

I'm fine with strengthening this language if the community thinks it's
necessary, but I thought I'd mention that there's a document on an upcoming
telechat with a DISCUSS on it specifically because there's a normative
downref to something that deserves consideration for advancement, so at
least the current IESG is observing the preference BCP 97 already states.

-MSK

---

From: John C Klensin <[email-masked]>
Date: Sun, 17 Oct 2021 22:41:55 -0400

--On Sunday, October 17, 2021 18:53 -0700 "Murray S. Kucherawy"
<[email-masked]> wrote:

>> I actually think that suggests something that should probably
>> be considered for BCP97bis: that the downref procedure and
>> registry should be used only when there are substantive
>> reasons why the relevant document cannot be upgraded or that
>> doing so would require an unreasonable amount of effort.
>> That would strengthen the text that now appears in the last
>> paragraphs of Murray's Sections 4.2 and 5 but even the
>> current text suggests to me that "trivial" is not a good
>> enough reason for the use of that registry.
>> 
>> special procedures (or different procedures) need be followed
>> to use the reference in another document, the registry should
>> explanation that seems necessary -- that information should
>> not just be in the Last Call.  My instinct tells me that the
>> RFC Editor Function should be responsible for the registry
>> itself, but that might raise issues I have not thought of yet.
>> 
> 
> Describing the registry's current structure and maintenance is
> easy enough.  I'll add that.

Thanks.

> It would probably be a nightmare to add that retroactively for
> the entries already present, but the publication of this
> revision to BCP 97 could stipulate that, going forward, the
> reason for downref permission needs to be recorded and made
> visible for future entries, and who the responsible AD was.

Works for me.  I think it is far more important that we avoid
creating additional messes in that future than that we clean up
ones that have been there for years.

> What did you mean by "in which document's categories the

In essence, the descriptions in your sections 4.2, 5, and 6
create categories of downrefs. I'd be inclined to divide 6 into
two subcategories, one for what we have called "recognized
standards bodies" in other contexts and one for other types of
external references.  Either way, I think that categorization
could be very helpful going forward and I was suggesting that it
should be included in the registry.

best,
   john

---

From: Michael Richardson <[email-masked]>
Date: Sun, 17 Oct 2021 22:44:20 -0400

Murray S. Kucherawy <[email-masked]> wrote:
    > } available copies of the target documents for use by all anticipated
    > } reviewers
    > } participants, any member of the community that chooses to participate in
    > } Last Call discussions, area review teams, IANA expert reviewers, and members
    > } of the IESG.

    > sure how we could do that programmatically for all cases.

Programmatically? Not what I meant.

Freeform text in the datatracker... as in, "email bob and tell you are
                        reviewing FOO, in order to get document BAR"

    > reviewers, e.g., the review teams) need the referenced material to be able

If people who work on ITU-T,3GPP,IEEE,ANSI documents come to us with a
document for us to publish, it seems like they ought to make all the
background available to us.

Are we getting this kind of minimum?

    > What do you suggest in terms of text changes here to address what you're

} At a minimum,

**prior to WGLC and sectorial review**

} authors/editors of source documents need to secure freely
    > } available copies of the target documents for use by all anticipated
    > } reviewers
    > } participants, any member of the community that chooses to participate in
    > } Last Call discussions, area review teams, IANA expert reviewers, and members
    > } of the IESG.

Maybe then add:
"Access to the documents should be noted in the Shepherd write up."

---

From: John C Klensin <[email-masked]>
Date: Sun, 17 Oct 2021 23:03:11 -0400

>> enough reason for the use of that registry.

Good... and consistent with what I had understood and expect.

> I'm fine with strengthening this language if the community
> thinks it's necessary, but I thought I'd mention that there's
> a document on an upcoming telechat with a DISCUSS on it
> specifically because there's a normative downref to something
> that deserves consideration for advancement, so at least the
> current IESG is observing the preference BCP 97 already states.

My concern is not with the current IESG.  If they did not agree
with the principles you are describing, I assume that the I-D
would go nowhere.  But, if the text is not crystal clear, I
think it is reasonable to be concerned about someone or
something coming up in the future and insisting on a downref as
a shortcut.  If (or more likely when) that happens, it would be
good for that future IESG to have as much reinforcement as
possible for saying "no, we are going to Do The Right Thing".

And, fwiw, what started my thinking in that direction was
precisely Russ's "trivial" comment.  It would have been trivial,
but it would be the wrong thing to do.  At least IMO, reinforced
by your text and the current BCP 97 text.

   john

---

From: John C Klensin <[email-masked]>
Date: Sun, 17 Oct 2021 23:38:30 -0400

--On Sunday, October 17, 2021 22:44 -0400 Michael Richardson
<[email-masked]> wrote:

>...
>     > Really how you go about getting it is somewhat
> orthogonal to the point     > we're trying to address with
> this update, which is: If you're going to make     > a
> normative reference to something access to which is
> restricted, that has     > to be resolved somehow before it
> goes to the IESG; we (and any other     > reviewers, e.g., the
> review teams) need the referenced material to be able     > to
> do our jobs.
 
> If people who work on ITU-T,3GPP,IEEE,ANSI documents come to
> us with a document for us to publish, it seems like they ought
> to make all the background available to us.

I'm trying to get my head together around a specific and
constructive suggestion, but, in the interim, I think the above
misses something and that is that there are really two sets of
cases:

(1) Someone is involved in working on a spec in one of those
SDOs, or something like them, or maybe in an industry consortium
that doesn't do standards as we normally understand them.  They
then want the IETF to adopt a specification that depends on that
external one.  Certainly they should be able (and willing) to
make any needed background available to us.  They may or may not
be able to make the actual standard freely available (and/or
available for free); that may be tied up with other issues.

(2) There is a standard out there, typically a well-known and
well-established one, that we want to reference as part of our
own work.  We may have no involvement in its creation, but we
are smart enough to understand that trying to fork it with our
own competing spec would not be good for anyone.  Because, as
Carsten pointed out, it is the only spec from that era (long
pre-IETF among other things) that we are still using, let me
take RFC 20 as an example.  There are now, and were then, many
supplemental references that accurately describe what ASCII is
all about but there was really one authoritative one from which
everything else was derived and it was the ANSI/X3
specification.  I am quite confident that Vint was not one of
the people who participated actively in creating the ASCII spec,
so the "people who work on... come to us" bit does not apply.  

So, assuming that ANSI's rules about selling documents circa
1968 were still in effect today as was their occasional
inclination at the time to pursue copyright violation lawsuits,
but we were processing the document for the first time today,
what would you have had him do?  Make the requirement that he go
out and buy up enough copies to supply the IESG, the document
shepherd, and maybe everyone who claimed to be active in a
particular WG, and perhaps a few dozen more people?  Point to
one of the many freely available summaries of that document and
its tables rather than the authoritative source?  Sometime else?

Before you answer, remember that details of the definition of
ASCII (including its name) were changed between 1968 (and 1969)
and the present, that ANSI has traditionally been very reluctant
to make copies of superceded standards available (even for a
fee) and that the vast majority of the descriptions, excerpts,
and summaries of that standard are not too fussy about keeping
track of its status at any particular time before the present.

And while the ASCII Standard is a relatively short and
inexpensive document, consider RFC 2156 ("MIXER") as another
example.   I'd guess that the list of CCITT (ITU) and ISO
documents it references could easily have costs hundreds of US
dollars or equivalent for a set of copies for each of those
reviewers.  Should we have told those working on that what
became 2156 that the price for having it considered was to
promise to put up several tens of thousands of dollars as an
admission fee?   Or that, because ISO and ITU had not decided to
make all of their standards available for free, the RFC could
not be published?

I'm all in favor of putting whatever pressure we reasonably can
on other standards bodies to make their documents freely
available and free.  But we've best be very careful that we
don't shoot ourselves in the foot in the process.

   john

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sun, 17 Oct 2021 21:28:45 -0700

> should be included in the registry.

Do we have a good definition of "recognized standards bodies" anywhere?
This has come up recently in other contexts (e.g., RFC 6838) recently and
I'm wondering if there's an actual definition somewhere that I don't know
about.  In particular, what defines "recognized"?

Has there been a need previously to have a downref registry for things
external to the IETF?  The current registry only has RFCs in it.

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sun, 17 Oct 2021 21:30:56 -0700

> **prior to WGLC and sectorial review**

Ah, I see what you mean.  I thought that was clear from the current text,
but...

> "Access to the documents should be noted in the Shepherd write up."

...this seems like a good idea.  Added.

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sun, 17 Oct 2021 21:52:57 -0700

On Fri, Oct 15, 2021 at 10:12 AM Murray S. Kucherawy <[email-masked]>
wrote:

> [...]

I've captured the feedback from several of you so far into -01 of this
document, which is now posted.  Please let me know if I messed anything up.

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sun, 17 Oct 2021 21:55:59 -0700

On Sun, Oct 17, 2021 at 9:52 PM Murray S. Kucherawy <[email-masked]>
wrote:

>> I've got a draft that seeks to update BCP 97, which is the guidance
>> around how we handle normative downward references.  It's currently made up
>> of three separate RFCs and an erratum, so this will consolidate those into
>> a single document.  The main mission here, though, is to update the
>> guidance around normative references to external documents, especially
>> those behind paywalls.
>>

It occurs to me that the MIB reference is correct but rather stale by now.
Can anyone dream up a replacement comment about something modern like, say,
YANG?

-MSK

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 18 Oct 2021 09:02:35 +0200

> This procedure is not to be used if the proper step is to move the document to which the reference is being made into the appropriate category. It is not intended as an easy way out of normal process. Rather, the procedure is intended for dealing with specific cases where putting particular documents into the required category is problematic and unlikely ever to happen.

So does this now end the practice of certain areas to publish a normative specification as Informational first, and then do the proper vetting (but with a weaker process) of the document at the time it is first used in a downref?

The third level of maturity that this creates (pre-PS :-) is actually quite useful to give specifications some time to mature before we fully commit to them.
It is a variant of what Experimental was meant to be, but that label has acquired a mostly pejorative meaning, just as

  “We are not sure this is a good idea”

can mean that you are really just not sure (so you make it Informational in the above undocumented process) or that a sizable part of the community thinks it will turn out not to be a good idea (so you make it Experimental).

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 18 Oct 2021 09:09:14 +0200

> At a minimum, authors/editors of source documents need to secure freely available copies of the target documents for use by all anticipated reviewers during the source document's life cycle, which includes working group participants, any member of the community that chooses to participate in Last Call discussions, area review teams, IANA expert reviewers, and members of the IESG. The mechanism for acquiring access to those documents is to be be specified in the shepherd writeup. <https://www.ietf.org/archive/id/draft-kucherawy-bcp97bis-01.html#section-6-4>
As John Klensin has pointed out, this really can apply only to type-1 documents (and, actually, only if there is a certain power relationship).

I don’t really have a constructive suggestion, but I think if we enshrine this bit of existing policy/practice into a process document, we need to be more careful about what we actually require.
Maybe a downref needs to be explicit on whether this is a type-1 or a type-2.

Grüße, Carsten

---

From: tom petch <[email-masked]>
Date: Mon, 18 Oct 2021 10:35:31 +0100

> document, which is now posted.  Please let me know if I messed anything up.

I noticed and read this I-D before this thread started and was wondering 
where and how to comment.

As others have said, producing process documents is not the IETF's 
finest hour and this I see as an exemplar of that.  (I wonder if those 
commenting on it are commenting on what they think it says as opposed to 
what it does:-(

The terminology I find unhelpful; the document cries out for a section 
on Terminology.  In most places, I do not know what the document is 
referring to.  Thus when it talks of what must be in an RFC, I think 
good, I can ignore all this as it is up to the RFC Editor to add 
annotating text!  It may be of course that the text here means to refer 
to I-D, as at Last Call, as well but that is not what it says.  Likewise 
the word standard appears in many places.  Most on this list will know 
what a Standard is; is that intended or does it mean a document on the 
standards track which most on this list will know is different?  And the 
focus seems to be RFC ignoring the role of I-D in the IETF process.  The 
document introduces the concept of source document and target document 
for clarity and then fails to use them when clarity is called for.  And 
then most will be familiar with Normative and Informative; here the 
author uses normative and informative.  Are they intended to have the 
same meaning?  Who knows (apart from the author:-)?

To me it is symptomatic that an example is made of a MIB when the IESG 
made YANG the standard for network management many years ago and the 
example would have exactly the same force were it to refer to YANG 
(RFC8407).

Tom Petch

---

From: <[email-masked]>
Date: Mon, 18 Oct 2021 09:38:53 +0000

Hi Murray,

Good idea. You may echo this text from Section 3.9 of RFC8407:

   For every import or include statement that appears in a module

   contained in the specification that identifies a module in a separate

   document, a corresponding normative reference to that document MUST

   appear in the Normative References section.

Cheers,
Med

De : ietf <[email-masked]> De la part de Murray S. Kucherawy
Envoyé : lundi 18 octobre 2021 06:56
À : ietf <[email-masked]>
Objet : Re: BCP97bis

On Sun, Oct 17, 2021 at 9:52 PM Murray S. Kucherawy <[email-masked]<mailto:[email-masked]>> wrote:
On Fri, Oct 15, 2021 at 10:12 AM Murray S. Kucherawy <[email-masked]<mailto:[email-masked]>> wrote:
I've got a draft that seeks to update BCP 97, which is the guidance around how we handle normative downward references.  It's currently made up of three separate RFCs and an erratum, so this will consolidate those into a single document.  The main mission here, though, is to update the guidance around normative references to external documents, especially those behind paywalls.

I've captured the feedback from several of you so far into -01 of this document, which is now posted.  Please let me know if I messed anything up.

It occurs to me that the MIB reference is correct but rather stale by now.  Can anyone dream up a replacement comment about something modern like, say, YANG?

-MSK

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

---

From: [email-masked]
Date: Mon, 18 Oct 2021 05:49:08 -0700 (PDT)

> past but things have changed.

Speaking as a media type reviewer, I require free access to the relevant
standards before I'll review a media type in the standards tree. This has led
various special arrangements to access specifications that would otherwise cost
$$$.

Note that this doesn't mean that free access is required for everyone. The
requirements for media types in the standards tree are that there be a  stable,
publicly available specification. Free availability is not part of it.

> References to resources that are available now but you're worried they will
> be moved or exhobitantly paywalled or lost -- keep "if needed" copies.

While I'm not aware of it happening, it's theoretically possible for a
standards organization to fail and take all of its specifications with it,
just like any other organization.

In the case of media types, what constitutes a standards organization able to
register types in the standards tree is up to the IESG to access. In theory
that assessment includes an accessment of the organizations likely stability.

				Ned

> Just to take "stability of reference" off the table. I read of legal
> documents and judements being subject to this for similar archival goals.

---

From: John C Klensin <[email-masked]>
Date: Mon, 18 Oct 2021 09:07:48 -0400

--On Monday, October 18, 2021 09:02 +0200 Carsten Bormann
<[email-masked]> wrote:

>> This procedure is not to be used if the proper step is to
>> move the document to which the reference is being made into
>> the appropriate category. It is not intended as an easy way
>> out of normal process. Rather, the procedure is intended for
>> dealing with specific cases where putting particular
>> documents into the required category is problematic and
>> unlikely ever to happen.
> 
> So does this now end the practice of certain areas to publish
> a normative specification as Informational first, and then do
> the proper vetting (but with a weaker process) of the document
> at the time it is first used in a downref?
> 
> The third level of maturity that this creates (pre-PS :-) is
> actually quite useful to give specifications some time to
> mature before we fully commit to them. It is a variant of what
> Experimental was meant to be, but that label has acquired a
> mostly pejorative meaning, just as
> 
>   "We are not sure this is a good idea"
> 
> can mean that you are really just not sure (so you make it
> Informational in the above undocumented process) or that a
> sizable part of the community thinks it will turn out not to
> be a good idea (so you make it Experimental).

Actually, Carsten, a different way to look at that practice is
not that it substitutes for the traditional use of Experimental,
but that it restores the three-step/ maturity level standards
process.  Originally, Proposed Standard was fairly close to
"this is normative, we think it is a good idea but some of the
details might not be quite right, it needs additional vetting
and test implementations, and no one sane would deploy it at
scale into a production product before those other steps occur".
My first task as a AD turned out to be explaining to a major
vendor who complained that, by changing some details in a PS we
had made their already-shipped product non-conforming, they were
not going to get any sympathy and should go back and read the
definition of PS.

So, if this is what is now being done with Informational specs
that are treated as normative, deja vu.

    john

---

From: John C Klensin <[email-masked]>
Date: Mon, 18 Oct 2021 09:33:38 -0400

Hi.

In looking through the new -01 draft (even though this text has
not changed) I noticed something that I sort of hinted at
yesterday in responding to other comments.

You need to define "freely available" and do so precisely.  

We have historically considered printed books and articles in
established journals to be suitable for normative references
from the RFC Series ("down" really has nothing to do with that
criterion) even if buying the book or obtaining the journal was
expensive.  In theory, there was always a trip to the library.
Some of the standards from other SDOs have the same property:
they are often very expensive unless one's organization is a
member that gets them for free, but many libraries and other
repositories do have them available.

Of course, some of us have access to better technical libraries
than others. That is an economic and cultural problem I don't
know how to fix, but I'm fairly sure that pushing in the
direction of "must be available online, with no restrictions and
no cost" would be quite self-destructive for the IETF.

"Freely available" does not necessarily imply "free" (zero cost).

By contrast, one can imagine a reference to a restricted
corporate document, some types of prepublication drafts, and, if
the world continues to fragment, even the detailed description
of how some equipment operates.  In those cases, the document
may just not be "available" to many IETF participants even
though, if someone were allowed to access it, it would be at no
cost.

So the I-D should be very clear about what it is talking about.
Then, if needed, we can have a better discussion about the
requirements.

best,
    john

---

From: John C Klensin <[email-masked]>
Date: Mon, 18 Oct 2021 09:55:13 -0400

--On Monday, October 18, 2021 05:49 -0700
[email-masked] wrote:

>...
> Speaking as a media type reviewer, I require free access to
> the relevant standards before I'll review a media type in the
> standards tree. This has led various special arrangements to
> access specifications that would otherwise cost $$$.

Presumably, if someone lent you a copy for the duration of your
review process, that would suffice.  Correct?

We had some discussions a long time about about the RFC Editor
requiring copies of everything outside the RFC Series that was
normatively referenced and then maintaining their own
repository.  For better or worse, the idea never went anywhere
but it would have made all of these documents freely available
and reading them free if only one could get to Marina del Rey or
somewhere on the USC campus.  Easier for you than for me, I
suppose :-).

> Note that this doesn't mean that free access is required for
> everyone. The requirements for media types in the standards
> tree are that there be a  stable, publicly available
> specification. Free availability is not part of it.

Right.  And there is a huge difference between one copy,
available without cost to a media type reviewer, perhaps for a
limited time and, e.g., requiring that copies be distributed to,
or otherwise available for free to, the entire IESG, the
document shepherd, the relevant WG, and potentially anyone who
might want to do a Last Call review.  

We need to be very careful about how we define and scope
requirements in this area.

>...
> While I'm not aware of it happening, it's theoretically
> possible for a standards organization to fail and take all of
> its specifications with it, just like any other organization.
>...

Yes, but if the work of that standards organization was widely
enough used that we should have taken them seriously in the
first place, there would presumably be copies of the relevant
documents in libraries and other repositories all over the
world.  Having the SDO fail would be very bad for any
expectations about maintenance of those documents and probably
for their credibility but would have little impact on
availability of archival copies.

     john

---

From: "Salz, Rich" <[email-masked]>
Date: Mon, 18 Oct 2021 14:00:51 +0000

The only thing certain in this world is uncertainty, as someone has said.  Putting a date-link in a URL is no guarantee either. And of course publications have been known to go out of print :)

---

From: John C Klensin <[email-masked]>
Date: Mon, 18 Oct 2021 10:20:22 -0400

--On Sunday, October 17, 2021 21:28 -0700 "Murray S. Kucherawy"
<[email-masked]> wrote:

>> create categories of downrefs. I'd be inclined to divide 6
>> into two subcategories, one for what we have called
>> "recognized standards bodies" in other contexts and one for
>> other types of external references.  Either way, I think that
>> categorization could be very helpful going forward and I was
>> suggesting that it should be included in the registry.

> Do we have a good definition of "recognized standards bodies"
> anywhere? This has come up recently in other contexts (e.g.,
> RFC 6838) recently and I'm wondering if there's an actual
> definition somewhere that I don't know about.  In particular,
> what defines "recognized"?

I think it went into the vocabulary with media type
registrations (the predecessors of 6839) in which requests from
such standards bodies, presumably for identifiers to support
their work or defined by their standards, were (and are) given
special treatment.  Who qualified (or not) was left to the
discretion of the IESG.  The assumption was that the IESG would
consider many of the issues we have been talking about in the
context of BCP97bis: Is the body established, recognized, and
credible?  It is likely to be around for a while? Are their
specifications "widely and freely available" (even if not free)?

Should the criteria for "recognized" be the same for downrefs as
for media types?  I assume at least similar but don't know
beyond that (and suspect we would get into trouble by trying to
spell things out too much).

As we try to make these things more precise and mechanical,
there is another issue with "freely available" for which it
might be useful to draw on the thinking in our patent policies.
A WG is allowed to consider a possible standard built on
patented and highly restricted technologies (e.g., you are going
to need a license that may cost big bucks, and you have no
guarantees that you will get one or that you will get the same
terms as your competitor).  They are expected to carefully
consider less restricted technologies but, if they (and the
community) conclude that the restricted technology is enough
better than any of the alternatives, they are allowed to go
ahead.  Perhaps we need provision for exceptions, on a similar
basis to the downref rules.  Trying to lay out rigid rules that
have no exceptions may be a bad idea.

best,
   john

> 
> Has there been a need previously to have a downref registry
> for things external to the IETF?  The current registry only
> has RFCs in it.
> 
> -MSK

---

From: "Salz, Rich" <[email-masked]>
Date: Mon, 18 Oct 2021 14:28:41 +0000

*   At a minimum, authors/editors of source documents need to secure freely available copies of the target documents for use by all anticipated reviewers during the source document's life cycle, which includes working group participants, any member of the community that chooses to participate in Last Call discussions, area review teams, IANA expert reviewers, and members of the IESG. The mechanism for acquiring access to those documents is to be be specified in the shepherd writeup.

I understand the desire for everything that is defined by the IETF to be built on public freely available resources.  I think this goes WAY WAY WAY too far.

Can I buy a copy of a standard and require it to be postal mailed to everyone serially?

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 18 Oct 2021 07:33:21 -0700

On Mon, Oct 18, 2021 at 7:29 AM Salz, Rich <[email-masked]> wrote:

>
>    - At a minimum, authors/editors of source documents need to secure
>    freely available copies of the target documents for use by all anticipated
>    reviewers during the source document's life cycle, which includes working
>    group participants, any member of the community that chooses to participate
>    in Last Call discussions, area review teams, IANA expert reviewers, and
>    members of the IESG. The mechanism for acquiring access to those documents
>    is to be be specified in the shepherd writeup.
>
>
>
> I understand the desire for everything that is defined by the IETF to be
> built on public freely available resources.  I think this goes WAY WAY WAY
> too far.
>
>
>

That's not what we're going for.  It's not an insistence that everything be
built on public, freely available resources.  However, reviewers -- from
the WG to the IESG and everyone in between -- need to be able to do their
reviews while having access to the normative references.  So this says
that, at a minimum, those resources need to be made available somehow
during the document's development, but there's no requirement for that to
continue in perpetuity.

The IESG has had multiple cases during my time there where we haven't had
access to some normative reference, and so we can't do our job.  This has
added long delays to document processing.  That's what we're trying to
address here.

-MSK

---

From: "Salz, Rich" <[email-masked]>
Date: Mon, 18 Oct 2021 14:36:17 +0000

*   The IESG has had multiple cases during my time there where we haven't had access to some normative reference, and so we can't do our job.  This has added long delays to document processing.  That's what we're trying to address here.

I believe it is far more common for the IESG to review and progress documents without having all normative references tracked down and read.

---

From: John C Klensin <[email-masked]>
Date: Mon, 18 Oct 2021 10:39:43 -0400

--On Monday, October 18, 2021 14:00 +0000 "Salz, Rich"
<[email-masked]> wrote:

>     That makes it clear what is being referenced, which is
> good.     Absent other provisions/ requirements, it does
> nothing to     guarantee that the content will be there when
> someone goes     looking for it years (or decades) later.
> 
> The only thing certain in this world is uncertainty, as
> someone has said.  Putting a date-link in a URL is no
> guarantee either. And of course publications have been known
> to go out of print :)

And some out of print publications are in libraries, including
repository libraries, all over the world and some are not.

Hmm.  It occurs to me that a great deal of this discussion is
ultimately about the question of what constitutes a "stable
reference".  The criteria have traditionally been an RFC Editor
Function decision.  For the edge cases, there have (at least
usually) been discussions with the RFC Editor, not unilateral
actions by the IESG.  Part of the reason for that has been an
assumption that "stability" is actually an assessment requiring
some expertise, expertise the RFC Editor was expected to have
and the IESG might not.

I don't know whether the assumption that the RFC Editor Function
is the best place to make those decisions given the evolving
"future" plans.   However, if we assume that and hope for the
best, this I-D could kick much of this discussion down the road
by saying that all downward references (just like all other
normative references) are required to met RFC Editor criteria
for stability and availability.   That would not eliminate the
need to pin down whatever addition requirements we might have
for "free" or a special definition of "freely available", but
would, I think, get rid of other parts of this discussion...
including ones about books going out of print, SDOs collapsing,
and even documents changing in situ.   At worst, we could have
that discussion in one place rather than trying to settle it for
downrefs and then doing it again for other kinds of normative
references.

best,
    john

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 18 Oct 2021 07:39:59 -0700

On Mon, Oct 18, 2021 at 7:36 AM Salz, Rich <[email-masked]> wrote:

>
>    - The IESG has had multiple cases during my time there where we
>    haven't had access to some normative reference, and so we can't do our
>    job.  This has added long delays to document processing.  That's what we're
>    trying to address here.
>
>
>
> I believe it is far more common for the IESG to review and progress
> documents without having all normative references tracked down and read.
>

The role doesn't matter, does it?  As an Area Review Team member or even a
Working Group participant faced with a document with normative references
behind a paywall, you face the same problem.

Sure, you could make some assumptions that what's in the normative
reference is correct, or sane, or trustworthy, or whatever.  What if you're
wrong?  What if you want to be certain?

Shouldn't our processes err on the side of pushing for quality?

-MSK

---

From: "Salz, Rich" <[email-masked]>
Date: Mon, 18 Oct 2021 14:51:28 +0000

*   Sure, you could make some assumptions that what's in the normative reference is correct, or sane, or trustworthy, or whatever.  What if you're wrong?  What if you want to be certain?

I suppose asking the authors isn’t practical for some reasons. I’m mildly curious what those reasons are, but that’s a secondary point.

  *   Shouldn't our processes err on the side of pushing for quality?

Sure, as long as reality isn’t constrained.  I looked up X.509, the ITU standard for certificates.  Text on the website says:
                This text was produced through a joint activity with ISO and IEC. According to the agreement with our partners, this document is only available through payment. Please get in touch with [email-masked] for more information.

Are normative references to X509 now not proper? Similarly, the next version of FIPS is a “diff” document around a couple of ISO documents; the cheapest price I found for them is USD $180.  Are normative references to FIPS 140-3 now not proper?

We have a working group that supports IEEE point-to-point time protocol.  That standard costs USD $280.  What should the NTP (and predecessor TICTOC) working group do?

---

From: <[email-masked]>
Date: Mon, 18 Oct 2021 14:57:43 +0000

Hi all,

I agree with the concern raised below.

It is likely that securing access would be to pay the fees for every access. I’m not sure this can be fixed by write-up unless we have in mind something similar to the approach considered in my write-up at: https://datatracker.ietf.org/doc/draft-ietf-drip-reqs/shepherdwriteup/

===================================================
All normative references are published RFCs, except a reference to an
external specification ([F3411-19]).

Note that [F3411-19] is not freely available ($88), but an early
version is publicly available at
https://raw.githubusercontent.com/opendroneid/specs/master/
OpenDroneID_Bluetooth_0.64.3.pdf.  A pointer to this freely available
document is also cited in the document.  This is clearly called in
the draft (see the text right after Figure 2).
===================================================

Cheers,
Med

De : ietf <[email-masked]> De la part de Salz, Rich
Envoyé : lundi 18 octobre 2021 16:29
À : Carsten Bormann <[email-masked]>; Murray S. Kucherawy <[email-masked]>
Cc : ietf <[email-masked]>
Objet : Re: BCP97bis

---

From: tom petch <[email-masked]>
Date: Mon, 18 Oct 2021 16:31:43 +0100

> IETF 112 begins next month.

I have called out a number of downrefs in the past year or two, of 
Normative References to IETF documents that are not Standards Track RFC 
or I-D, and the response has sometimes been for the author to revise the 
I-D to make it an Informative Reference.  This is a nonsense; the I-D 
cannot be understood without the Reference in question.

I can think of two explanations.

1) the author is not familiar with the processes and thinks that I-D 
Nits it ordering the author to reclassify the reference

2) the author is familiar with the processes and is trying to game the 
system.

Either way, I think that the process is failing in  way that this I-D 
does not address.

Tom Petch

---

From: John C Klensin <[email-masked]>
Date: Mon, 18 Oct 2021 11:37:56 -0400

--On Monday, October 18, 2021 14:36 +0000 "Salz, Rich"
<rsalz=[email-masked]> wrote:

>   *   The IESG has had multiple cases during my time there
> where we haven't had access to some normative reference, and
> so we can't do our job.  This has added long delays to
> document processing.  That's what we're trying to address here.
> 
> I believe it is far more common for the IESG to review and
> progress documents without having all normative references
> tracked down and read.

Murray, I think Rich is right.  Maybe it should be different,
but when I have written, been shepherd for, or otherwise been
heavily involved with documents that make normative references
outside the RFC Series in recent years, I think it has been
unusual if even one IESG member other than the responsible AD
studied all of the references documents carefully.

Another distinction may be worth making.  There are two kinds of
normative reference to documents outside the RFC Series.   We
encountered an important example earlier this year although I
did not figure out how to formulate the distinction until now.
In one case, the referencing document actually contains
everything one needs to know to implement whatever is being
specified.  RFC 20 is a perfect example of that: it contains
copies of the relevant tables and the definitions that were
considered relevant.  X3.4-1968 is a normative reference (even
though we did not make the distinction at that time), but the
reference is needed only to identify the authority for what the
RFC says.  If one needed to actually read and understand
X3.4-1968, it would only be to verify that its relevant contents
are accurately portrayed.  Depending on definitions that, AFAIK,
have never been spelled out explicitly enough that one could use
them to split hairs, one could even argue that reference is
Informative and not Normative.  

At the other extreme, we've seen documents that essentially say
"to do X, go look at [BigDoc] and do what it says" where
[BigDoc] is, e.g., something more than 1000 pages long and hard
for non-specialists to read, much less fully understand
references to specific sections (so even "[BigDoc] Section 999"
would not be of tremendous help).  Clearly a Normative
reference, but anyone claiming to have reviewed the proposed RFC
who has not studied [BigDoc] and understood the specification
there and its implications... well, really has not done so.

The solution to the latter type of situation is not to be sure
free copies of [BigDoc] are available to anyone who might (or
should) be interested in the hope they will read them, but for
the IESG to push things more in the direction of RFC 20 and good
summaries about exactly what the actual requirements are in
practice.  There are many much more recent examples that have
done just that.

And, if we could separate this issue from the contents of the
I-D, this document and the discussions of it would get easier.
Unfortunately, the requirement for that separation is that the
IESG become clear about the distinction and push back
aggressively on shortcuts like the "just look over there" one
implied by the second case.

best,
    john

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 18 Oct 2021 10:17:52 -0700

On Mon, Oct 18, 2021 at 7:57 AM <[email-masked]> wrote:

> It is likely that securing access would be to pay the fees for every
> access. I’m not sure this can be fixed by write-up unless we have in mind
> something similar to the approach considered in my write-up at:
> https://datatracker.ietf.org/doc/draft-ietf-drip-reqs/shepherdwriteup/
>

In the cases I remember, it's been possible to get free access to the
target document on request when the other SDO is made aware that we're
developing a standard that refers to it.  The authors/editors in those
cases have requested it and received a free copy with the understanding
that it won't be distributed other than to satisfy our reviews.

What you've done here would also be satisfactory (and for this IESG, it
was).

Personally I think swinging the other way is just as bad: The target
document might be the product of an SDO that we consider to be eminently
stable, but how do we know the document we're about to produce is correct
if we can't read the target?  Are reviewers just expected to take it for
granted?

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 18 Oct 2021 10:30:10 -0700

> studied all of the references documents carefully.

It's happened at least twice during my time on the IESG, which led me (or,
I suppose, us, meaning the current IESG) to believe this is a pattern that
might be worth addressing with revised guidance as there currently doesn't
seem to be any.  It's certainly true that this might be unusual overall,
but it has stood out in my own context.

[...]
> practice.  There are many much more recent examples that have

So then the way to resolve this problem might be to codify exactly the
practice you're proposing: If the primary target document is not freely
available, craft an RFC that captures the important requirements of it so
we can use that as our reference.  That document can then be normative for
our future work on that topic.  It's more work for the IETF, to be sure,
but if we say this is the process then at least working groups will know
what's expected of them as far ahead of time as possible.

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 18 Oct 2021 10:35:04 -0700

On Mon, Oct 18, 2021 at 7:51 AM Salz, Rich <[email-masked]> wrote:

> Are normative references to X509 now not proper? Similarly, the next
> version of FIPS is a “diff” document around a couple of ISO documents; the
> cheapest price I found for them is USD $180.  Are normative references to
> FIPS 140-3 now not proper?
>

"Not proper" is a little hyperbolic, I think.  It creates a review barrier,
and that's the problem I'm trying to address.  A resolution to that barrier
doesn't have to involve declaring such a reference to be improper, and I
don't think that's what this is saying.

 We have a working group that supports IEEE point-to-point time protocol.
> That standard costs USD $280.  What should the NTP (and predecessor TICTOC)
> working group do?
>
>
Ask the IEEE for a freely available copy we can use, with restricted
distribution, to confirm compatibility between their standard and ours.
That's worked before (at least, I think it was the IEEE; might've been the
ITU).

-MSK

---

From: Warren Kumari <[email-masked]>
Date: Mon, 18 Oct 2021 13:39:59 -0400

> behind a paywall, you face the same problem.

Yes. No. Maybe.

There are some protocols where the only people
implementing/interoperating/whatever are all part of a small group that
already knows and understands the <whatever is in the paywalled document>,
and / or are willing to shell out the money to pay for it.

A recent example of this is
https://datatracker.ietf.org/doc/draft-ietf-6lo-nfc/ -- there are likely to
only be a very small number of people who will implement the tranport part,
and they already understand the "NFC Logical Link Control Protocol version
1.3" and / or would be willing to purchase it.

Yes, in an ideal world, this would be an open, free, etc document -- but,
absent an ideal world, I still think that it's better that how IPv6 is
transmitted over NFC using 6LoWPAN is documented in an RFC than having an
undocumented protocol....

W

-- 
The computing scientist’s main challenge is not to get confused by the
complexities of his own making.
  -- E. W. Dijkstra

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 18 Oct 2021 10:45:15 -0700

> same meaning?  Who knows (apart from the author:-)?

Very little of the text here is changed from the other documents that
already comprise BCP 97.  Those definitions aren't there now; we generally
seem to know what those terms mean.  Perhaps the text expects familiarity
with RFC 2026 (a normative reference), which defines some of them, and the
rest are defined in this document ("normative" and "informative" are
described in Section 1.1, for instance) or in the Guidelines for I-D
Authors.  We could add a reference to the latter if people think that would
be helpful.

I'd be happy to add a Terminology section if we think it's needed, but I'm
wary of the risk of definitions here and those elsewhere diverging.

> example would have exactly the same force were it to refer to YANG

I have already asked for (and now received) some text to replace that
reference.

-MSK

---

From: Lars Eggert <[email-masked]>
Date: Mon, 18 Oct 2021 20:55:26 +0300

Hi,

> The only thing certain in this world is uncertainty, as someone has said.  Putting a date-link in a URL is no guarantee either. And of course publications have been known to go out of print :)

some academic circles have started to cite URLs with archive.org links, adding any pages that aren't already archived first.

Thanks,
Lars

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 18 Oct 2021 11:00:53 -0700

On Mon, Oct 18, 2021 at 10:40 AM Warren Kumari <[email-masked]> wrote:

> undocumented protocol...

I agree, but we're not talking here about making that a hard requirement.
We just need to ensure meaningful reviews are possible when that's not the
case.

-MSK

---

From: Warren Kumari <[email-masked]>
Date: Mon, 18 Oct 2021 15:21:37 -0400

> We just need to ensure meaningful reviews are possible when that's not the

Yup, acknowledged -- I was just trying to provide some background / explain
why sometimes we need to escape valve.
W

---

From: "Salz, Rich" <[email-masked]>
Date: Mon, 18 Oct 2021 19:36:16 +0000

So I’m notice that a single-author AD-sponsored BCP revision is getting a relatively lot amount of pushback on one particular area from a number of very experienced IETFers (to be clear: other than me).

Perhaps an AD sponsoring a BCP written by another AD is not the best way forward for something that has such a broad impact.

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 19 Oct 2021 09:04:57 +1300

I think the original concern was indeed standards that (for 
proprietary or other reasons) were actually kept secret.
So "freely" didn't imply "free of charge"; it meant available
to the general public. In that sense it's closely related
to "open standards". Those are standards that are open to
the general public. I think that's what we insist on, and
"free of charge" is desirable, but not essential.

"Open standards that are openly developed" means standards
whose development process is open to the general public.
We don't insist on that for external references.

Regards
   Brian Carpenter

---

From: John C Klensin <[email-masked]>
Date: Mon, 18 Oct 2021 16:25:12 -0400

--On Monday, October 18, 2021 10:30 -0700 "Murray S. Kucherawy"
<[email-masked]> wrote:

>> the IESG to push things more in the direction of RFC 20 and
>> good summaries about exactly what the actual requirements are
>> in practice.  There are many much more recent examples that
>> have done just that.
>> 
> 
> So then the way to resolve this problem might be to codify
> exactly the practice you're proposing: If the primary target
> document is not freely available, craft an RFC that captures
> the important requirements of it so we can use that as our
> reference.  That document can then be normative for our future
> work on that topic.  It's more work for the IETF, to be sure,
> but if we say this is the process then at least working groups
> will know what's expected of them as far ahead of time as
> possible.

I think so.  I would hope that the language used would still
leave the IESG with discretion for unusual cases but that seems
to me a better way to go.  We do need to be careful about one
thing, which is that, if the externally referenced document
changes out from under our specification, e.g., X3.4-1968
evolves into ANSI INCITS 4:1986, the development of any new IETF
document that depends on the original RFC should include a check
to be sure the changes are not significant wrt our work and, if
they are, should include, an explanation and necessary updates.

To pick a somewhat more recent example than RFC 20, RFC 3339
contains the sort of summary I think we should be looking for.
It defines a profile of ISO 8601:1988 but contains sufficient
ABNF and explanation that any reader who was not trying to check
up on the authors would not need to review 8601:1988 itself.  I
note that ISO 8601:1988 had already been replaced by ISO
8601:2000 by the time the RFC was published.  That is noted by
an un-cited reference to the latter; I am suggesting that it
should have been required to contain a short section or appendix
that either stated that there were no substantive changes in the
ISO documents that would affect the specification between the
1988 and 2000 versions or a brief discussion of the changes and
their effects.  And, of course, the implications of those issues
came up in the discussion prior to chartering of SEDATE and,
while draft-ietf-sedate-datetime-extended-00 still references
8601:1988 as normative, without some explanation that is an
invitation to future problems if only because, AFAIK, one can
not even obtain the 1988 specification from ISO except, perhaps,
for historical research purposes.

    john

---

From: Michael Richardson <[email-masked]>
Date: Mon, 18 Oct 2021 16:40:01 -0400

> repositories do have them available.

If I could go to my local university library and access (and checkout: I have
a card) a copy of that SDO's work (as paper or CDrom), then it would be okay.
Many universities can not afford these subscriptions anymore.
Even the soft copy only ones.   But, not everyone has that kind of access.
(I was the only 15 year old in my school who had a library card at nrc.ca,
and I had to get a letter from my librarian)

Sometimes there are IP address limits, and if I can get on the network there,
then I can read them online.

    > though, if someone were allowed to access it, it would be at no

Agreed.
(I did implement RFC822 when I was ~15 for the Amiga)

---

From: John C Klensin <[email-masked]>
Date: Mon, 18 Oct 2021 16:41:51 -0400

> "free of charge" is desirable, but not essential.

Right.  But, to see if we are all on the same page about that,
it is very different from justifying a rule that might require
that authors or WGs go out and purchase a large quantity of
copies of a document and distribute them.  

john

---

From: "Scott O. Bradner" <[email-masked]>
Date: Mon, 18 Oct 2021 16:42:50 -0400

that is what I remember as well (“freely” != “free”)

Scott

> On Oct 18, 2021, at 4:04 PM, Brian E Carpenter <[email-masked]> wrote:
>

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 19 Oct 2021 10:15:59 +1300

On 19-Oct-21 08:36, Salz, Rich wrote:
> So I’m notice that a single-author AD-sponsored BCP revision is 
getting a relatively lot amount of pushback on one particular area from a 
number of very experienced IETFers (to be clear: other than me).
> Perhaps an AD sponsoring a BCP written by another AD is not the best way forward for something that has such a broad impact.

ADs (or recent IESG alumni) tend to be the subject matter experts on arcana of the standards process. I think it's entirely to be expected that they will draft process documents, and very likely that they will be motivated to do so.

As this discussion shows, there is plenty of daylight on what they are doing, so I don't see any harm.

    Brian

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 19 Oct 2021 10:22:41 +1300

Tom,
> 2) the author is familiar with the processes and is trying to game the 

There's a possible 3rd explanation:

3) the reference didn't need to be normative in the first place.

I certainly saw that a number of times as a Gen-ART reviewer.

> Either way, I think that the process is failing in  way that this I-D 

If there's a failure, surely it's a failure of WG Last Call or IETF
Last Call reviewers to detect that an Informative reference should
be a Normative downref? That's not a process bug, it's poor
execution of the process.

   Brian

---

From: Michael Richardson <[email-masked]>
Date: Mon, 18 Oct 2021 17:31:10 -0400

> 1.3" and / or would be willing to purchase it.

I think that actually, this document is a really good example of how things
go badly.

Because only a small number of people have access to that background
document, nobody can meaningfully review it.
Maybe that's why the document has been around for *7* years.

The shepherd write up does not tell me where to get that background document,
although apparently I'm among those who discussed the document.  I don't know
the background for one, and I certainly couldn't implement without the
background.

    > undocumented protocol....

I mostly agree with you.

---

From: Sandy Wills <[email-masked]>
Date: Mon, 18 Oct 2021 18:37:29 -0400

Perhaps add that definition in the document:

For RFC references, "Freely Available" is defined as "Obtainable on the 
open market without violating copyrights, corporate policies, or 
government laws or regulations.  There may be a financial cost to 
obtaining such a reference."

-- 
Sandy Wills
[email-masked]

---

From: Michael StJohns <[email-masked]>
Date: Mon, 18 Oct 2021 18:39:31 -0400

> We don't insist on that for external references.

It's possible I missed someone else referencing this, but RFC6852 is 
probably a good reference for this discussion.

We have "freely available" which really means "accessible to all" not 
"available free of charge".

We have open standards which may (or may not) mean "processes are open 
to all interested and informed parties"  but generally does mean 
"non-proprietary" and "implementable by all" and probably means 
"collectively created by parties with varying goals".  All standards 
organizations - including the IETF - have barriers to entry by 
individuals.  For us it's just that the barriers are more cultural than 
formal or even financial. Compare and contrast with the ITU that votes 
by country, or OASIS which has both organizational and personal 
memberships, but votes by individual. I find it hard to say where the 
line would be drawn between a common understand of what open and 
non-open standards organization processes are.

We have "voluntary adoption" which contrasts with things like NIST FIPS 
documents which are directive on the US government, can be directive on 
US government contractors, and in many cases tend to be voluntarily 
adopted by other groups and organizations.

Nuances are important here I think.  For the purpose of this discussion, 
the fact that some documents may need to be paid for by the general 
public shouldn't disqualify them as references. That said, leaving it to 
the document authors to negotiate with the external standards 
organizations for access, or worse to pay for access,  is probably not 
in our best interests.

Perhaps an updated version of 6852 where we can get MOAs with various 
organizations that we're willing to do external references to and an 
agreement that they will provide copies of the referenced standards for 
our processes might  be in order.   As would an agreement for stable 
references for archival retrieval of referenced documents.  That would 
be a change to what Murray has currently drafted for section 6 of the 
document.

Later, Mike

---

From: George Michaelson <[email-masked]>
Date: Tue, 19 Oct 2021 09:03:27 +1000

During the X.4xx/X.5xx timeframe, I believe an exception was carved
out for what X.509 represents. Not that other standards couldn't be
found without paying, but that X.509 was formally put out there for no
cost.

I recall across the timeframe that my employer paid for the CCITT
fascicles, they were printed on exceptionally thin paper, and occupied
several feet (oh wait, maybe I mean "almost a meter") of space. But
subsequently I was told (perhaps incorrectly) that the filthy-lucre
side of the CCITT/ITU standards publishing enterprise was persuaded,
(perhaps as an experiment, or for some other reason) to break out what
X.509 represents. We had both the red series and the blue series. I
forget why the colour changed and what it meant. Wait.. is this
british passports?

So, the existence of some X-series documents outside the paywall is
the exception, not the normality, as I understand it.

If you were a contributor to the process inside your national
standards body, "free" (lets discount any local fees to be in the
standards process) were normal. So a lot of people had the documents,
even if they weren't implementing.

-G

---

From: Randy Presuhn <[email-masked]>
Date: Mon, 18 Oct 2021 17:25:58 -0700

Hi -

On 2021-10-18 4:03 PM, George Michaelson wrote:
...
...

 From https://www.itu.int/en/ITU-T/publications/Pages/recs.aspx :
"The vast majority of all Recommendations are available in electronic 
(PDF) form, free of charge to all, once the final, edited version is 
published."

For example, lots of interesting stuff is listed on
https://www.itu.int/itu-t/recommendations/index.aspx?ser=X

Randy

---

From: George Michaelson <[email-masked]>
Date: Tue, 19 Oct 2021 10:28:45 +1000

Good to see. I'd say "wow, I think thats a change of posture" but who
knows: maybe I was woefully misinformed at the time?

It was a long time ago. nearly 30 years ago.

-G

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 19 Oct 2021 15:58:46 +1300

> knows: maybe I was woefully misinformed at the time?

No, it was a significant change on the ITU side. Especially significant
because it was around June 1991 when Carl Malamud** and Tony Rutkowski
started project BRUNO to get the relevant CCITT standards on-line for
free.  The experiment didn't survive very long, and its 30th anniversary
is already past. The eventual change of policy was in 2007, says Google.

    Brian

** If you haven't read Carl's book "Exploring the Internet" you've missed
a treat, and he covers Project Bruno among much else.

---

From: "Salz, Rich" <[email-masked]>
Date: Tue, 19 Oct 2021 11:02:33 +0000

On 10/18/21, 6:38 PM, "Sandy Wills" <[email-masked]> wrote:

This is a good idea.  I think "readily available" should be used, to avoid the overloaded term.

---

From: Michael Richardson <[email-masked]>
Date: Tue, 19 Oct 2021 07:26:51 -0400

> I certainly saw that a number of times as a Gen-ART reviewer.

How can a third party point this out, unless they can read the document?
The converse is worse: some informative reference that actually is normative,
but only implementers two years later recognize this.

---

From: Michael Richardson <[email-masked]>
Date: Tue, 19 Oct 2021 07:30:29 -0400

Sandy Wills <[email-masked]> wrote:
    > For RFC references, "Freely Available" is defined as "Obtainable on the open
    > market without violating copyrights, corporate policies, or government laws
    > or regulations.  There may be a financial cost to obtaining such a
    > reference."

For a number of organizations, workers are *not* allowed to click through
any terms and conditions, regardless of financial cost.

They have to ask the lawyers to review because of concerns about IPR "infection"

I'm told one Big Tech company has a 9-month backlog on that, so effectively,
those people can never follow the link.

---

From: [email-masked]
Date: Tue, 19 Oct 2021 06:33:31 -0700 (PDT)

> review process, that would suffice.  Correct?

Yes, that's correct.

I should also add that having the specifications available is essential to
doing proper review, even when all we're looking at is a media type. There was
a registration recently that seemed like it made sense, but without the
specifications I couldn't tell what the security considerations were. But after
getting some of the relevant standards - which wasn't easy - it became clear
that the registration actually made no sense at all: They were attempting to
register what amounted to an entire protocol stack as a media type.

> requirements in this area.

I wasn't advocating using the media types approach for other IETF review
processes, but now that the idea has been broached, I don't see why this
wouldn't work in cases where it's a small group doing the review.

Extending it to, say, last call reviews is another matter. Some of
the other standards bodies I've dealt with likely wouldn't have a problem
with making the specification publicly available for a limited time -
and recognizing that the genie can't really be put back in the bottle - but
others most certainly would.

And this isn't always about the money, at least not in the way you'd think. In
some cases we're dealing with a draft specification for something in a highly
competitive area. Early release of such documents could potentially lead to
serious interoperability problems in the event someone jumps the gun
on implementation and late changes need to be made.

> availability of archival copies.

Seriousness is more or less what the IESG is looking for.

				Ned

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 20 Oct 2021 09:46:59 +1300

> but only implementers two years later recognize this.

Well yes. That's exactly why, during my career as a Gen-ART reviewer, I tried
to pay attention to this whole issue. But there is scope for human error here,
so it's going to happen from time to time.

    Brian

---

From: tom petch <[email-masked]>
Date: Wed, 20 Oct 2021 09:57:07 +0100

> is already past. The eventual change of policy was in 2007, says Google.

And posted to the IETF list by Yaakov Stein 17 October 2007 at 14:02.

Tom Petch

> .

---

From: tom petch <[email-masked]>
Date: Wed, 20 Oct 2021 11:46:06 +0100

Brian

  Ah yes, I did not consider the third option.

To make this concrete, a recent I-D has a Normative Reference to a 
protocol RFC that was clearly Normative but was now being updated to a 
-bis.  I commented that a -bis existed and was well on its way to RFC so 
had the authors considered referencing that.  A revised I-D appeared 
with an Informative Reference to the -bis!  Mmmm; I suspect that some 
flag in the process pushed the authors into that downgrade.

A regular example is the data model with a reference to the information 
model that is Informational. To me it is obvious that information models 
should be Standards Track since the first reference to them will be 
Normative but WG Chairs do not see that :-(

Tom Petch