Subject: letting IETF build on top of Open Source technology

From: Alia Atlas <[email-masked]>
Date: Sun, 29 Oct 2017 23:02:50 -0400

After some discussion, my co-authors and I have written down a proposed
policy that would clearly and specifically allow standards-track RFCs to
have external non-SDO normative references.  It's not disallowed now - but
determining early when it is ok is unclear.

My hope is that this will facilitate improved standard technology that can
use "de-facto standards" that are suitably open and mature.

Please take a read (between all the fun technical drafts) and let us know
what you think.

Thanks,
Alia

---------- Forwarded message ----------
From: <[email-masked]>
Date: Sun, Oct 29, 2017 at 10:36 PM
Subject: I-D Action: draft-atlas-external-normref-00.txt
To: [email-masked]

A New Internet-Draft is available from the on-line Internet-Drafts
directories.

        Title           : Normative References in RFCs from Open Source
        Authors         : Alia Atlas
                          Eliot Lear
                          Joel Halpern
                          Heather Flanagan
                          Jeff Tantsura
        Filename        : draft-atlas-external-normref-00.txt
        Pages           : 6
        Date            : 2017-10-29

Abstract:
   IETF procedures generally require that a standards track RFC may not
   have a normative reference to a non standards track specification
   except those from other
   standards bodies.  This document creates an External Specification
   registry, similar to the DownRef registry that has been created based
   on [RFC3967] and permits normative references to community accepted
   external specifications.

The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-atlas-external-normref/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-atlas-external-normref-00
https://datatracker.ietf.org/doc/html/draft-atlas-external-normref-00

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

_______________________________________________
I-D-Announce mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/i-d-announce
Internet-Draft directories: http://www.ietf.org/shadow.html
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

---

From: Martin Thomson <[email-masked]>
Date: Mon, 30 Oct 2017 14:15:18 +1100

On Mon, Oct 30, 2017 at 2:02 PM, Alia Atlas <[email-masked]> wrote:
> draft-atlas-external-normref-00

Why is this newly proposed process necessarily distinct from the
exsiting downref process?  The draft doesn't say this.

---

From: Alia Atlas <[email-masked]>
Date: Sun, 29 Oct 2017 23:18:32 -0400

Hi Martin,

> exsiting downref process?  The draft doesn't say this.

The existing downref process (RFC 3967) doesn't apply. That is to handle
references to
an RFC that is at a lower maturity - such as a normative reference to an
Informational or
Experimental RFC - or from an Internet Standard to a Proposed Standard.

Regards,
Alia

---

From: Martin Thomson <[email-masked]>
Date: Mon, 30 Oct 2017 14:27:12 +1100

> Experimental RFC - or from an Internet Standard to a Proposed Standard.

I couldn't find anything in 3967 to support that view (other than the
parenthetical in the abstract).  Maybe it's implicit.

---

From: Alia Atlas <[email-masked]>
Date: Sun, 29 Oct 2017 23:39:34 -0400

> parenthetical in the abstract).  Maybe it's implicit.

Well, certainly if you look at how the DownRef registry was incorporated
into datatracker
(https://datatracker.ietf.org/doc/downref/ ), the first item is RFC and the
second is the
referring draft.   At the least, it is clearly how it has been interpreted.

For that matter, the abstract clearly says:

"IETF procedures generally require that a standards track RFC may not
   have a normative reference to another standards track document at a
   lower maturity level or to a non standards track specification (other
   than specifications from other standards bodies).  For example, a
   standards track document may not have a normative reference to an
   informational RFC.  Exceptions to this rule are sometimes needed as
   the IETF uses informational RFCs to describe non-IETF standards or
   IETF-specific modes of use of such standards.  This document
   clarifies and updates the procedure used in these circumstances."

I don't see how this could be interpreted as allowing references to
specifications
from something that isn't from a standards body - though RFC2026 does talk
about
references to proprietary specifications - which has turned into asking for
re-publication
on the ISE stream.  Indeed RFC 3967 says "A standards document may need to
refer
to a proprietary protocol, and the IETF normally documents proprietary
protocols using
informational RFCs."

Personally, I have not seen it work well or be generally perceived as
anything more
than a waste of time to ask well specified and widely available mature open
source
work to come and be republished as an Independent Stream Informational RFC.
The
situation can be different for proprietary solutions and there are also
different assumptions
then in terms of control of the technology.

Can you explain your different interpretation?

Regards,
Alia

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 30 Oct 2017 16:48:36 +1300

> parenthetical in the abstract).  Maybe it's implicit.

The quote from RFC2026 in section 1 of RFC3967 is pretty unambiguous
when read in context - throughout RFC2026, "standards track" refers
to the IETF standards track, and "standards track specifications"
refers to IETF documents. And in any case the Abstract of RFC3967
sets the scope thus: "Exceptions to this rule are sometimes needed as
the IETF uses informational RFCs to describe non-IETF standards or
IETF-specific modes of use of such standards."

My main concern with the new proposal is other:

>  1.  Is it stable, mature, and immutable (except for errata)?

Those are not well known characteristics of open source projects.
Even if we tie a reference down to a very specific version,
I'm unclear how that can truly act as a stable reference. It may
be stable in a mathematical sense, but it may also be seven years
behind deployed reality.

    Brian

---

From: Martin Thomson <[email-masked]>
Date: Mon, 30 Oct 2017 14:54:17 +1100

> Can you explain your different interpretation?

I see "Standards Track", which clearly means RFC, and "not Standards
Track".  Assuming that to also mean RFC would be a logical error.  If
the entire premise of 3967 is to describe how RFCs refer to other RFCs
of a lower maturity levels, it never outright says it.  I agree that
this would appear to be the intent and interpretation, but it seems
like 3967-bis might be a neater way to address your central concern
(that we also refer to things other than RFCs, OMG, NIH!)  FWIW, the
narrow scope here (and in 3967) does us all a disservice when a
slightly larger mallet would do the job more definitively.

---

From: Alia Atlas <[email-masked]>
Date: Mon, 30 Oct 2017 00:09:37 -0400

> slightly larger mallet would do the job more definitively.

It has frequently been the case that getting large policy changes through
the IETF
is challenging and causes the non-controversial first steps to never
happen. There
was a lot of work in NEWTRK that had such issues.

The issues around IPR considerations are different for a generic reference
from
referencing another RFC or external standards body document, where the IPR
policies
are clear and well-known.

While I would be happy (of course) to discuss what broader scope you think
would be
preferable, the first question is whether there is community support for
this narrow scope.

Regards,
Alia

---

From: Martin Thomson <[email-masked]>
Date: Mon, 30 Oct 2017 15:14:36 +1100

> preferable, the first question is whether there is community support for

I'm am suggesting that the same discretion that you propose giving, is
extended to any documentation that meets the stated requirements.  I
fail to see how open source software is worthy of special treatment.
It's all words.  What matters are the questions you are addressing -
necessity, stability, etc...

---

From: Alia Atlas <[email-masked]>
Date: Mon, 30 Oct 2017 00:19:57 -0400

> necessity, stability, etc...

Ah - there is nothing in the document that talks about open-source!  It is
simply about referencing external non-SDO items.  I agree that there is no
reason to special case open source software - though it can fall in an
interesting
gap between proprietary standards and open standards discussed in RFC 2026.

Because the draft is a policy draft (and thus, strangely, dry to many
readers),
I am trying to clarify that the motivations for this draft at this time are
around
the challenges with mutual building with open source.

Regards,
Alia

---

From: Riccardo Bernardini <[email-masked]>
Date: Mon, 30 Oct 2017 10:10:43 +0100

On Mon, Oct 30, 2017 at 4:48 AM, Brian E Carpenter <
[email-masked]> wrote:

> "

Indeed, this was my first thought too. Although I like open source software
(and I find it often better than the proprietary counterpart, although with
maybe less bells and whistles), it is true that documentation in open
source projects have a huge variability (I am currently trying to write a
Linux module to be loaded at run-time and the biggest obstacle is the
scarce documentation... Moreover, Linux kernel API can change from one
release to the other, making that  tutorial written five years ago totally
useless today...)

Since the main feature of protocols should be interoperability,  I think
that a formal and unambiguous specification (and not just "as the GNU
implementation of version 3.2.4 of Foo protocol behaves") is necessary,
unless the cited open source protocol plays a very marginal role. Maybe a
solution could be to take a "snapshot" of the protocol in an Informational
RFC document to be written, possibly, in collaboration with the OS
community.  However, with this approach it seems to me that we are back to
the downref process.

Riccardo

---

From: Russ Housley <[email-masked]>
Date: Mon, 30 Oct 2017 09:19:15 -0400

Alia:

The SIDR WG wanted to reference rsync, but there was no stable reference that described the protocol, only the code.  A document was written to specify the rsync URL, and they is the only thing that is normatively referenced.  A reference to the protocol would be better, and I hope this BCP would allow that to happen in the future.

Section 2 is written as a list of questions.  In some cases it is unclear to me which answer to the question makes an acceptable normative reference.  A list of requirements would be more clear.

Russ

> On Oct 29, 2017, at 11:02 PM, Alia Atlas <[email-masked]> wrote:
> 
> After some discussion, my co-authors and I have written down a proposed policy that would clearly and specifically allow standards-track RFCs to have external non-SDO normative references.  It's not disallowed now - but determining early when it is ok is unclear.
> 
> My hope is that this will facilitate improved standard technology that can use "de-facto standards" that are suitably open and mature.
> 
> Please take a read (between all the fun technical drafts) and let us know what you think.
> 
> Thanks,
> Alia

---

From: Michael Richardson <[email-masked]>
Date: Mon, 30 Oct 2017 11:32:13 -0400

why is it called Normative References in RFCs **from** Open Source

Isn't this about references *to* open source?

--
Michael Richardson <[email-masked]>, Sandelman Software Works
 -= IPv6 IoT consulting =-

---

From: Eliot Lear <[email-masked]>
Date: Mon, 30 Oct 2017 16:53:28 +0100

Hi Russ,

> The SIDR WG wanted to reference rsync, but there was no stable reference that described the protocol, only the code.  A document was written to specify the rsync URL, and they is the only thing tha…

When you say, “the protocol”, to what are you referring?

> Section 2 is written as a list of questions.  In some cases it is unclear to me which answer to the question makes an acceptable normative reference.  A list of requirements would be more clear.

I think that's a fair point, but for a -00 the questions seem like a
good place to start.  Maybe you even have some ideas as to what good
answers should be.

Eliot

---

From: Russ Housley <[email-masked]>
Date: Mon, 30 Oct 2017 12:31:07 -0400

> When you say, “the protocol”, to what are you referring?

The rsync protocol.

Russ

---

From: Eliot Lear <[email-masked]>
Date: Mon, 30 Oct 2017 17:32:59 +0100

On 10/30/17 5:31 PM, Russ Housley wrote:
> What I meant was, do mean the document you referred to above or the code itself?

Eliot

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 30 Oct 2017 12:36:37 -0400

Two further comments on this rsync example:

1) As written, and to conform to our general notion of stable reference, 
if we want a normative reference to rsync, it would have to be a 
reference to a specific version, with a reliable way that folks could 
get to the definition of that version.

2) As written, and something I think is important, it would take a 
further exception to treat the code as a specification.  Code is NOT a 
specification for interoperable implementation.  And I would consider it 
a serious flaw if the only acceptable implementation came from a single 
source, even if that source was an open one.

Thus, if the rsync open source project included a clear and detailed 
description of the protocol, then this change would allow that to be a 
normative reference.

Yours,
Joel

---

From: Miles Fidelman <[email-masked]>
Date: Mon, 30 Oct 2017 14:44:27 -0400

>> When you say, “the protocol”, to what are you referring?

rsync is an ALGORITHM for synchronizing copies of files/directories, it 
has an implicit protocol for moving bytes between two copies, but it 
doesn't seem to be well documented

there have been implementations of the algorithm over various underlying 
transfer protocols - for example, zsync claims to use the rsync 
algorithm over HTTP (never used it myself)

and yes - it would be nice to have a standard protocol for 
file/directory synchronization  (it would also be nice to standardize 
things like exchanges between instances of distributed version control 
systems - for example the way subversion uses WebDAV & DeltaV)

somehow application layer developers don't think in terms of protocols, 
they keep coming back to client-server models & software-oriented APIs, 
rather than protocols

Miles Fidelman

-- 
In theory, there is no difference between theory and practice.
In practice, there is.  .... Yogi Berra

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 31 Oct 2017 08:03:31 +1300

On 31/10/2017 07:44, Miles Fidelman wrote:
...
> they keep coming back to client-server models & software-oriented APIs, 

If an API is well documented and implies a well-defined state machine,
I would have no problem referring to it as a stable reference.

   Brian

---

From: John C Klensin <[email-masked]>
Date: Mon, 30 Oct 2017 15:09:26 -0400

--On Sunday, October 29, 2017 23:02 -0400 Alia Atlas
<[email-masked]> wrote:

> After some discussion, my co-authors and I have written down a
> proposed policy that would clearly and specifically allow
> standards-track RFCs to have external non-SDO normative
> references.  It's not disallowed now - but determining early
> when it is ok is unclear.
> 
> My hope is that this will facilitate improved standard
> technology that can use "de-facto standards" that are suitably
> open and mature.
> 

Alia,

While others have said parts of this in different ways (I
believe and several of the comments since you posted your note
are really just about symptoms), I think it is important to come
back to the fundamental tension between at least one critical
objective for open source and the main objective of standards
and standardization.   

In the former case, the hope is that open source (and general
permission to make adaptation and modification) will permit the
community to rapidly evolve and improve systems, or at least
adjust them to different preferences and requirements.  By
contrast, standards are about interoperability (there should be
a high expectation that two conforming implementations will work
together) and interchangeability or substitutability (one should
be able to pull out one conforming implementation and replace it
with another with a minimum of fuss, disruption, and cost.
Those goals are often contradictory with local or frequent
changes, especially ones that are not very precisely documented.

IETF has traditionally put its primary focus on
interoperability.  One could argue that is an historical
accident of how we got here of that it is a necessity in a
network environment in which the main purpose of protocols is to
allow communication and smooth interworking of implementations
by different groups and often on different platforms.  Our
colleagues at ISO and ITU have tended to emphasize
substitutability.  Again, that could be an accident relating to
their history (especially in ISO's case) with supporting
definitions and procurement of physical products.  Their
interpretation of that principle for computer networking is
probably on the "why OSI failed" list somewhere.

But, ultimately, both are important.  The stable normative
reference requirement in anything we claim to be a standard is
ultimately just a corollary: if there is something I need to
understand or depend on in order to implement a specification in
an interoperable and substitutable way, than whatever that is
can't change out from under that standard spec.  At one point,
we wouldn't even allow a Full Standard to reference a Proposed
one. The relaxation to allow normative references to
Informational RFCs should be no threat because the IETF and the
RFC Editor control any changes and RFC numbers are absolutely
stable (we do have some issues about what happens when one
document references another that is replaced or becomes
obsolete, but that is mostly a separate topic).  It does impose
a requirement, which I'm not sure we are taking seriously
enough, to verify that the Informational document does not
depend normatively on anything that is less stable than our
standards-track documents.

The rule about recognized SDOs isn't a significant relaxation
either, assuming the IESG is informed and responsible enough to
"recognize" only those SDOs who take the stability and, clarity,
and dependencies of standards at least as seriously as we do.

But referencing something that may be a less-than-complete or
less-than-precise spec, that may be out of synch with the
software it supposedly describes (in standards terms, where the
software may not conform to the spec), or that may represent the
opinion of some semi-random party about what the spec should
actually be without adequate review by others --and many of us
have seen all of those with open source software and systems--
threatens the integrity of the IETF standards themselves.   Not
a good idea.

There is another issue about this that I don't think anyone has
mentioned.  We went through some very painful discussions and
times to get various standards bodies, governments, and even a
few corporations, to agree that IETF standards could be
referenced in their specifications, regulations, and procurement
requirements.  Some of that recognition was based on our
persuading them that our standards were stable,
carefully-considered, and hence actually meaningful.  I would
assume that, if normative references to unstable material were
allowed, those recognition decisions and agreements might well
come unglued.  I don't think we want to go there.

Just say "no".

    john

---

From: Miles Fidelman <[email-masked]>
Date: Mon, 30 Oct 2017 15:10:38 -0400

> I would have no problem referring to it as a stable reference.
Well sure, but an awful lot of APIs aren't defined in terms of 
well-behaved state machines, much less synchronized state machines. A 
procedural call or method invocation (local or remote) is a different 
conceptual model than an exchange of PDUs and keeping state machines 
synchronized.

Miles

---

From: Andrew Sullivan <[email-masked]>
Date: Mon, 30 Oct 2017 15:12:48 -0400

On Mon, Oct 30, 2017 at 02:44:27PM -0400, Miles Fidelman wrote:
> somehow application layer developers don't think in terms of protocols, they
> keep coming back to client-server models & software-oriented APIs, rather
> than protocols

Given that an _awfully_ large number of interconnections through what
I think of as protocols are effectively just APIs (usually using https
underneath), it seems that the "somehow" here is more that we protocol
nerds haven't figured out how to make our ways of making things work
attractive or even acceptable to application layer developers.  Sounds
to me like this is our problem, not theirs, and that we better figure
out a way to make it work.

I'm not sure the current proposal _is_ that way, but it's certainly
_a_ way.

A

-- 
Andrew Sullivan
[email-masked]

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 30 Oct 2017 21:40:58 +0100

On Sun, Oct 29, 2017 at 11:39:34PM -0400, Alia Atlas wrote:
> anything more than a waste of time to ask well specified and widely available mature open
> source work to come and be republished as an Independent Stream Informational RFC.

Do you have examples of this ? mature FOSS (attempted to be) republished as an informational
RFC where the process was unhelpfull ? Curious.

I would consider rsync an ideal example why i am disagreeing:

I think its very mature in theory, but every time you dare to
interoperate different versions, not even to say different implementations,
i was often in for a fail. Not to mention ASCI-8859-UDF issues and the like.

Aka: I would be happy if there was a good spec for rsync, and even more
so if there was an attempt of implementations to become compliant with a
good spec.

Toerless

---

From: Riccardo Bernardini <[email-masked]>
Date: Mon, 30 Oct 2017 22:29:45 +0100

> > anything more than a waste of time to ask well specified and widely

^^^^^^^^^^^^*

> > source work to come and be republished as an Independent Stream
I guess that  "well specified" is a cornerstone word here: _if_ the OS
protocol was well specified (and with this I mean in an unambiguous way,
with enough detail to allow interoperability with other implementations and
in a stably referenced document), I would agree that re-publishing it is a
waste of time and I would have no qualms in using it in an RFC.
However, my personal experience with OS software (both as users an --
partly -- as a contributor) is that documentation in some cases can be
really lacking and the source code is considered the ultimate documentation
(I remembered that few years ago I asked  details about the format of
octave data files and I was told to look at the source code).

One could think about considering a snapshot of a specific version of the
source code as the protocol specs; unfortunately source code is not good
for that: first, you need to reverse engineer it in order to deduce -- say
-- the format of a packet; secondly often is *not* unambiguous, for
example: if you declare a packet field as "int" in C, how many bits is?
Sure, the code will run smoothly with current processors since int is 32
bits everywhere, but what if you are going to communicate with a processors
whose int is 64 (or 16)-bit wide?  (and I did not mention "efficiency
tricks" that are ill-defined according to the C specs, but work as long as
you stick to gcc).

One could object that if there is an open source implementation, there is
no need for unambiguous specs since one can just use the existing
implementation. However, I see this as a  very weak solution: someone else
could need to create a new implementation, maybe in a different language,
maybe for a different architecture not compatible with the current
implementation.

Summarizing, if the cornerstone condition "well specified" is met, I think
that the OS protocol can be considered as a protocol developed by any
standard body (this suggest that condition "well specified" is quite
demanding), but if the condition is not met, the only solution I can think
of is to "snapshot" it (and clean it, if necessary) into an Informational
RFC.

Riccardo

---

From: Eliot Lear <[email-masked]>
Date: Mon, 30 Oct 2017 22:42:20 +0100

Hi Brian,

> be stable in a mathematical sense, but it may also be seven years

That's why point 4 is important:  Is it intended as a public interface? 
If the intent isn't there then this isn't the right mechanism.   We may
want to broaden out that intent statement to be a bit more explicit
about intent of stability.

Eliot

---

From: Stephen Farrell <[email-masked]>
Date: Mon, 30 Oct 2017 21:51:02 +0000

On 30/10/17 21:42, Eliot Lear wrote:
> That's why point 4 is important:Â  Is it intended as a public interface?Â 
> If the intent isn't there then this isn't the right mechanism.Â Â  We may

I like the idea, but would echo Brian's concerns.

Maybe some examples would help though? What examples have
you in mind (as authors) or can you point at some?

The ones that occur to me first wouldn't be so easy to
handle I think (in case you wonder those are: jabber/OTR
and ssh and pgp algorithm specs). It might be that there's
a good way to handle such, and that'd probably be a good
thing if there were, but I'm not sure TBH.

Ta,
S.

---

From: Russ Housley <[email-masked]>
Date: Mon, 30 Oct 2017 18:03:55 -0400

> On Oct 30, 2017, at 12:32 PM, Eliot Lear <[email-masked]> wrote:
> 
> 
> 

There was not a document that describes the rsync protocol.  The code has lots of comments, and it supports many versions.  The code is the real protocol definition.

Russ

---

From: Eliot Lear <[email-masked]>
Date: Mon, 30 Oct 2017 23:19:19 +0100

Hi Stephen,

> you in mind (as authors) or can you point at some?

The obvious example would be the vast quantities of YANG that are
cropping up everywhere. Another one that many reference outside of the
IETF context is ArcSight CEF, which is an overlay on SYSLOG that is
frequently used.  When there's active development on a project, this
isn't a good fit, due to feature velocity.  Even there, though, some may
wish to declare certain interfaces both public and stable.  I could
imagine the git team doing that, for instance, given the number of front
ends there are with it.

Eliot

---

From: S Moonesamy <[email-masked]>
Date: Mon, 30 Oct 2017 15:16:32 -0700

Hi Joel,
At 09:36 AM 30-10-2017, Joel M. Halpern wrote:
>1) As written, and to conform to our general notion of stable 
>reference, if we want a normative reference to rsync, it would have 
>to be a reference to a specific version, with a reliable way that 
>folks could get to the definition of that version.

 From what I recall the rsync case was a URI registration.  The RFC 
for that registration was referenced by an IETF RFC from SIDR.

>further exception to treat the code as a specification.  Code is NOT 
>a specification for interoperable implementation.  And I would 
>consider it a serious flaw if the only acceptable implementation 
>came from a single source, even if that source was an open one.

Were there any objections during the Last Call [1]?

Regards,
S. Moonesamy

1. Please see the rsync case.

---

From: Stephen Farrell <[email-masked]>
Date: Mon, 30 Oct 2017 22:36:13 +0000

Hiya,

On 30/10/17 22:19, Eliot Lear wrote:
> cropping up everywhere. 

I can see that something might work ok for yang, yes.

> Another one that many reference outside of the
> frequently used.Â  When there's active development on a project, this
> isn't a good fit, due to feature velocity.Â  Even there, though, some may
> wish to declare certain interfaces both public and stable.Â  I could

I'm not familiar with ArcSight.

I guess I'd suggest trying to collate a bunch of examples
(not necessarily in a -01 version, but somewhere) and it'll
be easier for folks to see what might work easily or be
hard.

Cheers,
S.

---

From: Alia Atlas <[email-masked]>
Date: Mon, 30 Oct 2017 19:09:06 -0400

Hi Ricardo & Brian,

On Mon, Oct 30, 2017 at 5:10 AM, Riccardo Bernardini <[email-masked]>
wrote:

> Indeed, this was my first thought too. Although I like open source
> software (and I find it often better than the proprietary counterpart,
> although with maybe less bells and whistles), it is true that documentation
> in open source projects have a huge variability (I am currently trying to
> write a Linux module to be loaded at run-time and the biggest obstacle is
> the scarce documentation... Moreover, Linux kernel API can change from one

Sadly, our RFCs can also be years ahead or behind of reality.  We also have
the ability to Update and Obsolete RFCs - with consequently impacted
referring documents.

In our draft, one requirement is that the work be intended to be a public
interface and another is that it is mature work.

Open source projects can vary from "I coded this up yesterday - go play" to
extremely mature projects that run parts of the Internet. A conversation
needs to recognize this range and the nuances.

> community.  However, with this approach it seems to me that we are back to

In general, the draft recommends using specifications. There can be cases -
such as algorithms - where the code is really the best descriptor.
I'm happy to hear suggestions for improving the text - of course!

Regards,
Alia

---

From: Alia Atlas <[email-masked]>
Date: Mon, 30 Oct 2017 19:11:39 -0400

> Isn't this about references *to* open source?

I do routing, not naming, well?

Good point & I see that the title will need changing too.

---

From: Miles Fidelman <[email-masked]>
Date: Mon, 30 Oct 2017 19:17:54 -0400

> I'm not sure the current proposal _is_ that way, but it's certainly
Right on!  And that really is (IMHO) and important mission for ISOC - 
migrating what we know about lower layer protocols into the application 
layer.

Miles Fidelman

---

From: Alia Atlas <[email-masked]>
Date: Mon, 30 Oct 2017 19:26:04 -0400

> RFC where the process was unhelpfull ? Curious.

It's more a question of motivation.  For instance, there are ecosystems
built on top of thrift and gRPC
or protobufs.  What is the motivation for folks to come and go through the
Independent Stream process?
There's an assumption that the folks who want to use a non-standard
technology are the same as are
appropriate to write an Informational RFC.

I'd prefer not to give an example here, but while it has come up, the main
issue is folks not even bringing or
thinking to bring re-use.

> i was often in for a fail. Not to mention ASCI-8859-UDF issues and the

Then that would be feedback from the WG not to build on top of it - or to
require a better specification.
This is a judgement call from the WG, not an anything goes.

> so if there was an attempt of implementations to become compliant with a

Sure - but still not the protocol police :-)

Regards,
Alia

---

From: Melinda Shore <[email-masked]>
Date: Mon, 30 Oct 2017 17:07:14 -0800

On 10/30/17 2:03 PM, Russ Housley wrote:
> There was not a document that describes the rsync protocol.  The code
> has lots of comments, and it supports many versions.  The code is the
> real protocol definition.

I'd like to think that ultimately we'll be able to include
code or other non-IETF-standard material in normative
references, but I think there are some key issues not
addressed in the draft, particularly around the stability
of the reference.  I think that revision control systems
probably can be used to address some of that ("branch
<whatever> on date <whenever>") but stuff gets moved in and
out of repos all the time and it's difficult to guarantee
the stability of material under someone else's control.  So,
I expect the question of the quality of the reference itself
probably needs to be addressed in Section 2 of the draft.
It also might be worth distinguishing between the quality of
the reference and the quality of the material being referenced
(basically the same point).

Melinda

---

From: John C Klensin <[email-masked]>
Date: Mon, 30 Oct 2017 21:34:21 -0400

--On Monday, October 30, 2017 19:26 -0400 Alia Atlas
<[email-masked]> wrote:

>> interoperate different versions, not even to say different
>> implementations, i was often in for a fail. Not to mention
>> ASCI-8859-UDF issues and the like.
>> 
> 
> Then that would be feedback from the WG not to build on top of
> it - or to require a better specification.

That would be a reasonable answer if our WGs were always
balanced among perspectives and interests and/or the IESG really
did a careful audit each time a document was sent to it as to
whether the WG has actually considered all of the issues related
to a particular spec _and_ no one every argued strongly that a
WG had approved of a piece of work and therefore the IESG and
the community had to standardize it unless harm could be
conclusively proved by those who objected.  Unfortunately, while
we often get those things right, we often don't and I've seen
very little evidence that the IESG is really willing to take on
that particular reviewer/ certifier/ gatekeeper job to the point
of rejecting -- not just asking for modifications to-- a WG
product when all of the ducks are not lined up.

Again, I see the normative reference rules as protecting the
IETF and the general perception of the quality and stability of
our standards, not just a question of the approval (or not) of
particular documents.

best,
    john

---

From: Eliot Lear <[email-masked]>
Date: Tue, 31 Oct 2017 10:33:22 +0100

Hi Melinda,

On 10/31/17 2:07 AM, Melinda Shore wrote:
> the stability of material under someone else's control.

To me, this is where intent comes in.  To put this in the negative,
because I think that's where we derive most value from the statement, if
someone doesn't intend an interface to be stable and mature, we should
not be referencing it.  I think that accounts for over 99% of all
interfaces.  Thus, starting with a statement that the interface is
intended to be stable is a big step.  At that point what should we be
looking for and how should we judge?

Eliot

---

From: "tom p." <[email-masked]>
Date: Tue, 31 Oct 2017 12:01:16 -0000

----- Original Message -----
From: "S Moonesamy" <[email-masked]>
To: "Joel M. Halpern" <[email-masked]>; <[email-masked]>
Sent: Monday, October 30, 2017 10:16 PM

> for that registration was referenced by an IETF RFC from SIDR.

sidr-arch

had

   [RSYNC]    Tridgell, A., "rsync", March 2008,
              <http://rsync.samba.org/>

as an Informative Reference which became

   [RSYNC]    rsync web pages, <http://rsync.samba.org/>.

in RFC6480 and then a Normative Reference in RFC6481.

Tom Petch

---

From: "Mehmet Ersue" <[email-masked]>
Date: Tue, 31 Oct 2017 17:05:49 +0100

Hi Alia, 

> Even if we tie a reference down to a very specific version, I'm unclear how
> that can truly act as a stable reference. It may be stable in a mathematical
> sense, but it may also be seven years behind deployed reality.

I agree with Brian. There are different type of non-SDO organizations having documents with diverse maturity levels and stability. Many of them don't qualify as normative references.

Obvious examples for normative references are usually from well-known SDOs like IEEE or W3C where the new standard-track RFC uses it as its basis or builds on-top of it (e.g. 	[W3C.REC-xml-20001006])

I also don't see an issue with MD5 and HMAC as these are Informational RFCs at IETF with a well-known stability.

Looking at section 1.1. of RFC 3967 and the statement: "a normative reference specifies a document that must be read to fully understand or implement the subject matter in the new RFC, or whose contents are effectively part of the new RFC, as its omission would leave the new RFC incompletely specified", 
can you please give examples of external documents from non-DOSs, which clearly qualify as normative references for a standard-track RFC at IETF? 
I think primarily on non-SDO documents where the new RFC would build on-top of it.

Thanks,
Mehmet

> -----Original Message-----
> From: ietf [mailto:[email-masked]] On Behalf Of Brian E Carpenter
> Sent: Monday, October 30, 2017 4:49 AM
> To: Martin Thomson <[email-masked]>; Alia Atlas
> <[email-masked]>
> Cc: IETF Discussion Mailing List <[email-masked]>
> Subject: Re: letting IETF build on top of Open Source technology
> 
> >> The existing downref process (RFC 3967) doesn't apply. That is to
> >> handle references to an RFC that is at a lower maturity - such as a
> >> normative reference to an Informational or Experimental RFC - or from
> >> an Internet Standard to a Proposed Standard.
> >
> The quote from RFC2026 in section 1 of RFC3967 is pretty unambiguous when
> read in context - throughout RFC2026, "standards track" refers to the IETF
> standards track, and "standards track specifications"
> refers to IETF documents. And in any case the Abstract of RFC3967 sets the
> scope thus: "Exceptions to this rule are sometimes needed as the IETF uses
> informational RFCs to describe non-IETF standards or IETF-specific modes of
> use of such standards."
> 
> Even if we tie a reference down to a very specific version, I'm unclear how
> that can truly act as a stable reference. It may be stable in a mathematical
> sense, but it may also be seven years behind deployed reality.
> 
>     Brian

---

From: Alia Atlas <[email-masked]>
Date: Tue, 31 Oct 2017 12:13:00 -0400

Hi Mehmet,

On Tue, Oct 31, 2017 at 12:05 PM, Mehmet Ersue <[email-masked]> wrote:

> I agree with Brian. There are different type of non-SDO organizations
> having documents with diverse maturity levels and stability. Many of them
> don't qualify as normative references.
>

Of course - and that's why it is necessary for the item to meet the
questions posed in the draft - which are the requirements (and we'll
rephrase more clearly as Russ suggested).

> Obvious examples for normative references are usually from well-known SDOs
> like IEEE or W3C where the new standard-track RFC uses it as its basis or
> builds on-top of it (e.g.  [W3C.REC-xml-20001006])
>
> I also don't see an issue with MD5 and HMAC as these are Informational
> RFCs at IETF with a well-known stability.
>

Sure - and that was one of the motivations for RFC 3967.

> Looking at section 1.1. of RFC 3967 and the statement: "a normative
> reference specifies a document that must be read to fully understand or
> implement the subject matter in the new RFC, or whose contents are
> effectively part of the new RFC, as its omission would leave the new RFC
> incompletely specified",
> can you please give examples of external documents from non-DOSs, which
> clearly qualify as normative references for a standard-track RFC at IETF?

An example would be potentially thrift, I think, for protocol encodings. I
have to hunt down the detailed reference for it.
I know that protobufs have good documentation also.  I am sure that there
are others and would be happy to hear folks chime in with more.

It's unclear whether there is a set of requirements acceptable to the IETF
Community that would work for WHATWG work.

Is your concern that you don't see useful open source work that would be
suitable to reference? Or that none would meet our set of requirements?
Or that it is risky to consider referencing non-SDO external work?

Regards,
Alia

---

From: "Mehmet Ersue" <[email-masked]>
Date: Tue, 31 Oct 2017 18:14:29 +0100

See ME:

Thanks,

Mehmet

From: Alia Atlas [mailto:[email-masked]] 
Sent: Tuesday, October 31, 2017 5:13 PM
To: Mehmet Ersue <[email-masked]>
Cc: Brian E Carpenter <[email-masked]>; Martin Thomson <[email-masked]>; IETF Discussion Mailing List <[email-masked]>
Subject: Re: letting IETF build on top of Open Source technology

Hi Mehmet,

On Tue, Oct 31, 2017 at 12:05 PM, Mehmet Ersue <[email-masked] <mailto:[email-masked]> > wrote:

Hi Alia,

Of course - and that's why it is necessary for the item to meet the questions posed in the draft - which are the requirements (and we'll rephrase more clearly as Russ suggested). 

An example would be potentially thrift, I think, for protocol encodings. I have to hunt down the detailed reference for it.

I know that protobufs have good documentation also.  I am sure that there are others and would be happy to hear folks chime in with more.

It's unclear whether there is a set of requirements acceptable to the IETF Community that would work for WHATWG work.

Is your concern that you don't see useful open source work that would be suitable to reference? Or that none would meet our set of requirements?

ME: Generally I don’t see an issue with referencing useful open source work. Though only in rare cases an IETF RFC has a dependency on or builds indeed on-top of such a document normatively. I think the main criteria for a normative reference is still based on the quoted text from RFC 3967 above.

I believe use cases and examples will help to define concrete requirements.

Once we have the requirements together such documents need to be looked at in a case by case basis and taken over into a registry.

Regards,

Alia

Thanks,
Mehmet

> From: ietf [mailto:[email-masked] <mailto:[email-masked]> ] On Behalf Of Brian E Carpenter
> To: Martin Thomson <[email-masked] <mailto:[email-masked]> >; Alia Atlas
> <[email-masked] <mailto:[email-masked]> >
> Cc: IETF Discussion Mailing List <[email-masked] <mailto:[email-masked]> >
> > On Mon, Oct 30, 2017 at 2:18 PM, Alia Atlas <[email-masked] <mailto:[email-masked]> > wrote:

---

From: Alia Atlas <[email-masked]>
Date: Tue, 31 Oct 2017 14:29:56 -0400

Hi Mehemet:

below in-line

On Tue, Oct 31, 2017 at 1:14 PM, Mehmet Ersue <[email-masked]> wrote:

> See ME:
>
>
>
> Thanks,
>
> Mehmet
>
>
>
> *From:* Alia Atlas [mailto:[email-masked]]
> *Sent:* Tuesday, October 31, 2017 5:13 PM
> *To:* Mehmet Ersue <[email-masked]>
> *Cc:* Brian E Carpenter <[email-masked]>; Martin Thomson <
> [email-masked]>; IETF Discussion Mailing List <[email-masked]>
> *Subject:* Re: letting IETF build on top of Open Source technology
>
>
>
> Hi Mehmet,
>
>
>
> ME: Generally I don’t see an issue with referencing useful open source
> work. Though only in rare cases an IETF RFC has a dependency on or builds
> indeed on-top of such a document normatively. I think the main criteria for
> a normative reference is still based on the quoted text from RFC 3967 above.
>
>

[Alia] Agreed - and I pulled the requirements around stability, maturity,
confidentiality, and IPR considerations largely from 2026.
My question is whether it is rare than an IETF RFC builds on top of open
source work because the path to do so is seen as too
challenging/long/expensive or simply not relevant. This draft is proposing
a policy assuming the former applies at least some of the time.

> Once we have the requirements together such documents need to be looked at
> in a case by case basis and taken over into a registry.
>

The requirements, I think, stem from our (the IETF community's)
understanding of what is necessary for a dependency of a standards-track
RFC and I am not proposing that those should change.  I'm trying to capture
what they are and how to evaluate a piece of work.

Rather than looking at the set of documents and creating a registry, the
draft suggests that when a WG wants to use such an item as a dependency,
then after that use has IESG Approval, then the item can be added to a
registry - so other WGs and other documents in that WG can benefit from the
evaluation that was done and feel more confident about using it.

Regards,
Alia

---

From: Toerless Eckert <[email-masked]>
Date: Tue, 31 Oct 2017 21:54:28 +0100

So, whats the right terminology here for at least the two classes
that i think are easy to destinguish:

a) In one case i have FOSS that implements a new protocol like rsync does,
   and interop means that you want two independent implementations support
   the protocol.

b) In another case, which i think are all your examples, you have some
   form of "service" which is providing some form of "API" and interop
   really means that only one implementation is required to show how
   it works with multiple clients of the service.

I also think that b) is easier to adopt because 
- interop does not require you to find other implementations of the
  same protocol/service
- An API that MUST be used by a third party client is more naturally
  documented than a protocol that was meant to be used between multiple
  instances of a single FOSS software (typical a) case).

Cheers
    Toerless

---

From: Keith Moore <[email-masked]>
Date: Wed, 1 Nov 2017 01:09:58 -0400

It seems to me that a fundamental problem with referencing Open Source software is that in much (not all) of the Open Source world, rapid innovation and evolution is valued more than stable protocol interfaces.   There is a tendency to rely on users to frequently update their software, which is seen as giving developers freedom to change interfaces on relatively short notice.  

Of course that's an oversimplification.  In practice good open source developers are careful to avoid abrupt changes to interfaces; instead they create new interfaces and deprecate old ones.   But since one of the oft-touted benefits of Open Source is to allow software systems to evolve rapidly, this sometimes also results in new protocol features being deployed rapidly to the point that it impairs interoperability between new and old implementations.   This causes a great many problems other than just protocol interoperability failures, including dependency hell.

So part of the effort to improve cooperation between IETF and Open Source developers - who tend to have similar motivations even if their methods differ - might be in convincing them that they would benefit from more stable interfaces - at least on the wire - or at least encouraging them to define protocols so that they are cleanly extensible, and defining a baseline for interoperability even as they extend protocols to accommodate new features.   And actually it might be the case that IETF could learn something about protocol extensibility from people who have struggled to accommodate rapid evolution of their software.

Keith