Subject: Re: Last Call: draft-farrel-rtg-common-bnf (Reduced Backus-Naur Form(RBNF) A Syntax Used in Various Protocol Specifications) toProposed Standard

From: "Adrian Farrel" <[email-masked]>
Date: Thu, 5 Feb 2009 12:44:56 -0000

Thanks Tom,

> A) The start of this I-D seems a little coy - 'various protocol 
> specifications'
> 'several protocols' - and this is reflected in the Abstract and 
> Introduction.
> Reading between the lines, this seems to have had its genesis in the 
> 'Sub-IP
> Area' specification; nothing wrong with that, but the coyness seems 
> misplaced.

Coyness is my speciality. This is a cultural thing, I fear - we can't all be 
brash Americans no matter how much we aspire to it ;-)

I won't finger the "Sub-IP Area". But I will note that all the RFCs 
concerned are in the Routing Area.

> More generally, I think that this I-D cries out for an Applicability 
> Statement.
> It makes brief reference to RFC5234 but contains no guidance that I can 
> see as
> to when this standard should be used or when RFC5234 should be.  The IETF 
> has a
> history of producing multiple standards and letting the market decide but 
> I
> think that we do a better job when we give guidance.

Agreed.
I have added a brief Applicablity Statement in a new Section 1.3.
It is clear to me what the applicability is for:
- interpretation of existing protocols that use RBNF
- extensions to existing protocols that use RBNF
- new protocols that are "closely associated" to existing protocols
   that use RBNF.

It is also clear to me that RBNF is not recommended for protocols that 
already use some other form of BNF.

As far as I am concerned, the use of RBNF for the development of any other 
new protocol is a choice for the working group concerned and out of scope 
for me. However, I have included text that leans toward using ABNF.

> B) Coyness again, in its definitions
> 'The basic building blocks of BNF are rules and operators'
> but what is a rule?  RFC5234 eg says
>
> "A rule is defined by the following sequence:
>         name =  elements crlf"
>
> and I think that something similar is needed here (or else make RFC5234 a
> normative reference:-)

Again I am coy. I wonder if it a virtue or a vice.

I think you are a little precipitous in your judgement (I am being coy in my 
criticism of you ;-)

The first sentence of section 2 is (as you quote) "The basic building blocks 
of BNF are rules and operators."

The second sentence is
  "At its
   simplest form, a rule in the context we are defining is a protocol
   object that is traditionally defined by a bit diagram in the protocol
   specification."

So I think you are being overly enthusiastic to ask so quickly what a rule 
is.

I don't propose any change for this.

> C) In a similar vein, to me, and perhaps to many in the IETF, it is 
> RFC5234 or
> its precursors that represent the 'standard' meta-syntactic language. 
> Some
> comparison of the functionality would be helpful, as an informative 
> Appendix.
> Is this a proper subset, if not, then where?

This was considered and rejected as the I-D was being written.

5234 might be the default/standard form of BNF in the IETF *now* but we are 
dealing in protocols that pre-date 5234. Indeed they pre-date (just) ABNF 
right back to 2234.

The Introduction explains why 5234 cannot be used in this case because of 
the cost of updating existing RFCs.

The question becomes how to work with the sentence:
   Unfortunately these specifications are not based on any specific
   formal definition of BNF and differ slightly from the definitions
   provided in other places.

My view was that it is necessary to provide a specification of RBNF for the 
uses identified. Such a definition should be stand-alone and does not 
require any comparison with any other form of BNF. Such a comparison would 
be a lot of work for (I believe) zero gain.

So, I don't propose to make any changes for this.

> D) As s.2.4 says.
>
> 'Precedence is the main opportunity for confusion in the use of BNF.'
>
> I think this should go further.  The underlying reason IMO is because the
> concatenation mechanism, the one with no operator, takes precedence over 
> the
> alternative operator, and this is counter-intuitive.  RFC5234 spells this 
> out
> ' Use of the alternative operator, freely mixed with concatenations,
>   can be confusing.'
> and, IPR permitting (I note that this was submitted pre-5378 but any 
> revision
> would not be:-), I suggest incorporating such text.

OK. Added a line on this.

Cheers,
Adrian

---

From: John C Klensin <[email-masked]>
Date: Thu, 05 Feb 2009 10:21:59 -0500

Adrian,

Having spent several years working with formal and semi-formal
definitions of programming language and being fairly unsatisfied
with IETF trends in the area, I had hoped to avoid looking at
this spec.  No such luck, obviously.

I presume this is not what you intended, but you have just made
a powerful argument for:

(1) Adding a clear statement to this that says, approximately,
"this is intended to document a BNF variant that was used in
some protocols instead of ABNF, for reasons that seemed
appropriate at the time.  Updating those protocols to use ABNF
is likely to be too costly and too prone to accidental errors,
so this description is provided to improve understanding.  RBNF
is not recommended for new protocol specifications unless it is
necessary to preserve compatibility and reduce confusion
relative to existing ones.

Without debating "coyness" or the lack thereof, I see no reason
to avoid being clear about this... and going well beyond an
applicability statement for a Proposed Standard while doing so.

(2) Publishing as Informational or direct-to-Historic.

See below.

--On Thursday, February 05, 2009 12:44 PM +0000 Adrian Farrel
<[email-masked]> wrote:

>...
> It is also clear to me that RBNF is not recommended for
> protocols that already use some other form of BNF.
> 
> As far as I am concerned, the use of RBNF for the development
> of any other new protocol is a choice for the working group
> concerned and out of scope for me. However, I have included
> text that leans toward using ABNF.
>... 
>> C) In a similar vein, to me, and perhaps to many in the IETF,
>> it is  RFC5234 or
>> its precursors that represent the 'standard' meta-syntactic
>> language.  Some
>> comparison of the functionality would be helpful, as an
>> informative  Appendix.
> 5234 might be the default/standard form of BNF in the IETF
> *now* but we are dealing in protocols that pre-date 5234.
> Indeed they pre-date (just) ABNF right back to 2234.

With the understanding that I am not a big fan of ABNF and never
have been, it dates from RFC 822.  None of these specifications
predate that.  On the other hand, however wise or unwise the
decision to use something else might have been at the time, it
is long since done.  Debating its wisdom now appears to me to be
a waste of time and getting this description published (in some
category) seems wise to me.

However, if you are documenting existing practice for
understanding and compatibility, rather than encouraging use of
this form, the use of strongly normative RFC 2119 language
directed to new uses seem inappropriate.

> The Introduction explains why 5234 cannot be used in this case
> because of the cost of updating existing RFCs.

While I am very sympathetic to this position and the explanation
made there (actually, all it says is "would require a lot of
work" which strikes me as more of an assertion than an
explanation),
I note that, when RFC 821 was updated to form 2821, the formal
syntax for describing commands was changed from BNF to ABNF.  It
is possible and the cost equation is somewhat subjective.
However, errors were made in the transition that made it into
2821 despite considerable checking.  The likelihood of
introducing errors is, to me, a much stronger argument for not
making changes than "lot of work".

> My view was that it is necessary to provide a specification of
> RBNF for the uses identified. Such a definition should be
> stand-alone and does not require any comparison with any other
> form of BNF. Such a comparison would be a lot of work for (I
> believe) zero gain.

> So, I don't propose to make any changes for this.

I partially disagree if you are trying for Proposed Standard.  I
get much more relaxed if you are aiming for Informational, as
suggested above.  See below.

<tirade>
Having multiple syntactic metalanguages floating around in the
same community is bad news.  That is especially true if
documents that use those metalanguages are not painfully clear
about what they are using.  If a reader looks at the
metalanguage in one document and believes that he is seeing one
that is specified differently from what is actually there, we
have at least the equivalent of a specification ambiguity.
"Painfully clear" is not going to occur here, since there is no
way to modify the existing RFCs to include normative references
to this spec.

While this horse has left the barn, I believe it is
irresponsible for the IETF to publish a document that uses a
metalanguage without a precise definition of that metalanguage
(either inline or by reference).

If one cannot eliminate the use of multiple forms, the advantage
of doing a comparison becomes clear: it is an important warning
to the user as to what to watch out for.   Since this doesn't
look like ABNF (which uses different operator syntax and uses
pointed brackets only when needed for clarity) it is likely that
the appropriate comparison should be to either to the original
BNF or to ISO EBNF but that doesn't eliminate the desirability
of documenting the major differences.  In that context, "similar
to a subset of Extended BNF" impresses me as the worst sort of
hand waving.  I guess it isn't actually an (unspecified) subset
and I don't know what "similar" means.  The whole purpose of
using formal metalanguage is to provide rigorous precision and
clarity; "similar to a subset" denies that entire concept.

Finally, the name of this thing is misleading at best.  BNF is
_very_ minimal and this is, in no sense, "reduced" BNF.  Perhaps
it is "reduced extended BNF", "reduced variant near-subset
extended BNF", or "slightly extended BNF", but "reduced BNF it
is not".
</tirade>

   john

---

From: "Adrian Farrel" <[email-masked]>
Date: Thu, 5 Feb 2009 17:14:40 -0000

John,

Good of you to write. It has been a while since I received an email from 
you.

> this spec.  No such luck, obviously.

Well, who knows who constrains you to certain acts? :-)

> I presume this is not what you intended, but you have just made

Why make that presumption?
It is pretty much explicit in the draft.

> appropriate at the time.

Yes, this form of BNF was used.
I am not sure that it was instead of ABNF since it seems to pre-date the 
completion of that work.
Who knows whether the reasons seemed appropriate at the time? And who cares?

We are where we are. A set of protocols use RBNF. The IESG insists that it 
be documented.

> Updating those protocols to use ABNF

Absolutely.

> so this description is provided to improve understanding.

Right.

> RBNF

I am neither recommending or not recommending its wider use.

I am recommending the positive of your statement. That is, that RBNF should 
be used for new protocol specifications that extend or are compatible with 
existing specifications that use RBNF.

> applicability statement for a Proposed Standard while doing so.

Given several people have asked what my intention is with regard to the use 
of RBNF in new documents, I feel it is helpful to include specific 
statements. Currently, I have:

   RBNF as defined in this document is primarily applicable for the
   protocols listed in the previous section. The specification may be
   used to facilitate the interpretation of the pre-existing RFCs that
   are referenced. It should also be used in the specification of
   extensions to those protocols.

   RBNF could also be used for the specification of new protocols. This
   is most appropriate for the development of new protocols that are
   closely related to those that already use RBNF. For example, PCEP is
   closely related to RSVP-TE and when it was developed, the PCE working
   gorup chose to use the same form of BNF as was already used in the
   RSVP-TE specifications.

   If a wholly new protocol is being developed and is not related to a
   protocol that already uses RBNF, the working group should consider
   carefully whether to use RBNF or to use a more formally specified and
   broader form of BNF such as ABNF [RFC5234].

   The use of RBNF to specify extensions to protocols that do not
   already use RBNF (i.e., that use some other form of BNF) is not
   recommended.

You feel I should be clearer?

> (2) Publishing as Informational or direct-to-Historic.

How can it be Historic given that the protocols that use RBNF are not 
Historic? I think that would be silly.

Should it be Informational? Well, if it only provided an explanation of 
published RFCs, I'd be happy. But the flow is as follows:
- extensions to protocol documents will often be standards track
- extensions to existing protocols should use the same formal
  language as the base documents
- if a specification uses a formal language, the definition of that
  language is normative
- downrefs are discouraged

[SNIP]

>>> it is  RFC5234 or its precursors that represent the 'standard'
>>> meta-syntactic language.  Some comparison of the functionality
>>> would be helpful, as an informative  Appendix.

Hmmm. Looks like you are right about the date of 822. Unfortunate that the 
RFC Editor index does not show 2234 obsoleting or updating 822.

Glad you agree that we should publish a description of RBNF.

> directed to new uses seem inappropriate.

Well, there *will* be new uses.
It would be wrong IMHO for an extension to RSVP to vary from the BNF used in 
RFC2205 whatever the history.
So, given that there will be new uses, we need to constrain those uses to do 
the right thing.
2119 provides a way of laying it on thick enough that some people might pay 
attention.

> work" which strikes me as more of an assertion than an

I am perfectly willing to have someone disprove me by doing the work.

> making changes than "lot of work".

Well, perhaps I meant that doing it is synonymous with doing it right. But 
you are correct that introducing errors would be a pain.

You'll note that I did not provide a comprehensive list of RFCs that use 
RBNF. I can see piles of them from where I am sitting.

>>    Unfortunately these specifications are not based on any
>>    specific formal definition of BNF and differ slightly from
>>    the definitions provided in other places.
>>

Just jumping in before the tirade :-)

It would have been possible to write a spec for RBNF as a variation from 
ABNF. Doing so would require that I fully understood ABNF *and* that I was 
certain to have caught all of the differences. Writing a spec for RBNF as it 
stands was somewhat simpler (for me).

> (either inline or by reference).

Which is largely why this document exists.
Protocol extension documents are being written, they use the same 
metalanguage as the base specs. This should be documented and be a normative 
reference.

> clarity; "similar to a subset" denies that entire concept.

You have pointed to text in the Introduction.
Introductions are not normally formal, but provide background. The text is 
not intended to make you understand the content, but to help you understand 
the context.

> extended BNF", or "slightly extended BNF", but "reduced BNF it

Then suggest a new and practical name.
I wanted to avoid "Routing Area BNF"
And I don't care what the BNF is called.
I would even be happy to call it "A metalanguage sometimes used"

Cheers,
Adrian

---

From: John C Klensin <[email-masked]>
Date: Thu, 05 Feb 2009 14:03:15 -0500

--On Thursday, February 05, 2009 5:14 PM +0000 Adrian Farrel
<[email-masked]> wrote:

> John,
> 
> Good of you to write. It has been a while since I received an
> email from you.
> 
>> definitions of programming language and being fairly
>> unsatisfied with IETF trends in the area, I had hoped to
>> avoid looking at this spec.  No such luck, obviously.
> 

I have been trying for several years to narrow the number of
things I try to worry about in the IETF so as to concentrate
efforts on those for which I have time.  My own choice, clearly.

>> I presume this is not what you intended, but you have just
>> made a powerful argument for:
> 

I was trying to suggest that, despite the draft's, and Last
Call, presumably of Proposed Standard, it might be more
appropriate to publish this as Informational, defining "RBNF"
for the context of a list of specific specifications and their
derivatives.  That doesn't require standards-track unless we do
want to recommend it.

> Yes, this form of BNF was used.

I know.

> I am not sure that it was instead of ABNF since it seems to
> pre-date the completion of that work.

ABNF has been around, and actively used in the IETF, for a very
long time.  During most of that time, specifications that used
it simply references 822, which contains a plausible definition.
The only thing that changed with the publication of RFC 2234 in
1997 were that the specification expanded a bit and got a little
better defined and the appropriate reference changed from 822 to
2234/4234/5234/STD68.  I think it is worth caring about because
it sets a context for this different form: unlike protocol
standards, there is little reason for the IETF to support or
permit multiple metalanguage standards unless one can be clearly
shown to be superior for a particular situation.  You have not
made the assertion that none of those specs could have been
written with ABNF instead without loss of quality, only that
changing them over at this point would be more trouble than it
is worth (a conclusion with which I agree).

>...
>> RBNF
>> is not recommended for new protocol specifications unless it
>> is necessary to preserve compatibility and reduce confusion

And I am suggesting that, if this is going to be a
standards-track document, such a recommendation is a
requirement.  That is a special case of the requests from others
for an applicability statement.

> I am recommending the positive of your statement. That is,
> that RBNF should be used for new protocol specifications that
> extend or are compatible with existing specifications that use
> RBNF.

If you would be comfortable inserting "only" in that sentence,
we would be in agreement, both about what is needed and what
should be said.

> Given several people have asked what my intention is with
> regard to the use of RBNF in new documents, I feel it is
> helpful to include specific statements. Currently, I have:
> 

While I believe that a stronger statement would be both more
appropriate and simpler to write, I could live with the above.

> How can it be Historic given that the protocols that use RBNF
> are not Historic? I think that would be silly.

I think Informational would be more appropriate for that and
other reasons.  On the other hand, "silly" has not been a bar to
IETF decisions in recent years :-(

> Should it be Informational? Well, if it only provided an
> explanation of published RFCs, I'd be happy. But the flow is
> as follows:

And that takes us down the usual path, e.g., questions about how
one tests interoperability of a metalanguage specification?
Perhaps we should be testing meta-interoperability, but I don't
know what that is.

>
> Hmmm. Looks like you are right about the date of 822.
> Unfortunate that the RFC Editor index does not show 2234
> obsoleting or updating 822.

Mutter.  It really doesn't because the metalanguage defined in
822 is correct for 822, presumably correct for documents and
reference 822 for the metalanguage, and slightly different from
the metalanguage specified by 2234/ 5234.  But maybe the
relationship is such that it should be identified as "updates"
because we have no better mechanism.

> Glad you agree that we should publish a description of RBNF.

I think having these things floating around undocumented is a
big mistake.  Indeed, I believe that the specs that use what you
are called RBNF should never have been published without a clear
definition.  If is a bit too late to fix that now.
 
>> understanding and compatibility, rather than encouraging use
>> of this form, the use of strongly normative RFC 2119 language
> It would be wrong IMHO for an extension to RSVP to vary from
> the BNF used in RFC2205 whatever the history.
> So, given that there will be new uses, we need to constrain
> those uses to do the right thing.
> 2119 provides a way of laying it on thick enough that some
> people might pay attention.

The way to get people to pay attention is for the IESG to make
firm decisions about requirements and publicize them.  If they
do that, then it doesn't make any difference what category this
document is published in, only that it be published.  If they
are not willing to do that, then I'm skeptical about the
attention.

>>> The Introduction explains why 5234 cannot be used in this
>>> case because of the cost of updating existing RFCs.
>> 
>> While I am very sympathetic to this position and the
>> explanation made there (actually, all it says is "would
>> require a lot of work" which strikes me as more of an
>> assertion than an explanation),
> 

Sorry.  I completely agree that it would be a lot of work (and I
have enough experience with such conversions to be able to
judge).  I am also convinced, based on the same experience, that
the odds of an error-free conversation are too low to be
tolerable.  I was merely pointing out that you had not
_explained_ the reason for not converting, merely asserted that
it would be a lot of work.  I was not disputing the assertion.

> It would have been possible to write a spec for RBNF as a
> variation from ABNF. Doing so would require that I fully
> understood ABNF *and* that I was certain to have caught all of
> the differences. Writing a spec for RBNF as it stands was
> somewhat simpler (for me).

And maybe we need to be pragmatic about this and say "some spec
for RBNF is clearly better than no spec for RBNF and this is all
that resources are going to permit".   I can live with that if
we are explicit about it.
 
> Protocol extension documents are being written, they use the
> same metalanguage as the base specs. This should be documented
> and be a normative reference.

As I tried to say above, we are in complete agreement on that
subject.

>> _very_ minimal and this is, in no sense, "reduced" BNF.
>> Perhaps it is "reduced extended BNF", "reduced variant
>> near-subset extended BNF", or "slightly extended BNF", but
>> "reduced BNF it is not".
> 

I'm not good at names and would prefer to leave this to others.
But I don't think we should call it "reduced" relative to
something that is smaller than it is.  And I think we would be
better off with a title that is as specific to function as
possible.

best,
    john

---

From: "Adrian Farrel" <[email-masked]>
Date: Thu, 5 Feb 2009 20:49:23 -0000

Thanks John,

It looks to me from your mail that we are in partially violent agreement.

Certainly that this form of BNF needs to be documented.
Also that the Applicability text I have added "will do."

We have two open issues:
- Use of 2119 language
- Standards Track or Informational

On the first, I take your point and am uncomfortable about using 2119 for 
what is not a protocol spec. Experience seems to be, however, it helps 
readers to understand that a rule is a rule.

On the second, I would like to defer to the IESG. I will raise it with the 
sponsoring AD (Ross) and get them to discuss it when they process the I-D.

Cheers,
Adrian

---

From: "Tom.Petch" <[email-masked]>
Date: Fri, 6 Feb 2009 13:55:14 +0100

----- Original Message -----
From: "Adrian Farrel" <[email-masked]>
To: "John C Klensin" <[email-masked]>; "Tom.Petch" <[email-masked]>;
<[email-masked]>
Sent: Thursday, February 05, 2009 9:49 PM
Subject: Re: Last Call: draft-farrel-rtg-common-bnf (Reduced Backus-Naur
Form(RBNF) A Syntax Used in Various Protocol Specification toProposed Standard

> sponsoring AD (Ross) and get them to discuss it when they process the I-D.

I agree with John that Standards Track is inappropriate for this I-D (and agree
that it does need publishing). I see either Informational or Historic as
appropriate and when this leads to Normative downrefs, then again, I see that as
appropriate.

I think too that there is a third issue, of a better name than RBNF.  John
clearly showed that this I-D is not reduced.  Historic? Deprecated?
Limited_applicability? Variant? Simplified?

Tom Petch

---

From: John C Klensin <[email-masked]>
Date: Fri, 06 Feb 2009 10:13:33 -0500

--On Friday, February 06, 2009 13:55 +0100 "Tom.Petch"
<[email-masked]> wrote:

>...
> I think too that there is a third issue, of a better name than
> RBNF.  John clearly showed that this I-D is not reduced.
> Historic? Deprecated? Limited_applicability? Variant?
> Simplified?

"simplified" has the same problem as "reduced", unless one
argues that one simplifies a metalanguage by adding more
operators.  "Variant" would work for me, and this actually is
much more of a variation on classic BNF (or ISO Extended BNF)
than ABNF is.

     john

---

From: Tony Hansen <[email-masked]>
Date: Fri, 06 Feb 2009 20:59:59 -0500

Going back to RFC 2205,

      These rules are specified using Backus-Naur Form (BNF) augmented
      with square brackets surrounding optional sub-sequences.

What do you think of BNFO, for "Backus-Naur Form with Options"?

or BNFB, for "Backus-Naur Form with Brackets"?

	Tony Hansen
	[email-masked]

---

From: John C Klensin <[email-masked]>
Date: Fri, 06 Feb 2009 23:26:10 -0500

--On Friday, February 06, 2009 20:59 -0500 Tony Hansen
<[email-masked]> wrote:

> Going back to RFC 2205,
> 
>       These rules are specified using Backus-Naur Form (BNF)
> augmented       with square brackets surrounding optional
> sub-sequences.
> 

Either one would work for me and would be much more descriptive
of what is going on than "Reduced".

    john

---

From: "Adrian Farrel" <[email-masked]>
Date: Sat, 7 Feb 2009 10:43:05 -0000

Since it isn't ABNF, perhaps BBNF?

----- Original Message ----- 
From: "John C Klensin" <[email-masked]>