Subject: Re: /.well-known (RFC5785) vs versioned APIs [and EST/RFC7030]

From: Nico Williams <[email-masked]>
Date: Wed, 1 May 2019 12:58:39 -0500

On Sun, Feb 24, 2019 at 03:12:19PM -0500, Martin Thomson wrote:
> On Mon, Feb 25, 2019, at 04:55, Michael Richardson wrote:
> > I'm not sure where this query goes, so [email-masked].

I thought [email-masked] was supposed to be most appropriate for these
sorts of questions, being the successor to apps-discuss, but it seems to
be very low traffic nowadays (compared to apps-discuss!), and a recent
post of mine there got no replies.  [email-masked] seems appropriate
given that.

> My view on this, which- in the interest of transparency - is not
> uniformly held, is that versioning in URLs is generally not advisable
> for the same reason x- isn't. See rfc 6648.
> 
> I see /api/v1 all the time, but don't see the point. What that does is
> bake semantics into the path, which is akin to an rfc 7320 violation.
> 
> The goal is to ensure that incompatible versions are properly
> distinguished. For that purpose, all you need to do is ensure that the
> URL for v1 is not the same as the URL for v2. To that end, /api and
> /apiv2 work amply.  Or you could name the protocol.

While I agree that I'd rather not see a version number in URIs if I
don't have to, I don't think there's much of a moral difference
between /api/vN/... and {/api/..., /apivN/...}.  Both end up with
version numbers in the URI local-part components, with the second
case avoiding them only for the first version -- hardly a huge win.

If you really want to avoid version numbers in URI local-part components
then you should have different URI local-parts for different versions.

Or register and use a new header to communicate version numbers.  Maybe
we should have standard, generic API version negotiation/indication
headers?

> For est, I tend to think that /.well-known/ is overused in general,
> because configuring a URL is usually adequate.  But I can't hold a

If you need to co-locate a discovery API with an existing web service,
then this works.  Of course, we should be using SRV or URI RRs for
discovery, but just using /.well-known/ at the domain apex is very
tempting, which is why it happens.  If we don't want this, then we
should have an RFC like 6648 but for discovery protocols.

> strong position on a protocol that I barely know. [...]

Well, me too.  I'm just guessing here that discovery was the driver
for using /.well-known/est.

Nico
--

---

From: Nico Williams <[email-masked]>
Date: Wed, 1 May 2019 13:08:42 -0500

On Mon, Feb 25, 2019 at 10:51:55AM -0500, Michael Richardson wrote:
> Mark Nottingham <[email-masked]> wrote:
>     > You could do /.well-known/foo/v1, but of course you could also just
>     > call it something else, e.g., /.well-known/bar. The point being that if
>     > there's backwards-incompatible change in the semantics, it really is
>     > another thing.
> 
> Registering /.well-known/bar requires an IANA action, and pollutes that
> namespace, so it seems like a bad idea.

You could register /.well-known/foo and then have sub-endpoints there
with different names for different versions: /.well-known/foo/{bar, baz}.

>     > And of course ../v1.0 doesn't make any kind of sense, because minor
>     > versions are supposed to signify backwards-compatible changes, but by
>     > changing the URL, they become incompatible. More about this at
>     > <https://www.mnot.net/blog/2012/12/04/api-evolution>.
> 
> I agree that semantic versioning in the URL seems dumb, because only the
> first digit should matter as far being compatible.
> 
> I suppose using /api/1.0.1 and getting a 404 is a pretty clear statement that a
> minor change is not supported, but I don't really get it.  So we agree on /v1
> vs /v2 makes more sense than 1.0.0 and 2.0.0

There can always be some other way to negotiate versions, like:

  GET /.well-known/foo/versions

Or encode the versions into MIME types and use Accept/Content-Type for
version negotiation/indication.

Alternatively we could have a standard header for generic API version
negotiation/indication.  Though Accept/Content-Type works for me.

For some applications this might make more sense because the URIs use
Accept/Content-Type to support both browser interfaces and API
interfaces, then URI portability between the two becomes desirable, and
so now you have URI local-part stability requirements that prevent you
from making backwards-incompatible changes to the API without some
header-based version negotiation.

>     > One of the things on my list is to write something explaining how to
>     > use OpenAPI for standards; it's designed more for single-deployment
>     > APIs (see
>     > <https://www.mnot.net/blog/2012/06/25/http_api_complexity_model>), not
>     > ones that have multiple, uncoordinated deployments, so there are some
>     > foot guns there.
> 
> I would welcome a discussion about OpenAPI in IETF specifications.
> Mechanically, how to do it (throw the YAML in as a an appendix? Inline?
> Split up the YAML?  Which text is normative...), and pieces of OpenAPI to
> avoid.
> 
> My EST extension protocol is a bit complex from an OpenAPI point of
> view. First Alice talks to Bob, and get something, and gives it to Frank, and
> Frank then speaks to Joe, and returns something to Alice, which returns to
> Bob with it. OAUTH2 has a similar level of complexity.  I don't see an
> obvious way in OpenAPI to express the coupling between API calls.

+1

Nico
--

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 1 May 2019 20:10:47 +0200

> case avoiding them only for the first version — hardly a huge win.

There is no technical difference, but there is a big difference in signaling. 
/api/v1 is just an affordance that is inviting you to exercise it.
As in “This protocol is designed to have many incompatible versions”.

There is a strong tendency of trying to transfer the experience from software versions to protocol versions.  The former are useful (and there is a de-facto standard for how to handle the numbering), the latter rarely are — a “protocol version” is mainly there to prevent communication (and even in the cases where the original design was actually considering evolution, the deployment experience often still is exactly that).

> we should have standard, generic API version negotiation/indication

I hope not.

Kick the habit.  Stop thinking in versions.  There are better ways to handle evolution.

Grüße, Carsten

---

From: Nico Williams <[email-masked]>
Date: Wed, 1 May 2019 13:23:53 -0500

On Mon, Feb 25, 2019 at 11:04:31AM -0500, Michael Richardson wrote:
> 
> Phillip Hallam-Baker <[email-masked]> wrote:
>     > I think that the IETF already has the answer to the problem you are raising.
>     > But it is spread out so that while the existing standards essentially
>     > constrain the space, there is no RFC that puts them together and so I wrote
>     > one myself:
> 
>     > https://tools.ietf.org/html/draft-hallambaker-web-service-discovery-00
> 
>     > This has expired but will be replaced in a few hours time. I am in fact
>     > updating the whole document set right now.
> 
> okay, I see it now, and I'll read it.
> 
>     > So my response to Michael is that you don't want your version numbers in the .
>     > well-known path, you want them in the DNS records because then they can be
>     > used to select the host to connect to according to capabilities.
> 
> Interesting. I think that I agree.

If you're not careful though you might end up having to do a large
number of DNS lookups for discovery.

That seems OK when you have only one version, and you don't notice.
Then you start having various alternatives (e.g., different protocol
choices) encoded in DNS.  Then you add a version and suddenly realize
there's a cartesian product in the discovery protocol.

Kerberos has this problem.  It sucks.  It limits protocol evolution.

I would prefer a discovery mechanism where you use SRV, TXT, or URI RRs
to discover a service set where you can use HTTP for the remainder of
the discovery process.

So, lookup _fooapp._discovery.$DOMAIN, use this to find N servers and a
URI local-part fragment (or else use a /.well-known URI local-part?),
then go GET /fooapp/discovery at any one of them, which should be a
static JSON/XML/whatever description of everything else you need.

>     > Lets say we have 10 hosts and we are upgrading them to a new version of the
>     > spec over a two year period (licenses sometimes cost money, people always
>     > do). So we might upgrade 2 hosts to start with and advertise the fact with
>     > host specific TXT records.

See above.  I'd rather do a single DNS lookup[*], GET a single static
resource from one of the services found via DNS, and find the "these
{8 hosts} use version 1, while {these 2 hosts} use versions {1, 2}"
information there.

[*] Well, it's more than one lookup, but one for the discovery RRs, and
    one to resolve each service authority domainname.  But what I really
    don't want is to have to do these lookups:

      _fooapp._http1.$DOMAIN
      _fooapp._http2.$DOMAIN
      _fooapp._http3.$DOMAIN

    and then realize we need a version negotiation and end up with NxM
    lookups for

      _$versionappname._$protocol.$DOMAIN

    Yeah, you can do these lookups concurrently, but you still need to
    wait for the most-preferred RRset back.  It's a lot of work compared
    to any scheme with just one DNS discovery query.

Nico
--

---

From: Nico Williams <[email-masked]>
Date: Wed, 1 May 2019 13:29:20 -0500

> As in “This protocol is designed to have many incompatible versions”.

That's very subjective.  I still see no difference.  I'm not defending
versions in URI local-parts -- I dislike that as much as anyone on this
thread.

> There is a strong tendency of trying to transfer the experience from
> software versions to protocol versions.  The former are useful (and
> there is a de-facto standard for how to handle the numbering), the
> latter rarely are — a “protocol version” is mainly there to prevent
> communication (and even in the cases where the original design was
> actually considering evolution, the deployment experience often still
> is exactly that).

Protocol and implementation versioning are generally decoupled, but
protocols do need to evolve -- sometimes radically, but yes, backwards-
compatible evolution is always preferable.

> > Or register and use a new header to communicate version numbers.
> > Maybe we should have standard, generic API version
> > negotiation/indication headers?
> 
> I hope not.
> 

Tell Michael :)  If I had to version an API, I'd consider using MIME
types and Accept/Content-Type, as I mentioned.

---

From: Nico Williams <[email-masked]>
Date: Wed, 1 May 2019 13:55:24 -0500

On Sun, Feb 24, 2019 at 11:29:10PM -0500, Phillip Hallam-Baker wrote:
> So lets take a DNS configuration for my Mathematical Mesh service:
> 
>    _mmm._tcp.example.com SRV host1.example.com 0 10 80 host1.example.com
>    _mmm._tcp.example.com SRV host2.example.com 0 40 80 host2.example.com
>    _mmm._tcp.example.com TXT "version=1.0-2.0"

Also, does _tcp imply HTTP/1.1 in your example?

And if you want HTTP/3, will you add:

|    _mmm._udp.example.com SRV host1.example.com 0 10 80 host1.example.com
|    _mmm._udp.example.com SRV host2.example.com 0 40 80 host2.example.com
|    _mmm._udp.example.com TXT "version=1.0-2.0"

?

Now you're doing quite a few DNS lookups.  If you need to convey that
host1 supports only version 1 and host2 only version 2, you might find
yourself with a transport x version cartesian product of DNS lookups to
do.  This is bad.

I'd much rather do something like this:

     _mmm._discovery.$DOMAIN SRV host1.example.com ...
     _mmm._discovery.$DOMAIN SRV host2.example.com ...
     ; path is /mmm on all of the hosts in the SRV RRset:
     _mmm._discovery.$DOMAIN TXT "path=/mmm"
     host1.$DOMAIN     A 10.0.1.1
     host2.$DOMAIN     A 10.0.1.2

then GET /mmm/discovery at either host1 or host2, which should be a
static resource with all the configuration information needed for the
mmm application.

Discovery of HTTP/2 support could be done by trying 1.1 and upgrading.
Discovery HTTP/3... idk.  Maybe always use HTTP/1 or /2 for discovery
and if you get told that /3 is also supported, then switch to /3 as
desired.

No chance of cartesian products there.  If you use a single A RRset for
all the servers, then you get a total of three DNS lookups for three
RRsets:

     _mmm._discovery.$DOMAIN SRV mmm.example.com ...
     _mmm._discovery.$DOMAIN TXT "path=/mmm"
     mmm.$DOMAIN     A 10.0.1.1
     mmm.$DOMAIN     A 10.0.1.2

and the one HTTP GET of /mmm/discovery.

Nice, simple, with constant discovery cost.  Well, /mmm/discovery might
get really large, so it's O(N), but still, the number of queries is
constant.

Nico
--

---

From: Kent Watsen <[email-masked]>
Date: Wed, 1 May 2019 20:58:52 +0000

Have you considered RFC 6415 with its /.well-known/host-meta for versioning?   This is what RFC 8040 (RESTCONF) uses for versioning.

Kent

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Thu, 2 May 2019 10:37:30 -0400

> |    _mmm._udp.example.com TXT "version=1.0-2.0"

Web services barely make any use of HTTP beyond the end point specifier. So
I don't really see a reason to use HTTP/3. But if you were going to use it,
that is how I would specify it.

A better outcome in my view would be to develop one (or possibly more)
presentation layer designed expressly for Web Services. So TXT
"version=1.0-2.0 tran=https,http3,nwsp"

https = http/1.1 + TLS
http3 = http/3 over TLS
nwsp = some new protocol possibly QUIC based or something else based.

People keep pointing out that layering everything over http is a bad idea.
Well, I was one of the original authors of HTTP and I agree. HTTP/1.0 is
arguably an OK but not great Web Services transport. None of the changes to
HTTP since have been designed to support Web Services.

> yourself with a transport x version cartesian product of DNS lookups to

I am doing fewer lookups than I would if I was also doing TLSA though. One
of my recommendations to the deployment workshop is to recognize that when
a WG output fails to thrive, this should be noted and the way cleared for
proposals that are designed for deployment. The DANE WG insulted the core
constituencies required for deployment success. The BULLRUN team couldn't
have sabotaged the effort better if they tried.

The way I see it, we need to adopt a two stage strategy. The first step
being to make is possible to express all the information we need to express
in the form of widely supported DNS records. TLSA is not and will never be
widely supported by Internet hosting providers as long as they sell DNS
registrations at cost hoping to make their profits off selling WebPKI
certs. Structuring DANE the way it was has not only killed deployment of
TLSA records, it has made it very very difficult to deploy any new records.

Stage two is to provide improved ways of delivering the DNS information.
The DNS data model is what it is and changing it would take decades at the
least. But we can and are replacing the client-resolver protocol and can
probably change the resolver-authoritative protocol.

What I proposed with Omnibroker is that we should replace all the
client-resolver lookups with a single request/response 'what do I need to
know to talk service X to service Y' and back would come a set of weighted
IP addresses and OCSP tokens.

So the resolver which is trusted but unaccountable today would expand its
function and remain trusted but be auditable and accountable.

> static resource with all the configuration information needed for the

I think that modulo the use of _discovery instead of _tcp, that approach is
already specified in the doc but not necessarily called out.

The key point is that we need to pick either one way to do this or a very
small number of ways to do this so that DNS resolvers can choose their
additional records wisely.

Incidentally, one change to DNS that could give immense leverage would be
an extension allowing multiple UDP responses to a single packet. I did this
in Omnibroker because I can always fit requests into a single packet (since
a DNS name is limited length) but I frequently need multiple responses.

So imagine if we had an extension to DNS so that a client can say 'I can
handle multiple responses'. And this would allow the service to send the
kitchen sink in return to a response. The first time a client makes a
request of the resolver, it returns an RR that says 'I support
multi-query'. The next time the client makes a request it goes like this

Client request: _mmm._discovery.example.com RR=DISCO
Resolver response:

Packet 1: 20+1 additional records
Packet 2: 23+1 additional records
Packet 3: 8+1 additional records

The basic idea being that instead of truncating responses that are too
long, we simply spread them across three packets that are all smaller than
the MTU so as to avoid fragmentation. DISCO is a portmanteau record that
means 'return me a complete host connection chain for this service'

The +1 record contains the information telling the client that there are
three packets in total to look for.

This approach will be blocked by some firewalls but this should not be an
issue since your DNS resolver should either be inside the firewall or be
tunneling. But every response still looks like a regular DNS response and
should not be blocked by most ISP configurations.

If I was going to spec this out, I would make sure the +1 record at least
has a slot for a TSIG

I have not done any measurements but I am sure someone will tell me if I am
wrong. It is my very strong suspicion that if a server returns a sequence
of 1-16 UDP packets that >95 % of the time either all of the packets will
get through or none will.

UDP was a lot flakier when DNS was designed and there was a real chance of
packet loss and so partial retransmit would have made sense. I strongly
suspect that for the modern internet, we can simply resend the entire
section.

It goes without saying of course that definition of DISCO should work with
DOH because it is a discovery chain that is going to be needed at the other
end.

---

From: Nico Williams <[email-masked]>
Date: Thu, 2 May 2019 14:32:52 -0500

> "version=1.0-2.0 tran=https,http3,nwsp"

Another option is to not bother with a TXT RR at all, just register a
subnamespace of /.well-known for discovery of arbitrary apps, then just
getaddrinfo() (DNS A RRset lookup), and then

    GET https://${domainname}/.well-known/app-discovery/${app_urn}

using HTTP/1.1, where ${app_urn} is a URN identifying the application,
which URN would be appropriately encoded, naturally.

So, one DNS lookup and one GET -> all the discovery you need.

> HTTP since have been designed to support Web Services.

It's perfectly fine for this very particular purpose.

[The rest of this is all about DNS rather than about discovery.]

> [...]

Adding DNS RR types has a last mile problem, and it's at the UIs.  So I
tend to agree with you that from here on out it's TXT all the way.

But... must we have that discussion here?  Is anyone objecting to the
use of TXT RRs for discovery?  At any rate, we don't even need those
(see above).

> probably change the resolver-authoritative protocol.

Just as HTTP/2 and /3 do not change HTTP semantics -- some things are
forever, and the DNS data model is one such thing.  It's not all that
wrong either.  DNS has warts, but it's not fundamentally wrong, not
given the reality of IP addressing and routing.  (If we could start from
scratch not just with DNS, but with TCP/IP, we might well build
something rather different, but we can't, so I'm not too inclined to
spend much time dwelling on it.)

> IP addresses and OCSP tokens.

That would be nice.  Note that that isn't fundamentally at odds with the
DNS data model, though it would help to have additional metadata
currently outside the scope of DNS to bind all the relevant RRsets
together.

> > [...]

Yes, exactly.  We can even do better than that (see above).

> a DNS name is limited length) but I frequently need multiple responses.

UDP is the main source of DDoS trouble in DNS... and also a large source
of DNS' success and endurance.

I'd rather get away from UDP, not only because of DDoS issues, but also
because of confidentiality protection reasons.  (Or: I wish we could
have done both, DJBDNS and DNSSEC.)

> [...]

We do have a DNS extension framework...

Nico
--

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Thu, 2 May 2019 16:20:01 -0400

Restarting the thread clean

1) Yes, I agree that the DNS model is OK for our purposes. Recognizing that
there is little we could do to change it means that we should just accept
the model as it is

1a) That does not mean we have to accept the DNS protocol. that is much
more malleable (see DoH)

2) We should decide on a division between service discovery and service
description and move everything that is not essential to discovery to
description.

2a) I still want to put key digests and even keys in the DNS however
because that is the only way that we can encrypt the host key exchange.

2b) Latency is really critical in applications but not necessarily for
discovery. The cost of a round trip is not very important if I can cache
the resulting key exchange on the client side for a year or more.

3) UDP is not a panacea for DDoS but it does allow the service to perform
its own DDoS mitigation without consuming system resources (TCP/IP does
not).

3a) Assume that there is regulation at some point that requires residential
gateways to perform egress filtering requiring the source address of all
packets sourced to be valid. I had discussions earlier this week with
senior people in various governments where I made that recommendation. I
think we can get it at some point.

3b) Before a service does any 'expensive' action, assume we make use of one
round trip to perform a challenge/response

3c) We can use a client-Host key exchange to encrypt the client-Service key
exchange protecting both the domain and service.

So the way I see this sort of thing working is this:

First, forget the conversations between the resolver and the authoritative.
Assume that these are performed by a caching resolution service that
aggressively pre-fetches records just before they expire from the cache. So
99% of the time or more, 1.1.1.1, 8.8.8.8 or whatever will have the
information any client asks for in cache. So the resolver/authoritative
latency is irrelevant. and indeed we can expect that at some point, there
will be a DNS protocol optimization to make this more efficient by pushing
updates out to subscribers in some sparse updates-only fashion.

Second, assume that hosts and services are different things with different
keys. The host keys are supplied by DNS discovery, the service keys are
provided by the host.

Third, assume that the endpoint here is a fully UDP based Web Service
protocol. The lower layer may be QUIC or something much simpler that only
supports short requests and responses.

_mmm._tcp.example.com SRV 1 1 6666 host1.example.com
_mmm._tcp.example.com TXT "v=mmm tls=KCM5-7VB6-IJXJ-WKHX-NZQF-OKGZ-EWVN"
host1.example.com A 10.0.0.1
_key.host1.example.com  TXT "v=key
hek="PAQH-PKX2-BAIJ-I7EC-XGAL-7GWT-DDFP-MB5S-R4AJ-3FBT-7NHO-T26Z-2E6Y-WFH4"

Client query is
_mmm._tcp.example.com DISCO

Back is returned the following responses:
Packet 1: SRV, Authority chain, DNSSEC signatures
Packet 2: A, TXT records

The client then performs a HEK request of the Host on port 6666 using the
Curve25519 key PAQH-PKX2-...

If the service is under load, it may well require the request to be
resubmitted with a DDoS cookie. But in ordinary circumstances, this is
unnecessary. After one round trip, the client has a WF128 master secret
that can be used to encrypt a Hello inquiry on the service to get the
implementation details and to perform a Curve448 key exchange for
production use.

For the Mesh Services, I expect that the discovery services would in any
case be simply handing the user off to a different host after the initial
connection as there is really no reason to have all the accounts on the
same host or to funnel all the requests through a first tier simply to
redirect to a second. Yes there are good security reasons for two tiered
models but that does not mean that every first tier service needs to be
able to support any user at all.