Subject: Re: Last Call: <draft-daboo-webdav-sync-06.txt> (Collection Synchronization for WebDAV) to Proposed Standard

From: Cyrus Daboo <[email-masked]>
Date: Wed, 14 Dec 2011 11:06:21 -0500

Hi Julian,

--On December 14, 2011 12:10:27 AM +0100 Julian Reschke 
<[email-masked]> wrote:

> On 2011-12-02 00:41, The IESG wrote:
>> ...
>
> Abstract
>
>     This specification defines an extension to WebDAV that allows
>     efficient synchronization of the contents of a WebDAV collection.
>
> -> Expand acronyms on first use.

Really, doesn't everyone know what WebDAV is by now :-) In any case fixed 
in my working copy.

>
>     Typically, the first time a client connects to the server it will
>     need to be informed of the entire state of the collection (i.e., a
>     full list of all member URIs that are currently in the collection).
>     That is done by the client sending an empty token value to the
>     server.  This indicates to the server that a full listing is
>     required.
>
> I think it would be more consistent with other specs not to send an empty
> token, but to send *no* token.

Existing implementations expect an empty sync-token element. I don't really 
see much difference either way on this, so would prefer to leave it as-is.

>     As an alternative, the client might choose to do its first
>     synchronization using some other mechanism on the collection (e.g.
>     some other form of batch resource information retrieval such as
>     PROPFIND, SEARCH [RFC5323] , or specialized REPORTs such as those
>     defined in CalDAV [RFC4791] and CardDAV [I-D.ietf-vcarddav-carddav])
>
> The CardDAV reference will need to be updated...

Already fixed that one in my working copy.

>     To implement the behavior for this report a server needs to keep
>     track of changes to any member URIs and their mapped resources in a
>     collection (as defined in Section 3 of [RFC4918]).  This includes
>
> Actually, RFC 4918 defines "member URL"; maybe worth adjusting or
> pointing out it's the same.

Will make that change in my working copy. BTW "member URI" does appear in 
4918 in two places...

>     Marshalling:
>
>        The request URI MUST identify a collection.  The request body MUST
>        be a DAV:sync-collection XML element (see Section 6.1), which MUST
>        contain one DAV:sync-token XML element, and one DAV:prop XML
>        element, and MAY contain a DAV:limit XML element.
>
>        The request MUST include a Depth header with a value of "1" or
>        "infinity".
>
> This report definition is in conflict with the definition of the REPORT
> method in RFC 3253
> (<http://greenbytes.de/tech/webdav/rfc3253.html#rfc.section.3.6>).
>
> Essentially, a report is applied to just the request URI (depth: 0 or no
> depth header field), the request URI and it's internal members (1) or all
> descendants (infinity).
>
> A report definition should define the response for the case of Depth: 0.
> The format for the other cases follows directly from RFC 3253:
>
> "If a Depth request header is included, the response MUST be a 207
> Multi-Status. The request MUST be applied separately to the collection
> itself and to all members of the collection that satisfy the Depth value.
> The DAV:prop element of a DAV:response for a given resource MUST contain
> the requested report for that resource."
>
> (Note that I have complained about this a long time ago,
> <http://lists.w3.org/Archives/Public/w3c-dist-auth/2007JulSep/0041.html>).
>
> To fix this, the report would need to define Depth: 0 processing on a
> collection to mean the changes just on that collection (which means
> membership changes, but not changes to member resources), and the other
> modes would then follow based on RFC 3253.
>
> It doesn't seem to be possible to make this change in a way that doesn't
> break existing implementations, as RFC 3253 requires the report response
> format to be well-formed XML (thus only one root element), and that
> format then to be used *inside* DAV:multistatus/DAV:reponse/../DAV:prop.
>
> Optimally, this should be fixed. If this can't be fixed I'd argue that
> the spec should be published as Informational, and that it should include
> an explanation of the incompatibilty (essentially requiring servers to
> special-case this report in case they already use a generic WebDAV REPORT
> framework).

I am not convinced the use of Depth in sync report is violating the 
definition in 3253. In some ways it is a matter of how you look at the 
sync. Your viewpoint is that the report asks the collection for its changes 
- in that case, yes, Depth:0 would apply. The other view is that the report 
asks each of the child resources to list their change status, in which case 
Depth:1 and Depth:infinity also makes sense. Which viewpoint is taken 
probably depends on actual implementation rather than any perceived 
protocol restrictions.

Given that I feel the sync report is vital for WebDAV operations, for in 
particular CalDAV and CardDAV, I want to ensure this draft proceeds as a 
standard, not informational. If you feel strongly about this use of Depth, 
in spite of my comments above, then I would be willing to make some 
changes, provided we also include a statement on backwards compatibility 
(i.e., in an appendix state that clients/servers MAY use/accept the old 
Depth approach). If we do that we would need to proceed as follows: state 
that sync report can only be used with Depth:0, add a new request header to 
define the scope of the sync (e.g. Sync-Depth with values 1 and infinite). 
If we require Sync-Depth to be present, then we have a means for servers to 
detect old clients and handle Depth in a backwards compatible manner.

>        The response body for a successful request MUST be a DAV:
>        multistatus XML element, which MUST contain one DAV:sync-token
>
> Maybe s/one/a single/

Not sure why - "one DAV:xxx" is used in several places and I think is 
pretty clear.

>        ...
>
>     Preconditions:
>
>        (DAV:valid-sync-token): The DAV:sync-token element value MUST be a
>        valid token previously returned by the server.  A token can become
>        invalid as the result of being "out of date" (out of the range of
>        change history maintained by the server), or for other reasons
>        (e.g. collection deleted, then recreated, access control changes,
>        etc...).
>
> Does the sync-token need to originate from the same collection?

Yes of course. Perhaps changing the first sentence to:

The DAV:sync-token element value MUST be a valid token previously returned 
by the server for a synchronization report executed on the same 
request-URI.

>
> 3.3.  Depth behavior
>
>     Servers MUST support both Depth:1 and Depth:infinity behavior with
>     the DAV:sync-collection report.  Clients MUST include either a
>     Depth:1 or Depth:infinity request header with the DAV:sync-collection
>     report.
>
> See above; this contradicts the base definition of REPORT.

As stated above, I really don't think this is a contradiction of 3253.

>     In the case where a mapping between a member URI and the target
>     collection was removed, then a new mapping with the same URI created,
>     the member URI MUST be reported as changed and MUST NOT be reported
>     as removed.
>
> The client could tell that this happened if the DAV:resourceid property
> was included.
>
>     A member URI MUST be reported as changed if its mapped resource's
>     entity tag value (defined in Section 3.11 of [RFC2616]) has changed
>     since the request sync-token was generated.
>
> It should be mentioned that this only works well with resources that are
> not content-negotiated.

Unless the content negotiation was done as part of the original full sync 
request?

>     For example, consider a server that records changes using a
>     monotonically increasing integer to represent a "revision number" and
>     uses that quantity as the DAV:sync-token value (appropriately encoded
>     as a URI).  Assume the last DAV:sync-token used by the client was
>     "http://example.com/sync/10", and since then 15 additional changes
>     have occurred.  If the client executes a DAV:sync-collection request
>     with a DAV:sync-token of "http://example.com/sync/10", without a
>     limit the server would return 15 DAV:response elements and a DAV:
>
> Why 15? Is the assumption that any change to the server causes exactly
> one resource to change? What if there were 15 changes to the same
> resource?

OK, I will clarify the example to imply that the 15 changes are for 
separate resources. Proposed change:

From: and since then 15 additional changes have occurred
To: and since then 15 additional changes to different resources have 
occurred

>
>     <!ELEMENT multistatus (DAV:response*, DAV:responsedescription?,
>                            sync-token?) >
>
>     <!-- DAV:multistatus originally defined in RFC 4918, Section 14.16
>          but overridden here to add the DAV:sync-token element -->
>     <!-- DAV:response defined in RFC 4918, Section 14.24 -->
>     <!-- DAV:responsedescription defined in RFC 4918, Section 14.25 -->
>
> Why do we have DAV: prefixes on some of the DTD elements?
>
>

I haver removed the DAV: prefix in the actual <!ELEMENT ...> definitions, 
but kept them in the comments. OK?

-- 
Cyrus Daboo

---

From: Julian Reschke <[email-masked]>
Date: Wed, 14 Dec 2011 17:29:16 +0100

On 2011-12-14 17:06, Cyrus Daboo wrote:
> Really, doesn't everyone know what WebDAV is by now :-) In any case
> fixed in my working copy.

Just being "pro-active" :-). The less happens ion AUTH48, the better.

> Existing implementations expect an empty sync-token element. I don't
> really see much difference either way on this, so would prefer to leave
> it as-is.

Would be interesting to know how they treat missing sync tokens.

> Will make that change in my working copy. BTW "member URI" does appear
> in 4918 in two places...

Errata, errata!

> definition in 3253. In some ways it is a matter of how you look at the

It does.

RFC 3253 defines the response format for Depth 0, and then states how 
the format for Depth > 0 is derived from that; essentially by packaging 
into <multistatus>.

This means that reports that do not support Depth 0 can not be defined.

It also means that a report that uses <multistatus> for Depth 0, and 
which supports depths > 0, will have to packages <multistatus> inside 
<multistatus>.

> sync. Your viewpoint is that the report asks the collection for its
> changes - in that case, yes, Depth:0 would apply. The other view is that
> the report asks each of the child resources to list their change status,
> in which case Depth:1 and Depth:infinity also makes sense. Which
> viewpoint is taken probably depends on actual implementation rather than
> any perceived protocol restrictions.

What Depth 0 means is just a matter of definition. If you want to make 
Depth 0 mean "request-URI and all of it's direct members", that's fine. 
It just will give funny results when you use it with other Depths *and* 
follow the RFC 3253 definition in these cases.

> standard, not informational. If you feel strongly about this use of
> Depth, in spite of my comments above, then I would be willing to make
> some changes, provided we also include a statement on backwards
> compatibility (i.e., in an appendix state that clients/servers MAY
> use/accept the old Depth approach). If we do that we would need to
> proceed as follows: state that sync report can only be used with
> Depth:0, add a new request header to define the scope of the sync (e.g.
> Sync-Depth with values 1 and infinite). If we require Sync-Depth to be
> present, then we have a means for servers to detect old clients and
> handle Depth in a backwards compatible manner.

Not happy having to add a new header field just for that. A simpler 
approach might be to just assign a different report name, and then allow 
all depths.

> Not sure why - "one DAV:xxx" is used in several places and I think is

I was confused for a moment because I read it to be one token per 
response element.

> The DAV:sync-token element value MUST be a valid token previously
> returned by the server for a synchronization report executed on the same
> request-URI.

+1

(I was asking because in many cases the token will be the same 
everywhere, and clients might think this is always the case)

> Unless the content negotiation was done as part of the original full
> sync request?

How would that work?

For instance, the common case for Conneg is using Accept:, but Accept: 
on REPORT specifies the expected media type for the REPORT response.

Yes, that's a problem with WebDAV in general.

> To: and since then 15 additional changes to different resources have

+1

> I haver removed the DAV: prefix in the actual <!ELEMENT ...>
> definitions, but kept them in the comments. OK?

OK.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Wed, 14 Dec 2011 18:19:11 +0100

On 2011-12-14 18:01, Ken Murchison wrote:
> Cyrus Daboo wrote:
>> changes - in that case, yes, Depth:0 would apply. The other view is
>> that the report asks each of the child resources to list their change
>> status, in which case Depth:1 and Depth:infinity also makes sense.
>> Which viewpoint is taken probably depends on actual implementation
>> rather than any perceived protocol restrictions.
>
> My $.02:
>
> I look the sync report as a filtered query for properties (e.g.
> CALDAV:calendar-query), with the filter being "only give me a
> DAV:response if the resource has been changed/removed since the
> specified sync-token".
>
> A Depth of 1 or infinity gives us exactly what is specified in the draft.
>
> A Depth of 0 refers to the collection itself, and assuming it exists,
> the DAV:multistatus response may or may not include a single
> DAV:response element, depending on whether the server maintains an
> entity tag on the collection which changes with its membership. In
> either case, the sync-token is returned per the extended grammar in 6.3.
>
> So, a sync-collection report with a depth of 0 might simply return the
> following:
>
> <D:multistatus>
> <D:sync-token>http://example.com/ns/sync/1234</D:sync-token>
> </D:multistatus>
>
>
> Does this make sense, or is my logic completely convoluted?

If this was designed from scratch, it wouldn't be using multistatus, but 
would include the properties of the collection (if changed).

<D:sync-result>
   ... other collection props the report asked for ...
   <<D:sync-token>http://example.com/ns/sync/1234</D:sync-token>
</D:sync-result>

---

From: Cyrus Daboo <[email-masked]>
Date: Wed, 14 Dec 2011 12:27:59 -0500

Hi Julian,

--On December 14, 2011 5:29:16 PM +0100 Julian Reschke 
<[email-masked]> wrote:

> RFC 3253 defines the response format for Depth 0, and then states how the
> format for Depth > 0 is derived from that; essentially by packaging into
> <multistatus>.
>

Well I don't consider the text there very clear at all. In fact I think it 
is very much tailored to the 3253 use cases and not flexible enough for 
other general purpose use of REPORT. It states this:

      The DAV:prop element of a DAV:response
      for a given resource MUST contain the requested report for that
      resource.

Which implies "packaging" of multistatus within the DAV:prop element, which 
seems completely wrong to me.

> follow the RFC 3253 definition in these cases.

So would it hurt to allow this spec to override the 3253 "nested 
multistatus" requirement in the interests of generating a compact response 
specific to this type of report?

> approach might be to just assign a different report name, and then allow

I think if you insist on requiring Depth => nested multistatus, then we 
either add a new header or perhaps a <depth> element inside the request 
body (and that would also be reasonable in terms of being able to support 
backwards compatibility). At this point I would prefer the later as being 
least disruptive to existing implementations.

> For instance, the common case for Conneg is using Accept:, but Accept: on
> REPORT specifies the expected media type for the REPORT response.
>

OK, so we should probably punt on per-resource content-negotiation within 
the report itself (though I could see us wanting to support that in the 
future, in which case it would probably be done through extensions elements 
in the request body).

So what do we need to do to address this? State that the etags returned in 
the report are always for the non-content-negotiated representation of each 
child resource? I think that is already implied by the definition of 
DAV:getetag, so perhaps we should state that we are referring to that 
value, which is the non-content negotiated entity tag.

-- 
Cyrus Daboo

---

From: Cyrus Daboo <[email-masked]>
Date: Wed, 14 Dec 2011 12:34:59 -0500

Hi Ken,

--On December 14, 2011 12:29:18 PM -0500 Ken Murchison 
<[email-masked]> wrote:

> I don't necessarily see a problem with sync report using multistatus, but
> perhaps the sync-token should be returned in a prop response element for
> the collection rather than as an added element of multistatus.

That won't work when the "limiting/truncation" option is used as the 
multistatus response for the collection indicates an overall status error, 
rather than using propstat.

-- 
Cyrus Daboo

---

From: Julian Reschke <[email-masked]>
Date: Wed, 14 Dec 2011 18:43:06 +0100

On 2011-12-14 18:27, Cyrus Daboo wrote:
> Well I don't consider the text there very clear at all. In fact I think
> it is very much tailored to the 3253 use cases and not flexible enough
> for other general purpose use of REPORT. It states this:
>

It states what it states :-)

> Which implies "packaging" of multistatus within the DAV:prop element,
> which seems completely wrong to me.

No, you only need to package <multistatus> inside <prop> in case the 
response to the Depth: 0 request uses <multistatus>.

> multistatus" requirement in the interests of generating a compact
> response specific to this type of report?

I think so, because it defeats WebDAV stacks from executing reports 
consistently. (And, I assure you, I have written a framework in the past 
that did exactly that, otherwise I probably wouldn't have noticed the 
problem).

> body (and that would also be reasonable in terms of being able to
> support backwards compatibility). At this point I would prefer the later
> as being least disruptive to existing implementations.

Yes, a new child element of the request body (<depth>) works for me.

> OK, so we should probably punt on per-resource content-negotiation
> within the report itself (though I could see us wanting to support that
> in the future, in which case it would probably be done through
> extensions elements in the request body).
>
> So what do we need to do to address this? State that the etags returned
> in the report are always for the non-content-negotiated representation
> of each child resource? I think that is already implied by the
> definition of DAV:getetag, so perhaps we should state that we are
> referring to that value, which is the non-content negotiated entity tag.

Yes, REPORT and PROPFIND should be consistent. Let's solve this puzzle 
another day.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 15 Dec 2011 00:27:59 +0100

On 2011-12-14 18:34, Cyrus Daboo wrote:
> multistatus response for the collection indicates an overall status
> error, rather than using propstat.

OK, so it would need to be part of the sync-result (and the other props 
be included with a DAV:prop container).

Best regards, Julian

---

From: Ken Murchison <[email-masked]>
Date: Wed, 14 Dec 2011 12:01:37 -0500

My $.02:

<D:multistatus>

-- 
Kenneth Murchison
Principal Systems Software Engineer
Carnegie Mellon University

---

From: Ken Murchison <[email-masked]>
Date: Wed, 14 Dec 2011 12:29:18 -0500

>   <<D:sync-token>http://example.com/ns/sync/1234</D:sync-token>

I don't necessarily see a problem with sync report using multistatus, 
but perhaps the sync-token should be returned in a prop response element 
for the collection rather than as an added element of multistatus.

---

From: Arnaud Quillaud <[email-masked]>
Date: Thu, 15 Dec 2011 09:47:23 +0100

On 14/12/2011 18:27, Cyrus Daboo wrote:
> within the report itself (though I could see us wanting to support 
> that in the future, in which case it would probably be done through 
> So what do we need to do to address this? State that the etags 
> returned in the report are always for the non-content-negotiated 
> representation of each child resource? I think that is already implied 
> by the definition of DAV:getetag, so perhaps we should state that we 
> are referring to that value, which is the non-content negotiated 
> entity tag. 
In the text above, we are only talking about a change of value, not 
about a specific value.
Are there really a lot of cases where the non-content negotiated etag 
would not change while a content negotiated one would ? Unless that is a 
common scenario, I would not clobber the text with this additional 
statement.
The opposite case (non content-negotiated changes while content 
negotiated does not) is even less problematic as it would just result in 
a false positive for clients using content negotiation.

Arnaud