Subject: Re: Last Call: Adding a fragment identifier to the text/csv media type (see <draft-hausenblas-csv-fragment-06.txt>)

From: Simon Perreault <[email-masked]>
Date: Fri, 11 Oct 2013 11:40:42 +0200

This draft's premise is interesting, but the implementation leaves to be
desired. That is, I like the idea of fragment identifiers for CSV, but
row/column/cell-based selection doesn't address my need.

My need is based on the CSV files generated from IANA registries. Here's
one:

https://www.iana.org/assignments/ipv6-unicast-address-assignments/ipv6-unicast-address-assignments.csv

Here's an excerpt:

2001:0200::/23,APNIC,1999-07-01,whois.apnic.net,ALLOCATED,
2001:0400::/23,ARIN,1999-07-01,whois.arin.net,ALLOCATED,
2001:0600::/23,RIPE NCC,1999-07-01,whois.ripe.net,ALLOCATED,

In general, for IANA registries, what would be useful would be fragments
pointing to a particular record or a set of records. It seems like
row-based selection would accomplish that purpose, right? Well, yes, but
not in a useful way, because the row *index* is used in the fragment.
The problem is that when IANA inserts new records, all records that come
after the new one are moved one row down, and the URL containing the
fragment is no longer valid and needs to be updated. That's not cool.
What we need is a fragment that points to the record without needing to
be updated when new records are added. That is, I want to be able to say
something like this:

.../ipv6-unicast-address-assignments.csv#2001:0200::/23

I think this is an important use case, and I would like the draft to be
updated to address it.

Thanks,
Simon

---

From: Barry Leiba <[email-masked]>
Date: Fri, 11 Oct 2013 11:52:23 -0400

> row/column/cell-based selection doesn't address my need.

This is an Independent stream document, and the IETF doesn't have
change control of the document.

The authors can certainly accept your comments at their discretion.
But this last call isn't for comments on the *document*.  It's only to
assess whether there are objections to the IESG's allowing the change
to the media types registry.

Barry, Applications AD

---

From: Simon Perreault <[email-masked]>
Date: Fri, 11 Oct 2013 18:00:40 +0200

> to the media types registry.

Ah, right, I didn't notice that in my enthusiasm.

No objection then. But I hope the authors are listening! :)

Thanks,
Simon

---

From: t.p. <[email-masked]>
Date: Sun, 13 Oct 2013 11:42:57 +0100

I find the security considerations in this registration rather weak.
What might have sufficed in 2005 seems to me inadequate for 2013.  I
would expect a clearer statement of what are or are not considered
threats or attacks and what mitigations there then are for them.

Tom Petch

----- Original Message -----
From: "The IESG" <[email-masked]>
To: "IETF Announcement List" <[email-masked]>
Cc: <[email-masked]>
Sent: Thursday, October 10, 2013 7:50 PM

The IESG has received a request to update the IANA registration of
the text/csv media type, adding an optional fragment identifier.
The request comes from a document in the Independent stream, and the
IESG is the change controller for the text/csv media type.

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action. Please send substantive comments to the
[email-masked] mailing lists by 2013-10-24. Exceptionally, comments may
be sent to [email-masked] instead. In either case, please retain the
beginning of the Subject line to allow automated sorting.

The document making the request can be obtained via
http://datatracker.ietf.org/doc/draft-hausenblas-csv-fragment/

---

From: Alexey Melnikov <[email-masked]>
Date: Mon, 14 Oct 2013 11:34:14 +0100

> http://datatracker.ietf.org/doc/draft-hausenblas-csv-fragment/
This looks like a good addition, so I am personally in favour of this.

---

From: Barry Leiba <[email-masked]>
Date: Mon, 14 Oct 2013 11:31:22 -0400

> threats or attacks and what mitigations there then are for them.

Tom, do you have specific suggestions for the authors in this regard.

Barry

---

From: t.p. <[email-masked]>
Date: Mon, 14 Oct 2013 17:21:36 +0100

----- Original Message -----
From: "Barry Leiba" <[email-masked]>
To: "t.p." <[email-masked]>
Cc: "IETF discussion list" <[email-masked]>
Sent: Monday, October 14, 2013 4:31 PM

> Tom, do you have specific suggestions for the authors in this regard.

Looking at an unrelated media/type RFC, it starts with
The main security considerations for the ....
   payload format defined within this memo are confidentiality,
   integrity, and source authenticity.
which is the sort of beginning I expect a Security Considerations to
have in 2013, then going on to say which are relevant here and
how they might be mitigated - CMS, IPsec, TLS; or not as the case
may be.

I see this type as one for database data and so requiring more
careful consideration than, eg, text/plain, as used for messages
like this one.

Tom Petch

---

From: Tim Bray <[email-masked]>
Date: Mon, 14 Oct 2013 10:28:30 -0700

I think this is sensible, could be useful, won’t break anything, and
probably hits a huge 80/20 points for the ways people might want to address
inside spreadsheets.  Go ahead and publish it.

---

From: Pete Resnick <[email-masked]>
Date: Mon, 14 Oct 2013 14:45:32 -0500

> threats or attacks and what mitigations there then are for them.

I don't know that everyone is really understanding the request that is 
being made here. It is a bit unusual.

RFC 4180 <http://tools.ietf.org/html/rfc4180> contains the current 
registration for text/csv. That registration has the "Change Controller" 
as "IESG", which is to say it's a registration from an IETF document. 
Barring any change, that registration would remain exactly as it is 
(including its current Security Considerations).

Someone outside of the IETF is publishing a document describing how to 
use fragment identifiers with text/csv. That document is being published 
in the Independent Stream by the RFC Editor. Since the publication of 
RFC 4180, "fragment identifier" was added to the media type registration 
procedures. <http://tools.ietf.org/html/rfc6838#section-4.11> The 
present document (draft-hausenblas) wants to update the existing IETF 
registration to include it's idea of fragment identifiers (which was 
absent from the RFC 4180 registration), though it will leave the IETF 
(via the IESG) as "Change Controller".

This Last Call is to find out if the IETF is OK with a non-IETF document 
updating an IETF registration. If the answer is "no", then we leave the 
4180 registration in place, or we tell the ISE that draft-hausenblas is 
not conforming to IETF processes and that we want it to be an 
IETF-stream document. If the answer is "yes", we go ahead with the 
registration change based on whatever the ISE publishes. We can send 
comments to the author and to the ISE asking for changes, but it's not 
an IETF document; IETF consensus is not required and the ISE can publish 
it anyway.

So, your Last Call comments are *simply* on the registration update. The 
document is not ours on which to comment.

pr

> From: "The IESG"<[email-masked]>
> To: "IETF Announcement List"<[email-masked]>
> Cc:<[email-masked]>

-- 
Pete Resnick<http://www.qualcomm.com/~presnick/>
Qualcomm Technologies, Inc. - +1 (858)651-4478

---

From: John C Klensin <[email-masked]>
Date: Tue, 15 Oct 2013 04:54:47 -0400

--On Monday, October 14, 2013 14:45 -0500 Pete Resnick
<[email-masked]> wrote:

> I don't know that everyone is really understanding the request
> that is being made here. It is a bit unusual.
> 
> RFC 4180 <http://tools.ietf.org/html/rfc4180> contains the
> current registration for text/csv. That registration has the
> "Change Controller" as "IESG", which is to say it's a
> registration from an IETF document. Barring any change, that
> registration would remain exactly as it is (including its
> current Security Considerations).
>...
> This Last Call is to find out if the IETF is OK with a
> non-IETF document updating an IETF registration. If the answer
> is "no", then we leave the 4180 registration in place, or we
> tell the ISE that draft-hausenblas is not conforming to IETF
> processes and that we want it to be an IETF-stream document.
> If the answer is "yes", we go ahead with the registration
> change based on whatever the ISE publishes. We can send
> comments to the author and to the ISE asking for changes, but
> it's not an IETF document; IETF consensus is not required and
> the ISE can publish it anyway.
>...

In a slightly broader interpretation of that question, I believe
that having Independent (or even IAB) stream documents modify
registrations that are required, either by the registration
procedure or the registration itself, to be under IETF change
control is a bad idea.  If we don't like that constrain, we
should modify the registration, not conduct odd Last Calls.  I
would strongly support processing this document in the IETF
Stream as an individual submission (and, while that slips over
into the substantive part, approving it for publication and
modification of the registration on that basis).  

My reasoning is that, while the change seems fine, the precedent
seems atrocious.  If this is approved via Independent Stream
publication and the next case that comes along is, unlike this
one, generally hated by the community, the amount of
hair-splitting required to deny that one having approved this
one would be impressive.. and bad for the IETF.

Of course, the ISE could go ahead and publish this document and
the IESG could block the registration, leading to a confusing
situation.  I hope we don't have to go there.

       john

p.s. W3C is circulating a draft charter for a WG that might
affect CSV on the web.  Because having a hard-to-change
Informational RFC and IANA registration that was inconsistent
with W3C recommendations would be a generally bad idea, some
coordination may be in order, especially to verify that the
current draft is not an end run around W3C efforts.

---

From: t.p. <[email-masked]>
Date: Tue, 15 Oct 2013 10:34:47 +0100

----- Original Message -----
From: "Pete Resnick" <[email-masked]>
To: "t.p." <[email-masked]>
Cc: <[email-masked]>
Sent: Monday, October 14, 2013 8:45 PM
> document is not ours on which to comment.

Indeed, I fully understand that.  It is the new registration for this
media type that I am commenting on, nothing else, that the new
registration, if approved, will cause the IANA entry to point to the
text from this 2013 (more likely 2014) RFC for which I would expect the
Security Considerations, an integral part of this registration
regardless of what else is in this I-D, to be up to the standards that
the IETF impose on work carried out in 2013, which are higher than those
that were in place 10 years ago.  As I know from having been part of the
process of producing RFC over the past 10 years.

So I regard this registration as unsuitable as it stands and would
expect the IESG to ask for an updated Security Considerations.

I am not commenting on any aspect of this I-D except insofar as it
provides text that will form the registration as it will appear in the
IANA reference.

Tom Petch

---

From: Barry Leiba <[email-masked]>
Date: Tue, 15 Oct 2013 09:31:49 -0400

> modification of the registration on that basis).

I understand your comment about precedent.  To that, I'll note that
RFC 6838 says this in Section 3.1, related to Standards Tree
registrations:

   Registrations published in non-IETF RFC streams are also allowed and
   require IESG approval.

That is, it explicitly allows Independent stream documents to register
in the Standards Tree.  It would seem odd to at the same time forbid
them from modifying registrations (with, of course, the same IESG
approval).

Also, this document was offered to the IETF community, and the
community was not interested in taking it up.  But there's a need for
it in some circles, and we've seen no expression of objection.  That's
why I sent the authors to the ISE back in May.  I think this document
is a good example of what the Independent stream is for, and I think
that RFC 6838 allows us to approve these sorts of things case by case.

As to precedent, if another document should come along and do a
similar thing, we would do a similar analysis and have a similar
discussion.  If that one should garner significant objections, its
path would be different.

> current draft is not an end run around W3C efforts.

Thanks for bringing that up.  I'll be sure to send this to the
liaisons, and make sure we're not interfering with their plans for
text/csv.

Barry

---

From: SM <[email-masked]>
Date: Tue, 15 Oct 2013 08:03:50 -0700

>one would be impressive.. and bad for the IETF.

I was unable to see whether this specification would be of use to 
data.gov as the site is still unaccessible.  There is an opendata 
site in Brazil ( http://dados.gov.br/), France ( 
http://www.data.gouv.fr/ ) and several other countries.  The 
specification may be relevant to opendata which is something of 
interests to governments.  It would have been better if the 
specification was processed in the IETF Stream but the community was 
not interested in taking it up (see msg-id: 
CAC4RtVAeTGpHFA01YX=[email-masked]).

The draft updates a specification in the IETF Stream.  It also 
updates a registration in the standards tree.  This has the hallmarks 
of "not a good idea" as it will be a problem down the line.  If it 
takes a minimal effort to avoid a small problem it is worth the 
effort.  This is the list of persons who commented on the Last Call:

   Tom Petch
   John Klensin
   Alexey Melnikov
   Simon Perreault
   Tim Bray

It's not like the expertise or experience is not there.  In fact, 
there's much more than the average IETF Last Call.  I remember filing 
the minority report against the attempt to kill the Independent 
Submissions Stream. :-)

Regards,
-sm

---

From: John C Klensin <[email-masked]>
Date: Tue, 15 Oct 2013 12:20:10 -0400

--On Tuesday, October 15, 2013 09:31 -0400 Barry Leiba
<[email-masked]> wrote:

>> In a slightly broader interpretation of that question, I
>> believe that having Independent (or even IAB) stream
>> documents modify registrations that are required, either by
>> the registration procedure or the registration itself, to be
>> under IETF change control is a bad idea.  If we don't like
>> that constrain, we should modify the registration, not
>> conduct odd Last Calls.  I would strongly support processing
>> this document in the IETF Stream as an individual submission
>> (and, while that slips over into the substantive part,
>> approving it for publication and modification of the
>> registration on that basis).
> 
> I understand your comment about precedent.  To that, I'll note
> that RFC 6838 says this in Section 3.1, related to Standards
> Tree registrations:
> 
>    Registrations published in non-IETF RFC streams are also
> allowed and    require IESG approval.
> 
> That is, it explicitly allows Independent stream documents to
> register in the Standards Tree.  It would seem odd to at the
> same time forbid them from modifying registrations (with, of
> course, the same IESG approval).

Yeah, I know.  I am somewhat acquainted with the authors.

We probably need to agree to disagree on this, but I don't see
the distinction as odd at all.  The text/* namespace is
essentially unlimited.  If someone wants to create a new one,
the key issue is whether it is fundamentally harmful in some
way, even if it is ignored in the marketplace.  "Standards Tree"
implies some level of endorsement, but not very much, and the
ISE process is (we hope) pretty good at determining whether a
document is competent as a document.   The bar for such new
registrations should be reasonably low and, a snit or two about
parameter relationships (like "charset=")  generally has been.

But a parameter-adding modification is different.  It should be
evaluated for whether or not it could have bad effects on
existing uses of the registration or in combination with other
parameters.  As an example for this particular case and drawing
on my long-ago experience with data format conversion systems,
some systems that convert CSV files to matrix (or ER)form use
named columns, either inventing names or taking them from the
first line of the CSV file (for this case, "text/csv;
header=present").  For some of those systems, the separation
occurs right after the data are received or as they are
streaming in, so processing of a fragment with row selection
using this approach requires that one preserve the information
needed to decrement row numbers in the fragment specification.
That example is actually about two things: a comment to the ISE
that the draft needs to have an explicit discussion about
interactions between row numbers and headers (or at least a
Security Considerations warning) and a comment to the IESG about
the dangers of this type of registration modification.

Other interactions with other registration extensions could be
much more serious.   I note, without recommending it, that a new
registration for text/csv-with-fragments that recommended the
gradual retirement of text/csv would have none of these problems
and, if successful, would have the potential to eliminate many
of the issues with different CSV interpretations that are noted
in both this spec and RFC 4180 itself.  I don't know that would
be an especially good idea, but it would eliminate the multiple
issues raised by modifying an existing spec.

> community was not interested in taking it up.  But there's a
> need for it in some circles, and we've seen no expression of
> objection.  That's why I sent the authors to the ISE back in
> May.  I think this document is a good example of what the
> Independent stream is for, and I think that RFC 6838 allows us
> to approve these sorts of things case by case.

At the risk of hair-splitting, Section 3.1 of RFC 6838 is
entirely about new registrations.  Section 5.5 is applicable in
this case.  First, if the IESG, as the representative of the
relevant standards body, is assumed to be the "owner" of RFC
4180 and the existing registration, then the IESG needs to take
responsibility for the evaluation discussed above.  If Yakov
Shafranovich is the owner as author of 4180, then he has a
significant voice in any decision here.   More important,
Section 5.5 says 

	"Significant changes to a media type's definition should
	be requested only when there are serious omissions or
	errors in the published specification."

Presumably this is a "significant change".  Whether the absence
of fragments is a "serious omission" or not is debatable and we
have heard little debate on the subject.   Again, this would be
a much more comfortable situation if the document were handed as
an individual submission in the IETF Stream (on which the
community could give advice during Last Call that had to be
considered) rather than as an independent submission (on which
the community could give advice and hope the authors are
interested).

> similar thing, we would do a similar analysis and have a
> similar discussion.  If that one should garner significant
> objections, its path would be different.

Again, I agree with your analysis for new registrations.  What
is giving me pause is the notion of modifying an existing one to
add features.

> liaisons, and make sure we're not interfering with their plans
> for text/csv.

If the IETF is uninterested in text/csv (as you imply above) and
W3C is interested (the media type is apparently intertwined
with their "web data" and "semantic web" efforts), another
possibility would be for the IESG to take the action described
in the last paragraph of Section 5.4 of RFC 6838, reassign
responsibility for text/csv to W3C (which I assume is a
"recognized standards-related organization") and let them sort
it out.  That would presumably even allow replacement of RFC
4180 with a reference to a stable W3C spec.

That solution would, of course, not automatically generalize to
other cases.

best,
   john

---

From: John C Klensin <[email-masked]>
Date: Tue, 15 Oct 2013 13:29:45 -0400

--On Tuesday, October 15, 2013 08:03 -0700 SM <[email-masked]>
wrote:

>> My reasoning is that, while the change seems fine, the
>> precedent seems atrocious.  If this is approved via
>> Independent Stream publication and the next case that comes
>> along is, unlike this one, generally hated by the community,
>> the amount of hair-splitting required to deny that one having
>> approved this one would be impressive.. and bad for the IETF.

> I was unable to see whether this specification would be of use
> to data.gov as the site is still unaccessible.  There is an
> opendata site in Brazil ( http://dados.gov.br/), France (
> specification may be relevant to opendata which is something
> of interests to governments.  It would have been better if the
> specification was processed in the IETF Stream but the
> community was not interested in taking it up (see msg-id:

As long as the question is "should the IETF approve registration
of an extension whose documentation is published via the ISE",
most of the above is massively irrelevant.  Hence the change in
subject line and the copy to Nevil (and I'm not likely to
discuss it further on this list).

>From the standpoint of retrieval of information from a database
(opendata or otherwise), my experience suggests that CSV
fragments, especially as specified here, are going to be fairly
irrelevant.  Not harmful, probably ok for those who think they
have a use for them, just useless for lots of retrieval
functions.  The problem is that, in general and with a dataset
of any real size, people don't think of things in terms of row
and column numbers (getting them to think that way is quite
error-prone).   Especially when there are _lots_ of columns,
retrieval by column number is usually a bad idea.

Using an example from the I-D to construct a different one,
  http://example.com/data.csv#col="temperature"
would make a lot more sense in many cases than
  http://example.com/data.csv#col=2
But the spec doesn't allow that and it is perhaps better handled
with a query rather than a fragment (although that opens the
problem of where queries are processed that has tied the URNbis
WG in knots).

It is also not unusual with statistical and scientific databases
(especially non-relational ones) to have named rows as well as
named columns, but, while RFC 4180 allows for ";
header=present", it makes no provision for 'rowNames="present"',
much less what many data analysis packages would really like to
see, which would be something like 'rowNames="present, col=NN"'
with the latter designating the "column" (or columns) of the CSV
file in which those names appeared, perhaps borrowing from
regular expressions and allowing "$" and "$-1" instead of NN.

There is no reason why one couldn't have those sorts of
arrangements with a CSV format and some systems do, but is isn't
the text/csv of RFC 4180.

For datasets of non-trivial size and complexity, fragments as
specified here are going to be really useful only when the
application retrieves what used to be called a "codebook" and
first, uses it to change row and column identifiers or other
query-supporting information into row and column numbers, and
then constructs the URI with this fragment ID.  I suspect that
won't be common for lots of reasons starting from the
observation that many modern database management and database
access technologies discourage detached codebooks.   

Those types of application situations also lead to another
problem with the fragment approach.  Again going back to the
example in the draft, if one had, instead of 

   date,temperature,place
   2011-01-01,1,Galway

one had 
   date, time,observed-melting-point
   2011-01-01,0900.3, 0.001
   2011-01-01,0901.2, 0.002
   2011-01-01,0901.8, -0.09

Then many systems would do the conversions if the last column to
floating point as the data were being read (and might convert
times or dates as well).  Depending on how much information was
kept, responding to 
    http://www.example.com/melting-points.csv#col=3
with
  0.001
  0.002
  -0.09
rather than 
  0.001
  0.002
  -0.090
or
  0.1 x10**-2
etc., as the spec seems to require, might require significant
work.  Some processors would care, others wouldn't, but you see
the problem.

Again, I don't see a big problem with this addition although
there are a number of things I'd like to see either more clear
or explicitly warned about.   I don't see arguments about what
problems it doesn't solve as relevant unless extravagant claims
are made for it, and the current draft avoids such claims.  But
those issues are quite separate from the issue of the IESG
passing responsibility for documentation and evaluation of a
registration modification request (for a registration for which
I believe the IESG to be the "owner") off to the ISE.

best,
     john