Subject: Re: Last Call: draft-freed-sieve-environment (Sieve Email Filtering: Environment Extension) to Proposed Standard

From: Sam Hartman <[email-masked]>
Date: Tue, 18 Mar 2008 04:51:42 -0400

This extension appears to conflate two unrelated things: information
about the interpreter context and information about the message.

I don't think these two sets of information are similar enough that the same interface should be used to get to both of them.

In particular I believe that the remote-host and remote-ip variables
are inappropriate and should not be standardized.

I believe an applicability statement should be added to the extension
making it clear that this extension is only for interpreter state and
that another extension should be designed for examining information
about the message.

I find the string "MUA" meaning anything that happens after delivery
confusing.  I'd suggest another string--possibly "POST-MDA" and
reserve "MUA" for sieve scripts actually executed inside a MUA.
Alternatively perhaps "MUA" could mean the script is executed at the
direction of the MUA.  That's not quite the same thing as
post-delivery.

Section 4.3.3 claims that experimental RFCs are an appropriate
mechanism to register non-standards-track variables intended for wide
use.  That seems wrong.  I recommend revisiting the registration
policy.

In conclusion, I object quite strongly combining message and
interpreter context information.  The other comments I'm making are
less serous.  However based on the number of comments I think this
document needs significant positive review before it is ready to be
published.

_______________________________________________
IETF mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ietf

---

From: Alexey Melnikov <[email-masked]>
Date: Tue, 18 Mar 2008 14:25:09 +0000

Hi Sam,

>reserve "MUA" for sieve scripts actually executed inside a MUA.
I don't think POST-MDA is any better. For typical Sieve "post delivery" 
is always MUA.
But now that there is a proposal to run Sieve in IMAP, there is another 
post delivery point of Sieve script execution: IMAP (or MailStore).

>direction of the MUA.
No, this is not what MUA specified in the draft means.

>That's not quite the same thing as
>post-delivery.
>  
>
Correct. A script generated by MUA and executed by MTA/MDA will have the 
value "MTA" or "MDA" respectively.

---

From: Alexey Melnikov <[email-masked]>
Date: Tue, 18 Mar 2008 14:52:50 +0000

>document needs significant positive review before it is ready to be
Speaking as the document shepherd: the document received a fair number 
of reviews on the Sieve WG mailing list and there is support to progress it.
Some comments raised would require a new revision before the document 
goes to IESG review. I will make sure that your comments are either 
addressed or answered+argued against by the author of the document.

---

From: John C Klensin <[email-masked]>
Date: Tue, 18 Mar 2008 11:07:34 -0400

Sam,

A few comments about differences in philosophy rather than
specific details of this proposal.  Once one gets past that
difference in philosophy, it is not clear to me how much of your
comments are major but...

--On Tuesday, 18 March, 2008 04:51 -0400 Sam Hartman
<[email-masked]> wrote:

>...
> In particular I believe that the remote-host and remote-ip
> variables are inappropriate and should not be standardized.
> 
> I believe an applicability statement should be added to the
> extension making it clear that this extension is only for
> interpreter state and that another extension should be
> designed for examining information about the message.
>...
> mechanism to register non-standards-track variables intended
> for wide use.  That seems wrong.  I recommend revisiting the
> registration policy.

The bottom line about Sieve is about providing a standard way to
let mail-receiving and mail-reading client machines communicate
with mail server systems about filtering and classification of
mail.  Both the client and server systems in the Sieve case
occur after what the mail transport standards refer to as "final
delivery".    For practical reasons, there has been general
consensus in the email protocol community for many years that
filtering and classification are better done on always-on/
always-connected/ always-active servers than on mail
receiving/reading machines... if only because of perceived
performance issues with the latter.   Part of the elegance of
Sieve is that it tries to be flexible on that subject: if an
operation cannot be specified to and performed on the server,
then it can be acted on by the client, using the same vocabulary
for describing what is to be done.  But the main goal, IMO, is
to permit client control of server-side filtering operations
with a standard vocabulary.

The operations and vocabulary that are being specified for Sieve
aren't doing anything new or anything we can prohibit by not
publishing pieces of the specification language or by making
registration difficult.  If we keep functionality out of Sieve,
we don't prevent that functionality from being used.  We just
impede filtering being offered as a service by multiple-user
email providers, keep those of us who have been doing
server-side filtering and classification for years from doing so
unless we run our own servers and can run scripts and custom
filters there, and push everyone else into client-side filtering
that slows down the perceived performance of mail reading.

Now, use of Sieve is unwise, and perhaps dangerous, if there is
not a fairly strong trust relationship between the client and
server and if the client doesn't have a good model of server
capabilities.  I would wish it were otherwise, but that
relationship reflects reality.  I personally believe those
issues are adequately covered in the base Sieve materials, but
perhaps you disagree and should be looking for more explanation
there (or elsewhere).

But, because of the nature of this particular beast, I believe
that the debate should be about "is this functionality needed
and in use, perhaps with different syntax and outside of Sieve".
If the answer is "yes", I believe that we should publish and
register if only to provide a common way of doing something.  We
might also think the particular function is dumb or dangerous.
However, if we do, the proper remedy, IMO, is to write documents
explaining why people should avoid doing these things --things
that are well-specified-- and what they should do instead,
rather than trying to avoid having them clearly specified.

      john

---

From: Cyrus Daboo <[email-masked]>
Date: Tue, 18 Mar 2008 09:57:57 -0400

Hi Sam,

--On March 18, 2008 4:51:42 AM -0400 Sam Hartman <[email-masked]> 
wrote:

> I don't think these two sets of information are similar enough that the
> same interface should be used to get to both of them.
>

I am confused as to which items you think relate to a message as opposed to 
the interpreter state. This extension is supposed to be all about 
interpreter state - tests against messages are handled in the SIEVE base 
and the other extensions.

-- 
Cyrus Daboo

---

From: Sam Hartman <[email-masked]>
Date: Tue, 18 Mar 2008 11:10:18 -0400

>>>>> "Cyrus" == Cyrus Daboo <[email-masked]> writes:

    Cyrus> Hi Sam, --On March 18, 2008 4:51:42 AM -0400 Sam Hartman
    Cyrus> <[email-masked]> wrote:

    >> This extension appears to conflate two unrelated things:
    >> information about the interpreter context and information about
    >> the message.
    >> 
    >> I don't think these two sets of information are similar enough
    >> that the same interface should be used to get to both of them.
    >> 
    >> interpreter state and that another extension should be designed
    >> for examining information about the message.
    >> 

    Cyrus> I am confused as to which items you think relate to a
    Cyrus> message as opposed to the interpreter state. This extension
    Cyrus> is supposed to be all about interpreter state - tests
    Cyrus> against messages are handled in the SIEVE base and the
    Cyrus> other extensions.

The remote IP address and remote hostname.  The draft explicitly talks
about how the remote source of the current message is part of the context:

   The "environment" extension provides a new environment test that can
   be used to implement scripts that behave differently when moved from
   one system to another, when messages arrive from different remote
   sources or when otherwise operated in different contexts.

---

From: Sam Hartman <[email-masked]>
Date: Tue, 18 Mar 2008 11:39:26 -0400

>>>>> "John" == John C Klensin <[email-masked]> writes:

    John> Sam, A few comments about differences in philosophy rather
    John> than specific details of this proposal.  Once one gets past
    John> that difference in philosophy, it is not clear to me how
    John> much of your comments are major but...

    John> --On Tuesday, 18 March, 2008 04:51 -0400 Sam Hartman
    John> <[email-masked]> wrote:

    >> ...  In particular I believe that the remote-host and remote-ip
    >> for examining information about the message.  ...  Section
    >> 4.3.3 claims that experimental RFCs are an appropriate

    John> The bottom line about Sieve is about providing a standard
    John> way to let mail-receiving and mail-reading client machines
    John> communicate with mail server systems about filtering and
    John> classification of mail.  Both the client and server systems
    John> in the Sieve case occur after what the mail transport
    John> standards refer to as "final delivery".  For practical
    John> reasons, there has been general consensus in the email
    John> protocol community for many years that filtering and
    John> classification are better done on always-on/
    John> always-connected/ always-active servers than on mail
    John> receiving/reading machines... if only because of perceived
    John> performance issues with the latter.  Part of the elegance of
    John> Sieve is that it tries to be flexible on that subject: if an
    John> operation cannot be specified to and performed on the
    John> server, then it can be acted on by the client, using the
    John> same vocabulary for describing what is to be done.  But the
    John> main goal, IMO, is to permit client control of server-side
    John> filtering operations with a standard vocabulary.

John, It sounds like you believe I have a problem with standardizing a
mechanism to look at the remote IP or host of the person submitting
the message at hand.  That's not the case; I simply don't think that
the environment extension is where we should expose that in sieve.

---

From: Ned Freed <[email-masked]>
Date: Tue, 18 Mar 2008 07:51:03 -0700 (PDT)

> about the interpreter context and information about the message.

Not really. The information environment supplies is, as the name implies, about
the environment things are operating in. This includes information about the
Sieve interpreter, the host it is running on, and the network. The last item
includes connection information, which is in no way specific to a particular
message - a connection can  relate to many messages.

> I don't think these two sets of information are similar enough that the same
> interface should be used to get to both of them.

Then by all means provide an example of some sort of information that requires
sufficiently different handling in the Sieve context to warrant having two
different extensions to obtain it. (Or even three if you wanted to separate
interpreter information from host information.)

Absent a compelling reason why different bits of environment information
require different extensions to access I think the present approach of simply
using a single extension is far preferable.

> are inappropriate and should not be standardized.

Why not? These pieces of information are commonly used in Sieve scripts. Right
now the approach people often take is to grub around in the outermost Received:
field looking for this stuff. But this approach is extremely unreliable given
the widely varying formats used by different implementations, to say nothing of
the fact that message flows within an implementation can also vary widely, and
leads to nonportable and unreliable scripts. The main goals of environment are
to improve script portability and reliability, both by providing contextual
information that can be used to tailor script actions but also by eliminating
the need to grub around for information the script needs.

Now, perhaps you object to scripts using IP addresses as some sort of layering
violation. If that's the case then I'm sorry, but this particular horse left
the barn decades ago. IP address and host name information is consumed
routinely as part of message filtering activities and while I along with many
others wish things didn't have to be done this way it is nothing short of
delusional to pretend it isn't.

> that another extension should be designed for examining information

Given that none of the information provided by any current defined item is
about the message per se I fail to see the point.

> direction of the MUA.  That's not quite the same thing as

Well, according to draft-crocker-email-arch-10.txt, an MUA is a thing that
"works on behalf of end-users and end-user applications. It is their
'representative' within the email service." This would seem to cover
essentially all message processing activities prior to submission and after
delivery, not just those done from the subset of software you personally
consider to be an MUA.

That said, I have no objection to there being a little more granularity in the
MUA space, although in most cases I'm pretty sure it is going to be a
distinction without a difference. However, I have to wonder if (a) This small
and simple Sieve extension is the right place to conduct such an exercise in
defining new agent taxonomy and (b) Given how difficult it has been to get
consensus on the very limited and general set of terms used in the email
architecture specification, how much harder it is going to be to get agreement
on these distinctions in user agent space.

Now, one legitimate issue that this does point oout is that there is in fact a
value missing from the current enumeration: Store. In particular, here is that
of attaching Sieves to various parts of draft-ietf-lemonade-imap-sieve-05.txt
extends Sieve to be applicable when actions are taken on messages in the
message store. This in turn leads to a case where a number of Sieve tests and
actions cannot be performed and others have somewhat different semantics. This
type of evaluation is therefore a distinction with significant difference and
warrants its own evaluation-agent value. In fact there are finer distinctions
in the context of the store Sieve evaluations, but the draft in question
addresses this by defining a number of additional environment items to access
this information. But the one thing it doesn't do is deal with the
evalluation-agent issue.

Now, there are several ways this could be handled, and I'm open to suggestions
as to which one makes the most sense. We could:

(1) Have the imap-sieve document update the environment specification with an
    additional evaluation-agent value.

(2) Make the evaluation-agent enumeration extensible and have imap-sieve add
    a value to the list.

(3) Simply add the "store" value to the list in the existing environment
    document, along with an informative reference to imap-sieve.

(3) seems like the simplest and the least amount of work so I'm inclinced to go
with it, but I'll go with whatever consensus emerges.

> use.  That seems wrong.  I recommend revisiting the registration

On the contrary, it would be very wrong for this to be changed. In particular,
the registration policy for environment items needs to be aligned with that for
Sieve extensions in general because it is expected that various extensions,
including but not limited to the imap-sieve one I've already mentioned, will
want to define their own environment items. If it isn't aligned  we create a
situation where some extensions can define environment items and others cannot,
which would be more than a little ridiculous.

RFC 5228 section 6.2 gives the policy for new Sieve extensions: Standards track
or experimental RFCs. Now, you may argue that experimental Sieve extensions
should not be allowed, but that ship has already sailed (and I might add it did
so during your own tenure on the IESG).

> document needs significant positive review before it is ready to be

Your assertion as to what is being combiner here is fallacious. I suggest you
reexamine what it means for something to be related to a message as opposed to
the evironment surrounding the message.

As for the number of comments being made, this is the first one I've seen that
I've viewed as anything other than editorial in nature.

				Ned

---

From: Ned Freed <[email-masked]>
Date: Tue, 18 Mar 2008 10:45:44 -0700 (PDT)

> delivery".

This actually isn't true. Some Sieve implementations, including ours, operate
before final delivery. (Usually just before.) Others operate just after. And in
the imap-sieve case there is now a situation where Sieve can be executate LONG
after final delivery.

Indeed, there's at least one Sieve extension - ereject - that requires
operation at or before final delivery in order to funciton (it calls for a 5yz
SMTP response - no way can that happen after final delivery).

> For practical reasons, there has been general
> performance issues with the latter.

Performance issues are part of it, but there's also the temporal context to
consider. Some filtering operations are highly time dependent - indeed, there's
a Sieve extension (date) specifically do deal with such things.

> Part of the elegance of

Exactly.

> registration difficult.

This is the key point. You can rail all you want about how the client IP
address or host name oren't legitimate things to base your filtering on, but
that isn't going to change the fact that people do this sort of filtering all
the time. Heck, there are entire services built around this model, and I could
see wanting to take the remote IP address an feed it into the external list
extension in order to access information from such a service.

> If we keep functionality out of Sieve,

Very nicely put. I completely agree.

> perhaps you disagree and should be looking for more explanation

I believe there are a number of things we got wrong that unessarily hinder
script portability and reliability, but the good news is almost all of them can
be address through extensions of one sort or another. Access to the sort of
informaton environment provides addresses one of these concerns. (Another
example: Currently scripts have a choice of requiring an extension or doing
without it - there's no way to use an extension if and only if it is available.
This one is addressed by the ihave extension defined in another one of my
drafts.)

				Ned

---

From: Dave Crocker <[email-masked]>
Date: Tue, 18 Mar 2008 11:49:44 -0700

Sam Hartman wrote:
> This extension appears to conflate two unrelated things: information about
> the interpreter context and information about the message.
> 
> In particular I believe that the remote-host and remote-ip variables are
> inappropriate and should not be standardized.

Sam,

In looking over your initial post and the responses, I think that a core 
component of your comment that is missing is the underlying technical basis that 
makes the split compelling.

So far, the only reason you provided in your original note was that the data 
were not "similar enough".  At the least, it will help to hear why that is 
important, from a technical standpoint.

We all have design preferences, and we are entitled to them.  Absent a 
compelling technical reason for choosing one, such as performance, reliability, 
code complexity, etc., etc., the preference is merely that.  Preference.

For example, I could imagine the claim that having separate interfaces for 
interpretor (control) and message (payload) would make independent handling of 
the different types of data easier.

Alas, I could imagine a response to this concern that environment variable names 
provide all the demultiplexing power that is needed, and that having two 
different paths (interfaces) for passing data actually adds unnecessary complexity.

> direction of the MUA.  That's not quite the same thing as

The values in the draft cite architectural modules.  Your proposed change mixes 
nomenclature and perspectives within a single term.

If your proposed change were adopted, the other two values would have to be 
something like "TRANSIT" and "PRE-DELIVERY" and "POST-DELIVERY".  Pretty 
baroque, but at least it would be consistent with what you seem to be proposing.

Better still is that "POST-MDA" leaves ambiguous the choice between an active 
message store (MS) versus the MUA.  Ned's terminology is explicit. His 
document's citing pre-vs.post- delivery is merely some pedagogical assistance.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net
_______________________________________________
IETF mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ietf

---

From: Dave Crocker <[email-masked]>
Date: Tue, 18 Mar 2008 12:15:46 -0700

Ned Freed wrote:
>  before final delivery. (Usually just before.) Others operate just after. And
> in the imap-sieve case there is now a situation where Sieve can be executate
> LONG after final delivery.

Indeed.

and FYI:

    <http://bbiw.net/specifications/draft-crocker-email-arch-10.html>

> Figure 5 shows a Sieve specification going from the rMUA to the MDA. However
> filtering can be done at many different points along the transit path and any
> one or more of them might be subject to Sieve directives, especially within a
> single ADMD. Hence the Figure shows only one relationship, for (relative)
> simplicity.

The document inches ever-closer to finalization, with the intent of 
standardization, but the contents represent pretty good community consensus IMO, 
and the portion about SIEVE isn't controversial.  I believe it never has been.

d/
-- 

   Dave Crocker

---

From: Tony Finch <[email-masked]>
Date: Wed, 19 Mar 2008 14:11:11 +0000

> message - a connection can  relate to many messages.

However the information about the interpreter and the host are essentially
static - they don't change over the run-time of the server or even longer
periods - whereas connection information is dynamic. The fact that
multiple messages can be delivered on the same connection is a detail that
usually doesn't matter to users or even admins, so it's usually simpler to
treat it as a property of the message not of the platform.

Otherwise I agree with Ned.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
FORTIES CROMARTY FORTH TYNE DOGGER: NORTHERLY BACKING WESTERLY 5 TO 7,
DECREASING 4 FOR A TIME, PERHAPS GALE 8 LATER IN FORTIES, CROMARTY AND FORTH.
MODERATE OR ROUGH. WINTRY SHOWERS, RAIN LATER. MODERATE OR GOOD.
_______________________________________________
IETF mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ietf

---

From: Alexey Melnikov <[email-masked]>
Date: Wed, 19 Mar 2008 15:29:14 +0000

Speaking as an individual:

Ned Freed wrote:

 [...]

>with it, but I'll go with whatever consensus emerges.
I am fine in either of 3 choices. I agree that (3) is the easiest. And 
considering that IMAP Sieve is going to IETF LC really soon now, the 
informative reference in Sieve environment should point to the correct 
document when Sieve environment is published.

---

From: Ned Freed <[email-masked]>
Date: Wed, 19 Mar 2008 13:25:04 -0700 (PDT)

> periods - whereas connection information is dynamic.

That may be the case in your implementation, but it most assuredly is NOT in
ours. Indeed, in one of our typical highly distributed deployments things like
the host where the interpreter is running will change almost as often as the
remote IP address. (There won't be nearly as many possible values, of course,
but we're talking rate of change here, not range of values.)

Hopefully the sieve product and interpreter version will be relatively stable
in most cases, but while we don't recommend it we do see large scale
deployments running mixtures of different software versions or even (shudder)
different products. I see ihave as a more effective tool for dealing with such
environments, but people actually writing scripts (or perhaps more importantly
the people writing code that generates scripts) may or may not agree with me on
this.

> The fact that

Well, I agree that it shouldn't matter, but the fact is it is not at all
uncommon for people to use the number of message delivered per session as a
filtering criteria. (Before anyone starts railing about how stupid this is,
please note that the current environment specification provides no means to
access this information.)

				Ned

---

From: Sam Hartman <[email-masked]>
Date: Thu, 20 Mar 2008 11:03:47 -0400

>>>>> "Alexey" == Alexey Melnikov <[email-masked]> writes:

    Alexey> Hi Sam,
    Alexey> Sam Hartman wrote:

    >> I find the string "MUA" meaning anything that happens after
    >> delivery confusing.  I'd suggest another string--possibly
    >> "POST-MDA" and reserve "MUA" for sieve scripts actually
    >> executed inside a MUA.
    >> 
    >> 
    Alexey> I don't think POST-MDA is any better. For typical Sieve
    Alexey> "post delivery" is always MUA.  But now that there is a
    Alexey> proposal to run Sieve in IMAP, there is another post
    Alexey> delivery point of Sieve script execution: IMAP (or
    Alexey> MailStore).

Under the current language in the draft, you'd use MUA for that case.
I find that very confusing and believe others would as well.

    >> Alternatively perhaps "MUA" could mean the script is executed
    >> at the direction of the MUA.
    >> 
    Alexey> No, this is not what MUA specified in the draft means.

No, the current language in the draft would include cases where the MUA directs something to run a sieve script within the MUA definition.
_______________________________________________
IETF mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ietf