Subject: Re: Mailing list membership.

From: Kazunori ANDO <[email-masked]>
Date: Thu, 2 Mar 2017 03:53:31 +0900

Hi, folks

I made note about these conflicts. FYI.

https://www.facebook.com/notes/%E5%AE%89%E8%97%A4-%E4%B8%80%E6%86%B2/memo-about-some-conflicts-between-mailing-lists-and-dmarc/1332635393468971
-- 
Kazunori ANDO / [email-masked]

On 2017/03/01 13:49, Theodore Ts'o wrote:
> On Tue, Feb 28, 2017 at 05:29:24PM -0300, willi uebelherr wrote:
>>
>> related to the problem, what Khaled explained, what is your proposal?
>>
>> What are your "compatible with internet mailing lists" mail systems?
>
> RFC 2821, Simple Mail Transfer Protocol, section 3.10.2
>
>    "To expand a list, the recipient mailer replaces the
>    pseudo-mailbox address in the envelope with all of the expanded
>    addresses.  The return address in the envelope is changed so that all
>    error messages generated by the final deliveries will be returned to
>    a list administrator, not to the message originator, who generally
>    has no control over the contents of the list and will typically find
>    error messages annoying."
>
> This is the SMTP Envelope From field.  The FROM field is not changed,
> but the SMTP return address is changed, so that bounces go to the
> mailing list administrator as opposed to the person who sends mail to
> the mailing list.
>
> Unfortunately, if you are using a system whose domain requests that
> all recipients enforce DMARC alignment, this specifically instructs
> recipients to bounce mail if the SMTP Envelope return address doesn't
> match the FROM field in the header.  This means that they won't see
> mailing list mail as defined by the IETF Standards Track RFC 2821,
> which specifically says that is acceptable (and in fact a good thing)
> to change the SMTP envelope return address so that bounces (caused by
> people changing where they work, etc.) go to an administrator who can
> deal with them.  But if the mailing list administrators gets too may
> bounces, and it's because the sending domain is requesting that mail
> be bounced, the only thing they can do is to unsubscribe the sender or
> the recipient.
>
> Hence mailing list systems that enforce DMARC, or request DMARC
> processing, are fundamentally incompatible with mailing lists as
> defined by section 3.10.2 of RFC 2821.
>
> If you want to participate in such mailing list, one of the best ways
> is to change to a mailing list system that doesn't do DMARC.
>
> Best regards,
>
> 						- Ted
>
>

---

From: Michael StJohns <[email-masked]>
Date: Wed, 1 Mar 2017 13:55:59 -0500

Question below

> is to change to a mailing list system that doesn't do DMARC.

Thanks for the clear description of the problem - I admit that I got a 
similar one from folks at the Seoul IETF.

In the short term, one thing I think would be helpful is to identify for 
the IETF participants which email providers are known to use DMARC and 
maybe also a list of providers that are known not to.   Maybe add it to 
the IETF mailing list page at https://www.ietf.org/list/ with an 
explanation of what's been happening.

Mike

---

From: Bob Hinden <[email-masked]>
Date: Wed, 1 Mar 2017 11:14:05 -0800

Mike,

> On Mar 1, 2017, at 10:55 AM, Michael StJohns <[email-masked]> wrote:
> 
>> 
> Thanks for the clear description of the problem - I admit that I got a similar one from folks at the Seoul IETF.
> 
> In the short term, one thing I think would be helpful is to identify for the IETF participants which email providers are known to use DMARC and maybe also a list of providers that are known not to.   Maybe add it to the IETF mailing list page at https://www.ietf.org/list/ with an explanation of what's been happening.
> 

This is a good idea.  Taking it a step further, perhaps a warning could be included when subscribing to an IETF list from an email provider who is known to use DMARC.

Bob

---

From: Michael StJohns <[email-masked]>
Date: Wed, 1 Mar 2017 14:18:19 -0500

On 3/1/2017 2:14 PM, Bob Hinden wrote:
>> In the short term, one thing I think would be helpful is to identify for the IETF participants which email providers are known to use DMARC and maybe also a list of providers that are known not to.   Maybe add it to the IETF mailing list page athttps://www.ietf.org/list/  with an explanation of what's been happening.
>>

I like that.

Actually, split the difference to start with - its probably a coding 
pass to personalize a warning based on provider.  So do a) Add a warning 
to the IETF mailing list page, b) add a warning to the confirm message 
that's sent out when you subscribe that contains the warning regardless 
of provider, and c) eventually customize the confirm message based on 
provider.

Mike

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 1 Mar 2017 20:21:56 +0100

On 1 Mar 2017, at 20:18, Michael StJohns <[email-masked]> wrote:
> 
> eventually customize the confirm message based on provider

Better: Send the subscription confirmation message with attributes that cause a DMARC-honoring provider to discard it :-)
(No programming required :-)

Grüße, Carsten

---

From: willi uebelherr <[email-masked]>
Date: Wed, 1 Mar 2017 16:29:01 -0300

Dear Theodore, many thanks for your explanation to the RFC 2821 (SMTP). 
The answer from Khaled i have included, because it goes to me as a 
private message.

Maybe, based on my bad english, i feel some confusion with the terms.

1) Khaled, like i and you, is member of the IETF discussion list. This 
means, he receive all emails that are distributed over the list.

2) Khaled, like i and you, use the maillist server mailman from IETF 
discussion list to distribute his messages to all members in the list.

3) The IETF discussion list don't follow the DMARC processing. This 
means, it act only outside.

4) Khaled, like i and you, use a mail box server system as the 
interconnection point to the list. Khaled use hotmail, you use mit.edu, 
i use riseup.net.

This means, the actors are the mailbox servers with the mailman maillist 
server IETF discuss in both direction.

I understand and agree absolutly, that the maillist server never change 
the From-line in the header. He create the Return-Path-line and/or 
Error-To-line for error response from the receiver mailbox server 
system. The bounce-information.

The mailman maillist server use bounce-counters for every member and 
some limits for this bounce-counter. If the limit exceeds, and the 
admin-group do nothing, then the maillist server mailman disable the 
delivery. It is not an unsubscription.

The admin-group have to follow the incremental increase of the 
bounce-counters to understand, what is the background. Maybe, the 
mailbox is full, or don't exist or is the result of this stupid DMARC 
processing.

The DMARC processing is defined in the DNS info. But we can ignore it, 
or not? The admin-group can inform the member to change her mailbox 
server to "avoid more errors" like Khaled wrote. The IETF discussion 
admin-group can only inform about the error sources. The members have to 
change her mailbox servers, or not?

Based on that process, we can clean all this nonsense in our IETF lists 
environment and work strong based on the RFC 2821, like mailman do it.

What do you think about?

many greetings, willi

On 01/03/2017 07:50, Khaled Omar wrote:
 > Hi Willi,
 >
 >> Mailman never change the "From"-header. Therefore, the From-Header 
always points to the author of the email. What you think, is that the 
correct, compatible way? I think, yes.
 >
 > Such case is out of our hands, other e-mail service providers are 
welcome to be used just if this will add a value and avoid more errors.
 > Best Regards,
 > Khaled

On 01/03/2017 01:49, Theodore Ts'o wrote:
> -Ted

---

From: David Morris <[email-masked]>
Date: Wed, 1 Mar 2017 11:34:09 -0800 (PST)

> (No programming required :-)

No, that is worse as it insures support costs and more noise on this list.

---

From: "John Levine" <[email-masked]>
Date: 1 Mar 2017 21:00:33 -0000

In article <[email-masked]> you write:
>> This is a good idea.  Taking it a step further, perhaps a warning could be included when
>subscribing to an IETF list from an email provider who is known to use DMARC.

You can tell who publishes DMARC policies, but without experimentation
you can't tell who follows them on inbound mail.

Of the large public mail providers, only AOL and Yahoo currently
publish DMARC policies, but Gmail certainly looks at DMARC, albeit as
part of the special sauce so a Gmail user might get mail with DMARC
problems or might not.  I'm reasonably sure that Hotmail and Comcast
also look at DMARC on incoming mail.  DMARC is surprisingly useful for
blocking phishes, which is a big issue for consumer mail providers, so
the providers that use it are not going to stop.

R's,
John

PS: I presume you all know that we've been working on workarounds.

---

From: willi uebelherr <[email-masked]>
Date: Wed, 1 Mar 2017 21:13:55 -0300

Dear Carsten,

super! But what are this attributes, that brings the "DMARC honoring 
providers" to hiccup? (in german Schluckauf).

many greetings, willi

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 2 Mar 2017 01:35:25 +0100

> On 1 Mar 2017, at 20:34, David Morris <[email-masked]> wrote:
> 
> 
> 

I apologize for trying to inject some humor into this serious matter.

Silently dropping subscription requests isn’t very nice.  So there probably should be a second (DMARC-unencumbered) message telling people to expect the confirmation request message and to use a different mail system if it does not arrive.

I do not agree that actively preventing DMARC-honoring addresses from subscribing would be worse than the current situation, where the subscriber thinks they are on, only to be kicked off after a short while (or, worse, simply not receiving some of the messages).

Grüße, Carsten

---

From: Barry Leiba <[email-masked]>
Date: Wed, 1 Mar 2017 16:36:37 -0800

> providers" to hiccup? (in german Schluckauf).

I'm not sure what you're asking, but let me explain "the DMARC
problem", in case that's what you want to know (and for any who aren't
already aware of it):

- Bob posts to a mailing list from <[email-masked]>, and example.com
is a domain that publishes a DMARC record with the "p=reject"
parameter.

- (The example.com domain does this because it doesn't like people to
send mail that says "From: <[email-masked]>" when the mail is not
actually from an address at example.com.  It's a brand-protection
issue, to oversimplify a bit.)

- The mailing list prepends "[listname]" to the subject line, and
probably sticks some "from the mailing list" text at the end of the
message... and then sends it on to the subscribers, leaving the
"From:" line unchanged (so it still says <[email-masked]>).

- Carol and Ted and Alice are all mailing list subscribers, so each of
them is sent a copy of the message.

- Alice is <[email-masked]>, and nicedomain.example looks up
and honours DMARC records.

- The nicedomain.example mail server checks SPF.  It looks up the SPF
record for example.com and it sees that the IP address the mail is
coming from (which belongs to the list sever) is not authorized to
send mail as example.com.  So the SPF check fails.

- The nicedomain.example mail server checks DKIM.  It finds the DKIM
signature in the message and tries to verify it.  But the changes the
list server made to the message (the subject line and the stuff at the
end) broke the DKIM signature.  So the DKIM check fails.

- The nicedomain.example mail server has not been able to authenticate
the message with respect to the domain in the "From" line
(example.com), so it looks up example.com's DMARC record to see what
example.com's policy says.  And it says "p=reject".

- Honouring that, nicedomain.example rejects ("bounces") the message.

- The bounce message goes back to the mailing list server.

- The mailing list server sees that a list message it sent to Alice
bounced, so it increments the bounce count for Alice.

- After a few such messages, Alice's bounce count exceeds the
threshold for the mailing list software, and she is unsubscribed from
the list.

Now, of course, Alice can re-subscribe, but the same thing will
eventually happen again... and again.

Some workarounds include asking Bob to post from an address at a
domain that doesn't publish "p=reject", and/or asking Alice to
subscribe from an address at a domain that doesn't reject messages
based on DMARC policies.  There are also workarounds that can be done
in the list server, each of which creates its own problems.  None of
these workarounds are ideal.

The DMARC working group is working on a protocol called ARC, which is
aimed at fixing some of these issues.

(Hoping this has helped some people to understand what's going on...)

-- 
Barry

---

From: Miles Fidelman <[email-masked]>
Date: Wed, 1 Mar 2017 18:04:06 -0700

> This is a good idea.  Taking it a step further, perhaps a warning could be included when subscribing to an IETF list from an email provider who is known to use DMARC.
Yes, but that warning should be on the subscription web page, or the 
list manager should be tuned to make sure that the warning notice 
doesn't get rejected by DMARC!

Miles Fidelman

-- 
In theory, there is no difference between theory and practice.
In practice, there is.  .... Yogi Berra

---

From: willi uebelherr <[email-masked]>
Date: Thu, 2 Mar 2017 17:08:55 -0300

Dear John,

"PS: I presume you all know that we've been working on workarounds."

Please John, explain your results or understanding in the IETF list, 
that all members in the list know, for what you work and what problems 
exist for you.

many thanks and greetings, willi

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 13 Mar 2017 18:03:32 +0100

More SPF/DKIM fun.

Just got:

  ----- The following addresses had permanent fatal errors ——
<[email-masked]>
   (reason: 450 4.7.26 Service does not accept messages sent over IPv6 [2001:638:708:30c9::12] unless they pass either SPF or DKIM validation (message not signed))

Whiskey Tango Foxtrot.

So they give their mail server an IPv6 address but then don’t accept certain messages on that that they would happily accept over IPv4.

Grüße, Carsten

---

From: Dave Crocker <[email-masked]>
Date: Mon, 13 Mar 2017 10:21:20 -0700

>    (reason: 450 4.7.26 Service does not accept messages sent over IPv6 [2001:638:708:30c9::12] unless they pass either SPF or DKIM validation (message not signed))
...
> So they give their mail server an IPv6 address but then don’t accept certain messages on that that they would happily accept over IPv4.

Email abuse across the open Internet remains impressively high. 
Typically above 90% of traffic.  There is a very strong undercurrent of 
desire amongst some email-receiving operators to require all mail to be 
authenticated, in order to facilitate accountability for email streams. 
That is, to have SPF and/or DKIM tests succeed.

Some of them have the view that the transition to IPv6 is a place to 
impose this stringent policy.

In one of the anti-abuse organizations, I've tried to point out the 
problems with imposing policy changes based on transport -- independent 
of whether the policy change is a good idea -- but to no avail.

d/
-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 13 Mar 2017 18:28:09 +0100

Hi Dave,

I sure understand the problem that the scarcity-based address reputation hack won’t work that well on IPv6.  But the “solution”…  And NIST isn’t exactly Mom and Pop’s Bait and Internet Shop.

On 13 Mar 2017, at 18:21, Dave Crocker <[email-masked]> wrote:
> 
> In one of the anti-abuse organizations, I've tried to point out the problems with imposing policy changes based on transport -- independent of whether the policy change is a good idea -- but to no avail.

Well, to add insult to injury, this was a DEFER code, so our mail server tried for five days to deliver this; of course using IPv6 all the time.

No idea whether there is code around that can react to a DEFER with a change of source address.

Grüße, Carsten

---

From: "John R Levine" <[email-masked]>
Date: 13 Mar 2017 18:43:21 +0100

> I sure understand the problem that the scarcity-based address reputation hack won’t work that well on IPv6.  But the “solution”…  And NIST isn’t exactly Mom and Pop’s Bait and Internet Shop.

Nor is gmail, which also requires that incoming IPv6 mail be authenticated 
with SPF or DKIM.  They know what they are doing, and they have decided 
that the amount of legit mail they will lose by doing this is 
insignificant compared to the improvement in the amount of spam and 
malware they will be able to filter.

You can complain about how stupid they are or you can upgrade your mail 
system to use authentication methods that have been IETF standards for a 
decade and get your mail delivered.  Your choice.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail. https://jl.ly

> change of source address.

Unless it changes to an IPv4 address, it won't help.

---

From: Dave Crocker <[email-masked]>
Date: Mon, 13 Mar 2017 11:21:24 -0700

> decade and get your mail delivered.  Your choice.

On the other hand, perhaps the IETF would consider formulating a 
suggestion to those who are embracing this policy change that they 
document it properly and publish it openly.

I suspect most/all of the adopters participate in M3AAWG and that a 
draft BCP effort from M3AAWG, submitted to the IETF either for BCP 
status or Independent Stream Informations, would be constructive.

So one might even consider having the IETF ask this to be communicated 
through the IETF/M3AAWG liaison(s)...

d/

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 13 Mar 2017 20:14:12 +0100

On 13 Mar 2017, at 18:43, John R Levine <[email-masked]> wrote:
> 
> You can complain about how stupid they are or you can upgrade your mail system to use authentication methods that have been IETF standards for a decade and get your mail delivered.  Your choice.

The most likely outcome is that local admins will “upgrade" our mail system by turning off IPv6.  (We have had IPv6 deployed for some 18 years now.)

Grüße, Carsten

---

From: Viktor Dukhovni <[email-masked]>
Date: Mon, 13 Mar 2017 19:47:07 +0000

> The most likely outcome is that local admins will �upgrade" our mail system
> by turning off IPv6.  (We have had IPv6 deployed for some 18 years now.)

That's my approach at present:

    $ postconf -Mf smtp/unix
    smtp       unix  -       -       y       -       -     smtp
	-o inet_protocols=ipv4

-- 
	Viktor.

---

From: Philip Homburg <[email-masked]>
Date: Mon, 13 Mar 2017 21:22:23 +0100

>malware they will be able to filter.

That's not correct, unless gmail made an exception for my home mail server.

Getting mail delivered to gmail over IPv6 works most of the time without
ever setting up SPF or DKIM. Gmail does seem to be the single most
unreliable mail server that I know of, mostly due to their attempts
to be more strict on IPv6.

What gmail does require is reverse DNS for IPv6.

---

From: Brandon Long <[email-masked]>
Date: Mon, 13 Mar 2017 14:05:04 -0700

https://support.google.com/mail/answer/81126?hl=en#authentication see
"Additional guidelines for IPv6"

Right now, it's basically an OR between the two instead of AND, though as
with any rule, there are exceptions.  We also still do "best guess" style
SPF auth, so if you set your PTR records right, you may be getting SPF for
"free" from us.

And it's always subject to change based on spamminess.

If your work around is to use ip4 to send to us, that's fine.  The
percentage of spam mail on ip6 is higher than on ip4.  Our own outgoing
code makes choices on whether to use ip6 or ip4 based on expected
authentication level, and it is ugly.  I don't like ugly, but I don't like
spam either.

Brandon

---

From: Philip Homburg <[email-masked]>
Date: Mon, 13 Mar 2017 22:21:32 +0100

>authentication level, and it is ugly.  I don't like ugly, but I don't like

At the moment, my setup is identical for IPv4 and IPv6, with of course a
preference for IPv6.

If gmail would first return a 4xy then I would never notice, because
my mail server would retry over IPv4.

Unfortunately, every now and then gmail returns a 5xy for no reason on
my side. Then I resend the same e-mail from my MUA and it gets accepted.

I don't know why gmail seems to turn transient errors into permanent failures
at the SMTP level. As far as I can tell, that is unique to gmail.

---

From: [email-masked] (Martin Rex)
Date: Tue, 14 Mar 2017 20:23:55 +0100 (CET)

Carsten Bormann wrote:
> John R Levine <[email-masked]> wrote:
>> 
>> You can complain about how stupid they are or you can upgrade
>> your mail system to use authentication methods that have been
>> IETF standards for a decade and get your mail delivered.  Your choice.
> 
> The most likely outcome is that local admins will "upgrade" our mail system

turning off IPv6 is a surprisingly (depressingly?) successful mitigation
for a number of networking problems...

-Martin