Subject: Re: IETF Website Degradation

From: Adam Roach <[email-masked]>
Date: Mon, 03 Aug 2015 17:26:29 -0500

Incidentally -- and I don't know if you turned this on or if Cloudflare 
did -- but the interstitial page that appears when accessing IETF pages 
is described by Cloudflare like this: " Javascript and cookies are 
required for the tests and recording the fact that the tests were 
correctly passed."

I know some significant percentage of IETFers run with javascript turned 
off, so this may pose a hardship for them.

/a

On 8/3/15 17:03, Glen wrote:
> All -
>
> We are experiencing degradation on the websites served by the primary 
> IETF server group, including the IETF main website, the IAB and IRTF 
> sites, and the Datatracker.
>
> Page load times for all sites are extremely slow.
>
> We have engineers working on this now.
>
> The problem *seems* to be that Cloudflare is sending us hundreds of 
> simultaneous connections from hundreds of different IP addresses 
> across many netblocks.  We are contacting Cloudflare now to see why 
> this is happening, and what can be done to mitigate it, and we are 
> continuing to investigate options on our end to restore normal service.
>
> Other services and sites are not impacted, and one way or another we 
> will have service restored shortly.  Thank you for your patience in 
> this matter.
>
> Glen
> Glen Barney
> IT Director
> AMS (IETF Secretariat)
>

---

From: Glen <[email-masked]>
Date: Mon, 3 Aug 2015 15:33:41 -0700

Hi Adam -

Totally understood.

Alas, at the moment, without this mode, *nobody* could access the site.
:-)  So as a hopefully very temporary band-aid, I need to leave this on for
now.

To all IETF participants and all those impacted by this, I'm really sorry
for the inconvenience (and I am - it impacted me as well! ) and as soon as
we can get this shut off safely we will do so.  I'm hoping this won't be
long.  We will try turning it off several times this evening, and see if
the attack has stopped; however, given Cloudflare's east coast location, it
may be tomorrow morning (roughly 14 hours from now) before we can reach a
human at Cloudflare to find out what's really going on.

Also, as a reminder, please note Adam's example:  I'm not on the IETF
discussion list at the moment, so please email me directly if you need to
reach me, as Adam did.

Thanks again for your patience!

Glen
Glen Barney
IT Director
AMS (IETF Secretariat)

On Mon, Aug 3, 2015 at 3:26 PM, Adam Roach <[email-masked]> wrote:

> did -- but the interstitial page that appears when accessing IETF pages is
> described by Cloudflare like this: " Javascript and cookies are required
> for the tests and recording the fact that the tests were correctly passed."
>
> We are experiencing degradation on the websites served by the primary IETF
> server group, including the IETF main website, the IAB and IRTF sites, and
> the Datatracker.
>
> simultaneous connections from hundreds of different IP addresses across
> many netblocks.  We are contacting Cloudflare now to see why this is
> happening, and what can be done to mitigate it, and we are continuing to
> investigate options on our end to restore normal service.
>
> Other services and sites are not impacted, and one way or another we will
> have service restored shortly.  Thank you for your patience in this matter.
>
> Glen
> Glen Barney
> IT Director
> AMS (IETF Secretariat)
>
>
>