Subject: IAOC Meeting location selection

From: Bob Hinden <[email-masked]>
Date: Wed, 27 May 2009 22:15:46 -0700

Given the recent discusion on travel to IETF78, I wanted to summarize  
how the IAOC makes decisions on IETF meeting locations.

We first set the dates and general geographic location about three to  
four years in advance. For example, the future meeting calendar at http://www.ietf.org/meetings/0mtg-sites.txt 
  goes out through the end of 2013. We are maintaining on average a  
rotation between Asia, North America, and Europe with the goal of no  
more than two of three meetings in North America. We do this because  
we believe the community wants the dates set well in advance and to  
have a reasonable balance between NA, Europe, and Asia.

The IAOC's most important criteria for meeting locations is where can  
we have a successful IETF meeting. That is, hold a meeting where the  
IETF can get its work done. The next items are 1) having the meeting  
and hotel rooms in the same building or a conference center next to the
hotels, 2) direct international flights, 3) reasonable costs for the  
IETF and attendees, and 4) the ability to deploy an IETF style network  
in the venue. This list isn't ordered and we don't always achieve all  
of this for every meeting, but these are the goals. There are always  
tradeoffs.

If we have a host for a specific meeting and the host has a location  
preference we try to honor their preference. Having a host for a  
meeting significantly reduces the cost to have an IETF meeting. This  
is especially important for meetings outside of North America where  
the meetings rooms are not included with a block of hotel rooms. Due  
to the current Visa situation in the US we are planning many meetings  
outside of the US.

In the case of hosted meetings we might relax some of our criteria  
like direct flights or hotel and meeting rooms in the same building,  
but we will never pick a venue where we don't think we can have a  
successful IETF meeting. For non-hosted meetings, we try to pick  
convenient locations and vary the locations. That is, we try to not  
always meet in the same city in the same geographic area.

For example, we are currently trying to decide between holding a  
meeting in Vancouver and Quebec City. We are confident both could have  
a successful IETF meeting and both have hotel and meeting space for  
the dates we are looking for, but there are differences. Note, we  
looked at other Canadian cities, but these were the only cities where  
we could find acceptable hotel and meeting space for our dates. We  
have had meetings in Vancouver twice before and it meets most of the
above criteria, but at the same time Quebec City appears to be a nice  
new place to meet but doesn't have as many direct international  
flights. We decided to ask the community for feedback in the form of a  
poll. I would note that the current results show a strong leaning
toward Quebec City and this is somewhat at odds with some of the email  
thread on IETF78 in that many people seem to be willing to trade some  
travel inconvenience for a new venue. We will publish the results of  
the poll when it is completed on June 1.

In the case of Hiroshima, that is being hosted by WIDE, we know there  
will be challenges for some to travel between Tokyo and Hiroshima, we  
are working with the host to provide a lot of information to make this  
as painless as possible. There is already a link on the IETF web page
with a link to this information. We are planning something similar for  
Maastricht. Also note, that we were originally planning another  
location in Asia but the original sponsor was not able to do the  
meeting, the WIDE group helped us out to find a location in Japan.  
There are other possible locations in Japan, but Hiroshima was the  
only location that had availability on our dates.

For the Masstricht meeting we had a potential host who wanted to  
sponsor IETF78 in the Netherlands. Three venues in the Netherlands  
were available on the dates we wanted, but the other two were  
significantly more expensive. Consequently, we choose Masstricht
knowing that the local travel would be more complex.

The process inside of the IAOC for venue selection is two fold. We  
have a meetings committee that includes the IAD, IAOC members, NOC and  
community volunteers, and the secretariat. Most of the work happens in  
this committee and they produce a list of choices for a meeting in a
specific geographic location and meeting dates. There is normally a  
site visit if it is a new IETF location. The meeting committee  
prepares a summary of the choices including costs, along with their  
recommendation. These are presented to the IAOC as a whole. Based on  
this input the IAOC makes the final selection. Once that is done the  
IAD and the secretariat work with the venue to develop a contract. The  
IAOC will also approve the contract.

If there are people interested in joining the meeting committee please  
let me know. We are always on the lookout for people who have a lot of  
personal travel and meeting planning experience.

Please let me know if you have additional questions. I expect to get a  
lot of questions at the Stockholm plenary :-)

Bob Hinden
IAOC Chair

---

From: John C Klensin <[email-masked]>
Date: Thu, 28 May 2009 15:39:17 -0400

Bob,

I'd like to express my thanks for this information.  I believe
that, if the community had more information of this sort,
updated as needed, we would have fewer firestorms when meeting
sites are announced.

When you have time, I (and I believe others) would like to
understand better how you evaluate "reasonable costs for the
IETF and attendees".   I think it is general knowledge that it
is possible to trade IETF costs off against participant costs
(e.g., making things cheaper for the IETF and more expensive for
participants).  It would be good to know how the participant
costs are estimated and how the two are balanced.

Similarly, there is presumably a point at which sponsor
preferences have to give way to other considerations.  Could you
give us a sense as to when that occurs?  Put differently, at
what point might you decide that dealing with a particular
potential sponsor's preferences are sufficiently inflexible
and/or onerous that you lose interest in sponsorship from that
organization? It appears to me, for example, that the IAOC has,
more than once, dropped both the "same facility" and "direct
international flights" requirements, and pushed the boundaries
of the "participant costs" one in order to get a sponsor and
accommodate their needs.    That isn't necessarily the wrong
decision, but I believe the community should be better
informed... and that we would have much less 
ruckus if it were.

regards and thanks again,
    john

--On Wednesday, May 27, 2009 22:15 -0700 Bob Hinden
<[email-masked]> wrote:

> Given the recent discusion on travel to IETF78, I wanted to
> summarize how the IAOC makes decisions on IETF meeting
> locations.
> 
> We first set the dates and general geographic location about
> three to four years in advance. For example, the future
> meeting calendar at
> http://www.ietf.org/meetings/0mtg-sites.txt  goes out through
> the end of 2013. We are maintaining on average a rotation
> between Asia, North America, and Europe with the goal of no
> more than two of three meetings in North America. We do this
> because we believe the community wants the dates set well in
> advance and to have a reasonable balance between NA, Europe,
> and Asia.
> 
> The IAOC's most important criteria for meeting locations is
> where can we have a successful IETF meeting. That is, hold a
> meeting where the IETF can get its work done. The next items
> are 1) having the meeting and hotel rooms in the same building
> or a conference center next to the
> hotels, 2) direct international flights, 3) reasonable costs
> for the IETF and attendees, and 4) the ability to deploy an
> IETF style network in the venue. This list isn't ordered and
> we don't always achieve all of this for every meeting, but
> these are the goals. There are always tradeoffs.
> 
> If we have a host for a specific meeting and the host has a
> location preference we try to honor their preference. Having a
> host for a meeting significantly reduces the cost to have an
> IETF meeting. This is especially important for meetings
> outside of North America where the meetings rooms are not
> included with a block of hotel rooms. Due to the current Visa
> situation in the US we are planning many meetings outside of
> the US.
> 
> In the case of hosted meetings we might relax some of our
> criteria like direct flights or hotel and meeting rooms in the
> same building, but we will never pick a venue where we don't
> think we can have a successful IETF meeting. For non-hosted
>...

---

From: David Kessens <[email-masked]>
Date: Fri, 29 May 2009 15:24:14 -0700

John,

> costs are estimated and how the two are balanced.

While I appreciate your interest in such details, I believe we should
also be realistic in how precise such trade-offs can be made. Eg. the
IAOC just doesn't have the staff or budget to create detailed models
on the cost/benefit analysis and the IAOC would easily end up in a
situation where more money/energy gets spend to try to determine the
optimum site location and to fully inform the community on every
decision it makes. Basically, what I expect the IAOC to do is to make
a judgement call based on the various variables.

So far, I don't believe there is any evidence that IAOC has made
decisions to hold meetings in places that are completely out of the
question. At the same time, by occasionally holding a meeting in a
location that is somewhat unusual, we will continue to be able to
evaluate whether all our assumptions about the importance of airline
hubs or the desirability of a short train trip are really that
important for all participants.

Basically, instead of endless discussions on the merits of a required
short railtrip from 1 of 3 different very large airports that are all
listed very high in the european top 10 hub airports, let's go to
Maastricht and see how it will work out.

David Kessens
---

---

From: Fred Baker <[email-masked]>
Date: Fri, 29 May 2009 14:34:03 -0700

I don't think we have a specific set of requirements from the  
community. Dave Crocker and some friends started putting some notes  
together more than a decade ago, and last I spoke with him about it he  
had dropped it because he had been unable to form a consensus. Maybe  
that needs to be renewed. What we do understand we know as much as  
anything because we ourselves are IETF participants. As Bob put it,  
the first requirement is that the meeting be effective, and the direct  
implications of that include that it be reasonably accessible at a  
rational cost, that it have a meeting facility with an appropriate set  
of rooms, that it have enough hotels and restaurants and that they be  
rationally located with respect to the meeting facility, and so on.  
After that, we get into a lot of minutia of the costs of this and that  
and the desires of the sponsors. There are also some purely business  
issues that come up - sometimes we can horse-trade prices when  
comparing locations.

Taking one example that you mention in your note, I don't believe that  
the IAOC understands the community requirement to be "direct  
international flight" - we don't bend a rule there, we understand the  
rule to be something different. If we did think the rule was "direct  
international flight" and interpreted that literally - "for any given  
meeting, a participant from Australia, New Zealand, Korea, China,  
Japan, any European country, Canada, or the US needs to be able to get  
to the exact location using at most one international flight and take  
no additional leg afterwards" - I think the set of places we could  
meet would be very small, probably the null set, simply because of the  
number of places we come from and their various geographies. What we  
do understand the requirement to be is that the number, cost, and  
convenience of connections, along with other costs and issues, be  
optimized for the average participant, and yes we try to do that to  
the extent that we can.

In consideration of that, look at the question we asked about  
Vancouver vs Quebec City. Ray will give the results Monday when the  
poll closes, but at this instant we are running about 2:1 in favor of  
Quebec City. For our Asian colleagues, that means flying to a major US  
or Canadian airport like EWR, ORD, or YYQ, and taking another flight  
to YQB. For our European colleagues, there are a few direct flights  
from Europe, but for the most part they will similarly go to New  
Jersey, Chicago, Montreal, or Toronto, and then on to YQB; from ARN to  
YQB int.net finds 44 paths but only five that have a single stop, and  
those are all via ORD or EWR. In the US, folks will fly to one of a  
few major hubs and fly on to YQB. For me personally, itn.net tells me  
that I will fly from SBA to a major hub like LAX or SFO, from there to  
a hub near the US/Canadian border like ORD, DTW, YYZ, or EWR, and then  
proceed to Quebec City - three legs, and I will probably do it the day  
before rather than trying to do it in one day. I would expect that our  
ANZ colleagues will follow me - a long hop to LAX or SFO, another to  
the US border with Canada, and a final hop. Vancouver is a major  
international hub, but Quebec City is not; like Minneapolis, Dublin,  
Prague, San Diego, Dallas, Vienna, Stockholm, Hiroshima, Atlanta,  
Orlando, and Maastricht, it is a place one gets to one hop beyond a  
major hub. And what we are being told is - "Vancouver is nice, but  
let's go to Quebec City".

On May 28, 2009, at 12:39 PM, John C Klensin wrote:

> _______________________________________________
> Ietf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ietf

---

From: Andrew Sullivan <[email-masked]>
Date: Fri, 29 May 2009 23:23:26 -0400

On Fri, May 29, 2009 at 02:34:03PM -0700, Fred Baker wrote:

> our Asian colleagues, that means flying to a major US or Canadian airport 
> like EWR, ORD, or YYQ, and taking another flight to YQB. 

[other air routings elided]

Also, note, the passenger rail service to Québec from either Montréal
or Toronto is not frequent; not real fast; not convenient; and not
especially cheap, given that Canada partly came into existence as an
excuse to run rails across the continent.  So your alternative options
for travel are somewhat limited [1].

The above diversion aside, I think there are really two topics being
discussed, not always separately, in this thread, and I think the one
I (at least) think is more important is not getting exposed as well as
it might.

1.  The IAOC sometimes picks places that are $wrong_somehow.  I find
this issue hard to get worked up about, and I find I have little
sympathy with arguments talking about how inconvenient travel to
$location is.  I'm aware I don't have the onerous travel obligations
some do, and perhaps there is a real problem I don't understand.
Nevertheless, if we're going to have single large physical meetings of
international contributors, some people are always going to have to be
inconvenienced.  It's good for the community, I think, if sometimes
those of us from North America (and large North American cities, at
that) are the inconvenienced ones.  

2.  It is hard to understand what the IAOC is doing, and what factors
weigh in their decisions.  This is partly because the IAOC minutes
aren't published as quickly as perhaps some would like; and, when they
are, they do not contain as much detail as some might like, either.
(This basic issue is doing a lot of the important work in John
Klensin's arguments, for instance, and though he's stated this fairly
clearly, I don't have the impression it's getting as much attention as
it ought.)  To the extent we think this issue is a real one, it seems
to me that it's important to get this one fixed.  

If I am right about these two different topics, then (if we are to
continue this thread) I'd prefer to hear clearly stated arguments --
and suggestions for improvements -- about (2).  If (2) is true, then
(1) is just a symptom, and not a compelling one for some of us.

Best regards,

Andrew

[1] That said, if you can put up with the hassles of getting to the
train from wherever you land, and the fair chance that your train will
not have functioning climate control or will fail to run on time (or
both), some of the route is pretty and worth looking at from a train.
Also, Via Rail's seats are quite comfortable and, in my experience,
the staff are loads more cheerful than the surly lot that seems to
have taken over Air Canada in recent years.  Since this is not yet
officially the grouse-about-lousy-travel-options list, however, I'll
stop with my reviews now.  

-- 
Andrew Sullivan
[email-masked]
Shinkuro, Inc.

---

From: Andrew Sullivan <[email-masked]>
Date: Fri, 29 May 2009 23:29:03 -0400

On Fri, May 29, 2009 at 11:23:26PM -0400, Andrew Sullivan wrote:

> discussed, not always separately, in this thread, 

I should have said "these threads", of course -- I was implicitly
tying this discussion to the recent discussion of particular sites for
meetings.  But the general point is still that the process issue about
the community understanding of how things get decided is distinct from
(and more important than) the decision process itself or actual
selection.

A

-- 
Andrew Sullivan
[email-masked]
Shinkuro, Inc.

---

From: John C Klensin <[email-masked]>
Date: Sat, 30 May 2009 09:16:05 -0400

--On Friday, May 29, 2009 23:29 -0400 Andrew Sullivan
<[email-masked]> wrote:

> I should have said "these threads", of course -- I was
> implicitly tying this discussion to the recent discussion of
> particular sites for meetings.  But the general point is still
> that the process issue about the community understanding of
> how things get decided is distinct from (and more important
> than) the decision process itself or actual selection.

Yes, exactly.
    john

---

From: John C Klensin <[email-masked]>
Date: Sat, 30 May 2009 13:05:22 -0400

--On Friday, May 29, 2009 15:24 -0700 David Kessens
<[email-masked]> wrote:

>> (e.g., making things cheaper for the IETF and more expensive
>> for participants).  It would be good to know how the
>> participant costs are estimated and how the two are balanced.
 
> While I appreciate your interest in such details, I believe we
> should also be realistic in how precise such trade-offs can be
> made. Eg. the IAOC just doesn't have the staff or budget to
> create detailed models on the cost/benefit analysis and the
> IAOC would easily end up in a situation where more
> money/energy gets spend to try to determine the optimum site
> location and to fully inform the community on every decision
> it makes. Basically, what I expect the IAOC to do is to make a
> judgement call based on the various variables.

Please reread what I wrote.   I didn't ask for "precise
trade-offs".  I didn't ask for "details models on the
cost/benefit analysis".  

Bob said that they looked at "participant costs".   I'd like to
know what they examine in that process.   I care less about how
they make the calculation / judgment call than I do about
whether "participant costs" equals "hotel room rates plus
registration fee" or whether there is some effort to consider,
e.g.,

	* Costs of getting from the airport to the hotel.
	Sometimes those are significant, sometimes not.
	
	* Costs of eating around the venue.  We've been in
	places where the only real options are hotel restaurants
	or long cab rides and other places where the hotel and
	meeting facilities are surrounded by a large range of
	options.
	
	* Costs of transport to/from the venue.  I have no idea
	how to actually calculate that without a model of the
	IETF population and where we come from and am _not_
	suggesting constructing such a model.  But there are
	places to which airfares are traditionally very high
	relative to other places, either due to distance or to
	the vicinity being served by only one character, and I
	wonder whether IAOC is trying to consider it.

Now, in each of those cases, I am actually asking a very simple
question.  Both your note and several private ones I've gotten
seem to think I'm asking "please expose the complex mathematical
model you are using and what the parameters are for each site
considered".  I'm not and never have.  For each of the above and
anything else that the IAOC or its meeting committee think might
be part of "participant costs", I'd like to know:

	(1) Is this factor being considered?  
	
	(2) If so, has there been some attempt at data-gathering
	or estimation, or is the judgment purely subjective?

Please note that those two questions have "yes" or "no" answers.
They don't even require explanations.

It is possible that, once the community gets those answers,
there will be a discussion about whether the IAOC's optimization
criteria agree with those that the community prefers.  As an
example, if the "participant cost" category is considering only
hotel room costs, I think we might have a discussion about
registration fees.  I would consider that discussion
constructive as long as it was about principles rather than an
attempt at micromanagement.  I hope the IAOC would consider it
constructive too.

For completeness, I hope that the IAOC is also considering
"participant comfort" and "environmental livability".  I note
that we are now getting information about construction projects.
That is a significant improvement but, like it or not, it took
one of these flaps about the decision-making process before the
information started being shared with the community and, I
suspect (but don't know because we weren't told), before the
question started being asked aggressively.   We've also had long
discussions about air quality, safety, smoking conditions, and
availability of options for particular diets.   Again, my
concern is whether those questions are being asked in the IAOC
and the answered considered and _not_ what the answers are.
And, like the question above, the questions have simple, binary,
answers and do not require analysis and detailed models.

The _only_ question I've asked that can't be answered in such a
way is how "participant costs" and "IETF costs" are compared and
weighted.  Without answers to the questions above, that question
is meaningless (which is part of the reason for asking those
questions).  And I realize that the answer to this one is likely
to be entirely subjective... and have no problem with that.
But, come on... it really is possible for the IAOC to answer,
however subjectively and informally, whether a site would be
chosen because it saves the IETF a lot of money even if the
difficult-to-measure participant costs were very high.  I assume
the answer is "no", but then --absent the extensive models that
I'm _still_ not asking for, it is, IMO, reasonable for the
community to ask the folks in the selection process to say a few
words about how they think about that balance.
 
> So far, I don't believe there is any evidence that IAOC has
> made decisions to hold meetings in places that are completely
> out of the question. At the same time, by occasionally holding
> a meeting in a location that is somewhat unusual, we will
> continue to be able to evaluate whether all our assumptions
> about the importance of airline hubs or the desirability of a
> short train trip are really that important for all
> participants.

Absolutely.

As a partial response to Fred's comments, I don't believe that
the community has ever supplied the IAOC with consensus about a
consistent and well-defined set of criteria either.  I also
don't believe that convening a WG to try to develop such
criteria would be either productive or a good use of time.  The
effort by Dave Crocker and others that Fred mentioned was
strongly influenced by the advantages of picking a handful of
locations and simply rotating among them.  I think that many of
us, while recognizing the advantages of that approach, still
prefer a larger number of locations.   

As another example, there used to be a principle that we meet in
locations from which there were IETF participants and probably
still is.  But "location" seems to have evolved from "city" to
"country" to "continent".  I don't know how the community feels
about that and I don't think the IAOC does either.

But the only ways I know of to work these problems are either
for the community to assume that the IAOC will read our
collective minds (I don't think the IAOC members would claim the
ability to do that), and make the right decisions on that basis,
and have us quietly accept any such decision or for us to engage
in a running dialog that gradually refine and continually adjust
criteria and IAOC perceptions of them.  I hope that can be kept
constructive and can avoid getting into micro-details of
particular meetings or ongoing second-guessing.  

But I don't think that the risk that some members of the
community will go down those paths, or even the perception that
some have, is an adequate reason to try to eliminate the
discussions.  And I continue to believe that more spontaneously
and timely openness on the part of the IAOC --as BCP 101 appears
to me to require-- would head off most of the "why did you do
this and why aren't you telling us" part of the discussion.

> Basically, instead of endless discussions on the merits of a
> required short railtrip from 1 of 3 different very large
> airports that are all listed very high in the european top 10
> hub airports, let's go to Maastricht and see how it will work
> out.

David, again, please reread what I've said.  I haven't opposed
going to Maastricht.  I asked what happened to the international
airport criterion and pointed out (as did others) that asking
someone to transfer to a train connection after a long flight
are not quite the same as asking someone to change planes.  

But your comment does identify the other difficulty I see here.
"See how it will work out" is entirely reasonable if we are
learning from these experiences.  But, as far as I can remember,
the community has never been given a retrospective review on a
previous meeting and what has been learned from it.   Ray does
surveys after every meeting now, but I can't recall the
community ever being given a report on the results of those
surveys and what was learned from them.   I do remember one
conversation in which I encouraged a plenary discussion of some
issues with a meeting while it was still in progress and was
told it needed to be avoided because it would embarrass the host
but don't believe there was any follow-up, at least outside IAOC
private discussions.

It may be that the IAOC is doing those evaluations, learning
from them, and making adjustments.  I assume that they are.  But
the IAOC is not supposed to be operating in "trust us, we have
knowledge you don't and we are in charge" mode.   And it is
ultimately variations on that mode to which I'm objecting.

So, given your comment, I'd like to make a specific proposal.
Let's "go to Maastricht and see how it will work out".  Let's
also go to Stockholm and Hiroshima and see how they will work
out.  But let's also put a report from the IAOC on how things
worked out, what, if anything, was learned, and what adjustments
are contemplated for the future on the agenda for the IAOC
plenary report at IETF 76 for Stockholm and at IETF 77 for
Hiroshima.  To make time for that, let's ask the IAOC to move
the usual statistical and financial part of their report to
their web page at least a week before the IETF meeting starts
and call people's attention to it so that questions can be asked
if needed but no meeting time is taken up reciting numbers.

Does that makes sense, or do you still think I'm leading either
a effort to second-guess the Maastricht decision or to force the
IAOC into complex mathematical or financial modeling of every
site under consideration?

best,
     john