Subject: Meeting Venue Preference Survey

From: Ray Pelletier <[email-masked]>
Date: Fri, 27 Aug 2010 15:12:29 -0400

All;

Do you have IETF meeting venue preferences?  If so, the IAOC wants to know!

Please take this survey at:  https://www.surveymonkey.com/s/8HPLZGJ

Thanks!

Ray
IAD

---

From: Michael StJohns <[email-masked]>
Date: Fri, 27 Aug 2010 16:23:54 -0400

Hi Ray - 

I started to take this survey then bounced out of it on the second page.  This comes under the heading of bad survey design.

I object to the way gateway/secondary cities are defined here and specifically equating Maastricht with Minneapolis seems somewhat stacking the deck.

What I'm looking for in a meeting location is a venue with both formal and informal meeting spaces where I stand a good chance of having a good technical discussion with random people at pretty much any time of the day or night - that's my view of what has contributed to the IETF's success over the years. (Although the marathon session for the first draft of the Host Requirements document was probably stretching it) That generally means a central large hotel with attached conference space with access to non-hotel food and drink  in close proximity.  

With respect to tourism, at different times in my career, I've had different interests in the IETF.  Currently, I'm down to only a few WGs that I follow and as of the last meeting, none that I'm currently contributing to.  Considering that I'm now consulting as my main activity and paying for this on my own dime, I expect that my ratio of tourism to attendance will be somewhat skewed towards tourism, but wouldn't expect the IETF to cater to that.  My prime interest is still technical interaction and discussion.

With respect to getting there - I'm finding the trend of getting off an international plane in a gateway city and then getting onto a train for 2-5 hours somewhat worrisome.  I spent more time online for Maastricht trying to research how to get to Maastricht that I did reading IDs - and even then when I got to the Maastricht central train station, I had no luck buying a ticket for Maastricht Raandwyck.

I live in a gateway city and would prefer to go to another gateway city - but I realize that's not always feasible and not always the best venue.   

I don't know how to categorize Maastricht vs Minneapolis except to say that air connectivity is better to Minneapolis and the meeting venue has more of what I'm looking for in an IETF setup - and I can't see any way to indicate that on your survey.

To be honest, I don't think I'll find the output of this survey of much use in its current form.

Mike

At 03:12 PM 8/27/2010, Ray Pelletier wrote:
>Please take this survey at:  <https://www.surveymonkey.com/s/8HPLZGJ>https://www.surveymonkey.com/s/8HPLZGJ
>
>Thanks!
>
>Ray
>IAD 
>_______________________________________________
>Ietf mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/ietf

---

From: Michael Richardson <[email-masked]>
Date: Fri, 27 Aug 2010 16:52:32 -0400

>>>>> "Michael" == Michael StJohns <[email-masked]> writes:
    Michael> What I'm looking for in a meeting location is a venue with
    Michael> both formal and informal meeting spaces where I stand a
    Michael> good chance of having a good technical discussion with
    Michael> random people at pretty much any time of the day or night -
    Michael> that's my view of what has contributed to the IETF's
    Michael> success over the years. (Although the marathon session for
    Michael> the first draft of the Host Requirements document was
    Michael> probably stretching it) That generally means a central
    Michael> large hotel with attached conference space with access to
    Michael> non-hotel food and drink in close proximity.

+1

The hotel bars in:
    - Minneapolis
    - Chicago (sorta)
    - DC (all three meetings)
    - Atlanta
    - Vancouver
    - Memphis
    - San Francisco (2004)

were very good for this.

The lack of clear canonical bars in:
    - Vienna
    - Montreal (both times)
    - Phildelphia (I think)

were problems.  I reserve judgement on Prague (smoke), and on Pittsburgh
and Dallas, because I can't recall.  The rest I either wasn't there, or
I simply can't recall.

    Michael> With respect to tourism, at different times in my career,
    Michael> I've had different interests in the IETF.  Currently, I'm
    Michael> down to only a few WGs that I follow and as of the last
    Michael> meeting, none that I'm currently contributing to.
    Michael> Considering that I'm now consulting as my main activity and
    Michael> paying for this on my own dime, I expect that my ratio of
    Michael> tourism to attendance will be somewhat skewed towards
    Michael> tourism, but wouldn't expect the IETF to cater to that.  My
    Michael> prime interest is still technical interaction and
    Michael> discussion.

+1 on all points here.

    Michael> I don't know how to categorize Maastricht vs Minneapolis
    Michael> except to say that air connectivity is better to
    Michael> Minneapolis and the meeting venue has more of what I'm
    Michael> looking for in an IETF setup - and I can't see any way to
    Michael> indicate that on your survey.

I also have no problem with Minneapolis, and I really do want to take
the train there... 

-- 
]       He who is tired of Weird Al is tired of life!           |  firewalls  [
]   Michael Richardson, Sandelman Software Works, Ottawa, ON    |net architect[
] [email-masked] http://www.sandelman.ottawa.on.ca/ |device driver[
   Kyoto Plus: watch the video <http://www.youtube.com/watch?v=kzx1ycLXQSE>
	               then sign the petition.

---

From: Mary Barnes <[email-masked]>
Date: Fri, 27 Aug 2010 15:53:33 -0500

I had the same reaction to the Maastricht comparison to any of those
other cities in terms of equivalency. I added a comment in that
regards to my responses. I agree 100% that the question is pretty
useless if Maastricht is considered secondary.  A survey of the number
of hops (planes, trains and automobiles) that participants have to
take to each of those "secondary" venues would highlight the distinct
difference IMHO.

 I also added a comment about the fact that some of the differences in
responses in terms of tourism opportunities likely depends upon how
many sessions the individual needs to attend, how many WGs they chair
and how many WGs they are presenting in.  Asking folks that question
would really help with the analysis. My guess is that it's those of us
 that need to be in sessions pretty much solid starting as early as
7:30 am and going to beyond 10pm on the majority of the days are the
ones that are most concerned about efficiencies and the conveniences
in getting the basics of food, a safe/clean place to sleep and
Internet.

Mary.

On Fri, Aug 27, 2010 at 3:23 PM, Michael StJohns <[email-masked]> wrote:
> Hi Ray -
>
> I started to take this survey then bounced out of it on the second page.
> This comes under the heading of bad survey design.
>
> I object to the way gateway/secondary cities are defined here and
> specifically equating Maastricht with Minneapolis seems somewhat stacking
> the deck.
>
> What I'm looking for in a meeting location is a venue with both formal and
> informal meeting spaces where I stand a good chance of having a good
> technical discussion with random people at pretty much any time of the day
> or night - that's my view of what has contributed to the IETF's success over
> the years. (Although the marathon session for the first draft of the Host
> Requirements document was probably stretching it) That generally means a
> central large hotel with attached conference space with access to non-hotel
> food and drink  in close proximity.
>
> With respect to tourism, at different times in my career, I've had different
> interests in the IETF.  Currently, I'm down to only a few WGs that I follow
> and as of the last meeting, none that I'm currently contributing to.
> Considering that I'm now consulting as my main activity and paying for this
> on my own dime, I expect that my ratio of tourism to attendance will be
> somewhat skewed towards tourism, but wouldn't expect the IETF to cater to
> that.  My prime interest is still technical interaction and discussion.
>
> With respect to getting there - I'm finding the trend of getting off an
> international plane in a gateway city and then getting onto a train for 2-5
> hours somewhat worrisome.  I spent more time online for Maastricht trying to
> research how to get to Maastricht that I did reading IDs - and even then
> when I got to the Maastricht central train station, I had no luck buying a
> ticket for Maastricht Raandwyck.
>
> I live in a gateway city and would prefer to go to another gateway city -
> but I realize that's not always feasible and not always the best venue.
>
> I don't know how to categorize Maastricht vs Minneapolis except to say that
> air connectivity is better to Minneapolis and the meeting venue has more of
> what I'm looking for in an IETF setup - and I can't see any way to indicate
> that on your survey.
>

---

From: "James M. Polk" <[email-masked]>
Date: Fri, 27 Aug 2010 20:04:33 -0500

I'm going to pile on what Michael and Mary have already said, by 
saying the comparable list of cities (Minneapolis, Orlando, 
Vancouver, Barcelona, Prague) isn't even remotely close to including 
Maastricht. Each of the above cities are accessible internationally 
via air (as in: on intercontinental flights), and from many 
cities.  Maastricht has a very small airport that I'm not sure you 
can get to it outside of NL and Germany (I'm sure I'm wrong, but I'm 
not wrong by much). You certainly can't get to Maastricht from North 
America or Asian directly.

I agree with everything else Michael and Mary say as concerns, and 
mention that, like Michael, I'm not following as many WGs as I once 
did, however I am a WG chair, and have between 10-14 active IDs (in ~ 
3 to 6 WGs) at any given time - but what Michael described was very 
near what I look for in a venue/IETF destination.

James

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Mon, 30 Aug 2010 13:46:49 +0200

On 28 aug 2010, at 3:04, James M. Polk wrote:

> I'm going to pile on what Michael and Mary have already said, by saying the comparable list of cities (Minneapolis, Orlando, Vancouver, Barcelona, Prague) isn't even remotely close to including Maastricht. Each of the above cities are accessible internationally via air (as in: on intercontinental flights), and from many cities.  Maastricht has a very small airport that I'm not sure you can get to it outside of NL and Germany (I'm sure I'm wrong, but I'm not wrong by much). You certainly can't get to Maastricht from North America or Asian directly.

I've been critical about this beforehand, but let me defend Maastricht a little here.

You guys are applying American thinking here. Don't think of Maastricht as a town with an unusably small airport, but rather think of it as having a nice big airport (that would be schiphol, often called "amsterdam airport") that happens to be unusually far away from the city. If you fly into New York ground transportation is going to take a good while, too. From schiphol to Maastricht is worse, but only by a factor two or so.

Actually much of the confusion regarding travel was because there was more choice than usual: people were flying into three airports (AMS, FRA, BRU). From Frankfurt and Brussels the train travel was international, and as some people have experienced, the combination of international flying and international train travel is less than ideal. But apparently people preferred this to flying through schiphol. That's their choice. I'm pretty sure that as someone who doesn't drive going to the Anaheim meeting would have been more problematic for me than Maastricht.

Although I'm from the Netherlands I had never really visited Maastricht before, and I must say it's a very nice city. I'm looking forward to going back for a repeat visit.

The main thing I ended up disliking about this meeting venue was the location of the conference center in the middle of nowhere. Having to travel for at least 15 minutes just to buy a soda or a sandwich (outside lunch hours) was REALLY annoying.

All in all Maastricht is getting a passing grade from me, but I certainly hope that we can do a bit better in the future.

---

From: Dave CROCKER <[email-masked]>
Date: Sat, 28 Aug 2010 10:17:20 -0700

Folks,

We really need to get these surveys produced by someone with training in survey 
design.  The intent of the survey is quite reasonable, but that the construction 
of it is not.  Survey's are quite sensitive to wording and context. This suffers 
from serious problems with both.

 From the survey:

> 2. Meeting Preferences
>
> 1. It is important to me to have the meeting venue be easy to get to:
> It is important to me to have the meeting venue be easy to get to:
 > Very unimportant
> Slides

What does "slides" mean?  I'm guessing it's an extraneous entry, since it throws 
off the apparent model of a balanced 5-choice set of responses.

> 2. Do you prefer a meeting in a gateway city,

I believe the underlying problem with this question, as demonstrated by the 
postings about it so far, is the lack of consistent criteria for defining 
"gateway" and "secondary".

I'll offer the view that a "gateway" city is a principle hub of international 
air travel, while a "secondary" city should have at least some international air 
access.  I think that's a useful distinction, but it means that more than one of 
the examples of secondary, in the survey, really would be classed as tertiary or 
worse, and there's a reasonable chance that Vancouver would count as primary.

At the least, please clarify the criteria for secondary.

I should note that it's probably still possible to get useful data from that 
survey question, simply based on respondents' subjective reactions to the terms 
gateway and secondary.  Over the years, including recently, there's been enough 
chatter about the basic distinction to make the specific lists of cities 
secondary.  Just knowing folks' preferences between gateway and 'other' might be 
helpful.  That said, "primary hub" might be a better choice than "gateway"; I 
would not be surprised to find some inconsistency in the meaning different 
people impart to the word.

(There's also some question about sampling for this survey.  The main ietf list 
is widely subscribed to, of course, but not as widely as this survey ought to 
target.  I suggest sending the notice also out to ietf-announce, at the least. 
Perhaps some other lists, such as for nanog, apnic, and ripe...)

> 3. Do you prefer going back to venues or trying new venues?

As phrased, this question probably biases responses toward 'new', since they 
sound more interesting, and possibly biases it strongly.

Presenting a statement of implications about the tradeoffs -- e.g., risks of 
new, reliability of returns -- would have set the stage for the response much 
better.

> 5. Would you be willing to pay a higher registration fee to have the meeting in a gateway city?

This is a fundamentally biased (distorted) question.  It is predicated on a 
factual assertion that is unsubstantiated and very probably false.

Gateway cities have many more travel choices and many more lodging choices. 
This very probably means that total travel costs can be /lower/ than for 
secondary cities.  At the least, this means that the relationship between cost 
and city 'class' is an open question.

Further, the registration fee is only one of a set of costs.  What is important 
is the total cost, not just the narrow, localized registration fee.

The set of responses provided also is rather oddly constrained.

> 8. Would you attend if we held the IETF in Africa?
> 9. Would you attend if we held the IETF in South or Central America?

This is yet another example of a question lacking foundation.  What is the basis 
for having a meeting in a region that produces few IETF active participants? 
Perhaps the reason is compelling.  But a question like this, lacking any 
premise, can only get a random sampling of spontaneous reactions. And given the 
way humans provide such reactions, the odds are high that repeating the survey 
in a month would produce different answers to this question.

d/

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: Randall Gellens <[email-masked]>
Date: Fri, 27 Aug 2010 16:50:51 -0700

At 4:23 PM -0400 8/27/10, Michael StJohns wrote:

> specifically equating Maastricht with Minneapolis seems somewhat 
> stacking the deck.

I agree!  They are totally different in ease of access and 
availability of co-located hotel and meeting space.

>  What I'm looking for in a meeting location is a venue with both 
> formal and informal meeting spaces where I stand a good chance of 
> having a good technical discussion with random people at pretty 
> much any time of the day or night - that's my view of what has 
> contributed to the IETF's success over the years. (Although the 
> marathon session for the first draft of the Host Requirements 
> document was probably stretching it) That generally means a central 
> large hotel with attached conference space with access to non-hotel 

Yes, very well put.  I attend an IETF for the work.  I'll vacation on my own.

I'll add to this that, to me, ability to breathe is extremely 
important.  That means a smoke-free venue and some chance of finding 
a smoke-free restaurant somewhere, plus air pollution that isn't too 
severe.

Although personally I detest going to cold places, and would never do 
so for vacation, I'm happy to go to an IETF in Minneapolis or 
Vancouver in the winter, because it's not hard to get to, the venue 
works well, and restaurants are smoke-free.

>  With respect to getting there - I'm finding the trend of getting 
> off an international plane in a gateway city and then getting onto 
> a train for 2-5 hours somewhat worrisome.  I spent more time online 
> for Maastricht trying to research how to get to Maastricht that I 
> did reading IDs

Me, too, and I enlisted others to help, so it can't all be blamed on 
me being stupid.

>  I don't know how to categorize Maastricht vs Minneapolis except to 
> say that air connectivity is better to Minneapolis and the meeting 
> venue has more of what I'm looking for in an IETF setup - and I 
> can't see any way to indicate that on your survey.

Yes.

-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly selected tag: ---------------
The only normal people are the ones you don't know very well.
    --Joe Ancis

---

From: Randall Gellens <[email-masked]>
Date: Fri, 27 Aug 2010 16:54:52 -0700

At 3:53 PM -0500 8/27/10, Mary Barnes wrote:

>  I agree 100% that the question is pretty

It's not even the number of hops but the difficulty of figuring them 
out and doing them, plus elapsed time.

>  in getting the basics of food, a safe/clean place to sleep and

A good observation.  It's been perplexing how many people seem to 
prefer what I find to be difficult venues that don't work well for 
the core purpose.  I think your explanation makes sense: some people 
go for only a few WGs and hence have lots of time to be a tourist.

-------------- Randomly selected tag: ---------------
The irony of the Information Age is that it has given new
respectability to uninformed opinion.       --John Lawton

---

From: Randall Gellens <[email-masked]>
Date: Mon, 30 Aug 2010 10:40:00 -0700

At 1:46 PM +0200 8/30/10, Iljitsch van Beijnum wrote:

>  Although I'm from the Netherlands I had never really visited 
> Maastricht before, and I must say it's a very nice city. I'm 
> looking forward to going back for a repeat visit.

I think most people liked Maastricht as a city.  I can't think of 
anyone who said Maastricht as a city was ugly or smelled bad or had 
rude people.  There are a lot of truly delightful cities that would 
be great places to visit but are not good choices for an IETF.

-------------- Randomly selected tag: ---------------
Reporter (to Mahatma Gandhi): Mr Gandhi, what do you think of
     Western Civilization?
Gandhi: I think it would be a good idea.

---

From: Randall Gellens <[email-masked]>
Date: Mon, 30 Aug 2010 10:57:57 -0700

At 10:17 AM -0700 8/28/10, Dave CROCKER wrote:

>  What does "slides" mean?  I'm guessing it's an extraneous entry, 
> since it throws off the apparent model of a balanced 5-choice set 
> of responses.

I took it to mean "depends on other factors" but wasn't at all sure. 
The problem is that most of the questions had no clear answer in 
isolation, but depended on other factors.

>  I believe the underlying problem with this question, as 
> demonstrated by the postings about it so far, is the lack of 
> consistent criteria for defining "gateway" and "secondary".
>
>  I'll offer the view that a "gateway" city is a principle hub of 
> international air travel, while a "secondary" city should have at 
> least some international air access.  I think that's a useful 
> distinction, but it means that more than one of the examples of 
> secondary, in the survey, really would be classed as tertiary or 

 From an air travel point of view, Vancouver is a gateway.  There are 
non-stop flights on multiple major carriers within multiple alliances 
to multiple cities on multiple continents.  It's commonly used as a 
transfer point.  However, I'm not aware of any major carrier that 
uses it as a primary hub (e.g., LAX, DFW, ORD, JFK, LHR, AMS, HKG), 
so if this is the criteria then it doesn't qualify.

>>  9. Would you attend if we held the IETF in South or Central America?

Like the question on an earlier survey about Quebec City, I think it 
requires more information and more individual research to have a good 
answer.  Which venue in which city?  How hard is it to get to the 
city and venue?  Could I get an airfare that my company would 
approve?  Would we be in a central facility with a lot nearby, or 
would we be scattered around?  (I would personally want to know what 
the rules are for smoking, but I understand only a few other 
participants would care.)

-------------- Randomly selected tag: ---------------
Nice guys finish last, but we get to sleep in.  --Evan Davis

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Mon, 30 Aug 2010 20:20:14 +0200

> Like the question on an earlier survey about Quebec City, I think it requires more information and more individual research to have a good answer.  Which venue in which city?  How hard is it to get to the city and venue?

Basically the only thing the survey gives us is how many people would never go to a meeting on those continents regardless of the particular circumstances. There wasn't even a "why not".

A few months ago the IEEE had its ICC conference in Cape Town. I believe around 800 people attended. For me this was about 13 hours of flying (MAD-AMS-CPT), although there was no timezone change. For someone from North America that would probably be a lot longer. My flight was affordable, but only available during weekends so I was forced to stay a few extra days. I would say that the security situation in Cape Town was barely acceptable, the mobile phone infrastructure wasn't acceptable at all (almost impossible to make international calls over the mobile network, including calls to colleagues also in Cape Town) and internet access was also a huge problem but presumably the IETF or host would take care of that if we were to meet in such a place. If things are so problematic in the safer of the two biggest cities of the richest country of the continent I can't imagine the IETF having a succesful meeting elsewhere on the continent. Of course I can't know for sure after only visiting one city in Africa once.

---

From: Andrew Sullivan <[email-masked]>
Date: Tue, 31 Aug 2010 10:57:48 -0400

On Sat, Aug 28, 2010 at 10:17:20AM -0700, Dave CROCKER wrote:

> We really need to get these surveys produced by someone with training in 
> survey design.

Or stop using the surveys.  It isn't clear to me that the surveys are
a good idea at all, not just because of sampling biases and survey
design issues, but because they may lead to apparent statements of
preference that don't conform to the general principles that ought to
undergird IAOC decisions.

Surveys are a good way to make a decision look democratic, but there
are well-known and serious problems with collective preference
expression.  (If you don't believe this, you might start by looking up
Kenneth Arrow, whose exploration of this issue is the most famous.
There's a large body of work around this general topic, however.)

If we have general principles for venue selection (for instance), then
asking people to complete a survey is not the right thing to do:
either a given site can be shown to conform to the venue selection
principles, or it can't.  If it does conform, then a survey might
actually reveal a collective "preference" to go elsewhere, but that
would not be a rational preference.  (I wonder in fact whether the
decision to go to Québec will turn out this way.)

A

-- 
Andrew Sullivan
[email-masked]
Shinkuro, Inc.