Subject: Re: IETF 107 Virtual Meeting Survey Report

From: Abdussalam Baryun <[email-masked]>
Date: Fri, 17 Apr 2020 07:28:59 +0200

Hi Jay,

I am interested to know the percentage of who answered the survey from
Africa. Also to know percentage from South America, Asia and Europe if
available.

Best Regards
AB

On Fri, Apr 17, 2020 at 4:49 AM IETF Executive Director <
[email-masked]> wrote:

> The results from the post-meeting survey for IETF 107 Virtual are now
> available [1].  This is the default export format from our survey
> application with some minor redactions where text responses have been used
> to identify individuals or reference external websites.
>
> Given the extraordinary circumstances around IETF 107 Virtual and the
> meeting being entirely online, the questions were focused primarily on the
> online experience and the individual components that delivered that
> experience.
>
> There were 273 respondents, of which 227 participated in IETF 107 Virtual
> and we estimate the total number of participants for that meeting as 701.
>
> Some highlights include:
> * 53% of those that answered the survey are from the US or Canada
> * 73% (+/-9.5%) of those that attended IETF 107 Virtual were satisfied or
> very satisfied with the meeting.
> * The guides and testing sessions provided before the meeting were well
> used and well received.  While we did not ask directly, we can assume they
> contributed to the success of the meetings.
> * The combination of Webex/etherpad/jabber was largely satisfactory but
> significant improvements in the overall experience can be made, in
> particular in the area of integration, account management and client
> features (all of which Meetecho addresses).
>
> It is our plan for our larger surveys like this one, to make a sanitised
> version of the raw data available to enable better analysis by both us and
> the community, but this takes a lot of work and we are not in a position to
> do that for this survey. This survey was run in anonymous mode and so no
> data was collected other than that explicitly provided.
>
> The use of data for evidence-based decision making is a key strategic aim
> of the LLC and I would welcome contact from anyone who wants to help with
> that, whether it be with survey design, data transformation or results
> analysis.
>
> If you have any questions or comments then please feel free to contact me
> directly.
>
> [1]  https://www.ietf.org/media/documents/ietf-107-survey-results.pdf
>
> --
> Jay Daley
> IETF Executive Director
> [email-masked]
>
> _______________________________________________
> IETF-Announce mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ietf-announce
>

---

From: Abdussalam Baryun <[email-masked]>
Date: Fri, 17 Apr 2020 08:11:17 +0200

ok, by looking into percentages in the report [1], Africa 1.47 Asia 9.52
Europe 30.77 south America 2.56 but it seems not correct. I mean by Africa
all Africa including Egypt, I think many people don't use the word *middle
east* only in politics, and I don't think it is clear when collecting data
for analysis for communities. However, there is this Middle East issue, I
really don't think it is right, because some countries use this word and
some countries of this word *middle East* are from Asia and/or Africa and
maybe also Europe.  So it makes it difficult to know true percentages and
information. IMO, it is best to collect data and call places due
to continents of the world. Therefore, the percentage of 1.47 for Africa is
not correct/complete.

 IMO the analysis is very important for making information to knowledge and
then to ietf decision making. We can not know best practice and
satisfaction only by survey and analysis. I would like to participate for
the design and analysis.

best

AB

---

From: Bob Hinden <[email-masked]>
Date: Fri, 17 Apr 2020 09:58:58 -0700

Jay,

Thanks for sharing the results of the survey.

One thing you didn’t call out was Question 6 "Why didn't you participate in the IETF 107 Virtual meeting? (check all that apply)”,  The two responses that I found interesting were:

"There were no sessions of interest to me"  41%

The IESG choose to only schedule a few w.g. sessions, I suspect there would have been more participation if more w.g sessions has been scheduled.  For example, I only attended one BOF and the Plenary.

"The time of day of the meeting was too difficult for me to participate"  47%

Almost half of the responses said this.  This seems like an inherent problem with virtual meetings, probably accounts for the answers to question 1, "In what region do you live?”.   This showed 53% from US and Canada, and 31% from Europe.   Very little elsewhere.   I wish there was a solution to this, but it’s clearly going to have a significant effect on IETF attendance until we can start having face to face meetings again.

Bob

> On Apr 16, 2020, at 7:48 PM, IETF Executive Director <[email-masked]> wrote:
> 
> The results from the post-meeting survey for IETF 107 Virtual are now available [1].  This is the default export format from our survey application with some minor redactions where text responses have been used to identify individuals or reference external websites.
> 
> Given the extraordinary circumstances around IETF 107 Virtual and the meeting being entirely online, the questions were focused primarily on the online experience and the individual components that delivered that experience.
> 
> There were 273 respondents, of which 227 participated in IETF 107 Virtual and we estimate the total number of participants for that meeting as 701.
> 
> Some highlights include:
> * 73% (+/-9.5%) of those that attended IETF 107 Virtual were satisfied or very satisfied with the meeting.
> * The guides and testing sessions provided before the meeting were well used and well received.  While we did not ask directly, we can assume they contributed to the success of the meetings.
> * The combination of Webex/etherpad/jabber was largely satisfactory but significant improvements in the overall experience can be made, in particular in the area of integration, account management and client features (all of which Meetecho addresses).
> 
> It is our plan for our larger surveys like this one, to make a sanitised version of the raw data available to enable better analysis by both us and the community, but this takes a lot of work and we are not in a position to do that for this survey. This survey was run in anonymous mode and so no data was collected other than that explicitly provided.
> 
> The use of data for evidence-based decision making is a key strategic aim of the LLC and I would welcome contact from anyone who wants to help with that, whether it be with survey design, data transformation or results analysis.
> 
> If you have any questions or comments then please feel free to contact me directly.
>

---

From: "Adrian Farrel" <[email-masked]>
Date: Fri, 17 Apr 2020 18:23:57 +0100

> Almost half of the responses said this.  This seems like an inherent
> problem with virtual meetings, probably accounts for the answers
> to question 1, "In what region do you live?”.   This showed 53%
> from US and Canada, and 31% from Europe.   Very little elsewhere.
> I wish there was a solution to this, but it’s clearly going to have a
> significant effect on IETF attendance until we can start having face
> to face meetings again.

Very good points, Bob.
It may see perverse, but I found it hard to justify a time zone change (i.e., staying up late) for a two hour virtual meeting. It is always a lot easier to think about doing that for five days of wall-to-wall meetings with useful corridors and side meetings.
I suspect that if there had been more virtual meetings scheduled, I might have been more inclined to spend more time on West Coast time.

Madrid is a difficult measure to use on me because it's only an hour off my time zone. However, if I was to attend remotely, I would certainly like to see a full agenda. Whether that is necessarily over five days or perhaps a less rushed six or seven days, could be debated. 

I must say that I am finding the current tribble of "IETF 107" interim meetings a bit difficult. Maybe my fault for following so many WGs, but having to monitor when meetings are, finding agendas showing up late, and attending out-of-hours meetings every few days, is a bind. We could probably do better.

A

---

From: Keith Moore <[email-masked]>
Date: Fri, 17 Apr 2020 13:27:54 -0400

> Almost half of the responses said this.  This seems like an inherent problem with virtual meetings, probably accounts for the answers to question 1, "In what region do you live?”.   This showed 53%…

I suspect there's a way to structure virtual meetings to enable more 
effective participation from a wider variety of locations.

For example: divide up the meeting into separate presentation and 
discussion sessions.   Record the presentations at least a couple of 
days in advance and make them available for viewing and/or download.  
Schedule two discussion sessions on the same material, say 12 hours 
apart, so that at least one discussion session can likely be attended by 
everyone.   Record the discussion sessions also, and send transcripts to 
the mailing lists.   Consider making the discussion sessions chat 
sessions (but make sure that the chat sessions are web-accessible in 
addition to being jabber-accessible or whatever).

Keith

---

From: Stewart Bryant <[email-masked]>
Date: Fri, 17 Apr 2020 20:00:43 +0100

Sent from my iPad

> On 17 Apr 2020, at 18:33, Adrian Farrel <[email-masked]> wrote:
> 

I completely agree. Shambles is the word that comes to mind.

Stewart

---

From: Jay Daley <[email-masked]>
Date: Sat, 18 Apr 2020 07:35:45 +1200

Bob

> On 18/04/2020, at 4:59 AM, Bob Hinden <[email-masked]> wrote:
> 
> Jay,
> 

We need to be cautious about that question because the likely margin of error, which we can only guess at, is high. For example, if we assume the total population of those that could have attended but chose not to is 3000 (I really have no idea) then 41% is roughly 27%-55% and 47% is roughly 33%-61%. 

> The IESG choose to only schedule a few w.g. sessions, I suspect there would have been more participation if more w.g sessions has been scheduled.  For example, I only attended one BOF and the Plena…

Having said that, I agree with your analysis on this point and I should note that a more thorough survey about virtual meetings in general is being prepared to provide better data for planning purposes. 

> Almost half of the responses said this.  This seems like an inherent problem with virtual meetings, probably accounts for the answers to question 1, "In what region do you live?”.   This showed 53%…

I have not compared this distribution to that of other meetings to see if the difference is significant. 

Jay

-- 
Jay Daley
IETF Executive Director

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 18 Apr 2020 08:15:41 +1200

> effective participation from a wider variety of locations.

Sent at 05:27 in my time zone. Replying at 08:10 in my time zone, to observe
that this is exactly why the IETF's decision making occurs on mailing lists,
or the data tracker, not in meetings. We should make more use of that.

For example, I've seen people asking for "presentation" slots in virtual
meetings. No! We can all read drafts and slides at any time we choose.
I'd even accept slides with audio commentary. But I'm not going to get
up at 3 a.m. to hear someone reading out their slides. Shouldn't we be
 
> addition to being jabber-accessible or whatever).

Yes.
   Brian

---

From: Keith Moore <[email-masked]>
Date: Fri, 17 Apr 2020 16:28:09 -0400

On 4/17/20 4:15 PM, Brian E Carpenter wrote:

> up at 3 a.m. to hear someone reading out their slides.

One of the most important purposes of the f2f meetings has been that the 
3x yearly meetings have served as deadlines to revise work.   I suspect 
that we still need deadlines.

But we don't need presentation times.   They have always been a huge 
waste of f2f meeting time.   Now that we're not likely to be meeting f2f 
for another year or two, is a good time to get rid of them.   Even when 
we go back to meeting f2f, prepare the presentations in advance and let 
us view them before we show up at the meeting.   For that matter, skip 
the presentation entirely (it's much faster to read than to listen to 
speech) and just post the slides (which are, admittedly, sometimes 
easier to digest than the I-Ds).   Add comments to the slides if it helps.

Keith

---

From: Carsten Bormann <[email-masked]>
Date: Sat, 18 Apr 2020 03:21:38 +0200

On 2020-04-17, at 22:28, Keith Moore <[email-masked]> wrote:
> 
> But we don't need presentation times.   They have always been a huge waste of f2f meeting time.  

Yes, we have sat through sequences of bad presentations.

The problem with bad presentations is not that they are presentations, it’s that they are bad.

We are doing these meetings to advance consensus on technical issues.  These issues often are complex.  **A presentation is often the best way to make a complex technical argument; challenging the points of a presentation in real time is often by far the best way to make a counter-argument.**

> Now that we're not likely to be meeting f2f for another year or two, is a good time to get rid of them.   Even when we go back to meeting f2f, prepare the presentations in advance and let us view them before we show up at the meeting.  

Yeah, right.  This is a prime example of process confabulation.  We SHOULD all be perfectly prepared when going into meetings.  Well, if everyone were perfectly prepared, everyone already would agree on the right decisions and we would already have consensus!  That’s just not the way things actually work.

> For that matter, skip the presentation entirely (it's much faster to read than to listen to speech) and just post the slides (which are, admittedly, sometimes easier to digest than the I-Ds).   Add comments to the slides if it helps.

Yes, we can take a lot of things into the asynchronous domain, like we did with mail already.  Moving tutorial-like and status-update background stuff into “meeting materials” that must be consumed ahead of the meeting (did you notice the RFC 2119 MUST here?  Hah.  Let’s have entrance exams for WG meetings.).  Doing things synchronously in a meeting can be more efficient in ensuring there is a common basis for discussions.

I have been through many bad discussions.  Where most of the people at the microphone are missing some basic facts, not even understanding the question or the choices to be made.  Something a good presentation or two could have cleared that up before we actually go into the discussion.  

I wish we could squash that indiscriminate “presentations are bad” meme.  It is not making good use of the evidence about how our meetings actually work.  We should instead focus on finding ways to make good use of our synchronous time, which certainly includes eliminating the bad presentations, but really needs to focus on making informed decisions.

Grüße, Carsten

---

From: Keith Moore <[email-masked]>
Date: Fri, 17 Apr 2020 21:26:07 -0400

On 4/17/20 9:21 PM, Carsten Bormann wrote:

> I wish we could squash that indiscriminate “presentations are bad” meme.

I wish we could squash the notion that face-to-face meetings should be 
filled with presentations, with only a small amount of time left for 
discussion.

Keith

---

From: Carsten Bormann <[email-masked]>
Date: Sat, 18 Apr 2020 05:22:07 +0200

> On 2020-04-18, at 03:26, Keith Moore <[email-masked]> wrote:
> 
> I wish we could squash the notion that face-to-face meetings should be filled with presentations, with only a small amount of time left for discussion.

We’ve been through this 500 times before.  If somebody reminds people that presentations are useful, this scare crow is always cited as the inevitable outcome.  It’s not, folks, and the point is that we want to have the productive presentations and not the unproductive ones (where productivity is measured in the progress made in the ensuing discussion).

I don’t think anybody believes that no-discussion meetings are useful.

The fact that chairs sometimes fail to prevent the scary scenario you evoke indicates a lack of focus; having to end a discussion when it just started to generate interesting results is a serious problem.  Maybe we need more chair training on agenda setting.  And maybe some AD coaching.
I’ve seen agenda that didn’t even have planned slot durations in them; that has to stop (as well as the fixation on keeping to those durations, I might add).

It is much less of a problem to end a meeting early because the agenda was kept clean; even more so for online (“virtual”) meetings.  With good planning, items can also be pushed off the agenda dynamically if others have needed more time.

Yes, being a chair is **difficult**; I’m so happy that I once again handed over my WG chairing responsibilities to younger chairs.

Instead of re-hashing this age-old thread, I’d rather focus on the new tools we have available now.  Moving a whole university to digital tends to sharpen one’s mind on these.
(My wife, with her 40 years of university teaching experience, just uploaded her first narrated slides this week, and that may indeed be a tool we want to embrace.)

Grüße, Carsten

---

From: Keith Moore <[email-masked]>
Date: Sat, 18 Apr 2020 05:40:24 -0400

> We’ve been through this 500 times before.  If somebody reminds people that presentations are useful, this scare crow is always cited as the inevitable outcome.  It’s not, folks, and the point is th…

I find this way of framing the discussion ("presentations are good/bad") 
unhelpful because a back-and-forth ("are not", "are too") isn't going to 
reveal the subtleties required to make meetings more effective.

My point in bringing this back up is simply this: since we now have to 
meet differently, at least for a while, it might make sense to try to 
think about the most effective way of doing virtual meetings.   Carrying 
over habits from face-to-face meetings might not serve us well, 
especially since (IMO) those habits didn't serve us that well for face 
to face meetings.
> (My wife, with her 40 years of university teaching experience, just uploaded her first narrated slides this week, and that may indeed be a tool we want to embrace.)

yes, precisely, and good example.

Keith

---

From: Christian Hopps <[email-masked]>
Date: Sat, 18 Apr 2020 05:55:30 -0400

> On Apr 17, 2020, at 4:28 PM, Keith Moore <[email-masked]> wrote:
> 
> But we don't need presentation times.   They have always been a huge waste of f2f meeting time.   Now that we're not likely to be meeting f2f for another year or two, is a good time to get rid of them.   Even when we go back to meeting f2f, prepare the presentations in advance and let us view them before we show up at the meeting.   For that matter, skip the presentation entirely (it's much faster to read than to listen to speech) and just post the slides (which are, admittedly, sometimes easier to digest than the I-Ds).   Add comments to the slides if it helps.
> 
> Keith

I'd like to disagree here (regarding presentations at F2F meetings). Presenting work can be an important part of WG mechanics. LSR (IS-IS/OSPF) will have a smallish group of experts reading the drafts and their revisions. A much larger group of experts including ones with cross-wg/cross-area expertise will sit through a short presentation at a F2F meeting and sometimes provide valuable feedback. I appreciate those extra sets of eyes. 

Also, human nature being what it is, sometimes you have work that a WG doesn't like (or won't like), but the emails just don't seem to be getting that across. Having a mic line of actual human beings, having been seen to listen to the speaker, and then telling them directly, in person, why the proposal is not a good idea with the allowance of immediate interaction, can be the difference between ending a debate cleanly and protracted never ending email threads, upset authors, appeals, etc. This won't work near as well if you just have people walk up (no presentation) and tell the author they are wrong, as you miss out on the "these people listened to me" part then.

Thanks,
Chris.

---

From: Keith Moore <[email-masked]>
Date: Sat, 18 Apr 2020 06:18:57 -0400

> I'd like to disagree here (regarding presentations at F2F meetings). Presenting work can be an important part of WG mechanics. LSR (IS-IS/OSPF) will have a smallish group of experts reading the dra…

Yes, I've seen this happen.   But did the presentation really have to 
take up F2F meeting time?   It seems like there are other ways of 
getting such feedback, including old-fashioned email.

Though I will admit that the f2f meeting format in which people can fill 
up holes in their schedules between WG meetings that they need to 
attend, with WG meetings that they have less need to attend, does 
attract more eyeballs with broader perspectives. With virtual meetings 
(especially when each WG schedules its own virtual meetings) we have the 
challenge of attracting those eyeballs for cross-wg/cross-area review, 
and IMO that challenge should be addressed sooner rather than later.

> Also, human nature being what it is, sometimes you have work that a WG doesn't like (or won't like), but the emails just don't seem to be getting that across. Having a mic line of actual human bein…

I've seen this happen too, but it's often been quite disappointing to 
see many people's very expensive f2f meeting time used to educate a very 
few people (sometimes only one person).

Keith

---

From: Robert Raszuk <[email-masked]>
Date: Sat, 18 Apr 2020 12:40:24 +0200

All,

IMO the quality of presentations differs from author to author and we are
unlikely to change that. While in general I think Keith has a point if the
slides would be really to point the issues to be discussed perhaps with 1-3
overview it would be all cool.

On the other hand I and others often when asking questions or making
comment as presenter to move back to slide X. That helps to even state the
question. And that is equally applicable to f2f or online meetings.

But to be constructive I would like to make two suggestions:

* Slides for the meeting should be posted before the draft cut-off date or
together with the draft (in fact via the very same submission tool) if
there is intention to present the topic. Min 2 weeks. Just posting the
slides 12h or less (or in many cases after the meeting as they are
unicasted to chairs only) does not help.

* Agenda order for each meeting should be dynamic - sorted by the interest
of the WG participants before the meeting via doodle pool (which would
close say 24h before the meeting starts or before f2f IETF starts). That
way topics which are less important or have poor slides move to the end and
those which are really key to the WG are on top - and that order to be a
decision of the WG members.

Thx,
R.

---

From: [email-masked]
Date: Sat, 18 Apr 2020 05:50:58 -0700 (PDT)

> take up F2F meeting time?

If you want those presentations to be effective, the answer is yes.

Feedback from an audience is important, even if it's entirely nonverbal.
Knowing when you're going too fast or too slow, when the audience is bored or
confused, often can make the difference between a useful presentation and a
useless one.

This point has been driven home quite forcefuly by the recent shift to fully
remote meetings Speaking as someone who has attended meetings remotely more
often than not for many years, the difference between the same presenter having
an audience and not can be striking.

(As an aside, we need to get a lot more adept at using virtualization tools in
general and active use of two-way video in particular.)

>   It seems like there are other ways of

Not even close. Now, if you're singing a song to a kindergarten class, or if
you're teaching a upper-division physics seminar (for the Nth time), things are
different: You already know what works so the feedback isn't as important.

We're presenting complex material, usually for the first time. Feedback
matters.

> and IMO that challenge should be addressed sooner rather than later.

One of the advantages of virtual meetings is that you can reclaim time much
more easily. Although IME this is usually because the topic turns out not to be
of sufficient interesting, not because of bad prezos.

> > Also, human nature being what it is, sometimes you have work that a WG
> > doesn't like (or won't like), but the emails just don't seem to be getting that
> > across. Having a mic line of actual human beings, having been seen to listen to
> > the speaker, and then telling them directly, in person, why the proposal is not
> > a good idea with the allowance of immediate interaction, can be the difference
> > between ending a debate cleanly and protracted never ending email threads,
> > upset authors, appeals, etc. This won't work near as well if you just have
> > people walk up (no presentation) and tell the author they are wrong, as you
> > miss out on the "these people listened to me" part then.

> few people (sometimes only one person).

Often? Not in my experience, it hasn't. This bogeyman really needs to be
retired.

				Ned

---

From: "Andrew G. Malis" <[email-masked]>
Date: Sat, 18 Apr 2020 11:04:42 -0400

I think there's general agreement that at least some presentations are
useful (thanks, Carsten) while others are bad (thanks, Keith). But I don't
think we can agree that ALL presentations are useful or ALL
presentations are bad.

So? Let the WG chairs decide. If we require that presentations be made
available to the chairs well prior to the meeting, then they can review
them and see which would be useful to set the context for the technical
presentation, which others could just be made available for offline
reading, and they can set the meeting agenda appropriately.

Cheers,
Andy

On Sat, Apr 18, 2020 at 5:41 AM Keith Moore <[email-masked]>
wrote:

> > We’ve been through this 500 times before.  If somebody reminds people
> that presentations are useful, this scare crow is always cited as the
> inevitable outcome.  It’s not, folks, and the point is that we want to have
> the productive presentations and not the unproductive ones (where
> productivity is measured in the progress made in the ensuing discussion).
>
> > Instead of re-hashing this age-old thread, I’d rather focus on the new
> tools we have available now.  Moving a whole university to digital tends to
> sharpen one’s mind on these.
> > (My wife, with her 40 years of university teaching experience, just
> uploaded her first narrated slides this week, and that may indeed be a tool
> we want to embrace.)
>

---

From: Keith Moore <[email-masked]>
Date: Sat, 18 Apr 2020 11:47:11 -0400

On 4/18/20 11:04 AM, Andrew G. Malis wrote:

> useful (thanks, Carsten) while others are bad (thanks, Keith). But I 
> don't think we can agree that ALL presentations are useful or ALL 
> presentations are bad.

I think this misses the point, and is part of why I think the discussion 
was framed unhelpfully.   Quality of presentations is not the issue I'm 
concerned about.   The issue is whether the use of f2f meeting time to 
do presentations is a wise use of that very expensive time.  Clearly 
there are differing opinions on that, but I can more than once recall 
flying across an ocean and paying for a week's hotel room and the 
conference fee, only to find out that one of the principal WG sessions 
that I used to justify attending that meeting turned out to be almost a 
complete waste of time -- not because the presentation was bad, but 
because there was so little time available for actual discussion.   
Discussion, not presentation, IMO, is the reason we have (had?) f2f IETF 
meetings.

By contrast I don't have a problem with a few slides that are just used 
to frame specific points for discussion, when those discussion points 
are known in advance.

> available to the chairs well prior to the meeting, then they can 
> review them and see which would be useful to set the context for the 
> technical presentation, which others could just be made available for 
> offline reading, and they can set the meeting agenda appropriately.

I presume the WG chairs have already been deciding.    But I certainly 
agree with having the presentations* be made available for download (not 
just to the chairs) prior to the meeting. Then, perhaps, if the chair 
decides that it's worth using precious meeting time for that 
presentation, the speaker can still benefit from early feedback to 
improve the f2f presentation.

Keith

* IMO it's sufficient to prepare the slides in advance, ideally with 
accompanying notes, and make those available for download. I am not 
asking speakers to make video presentations available in advance.

---

From: Carsten Bormann <[email-masked]>
Date: Sat, 18 Apr 2020 18:05:45 +0200

> On 2020-04-18, at 17:47, Keith Moore <[email-masked]> wrote:
> 
>> I think there's general agreement that at least some presentations are useful (thanks, Carsten) while others are bad (thanks, Keith). But I don't think we can agree that ALL presentations are useful or ALL presentations are bad.
> 
> I think this misses the point, and is part of why I think the discussion was framed unhelpfully.   Quality of presentations is not the issue I'm concerned about.  

Of course not.  The highest quality presentation about pickling beets (or the exact shape of 8P8C sockets in network gear) would a “bad presentation” in the context of the WGs I’m interested in.

> The issue is whether the use of f2f meeting time to do presentations is a wise use of that very expensive time.  

Yes.  And most people agree there is no answer to that question unless you consider whether a specific presentation actually helped advance the business of the WG.  (Note that there are also useful and bad discussions, same thing.)

> I presume the WG chairs have already been deciding.    But I certainly agree with having the presentations*

I think you are making the common(*) mistake of confusing presentations with slide decks.

> * IMO it's sufficient to prepare the slides in advance, ideally with accompanying notes, and make those available for download. I am not asking speakers to make video presentations available in advance.

A narrated slide deck can in itself already be a good presentation (while a slide deck usually isn’t, unless it is not actually useful for a presentation).  I think most people would think a “video presentation” entails filming the presenter while narrating, which is indeed of limited value in most of our meetings, all the time creating much more effort for the presenter.

If we make watching these a prerequisite for meetings, I’m still serious about entrance exams.  If not, they are not quite as useful as process confabulation would suggest they *should* be.

Grüße, Carsten

(*) Common at least since Microsoft decided to call slide decks “presentations”, which is probably an indicator for the most striking symptoms of Powerpoint’s failings.

---

From: Keith Moore <[email-masked]>
Date: Sat, 18 Apr 2020 14:14:40 -0400

> Yes.  And most people agree there is no answer to that question unless you consider whether a specific presentation actually helped advance the business of the WG.  (Note that there are also useful…

I don't know who "most people" are, but I don't think you've established 
that.  And I think it's still missing the point somewhat.   Say a chair 
decides that a particular presentation is likely to be useful at 
advancing the business of the WG.    The important question then 
becomes, should that presentation take up precious f2f WG meeting time 
that could otherwise be used to identify salient issues or get closure 
on some issues?  Or would it suffice to ask participants to view that 
presentation outside of f2f meeting time, and free up that time to 
engage in discussion with the hope of getting closure?

I generally view the task of a chair in planning f2f WG sessions at a 
plenary meeting as:   How to get closure on as many important/thorny 
issues as possible, during that week of opportunity for high-bandwidth 
interaction?   And that involves making use of not only f2f meeting time 
but also the time between meetings.

> A narrated slide deck can in itself already be a good presentation (while a slide deck usually isn’t, unless it is not actually useful for a presentation).

I have rarely seen a face-to-face or videoed narrated presentation, on a 
technical subject, that the same information couldn't be conveyed in 
about one-sixth the time it took to watch the presentation, simply by 
outlining the high points in text or as images or maybe an occasional 
animation.   There are exceptions, of course.   But again, what makes 
the f2f meetings worth having is interaction, not one-way 
presentation.   The ability of participants to ask clarifying questions 
or point out problems is useful, as is the ability of the speaker to get 
verbal and nonverbal feedback from the audience.    But that 
face-to-face time is very expensive, both in terms of what it costs the 
participants and/or their sponsors, and also in terms of what it costs 
the WG in its ability to get timely closure and publish documents while 
they're still helpful to the Internet community. So it makes sense to 
use that time judiciously.   If WG participants were expected to read or 
watch brief presentations prior to the f2f meetings, and perhaps to 
submit feedback in writing before the meetings, thus freeing up meeting 
time to actually answer those questions and have discussions, would that 
allow WGs to get more done in the time they have?   Based on my own 
prior experience, I believe it would.   But it might require the 
community to get used to doing things a bit differently.

Of course in the near term most of this discussion is moot, since it's 
likely to be well over a year before it's safe to travel and meet again.

Keith

---

From: Michael Richardson <[email-masked]>
Date: Sat, 18 Apr 2020 21:28:07 -0400

> inevitable outcome.  It’s not, folks, and the point is that we want to
    > have the productive presentations and not the unproductive ones (where

    > The fact that chairs sometimes fail to prevent the scary scenario you
    > evoke indicates a lack of focus; having to end a discussion when it
    > just started to generate interesting results is a serious problem.
    > Maybe we need more chair training on agenda setting.  And maybe some AD
    > coaching.

Yes, the problem isn't that there are bad presentations, but rather that there
is poor time management decisions made.    AD coaching is occuring, but
inconsistently, and with different emphasis.

    > I’ve seen agenda that didn’t even have planned slot durations in them;
    > that has to stop (as well as the fixation on keeping to those
    > durations, I might add).

The number of virtual interim meetings that had really bad agendas was fewer,
but there were a number where the agenda was six lines consisting of some
draft names.
No links (etherpad, jabber, conference link), few or no time allocations, no ...

I really am expecting ADs to notice and have a word with those chairs.

    > It is much less of a problem to end a meeting early because the agenda
    > was kept clean; even more so for online (“virtual”) meetings.  With
    > good planning, items can also be pushed off the agenda dynamically if
    > others have needed more time.

+1

    > Yes, being a chair is **difficult**; I’m so happy that I once again
    > handed over my WG chairing responsibilities to younger chairs.

:-)

    > tools we have available now.  Moving a whole university to digital
    > tends to sharpen one’s mind on these.

    > uploaded her first narrated slides this week, and that may indeed be a
    > tool we want to embrace.)

So basically a kind of screencast?

{as a technology thought: It seems like it ought to be possible to make every
slide an IFRAME of a video, have no delta frames, and just audio.  The result
should be very high fidelity.  Even better if we could store the vector
version of the slide (whether PDF or SVG) into the video.}

--
]               Never tell me the odds!                 | ipv6 mesh networks [
]   Michael Richardson, Sandelman Software Works        |    IoT architect   [
]     [email-masked]  http://www.sandelman.ca/        |   ruby on rails    [

--
Michael Richardson <[email-masked]>, Sandelman Software Works
 -= IPv6 IoT consulting =-

---

From: Warren Kumari <[email-masked]>
Date: Mon, 20 Apr 2020 17:10:51 -0400

On Fri, Apr 17, 2020 at 4:30 PM Keith Moore <[email-masked]> wrote:
> us view them before we show up at the meeting.

<dead horse num_hats=0>
This only works if people actually do view them before they show up at
the meeting.
If one asks "How many people have read the draft?" you often only get
a very small number of hands, even for short, easy documents. This is
especially true for new documents - before people have seen a
presentation about it, they don't seem to know enough to object -
Cunningham's Law ("the best way to get the right answer on the
internet is not to ask a question; it's to post the wrong answer.")
works even more for documents - the best way to get feedback is to say
something wrong in the draft / presentation.

If people won't read the document or even the abstract ahead of time,
why do you think that they will sit and watch a video / read the
slides? Yes, in an ideal world people would post their -00, and within
a day or two get twenty seven eight-by-ten colour glossy pictures with
circles and arrows and a paragraph on the back of each one explaining
what's wrong, and replacement text -- but, sadly, that's not the world
we live in.

Presentations make people understand **why** they dislike a draft
enough that they should spend the time working to make it less
objectionable...

W
</dead horse>

> For that matter, skip

-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

---

From: Keith Moore <[email-masked]>
Date: Mon, 20 Apr 2020 17:20:04 -0400

On 4/20/20 5:10 PM, Warren Kumari wrote:

> slides?

Slides are easy to read, usually can be consumed in a few minutes, and 
are generally a good way of knowing whether it's worth the trouble to 
read the draft, comment on the draft, or for that matter, to attend the 
WG meeting.

(Some abstracts are so, uhh, abstract, that you can't really tell what 
they're talking about.)

I'm not asking for videos.   Not only is a good video a LOT of work to 
produce (especially for people who don't do it all the time), it's at 
least as inefficient use of people's time as face to face presentations are.

>   Yes, in an ideal world people would post their -00, and within
Nor have we just had Thanksgiving dinners that couldn't be beat.
> enough that they should spend the time working to make it less

Right, but if we can get most of the benefit by reading the slides, then 
why shouldn't we do that?

Keith