Subject: Revised IETF LLC Draft Strategic Plan 2020 to address feedback raised

From: Jay Daley <[email-masked]>
Date: Fri, 29 May 2020 16:34:11 +1200

All except for three of the issues raised in response to feedback are now addressed by a revised version.  This includes major rewrites of a number of areas (tools, engagement), a new linkages section to address a set of related issues, and multiple other changes.

You can either read the whole thing in one go

	https://github.com/ietf-llc/strategy-2020-consultation/blob/latest-updates-from-consultation/DRAFT%20Strategic%20Plan%202020.md

or you can read the individual commits, each of which tackles a related set of issues

	https://github.com/ietf-llc/strategy-2020-consultation/commits/latest-updates-from-consultation

I would be grateful if those of you who provided feedback could let me know if these rewrites address your feedback or if more work is needed, and I welcome new feedback from anyone.

For the remaining three issues

1. https://github.com/ietf-llc/strategy-2020-consultation/issues/11 "Needs to explicitly reference the mission statements of IETF/IRTF/IAB"
The submitter has agreed with me that this is handled by other changes

2. https://github.com/ietf-llc/strategy-2020-consultation/issues/2 "Emphasis on accessibility for non-native english speakers needed"
Is out of scope for the LLC to decide on this and to do it without clear community guidance

3. https://github.com/ietf-llc/strategy-2020-consultation/issues/3 "No reference to support being developed for community members who promote the IETF in different communities"
We have done this some time ago but not clear if that was off our own back (and therefore out of scope) or following community lead. Checking.

thanks
Jay

-- 
Jay Daley
IETF Executive Director
[email-masked]

---

From: Eric Rescorla <[email-masked]>
Date: Fri, 29 May 2020 07:37:48 -0700

Jay,

Thanks for sending this. Some comments below. I've also sent
you a PR to convert the tables to Markdown, which would
make this rather easier to read.

> 6. To deliver a toolchain that is up-to-date and well regarded by users.

This seems in conflict with "evidence-led". Suppose the toolchain
was well-regarded by users but empirically less efficient than
other toolchains.

> Sponsors, in addition to supporting the IETF for the value it
> delivers, are also increasingly concerned about how the organisations
> they sponsor operate, how they treat their volunteers and staff and
> what opportunities they provide for a diverse range of new
> participants.  To be able to explain this, we need to document the
> participant journey, a map of the different stages of participation
> (e.g. newcomer, leadership), at what stages people start their
> participation in the IETF, how they transition between them and at
> what stages they end their participation.

There seems to be an implicit assumption here that people should
have a "career arc" in IETF that starts with newcomer and
ends with leadership, but I don't think that's obviously
true. Many of our most valuable contributors have never served
in leadership and some of them do lots of reviewing but not
a lot of RFC writing.

> With particular regard to the services that are provided to
> participants by the LLC, a culture has built up where decisions are
> taken more on the basis of opinion and debate than evidence.  For
> example, someone has a good idea that we should provide X for
> participants and rather than ask participants if they want X, how they
> already obtain it and if it is a priority for them, a debate takes
> place on the merits of us providing it or not.  Sometimes this debate
> results in stalemate and issues get put into the “too hard” basket as
> a result.
>
> We need to shift from this to a culture where data is the starting
> point for identifying what needs to change and why and what the impact
> will be.  As the transformation says, we need to shift people from
> lining up next to people with the same opinions and standing opposite
> those who disagree, to lining up together facing the data and working
> together to interpret it.
>
> We already have some data available, such as stats on meeting
> attendees and the post-meeting survey, but making this cultural change
> requires much more data.  For example, there is a big set of questions
> on non-participation and barriers to participation which are crucial
> to providing better service to the community overall. Obtaining this
> data will require a wide ranging plan backed up with a major programme
> of surveys and technical analysis.

Well, maybe. I certainly do think that data-driven decision
making is important, but in my experience it is often
infeasibly expensive to make one's decisions based entirely
on data. In many cases, the cost of gathering the data to
make a decision whether to implement a feature far exceeds the
cost of implementing the feature. This can be necessary
in cases where the feature might have large negative
impacts, but generally when the cost is just the feature
itself, this is not true. Good product management is about
judgement that is informed by data, not by replacing judgement
with data, as this sort of suggests.

> | 15 | Lack of clarity on why we need additional income, how much and in
what form |

Perhaps this should be framed in a less loaded way in terms
of lack of clarity on how much income is needed. If it's unclear,
perhaps we need *less* income (though I doubt that is true).

> Running a cycle of three global meetings per year is a difficult and
> expensive task, and one that is going to get harder because of four
> growing issues.

This seems to be a list of three issues (1) meeting requirements (2)
carbon and (3) COVID

-Ekr

On Thu, May 28, 2020 at 9:35 PM Jay Daley <[email-masked]> wrote:

> All except for three of the issues raised in response to feedback are now
> addressed by a revised version.  This includes major rewrites of a number
> of areas (tools, engagement), a new linkages section to address a set of
> related issues, and multiple other changes.
>
> I would be grateful if those of you who provided feedback could let me
> know if these rewrites address your feedback or if more work is needed, and
> I welcome new feedback from anyone.
>
>
> 1. https://github.com/ietf-llc/strategy-2020-consultation/issues/11
> "Needs to explicitly reference the mission statements of IETF/IRTF/IAB"
> 2. https://github.com/ietf-llc/strategy-2020-consultation/issues/2
> "Emphasis on accessibility for non-native english speakers needed"
> 3. https://github.com/ietf-llc/strategy-2020-consultation/issues/3 "No
> reference to support being developed for community members who promote the
> IETF in different communities"
> We have done this some time ago but not clear if that was off our own back
> (and therefore out of scope) or following community lead. Checking.
>
>
> thanks
> Jay
>
> --
> Jay Daley
> IETF Executive Director
> [email-masked]
>
>

---

From: "Pete Resnick" <[email-masked]>
Date: Fri, 29 May 2020 13:34:50 -0500

On 29 May 2020, at 9:37, Eric Rescorla wrote:

>> 6\. To deliver a toolchain that is up-to-date and well regarded by 
>> users.
>

I've got no problem with "efficient" being added to list of "up-to-date" 
and "well-regarded", but I wouldn't want to see "well-regarded" removed 
(at least with out something else that captures the right sense). We 
could get a data-driven result that using a particular toolchain is 
incredibly efficient, but if people are unwilling to use it for (perhaps 
irrational) reasons of not liking the look-and-feel of it, or even just 
because it uses a new way of thinking that people are resistant to, that 
weighs against trying to deliver it. I think it's OK for those 
principles to be in tension and the LLC has to use its judgment to 
figure out what's best (in consultation with the community of users, of 
course). But we still want to take that "regard" into account.

And like I said, a different term than "well-regarded" that captures my 
concern is perfectly OK.

pr
-- 
Pete Resnick https://www.episteme.net/
All connections to the world are tenuous at best

---

From: "Salz, Rich" <[email-masked]>
Date: Fri, 29 May 2020 18:46:30 +0000

I think Pete's concern is important.  IF we need to figure out how to measure well-regarded, we can do that.

---

From: Jay Daley <[email-masked]>
Date: Sat, 30 May 2020 08:36:55 +1200

Eric

> On 30/05/2020, at 2:38 AM, Eric Rescorla <[email-masked]> wrote:
> 
> 
> Jay,
> 

I personally find html tables far easier to read and manipulate than markdown tables, which is why I used them but I’ll look at the PR to see how easy you’ve made them to read.

> was well-regarded by users but empirically less efficient than

I’m not convinced that there is another toolchain we can measure against or that measured user satisfaction is that distant from empirical observation of efficiency. However, I’m happy to add something about efficiency. 

https://github.com/ietf-llc/strategy-2020-consultation/issues/36

> in leadership and some of them do lots of reviewing but not

No implicit assumption at all. This is simply documenting what exists though admittedly that information could be used for that purpose if the community so desired (I have no views at all on whether it should or should not). 

> with data, as this sort of suggests.

I agree and this does push too far in one direction. 

https://github.com/ietf-llc/strategy-2020-consultation/issues/37

> 
> > | 15 | Lack of clarity on why we need additional income, how much and in what form |
> 

Yes good point. 

https://github.com/ietf-llc/strategy-2020-consultation/issues/38

> This seems to be a list of three issues (1) meeting requirements (2)

Noted. 

https://github.com/ietf-llc/strategy-2020-consultation/issues/39

Thanks for the review. 
Jay

-- 
Jay Daley
IETF Executive Director

> -Ekr
> 
> On Thu, May 28, 2020 at 9:35 PM Jay Daley <[email-masked] <mailto:[email-masked]>> wrote:
>         https://github.com/ietf-llc/strategy-2020-consultation/blob/latest-updates-from-consultation/DRAFT%20Strategic%20Plan%202020.md <https://github.com/ietf-llc/strategy-2020-consultation/blob/latest-updates-from-consultation/DRAFT%20Strategic%20Plan%202020.md>
> 
>         https://github.com/ietf-llc/strategy-2020-consultation/commits/latest-updates-from-consultation <https://github.com/ietf-llc/strategy-2020-consultation/commits/latest-updates-from-consultation>
> 
> 1. https://github.com/ietf-llc/strategy-2020-consultation/issues/11 <https://github.com/ietf-llc/strategy-2020-consultation/issues/11> "Needs to explicitly reference the mission statements of IETF/IRTF/IAB"
> 2. https://github.com/ietf-llc/strategy-2020-consultation/issues/2 <https://github.com/ietf-llc/strategy-2020-consultation/issues/2> "Emphasis on accessibility for non-native english speakers needed"
> 3. https://github.com/ietf-llc/strategy-2020-consultation/issues/3 <https://github.com/ietf-llc/strategy-2020-consultation/issues/3> "No reference to support being developed for community members who promote the IETF in different communities"
> [email-masked] <mailto:[email-masked]>
>

---

From: Eric Rescorla <[email-masked]>
Date: Fri, 29 May 2020 13:59:51 -0700

On Fri, May 29, 2020 at 1:37 PM Jay Daley <[email-masked]> wrote:

> I personally find html tables far easier to read and manipulate than
> markdown tables, which is why I used them but I’ll look at the PR to see
> how easy you’ve made them to read.
>

In my experience, they're much harder to diff.

> I’m not convinced that there is another toolchain we can measure against
> or that measured user satisfaction is that distant from empirical
> observation of efficiency.
>

As someone who has been at organizations that go through toolchain changes,
I can tell you that very often the stakeholders have minimal experience
with other toolchains and therefore their level of satisfaction is not
related to whether those toolchains would be more efficient.

No implicit assumption at all.

The phrase "the participant journey" and the description of different
stages implies that.

This is simply documenting what exists though admittedly that information
> could be used for that purpose if the community so desired (I have no views
> at all on whether it should or should not).
>

Well, I don't know what it is you propose to document, so it's hard to know
what I think about it. But given that this is unclear, perhaps either
clarify it or document it.

-Ekr

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 1 Jun 2020 09:08:45 +1200

>> was well-regarded by users but empirically less efficient than

There are people here with experience of multiple tool chains for
document drafting/review/approval/publishing processes. And there
are people here with experience of other SDOs' processes for
creating/running/destroying WGs, and of handling very large volume
email or other interactions. Everybody is an expert now on Meetecho
vs Webex vs Zoom. So although it might be more work than the results
would justify, I think comparisons are very possible.

   Brian