Subject: Re: When is a 3933 experiment necessary? [Was: Last Call: <draft-farrell-ft-03.txt> (A Fast-Track way to RFC with Running Code) to Experimental RFC]

From: SM <[email-masked]>
Date: Fri, 01 Feb 2013 02:23:41 -0800

At 16:25 31-01-2013, Barry Leiba wrote:
>We often pick on every suggested change and point out every possible
>flaw, with different people holding out behind different flaws, and we
>get stuck there.  There seems to be some assumption, when we do this,
>that our current process doesn't also have significant flaws.  But the
>very reason we're trying to change something is that we *already* have
>a process with flaws.
>
>We know that no process we ever come up with will be perfect, and
>agreeing to change is a question of deciding what the flaw balance is.
>  We need to allow ourselves to move from one (known, "comfortable")
>set of flaws to a new set, *if* we can come to a reasonable agreement
>that we've improved things in a useful way.

I missed RFC 5111 which was a 3933 experiment.  The success metrics were:

   "Progress on the basic milestones is measured by whether they are
    completed within the time-frame specified in the initial
    Exploratory Group Charter  ..."

   "an Exploratory Group whose mailing list shows minimal traffic would
    probably not be a good candidate for milestone extension."

There is also RFC 4633 about long-term suspensions.  There isn't any 
success metric.  The following is worth highlighting:

   "In no event shall a suspension last for more than six months nor shall any
    suspension last beyond the end of the RFC 4633 experiment."

The following are selective quotes:

   "During the Gen Area meeting today, it was asserted that if this
    experiment is successful, this document might become a BCP essentially
    as written."

   "I think the spirit of RFC 3933 is to get things tried out quickly,
    so I don't buy this. The community has been discussing related topics
    in the recent past. Of course, if many comments come in during LC,
    Sam would need to rethink."

   'I see "going back to BCP83" as so unpleasant to IESG members that the
    "experiments" will be extended.'

There are things that were said which were never done.  There are 
things that were predicted and that never happened.  Maybe it is 
because of process flaws.  Maybe it is because it is easier to give 
up instead of trying.

Regards,
-sm

---

From: Sam Hartman <[email-masked]>
Date: Fri, 01 Feb 2013 11:23:10 -0500

>>>>> "SM" == SM  <[email-masked]> writes:

    SM> There are things that were said which were never done.  There
    SM> are things that were predicted and that never happened.  Maybe
    SM> it is because of process flaws.  Maybe it is because it is
    SM> easier to give up instead of trying.

Perhaps it's because the process works well enough these days that some
of us involved in process work in the past believe things are working
well enough now that the risk of change is greater than any likely
benefit.  That's generally my attitude these days. I generally haven't
supported process change proposals lately because I think things are
working quite well and I think the proposed improvements aren't worth
the risk.  For the most part I've stayed quiet because I think the risk
of harm is not huge either and if people can get together enough effort
to make a change it's probably not that horrible.  A few times I've
spoken up because something seemed like a really bad idea.  Fewer times
I've spoken up because something seemed like a really good idea.
In general though I'd like to be counted as a voice for the status quo;
I consider that a relatively well-considered opinion.

---

From: John C Klensin <[email-masked]>
Date: Sun, 03 Feb 2013 11:43:03 -0500

--On Wednesday, January 30, 2013 09:15 +0000 Adrian Farrel
<[email-masked]> wrote:

>...
> So, my conclusion: it would be good to have more process
> experiments if people feel the process needs to change.
> However, it would appear that such experiments need:
> - Thorough debate on an appropriate mailing list first
>    Do we need a "process experiments" mailing list?
> - Very clear expression of the purpose of the experiment
>    and how it will be evaluated
> - Very tight scoping to a portion of the IETF or of the IETF's
>    work so that the risk of the experiment simply changing
>    the IETF's process by default is removed

For whatever it is worth, I agree with this summary and
especially with the third point (which is where I think this
effort may have gotten into trouble).  I also agree with Dave's
observation about getting a "committed core" together.  I also
think it is useful to differentiate between the elements of the
experiment that constitutes a process variation and hence makes
community approval desirable and possibly necessary and the
elements that can be tested within the discretion of ADs, WG
Chairs, etc., and make that differentiation clear.  In some
cases, one of those categories will be unpopulated but I note
that one of the things that complicated the recent discussion
was a lot of "nothing is new here, why not just do it on a
selective basis and report back" comments.

All of this could be summarized as "it is better to have one's
ducks lined up before starting a community-wide discussion".

Re Spencer's analysis...

>> I was John's co-author on RFC 3933
>> (http://tools.ietf.org/html/rfc3933). When we were working on
>> the draft, the problem I thought we were solving, was that
>> the IESG needs to update the IETF's BCP processes from time
>> to time, but (1) it was like 32 simultaneous root canals to
>> actually get community consensus to modify the IETF's process
>> BCPs including untried changes that might be improvements, so
>> (2) the IESG was using informal IESG statements developed
>> with much less community involvement than was expected for
>> process BCP changes, in order to get anything done at all.
>> 
>> I (and, I think, John as well) saw RFC 3933 process
>> experiments as a middle path between lightweight IESG
>> decisions and full process BCP revisions. That's what I
>> thought Section 2 of RFC 3933 was trying to say.

Yes.  And I (and I think Spencer) hoped that the IESG would use
this mechanism in preference to simply ignoring a BCP process
provision that had become outdated.  To be a bit more explicit
about the language in 3933 on that subject, I (at least) felt
that there was always going to be a need for IESG judgment on
when statements were appropriate and when 3933 was needed.
Trying to write rules to establish a precise boundary seemed
impossible and likely to lead to more trouble.  What 3933
doesn't say explicitly, but could have, is that, if the
community disagrees with the pattern of the IESG's judgments of
such things, that ought to be a matter for discussion with the
Nomcom (or, at least in theory, appeals or recalls for specific
egregious cases).

>> I had assumed that if the IESG clearly had discretion to do
>> something under the current process BCPs, and thought it was
>> the right thing to do, they would just do it, rather than set
>> up a process experiment.

I assumed that too.  See comment above about differentiating a
3933 experiment (or 3933 part of a larger experiment) from a
discretionary one.  Interestingly, one of the things that has
changed since we wrote what because 3933 is that, at the time,
the IESG typically issued statements without prior community
discussion.  In more recent years, and for matters where a
statement has been more than the IESG telling the community how
it is doing things, such statements have more often been exposed
to community comment before taking effect.  So, where we had
three possibilities when 3933 was adopted, more or less:

	* IESG Prouncement
	* 3933 Experiment
	* BCP

We now have "IESG Statement after soliciting and considering
community opinions" as another possibility for which there are
worked examples.  That may reasonably narrow the number of cases
in which 3933 is necessary.

best,
   john

---

From: Dave Crocker <[email-masked]>
Date: Sun, 03 Feb 2013 08:55:49 -0800

> ducks lined up before starting a community-wide discussion".

for the IETF, you mean cats shepherded, not ducks lined up...

d/
-- 
  Dave Crocker
  Brandenburg InternetWorking
  bbiw.net