Subject: Re: Discussion of draft-hardie-advance-mechanics-00.txt

From: Barry Leiba <[email-masked]>
Date: Mon, 4 Oct 2010 16:16:04 -0400

> A periodic call for comments, say at  2 and 5 years out, with those
> judged to be still useful moving up the ladder,
> for example?
>
> I understand that "objection based processing" was not the most polite
> way to word this in my draft,
> and I'm sufficiently chastened to change it in the next version.  But
> I think any system we base on a periodic
> assessment like the above has to have a default of "advance".  The
> current Proposed standards
> get a lot of scrutiny and generally are pretty good.  If we don't do
> "default advance", we are adding
> friction to the common case, where they should move forward.

While this is true as far as it goes, I'd like to point out a good
example of where "the common case" may be less common than we'd like
to think.

ACAP, RFC 2244, is a Proposed Standard put out in 1997.  There's a
core of us who think it's a good protocol and a useful one, but it's
had very scanty implementation.  At this point, even most of its
champions think it might as well go to "Historic".

On the other hand, it suffers far more from apathy than from
antagonism, and should it have routinely come up for any sort of
review that had a default answer of "advance", it would almost
certainly have been advanced.  I doubt there would have been people
who really cared to fight to block it because hardly anyone uses it.
None of us ever tried to move it along the track because we knew
(know) that there's not much point.

There's no doubt in my mind that, much as I like it, ACAP should not
be at a higher current maturity level than PS.  There's some value in
saying that there have to be people who have the energy to push a rock
at least a little way up a hill in order to make something advance, to
show that there's some critical mass of support and implementation.
Otherwise, we may wind up with a bunch of little known and lesser
deployed "pet rocks" (to strain a metaphor) that get advanced by
default, providing little service or benefit to the Internet.

Barry

---

From: Keith Moore <[email-masked]>
Date: Mon, 4 Oct 2010 16:54:38 -0400

> default, providing little service or benefit to the Internet.

I think part of the problem is that our current "maturity" labeling scheme conflates several things:

- quality of the protocol itself (is the design good and without significant known flaws relative to its intended/expected use?)
- quality of the specification (is it written clearly and precisely enough to encourage interoperability?)
- currency of the specification (does the written specification still reflect current and/or desirable practice?)
- applicability of the protocol (how broadly should it be used?)
- initial/continued acceptance/deployment of the protocol

...which is why, instead of "advancing in grade" from PS to DS to FS, I'd like to see a periodic re-evaluation of the last three.  There are protocols today which are Full Standard for which the specifications are not being well-maintained and which don't reflect good current practice.  There are also protocols which are Proposed Standard for which the specifications are reasonably current and reflective of good practice.  There's a disconnect between our labeling of these standards and their "goodness" to the community, and that harms IETF's credibility.

Keith

---

From: Ted Hardie <[email-masked]>
Date: Tue, 5 Oct 2010 07:13:46 +0900

Comments  inline, some content snipped.
> - initial/continued acceptance/deployment of the protocol

I think this is a key point, both to this discussion and to the "how
many grades" discussion.
The current labels represent more than one axis of evaluation, and we
may have to agree
on which one is the most important in order to get clear labeling.
This may mean that
we add a second label (or potentially a third) to get full
descriptions.  But I don't think
even that would require us to go the full "commentary document" route
where the core
spec was surrounded by attached commentary built up over the years.

I think the "document quality" metric, assessed in relation to interoperabity
 is at the core of this.  Can you go from this document to an
implementation?  We can assess that in a variety of ways, but that's my
take on the most important bit to retain for the system.

The "is it in use" question could be handled by an increase in statuses like
"Historic" which could be applied simultaneously to the document set.
So you could have ACAP be both a full standard and historic; the meaning
would be "The IETF strongly believes that you could go from this document
to an interoperable implementation, but that you would find few other
implementations
in use".    Historic is probably not the right designation, given the
past experience.
But something like "FULL, Niche deployment" or "FULL, Wide deployment" could
be useful.

To my way of thinking, the "quality of the protocol" question is
inevitably bound up with
the "intended/expected use" issue--which means that the same protocol
could have multiple
different evaluations.  HTTP would get one as a hypertext transport,
for example, and completely
different one for streaming--both uses in the wild now.

The actual energy in the community to make change here actually seems
to me pretty low,
though, both for this and Russ's document.  I'm not sure whether a
proposal like either of
them should "default advance" with this level of energy.

regards,

Ted

---

From: Martin Rex <[email-masked]>
Date: Tue, 5 Oct 2010 01:36:48 +0200 (MEST)

Ted Hardie wrote:
> judged to be still useful moving up the ladder, for example?

There should be at least an IETF Last Call before any such
automatic advancement in order to figure out whether the community
thinks there is sufficient implementation experience that supports
advancing the document.

Example:  The "SPNEGO - the GSS-API negotiation mechanism"

http://tools.ietf.org/html/rfc4178#appendix-C

started at proposed with rfc-2478 (Dec-1998) was reviewed
and serious problems fixed with rfc-4178 (Oct-2005).

The basic problem was there there existed only a single implementation
(from Microsoft) and the original implementor did either not
sufficiently think about the spec while implementing it or forgot
to tell the IETF CAT WG about the problems of the spec he found.

Given the right mindset and some level of development experience,
it is possible to find most issues and all serious issues in a spec
doing only one single implementation.  But there are issues that
may significantly impair this process, such as implementation
deadlines.

My complaint in 1997 about the original spec was that it used
ASN.1 all over the place where the same could have been easily
accomplished entirely without ASN.1, and I predicted that this
will reliably prevent serious review at the theoretical level
for most IETF participants -- this certainly detered me from
reviewing the protocol itself.   (Discussing and reviewing the
underlying solution architecture was no problem.)

Having reviewed GSS-APIv2 before SPNEGO, I was _not_ surprized the
least about the number of issues that were found and fixed in rfc-4178.

-Martin