Subject: PS to IS question from plenary

From: IETF Chair <[email-masked]>
Date: Tue, 30 Jul 2013 16:41:06 +0200

Last night there was a question in the plenary about how many PS->IS transitions have occurred since RFC 6410 was published in October 2011. That RFC changed the three-step standards process to two steps. There was also a question of how this compared to previous times before that RFC got approved.

Looking at the timeframe from October 2011 to today (22 months), there have been four such protocol actions. These results are given by searching the IETF Announce mail archive:

0.1000 	12159 	07/16/2013 	Protocol Action: 'Specification of the IP Flow Information eXport (IPFIX) Protocol for the Exchange of Flow Information' to Internet Standard 	[email-masked] 
0.1000 	12036 	06/04/2013 	Protocol Action: DomainKeys Identified Mail (DKIM) Signatures to Internet Standard 	[email-masked] 
0.1000 	11609 	01/25/2013 	Protocol Action: 'Extension Mechanisms for DNS (EDNS(0))' to Internet Standard (draft-ietf-dnsext-rfc2671bis-edns0-10.txt) 	[email-masked] 
0.1000 	11510 	01/03/2013 	Protocol Action: 'Automated Updates of DNSSEC Trust Anchors' to Internet Standard (RFC 5011) 	[email-masked] 

Prior to the publication of RFC 6410, in the preceding 22 months there were these 20 actions raising standards to either Draft Standard or Full Standard:

0.1000 	9572 	07/12/2011 	Protocol Action: 'DomainKeys Identified Mail (DKIM) Signatures' to Draft Standard (draft-ietf-dkim-rfc4871bis-15.txt) 	[email-masked] 
0.1000 	9427 	06/06/2011 	Protocol Action: 'Transport Layer Security (TLS) Transport Model for the Simple Network Management Protocol (SNMP)' to Draft Standard 	[email-masked] 
0.1000 	9402 	05/31/2011 	Protocol Action: 'Transport Security Model for the Simple Network Management Protocol (SNMP)' to Draft Standard 	[email-masked] 
0.1000 	9401 	05/31/2011 	Protocol Action: 'Transport Subsystem for the Simple Network Management Protocol (SNMP)' to Draft Standard 	[email-masked] 
0.1000 	9400 	05/31/2011 	Protocol Action: 'Simple Network Management Protocol (SNMP) Context EngineID Discovery' to Draft Standard 	[email-masked] 
0.1000 	7410 	01/25/2010 	Protocol Action: 'ESMTP and LMTP Transmission Types Registration' to Draft Standard 	[email-masked] 
0.1000 	6858 	08/17/2009 	Protocol Action: 'Guidance on Interoperation and Implementation Reports for Advancement to Draft Standard' to BCP 	[email-masked] 
0.1000 	6819 	07/29/2009 	Protocol Action: 'Cryptographic Message Syntax (CMS)' to Draft Standard 	[email-masked] 
0.1000 	6816 	07/29/2009 	Protocol Action: 'TCP Congestion Control' to Draft Standard 	[email-masked] 
0.1000 	6256 	03/05/2009 	Protocol Action: 'RPC: Remote Procedure Call Protocol Specification Version 2' to Draft Standard 	[email-masked] 
0.1000 	6069 	01/14/2009 	Protocol Action: 'Capabilities Advertisement with BGP-4' to Draft Standard 	[email-masked] 

0.1000 	10108 	12/05/2011 	Protocol Action: 'The Multipart/Report Media Type for the Reporting of Mail System Administrative Messages' to Full Standard 	[email-masked] 
0.1000 	9781 	09/06/2011 	Protocol Action: 'Message Submission for Mail' to Full Standard (draft-ietf-yam-rfc4409bis-03.txt) 	[email-masked] 
2.3656 	8034 	06/21/2010 	Protocol Action: 'Cryptographic Message Syntax (CMS)' to Full Standard 	[email-masked] 
2.4437 	7668 	03/15/2010 	Protocol Action: 'SMTP Service Extension for 8-bit MIME Transport' to Full Standard 	[email-masked] 
2.2366 	6876 	08/19/2009 	Protocol Action: 'Extensible Provisioning Protocol (EPP) Transport over TCP' to Full Standard 	[email-masked] 
2.4694 	6784 	07/20/2009 	Protocol Action: 'Extensible Provisioning Protocol (EPP) Contact Mapping' to Full Standard 	[email-masked] 
2.2972 	6783 	07/20/2009 	Protocol Action: 'Extensible Provisioning Protocol (EPP) Host Mapping' to Full Standard 	[email-masked] 
2.1632 	6782 	07/20/2009 	Protocol Action: 'Extensible Provisioning Protocol (EPP) Domain Name Mapping' to Full Standard 	[email-masked] 
2.5992 	6781 	07/20/2009 	Protocol Action: 'Extensible Provisioning Protocol (EPP)' to Full Standard 	[email-masked] 

I should insert here the Standard disclaimer about possibly faulty search methodology or records, misunderstanding the question, or the hasty interpretation of results. In particular, the above search was not easy on ARO, involved manual steps, and I might have easily missed something. And I wish I had been able to do a database query instead. Feel free to repeat & verify my results...

Jari

---

From: John C Klensin <[email-masked]>
Date: Tue, 30 Jul 2013 14:27:56 -0400

--On Tuesday, July 30, 2013 16:41 +0200 IETF Chair
<[email-masked]> wrote:

> Last night there was a question in the plenary about how many
> PS->IS transitions have occurred since RFC 6410 was published
> in October 2011. That RFC changed the three-step standards
> process to two steps. There was also a question of how this
> compared to previous times before that RFC got approved.
> 
> Looking at the timeframe from October 2011 to today (22
> months), there have been four such protocol actions. These
> results are given by searching the IETF Announce mail archive:
>...
> Prior to the publication of RFC 6410, in the preceding 22
> months there were these 20 actions raising standards to either
> Draft Standard or Full Standard:
>...
> I should insert here the Standard disclaimer about possibly
> faulty search methodology or records, misunderstanding the
> question, or the hasty interpretation of results. In
> particular, the above search was not easy on ARO, involved
> manual steps, and I might have easily missed something. And I
> wish I had been able to do a database query instead. Feel free
> to repeat & verify my results...

Jari,

Thanks for this.

Disclaimers and possible small classification errors aside and
being careful to avoid making causal assumptions, I believe that
the implication of the above is that there is no evidence that
the 3 -> 2 transition has increased the number of documents
being moved or promoted out of Proposed Standard.   If one were
to assume a causal relationship and an absence of external
confounding variates or processes, one might even conclude the
the 3 -> 2 transition has made things quite a lot worse.
Conversely, it seems to me that one could argue that the change
has made things better only by demonstrating the existence of a
process that would have led to considerably fewer than four
documents being moved out of Proposed Standard in the last 22
months in the absence of the change.

While the apparently-significant reduction in documents moved
out of Proposed Standard is far worse than we expected, is it
time for Scott Bradner and myself to review
draft-bradner-restore-proposed-00, issue a new version, and
start a serious discussion about that model of a solution?
Would be willing to sponsor such a draft or, if you prefer,
organize a WG or equivalent to consider it?

thanks,
   john

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 31 Jul 2013 08:19:04 +1200

> organize a WG or equivalent to consider it?

I would rephrase it as performing the inverse operation to RFC 4450.
I don't think it needs to be a fully-fledged experiment under RFC 3933;
just a large batch of PS->IS promotions in one go under RFC 6410.

It is a good idea.

    Brian

---

From: SM <[email-masked]>
Date: Tue, 30 Jul 2013 14:00:00 -0700

>months in the absence of the change.

"Changing the Internet Standards Process from three maturity levels 
to two is intended to create an environment where lessons from 
implementation and deployment experience are used to improve 
specifications".   The change could be rated as a non-change if there 
were only four specification moved to Internet Standard since then.

The hurdle in moving a specification (not a RFC) from PS to IS is 
that the draft goes through IESG Evaluation again.  As for public 
review, it can be a hurdle too as the pervious discussions can be 
rehashed.  A PS specification which sticks to what goes over the wire 
turns these hurdles into a lesser effort.

draft-bradner-restore-proposed-00 proposes a nice fix and it might 
even help lessen time to publication.

Regards,
-sm

P.S. Olaf asked the question to the correct body.

---

From: Dave Crocker <[email-masked]>
Date: Tue, 30 Jul 2013 23:11:09 +0200

On 7/30/2013 11:00 PM, SM wrote:
> The hurdle in moving a specification (not a RFC) from PS to IS is that
> the draft goes through IESG Evaluation again.  As for public review, it
> can be a hurdle too as the pervious discussions can be rehashed.

And, of course, if it "can" be reshashed, in the IETF it will be.

However the specification for the new two-stage model provides criteria 
for the second stage, and it does not include re-evaluating the 
technology or its details.  Instead it focuses on adoption and use.

No doubt, there will continue to be IESG and other IETF participants who 
insist on treating consideration of Full standard as an excuse to 
discuss first principles, but it's not within scope.

Over time, as more specifications are processed to Full, the community 
will learn to be a bit more efficient about it.

d/

-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: Michael Richardson <[email-masked]>
Date: Tue, 30 Jul 2013 21:03:47 -0400

Just to make sure we have good data, can we go back a few more years?
Specifically, did we not previously have a restriction forbidding references
FS->DS, and {FS,DS}->PS?.... RFC3967 was in Dec. 2004, but I thought that we
had some other work more recently (2008?) that attempted to unjam things.

What I'm wondering is if the 20 actions prior to 6410 was an anomaly,
and really the historical rate of upgrade is really lower.

I think it is also important to understand how many new PS could have been
even ready at a given point for an upgrade.  I don't have a good/simple
metric for determining this, I think it takes some significant semi-expert
review.  Perhaps someone has an Information Science (?!) or Economics
graduate student who wants to mine the data and write a paper :-)

--
Michael Richardson <[email-masked]>, Sandelman Software Works

---

From: IETF Chair <[email-masked]>
Date: Wed, 31 Jul 2013 10:36:51 +0200

> and really the historical rate of upgrade is really lower.

A quick couple of seconds with mail search and grep says that in the four years preceding publication of RFC 6410, we made 28 actions. As noted earlier, 20 of those happened in the 22 months preceding the publication of RFC 6410. So in the 2+ years before that only 8 were produced.

I'd love to have a graph that plots DS/FS/IS progression per year, over time. But I don't have time to make that today. Unless someone else wants to...

Jari

---

From: SM <[email-masked]>
Date: Wed, 31 Jul 2013 02:28:43 -0700

Hi Dave,
>And, of course, if it "can" be reshashed, in the IETF it will be.

Agreed.

>However the specification for the new two-stage model provides 
>criteria for the second stage, and it does not include re-evaluating 
>the technology or its details.  Instead it focuses on adoption and use.

RFC 5011 is one of the rare RFCs which does not have any 
erratum.  The move wasn't problematic.  It is not clear whether RFC 
6891 is an odd case or symptomatic of the difficulties in updating a 
specification.

The easier way not to have a re-evaluation is not to reopen the RFC 
(e.g. RFC 5011).  Anything else means having two consensus calls on 
the text.  draft-bradner-restore-proposed-00 proposes an alternative 
where the specification is not expected to be complete and comprehensive.

Here's a minimum standards profile published in 2009:

   SMTP IETF STD 10 (RFC 821, 1869, 1870)

   DNS IETF STD 13 (RFC 1034, 1035)

   HTTP v1.1 (RFC 2616), URL (RFC 1738), URI (RFC 2396)

   IPv4 (RFC 791, 792, 919, 922, 1112)

   BGP4 (RFC 1771)

The ex-IAB Chair explains that RFC 1035 is an Internet Standard.  He 
then explains that an Experimental Standard (RFC 1183) updates that 
Internet Standard.  The Internet Standard is updated by a Best 
Current Practices (RFC 6895).  There is also a Proposed Standard ( 
RFC 1982) which updates the Internet Standard.  He recommends RFC 
2026 for further information.  He receives an enquiry afterwards 
about immature specifications.  He decides to ask the question at the 
plenary.  The community thanks him for his valiant efforts.

The alternative is to settle for Informational Standards as that 
requires less effort.

Regards,
-sm