Subject: Re: Updated IESG Statement "IESG Processing of RFC Errata for the IETF Stream"

From: John C Klensin <[email-masked]>
Date: Mon, 10 May 2021 08:45:53 -0400

Spencer, 

Trimming further -- your explanation about what has gone on / is
going on in the CELLAR WG was interesting and helpful, but I
have little or nothing to add that is specific to it.  On the
other hand, a situation where there is an active WG that
developed the document on which errata were filed seems to me
much different -- and with far more opportunity for real
consensus about decisions-- than the more usual errata
situations.  See below.

--On Sunday, May 9, 2021 16:45 -0500 Spencer Dawkins at IETF
<[email-masked]> wrote:

>...
> So, ISTM that what our current errata system was set up to do
> (and I can't emphasize enough that I wasn't there when it was
> set up), was to Reject stuff that wasn't REALLY an errata or
> obviously not the right correction to make, Verify obvious
> corrections, and Hold Everything Else For Document Updates, so
> that a working group has a chance to talk about what The Right
> Thing To Do is, given that they missed something Technical
> when they looked at this for the first time, as opposed to
> having an AD decide *now* what The Right Thing To Do will be
> at some point in the future, and constraining the working
> group when they *do* have enough brain cells to update the
> document. (If that's not the case, my apologies to the people
> who set it up in the first place).

But that view suggests that Bob's case of a proposed erratum
correctly identifying a problem but not the right fix should
lead to a Reject conclusion, and that does not seem quite right
either.

> If we wanted to make life easier for people using IETF
> specifications (as opposed to going to the Github repos and
> "making" their own specifications from Master/Main branches,
> we might think about
> 
>    - How much we want to "warn people away" from using the
> really helpful    errata inlining mechanism that is already
> available from the RFC Editor,    and
>    - How much stuff that's not Verified would be useful to
> present to the    people who use our documents - especially
> stuff that's Rejected because it    doesn't reflect the
> thinking of (my own favorite) the NFSv4 working group    more
> than a decade ago, but is really close to what they are
> thinking today.
> 
> As always, thanks for listening. And Do The Right Thing.

For a situation where there is an active WG with the best
document as part of its charter, let me suggest something else,
starting with wondering whether the errata system is the right
approach at all.  If, as I gather is the case you are talking
about, the WG gets a document published and then immediately
starts discovering bugs in it, isn't The Right Thing for the WG
to put a revision onto its agenda, get a new I-D started, and
put the corrections there?  THere makes the status immediately
clear as "WG work in progress" without any handwaving about what
"Verified" actually means.  A "Changes from RFC NNNN" section,
which would be desirable anyway, could explain things to the
degree desired.  If the WG wanted to put a note on that I-D that
says that everything in it represents WG consensus, that would
be much more clear than the present errata process [1].

The missing link is something that might be desirable for many
other reasons: It seems to me that, if an RFC is actively under
revision in an IETF WG, an "under revision" note on the RFC
Editor's page for that RFC with a pointer to the WG charter
and/or the relevant I-D would be a big help to readers.  It
would also capture the common case in which WGs make many
changes to a document on which errata are not filed.  

However successful we are (or are not) at convincing people that
I-Ds are not standards, pointing a reader to a revision in
progress makes it clear that something is going on, may give
insights into the situation with the existing RFC. In the best
of all possible worlds, it might even encourage people who think
they are finding problems in a document to participate in the
relevant WG and discuss their issues.

Finally, if issues are addressed in a WG that is active and
responsible for a document anyway, that reduces the workload on
everyone who would need to be part of a round trip through the
errata system before the issue gets back to that WG anyway.

With the understanding that the situation in particular WGs may
be different, does that sound to you like possibly a better
approximation to The Right Thing for many or most cases?

   best,
    john

[1] While I would hope ADs or Chairs of active WGs with
responsibility for a document against errata were filed would
consult with the WG about the erratum, there is nothing in the
IESG statement that requires that.

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Mon, 10 May 2021 17:18:54 -0500

Hi, John,

> lead to a Reject conclusion, and that does not seem quite right

I'm sorry if I wasn't clear - the understanding I was describing was my
own, in general, and definitely not talking specifically about CELLAR.

Your point above - about an active working group making informed decisions
about a document that they had produced - is very much accurate for CELLAR.
Reported errors (not from the errata system, but from implementers who
likely would have no idea where to go to file an errata report on an RFC)
are discussed promptly on the mailing list, and if it seems like there's
more than one opinion, in the next monthly meeting (CELLAR has about 10
meetings per year), and pretty quickly, they know what the answer should
be. But see below.

> be much more clear than the present errata process [1].

So, let's keep solving MY problem, which isn't what to do when CELLAR gets
an error report, but about what to do after they agree that they need to do
something to fix the specification.

Two things about that.

First, their RFC describes a markup language that they use to describe
Matroska media containers, and their soon-to-be-PUB-REQUESTED draft
describes pre-IETF versions of an archival video codec. What they really
WANT to do is finish Matroska, which is (as best I can tell) their premier
deliverable, and work on a new version of that archival video codec. They'd
like their published specifications to be correct, but any effort they
spend on corrections is taking away from effort that they can spend on
their next deliverables. So, if the errata system worked the way we wish it
would, that would be fine, but it just doesn't work the way we wish it
would (for more reasons than I know).

So, the other thing is - I am not sure whether you are thinking of an
UPDATES RFC NNNN WITH ERRATA draft, or an RFC NNNN-bis draft that replaces
RFC NNNN, but each of these has its own special agony.

   - If one were to take a look at
   https://datatracker.ietf.org/doc/rfc8540/ballot/, for an UPDATES RFC
   4960 WITH ERRATA draft, they will  be impressed that an Informational
   document saying "this is what the working group thinks of its errata"
   managed to gather (counting again) *9* abstains from ADs who (summarizing)
   said roughly "no value in publishing this, Should be a wiki page. Where's
   the RFC 4960-bis draft?" I don't know what the current IESG thinks about
   docs like this, but what the 2018 IESG thought about them is
   well-documented(1). Note that "publishing what the working group thinks
   about errata in a wiki page" is exactly what CELLAR is trying to avoid, and
   (for extra credit) might not even cross the inbox of an AD.
   - If one were to take a look at
   https://datatracker.ietf.org/doc/draft-ietf-tsvwg-rfc4960-bis/, the
   requested -bis draft, they would note that the UPDATES RFC 4960 WITH ERRATA
   draft was published almost exactly two years ago, and the REPLACES RFC 4960
   draft is still sitting comfortably in the working group.

So, we still have some work to do here.

"We" being the IETF, and not just dumping this on the current IESG, unless
they have a lot of time and energy on their hands.

I should semi-apologize to you for forgetting that you had written
https://datatracker.ietf.org/doc/html/draft-klensin-newtrk-8540style-harmful-00
on this EXACT situation, but I even commented on it at the time (in
discussion on IETF mailing list). So I know you see the problem pretty
clearly.

(1) To be clear, this is NOT a knock on the 2018 IESG. I'm just pointing to
a document I'm familiar with, as the then-responsible AD.

> approximation to The Right Thing for many or most cases?

Modulo how much I wish we'd gotten
https://datatracker.ietf.org/doc/html/draft-ietf-newtrk-repurposing-isd-04
out of the NEWTRK working group in 2006 because we could have stopped
thinking about this more than a decade ago, yes!

Best,

Spencer

---

From: Lloyd W <[email-masked]>
Date: Tue, 11 May 2021 15:36:26 +1000

Let me outline how this is all going to work in future. Prove me wrong.

Every git pull request on a document being edited under git revision control is by its very nature an errata report. errata are verified by being committed to main branch.

there are no final published documents, only git repositories with checkpoints, which are eventually given to the RFC Editor function to maintain as workgroups close. Implementations are based on document checkpoints, timestamped.

If you want to read an RFC or internet draft, you need to fire up git to go get it. Or rather, its up to date authoritative main revision repo tree.

And you thought the fuss over turning off ftp access to RFCs was bad...

Lloyd Wood
[email-masked]

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Tue, 11 May 2021 02:21:27 -0500

Hi, Lloyd,

On Tue, May 11, 2021, 00:36 Lloyd W <[email-masked]> wrote:

> Every git pull request on a document being edited under git revision
> control is by its very nature an errata report. errata are verified by
> being committed to main branch.
>
> there are no final published documents, only git repositories with
> checkpoints, which are eventually given to the RFC Editor function to
> maintain as workgroups close. Implementations are based on document
> checkpoints, timestamped.
>
> If you want to read an RFC or internet draft, you need to fire up git to
> go get it. Or rather, its up to date authoritative main revision repo tree.
>

I'm most familiar with WHATWG, and their "living standards", but I think
that's reasonably close to what you're describing.

https://whatwg.org/

So I wouldn't dare argue that what you sketched out is unlikely. ISTM we're
just trying to figure out how close, or how far away, that's going to be to
what the IETF is doing in ten years.

Best,

Spencer

---

From: "[email-masked]" <[email-masked]>
Date: Tue, 11 May 2021 07:29:31 +0000 (UTC)

Spencer,

Ah, WHATWG. The memories!

It was interesting watching the draft process and working group interact with Hixie's preferred way of working; every minor tweak to his draft resulted in a new version being pushed onto everyone, so no-one knew what they were discussing and we were just flooded with noise.

https://datatracker.ietf.org/doc/html/draft-hixie-thewebsocketprotocol

76 versions before the 'cut it out' contingent won out.

Of course, this was pre-heavy git use, where anyone just on the mailing list wouldn't even notice what was happening in git or be in the loop.

yes, WHATWG was a horror show.

Lloyd Wood
[email-masked]

there are no living standards. There are dying standards, and undead standards.

On Tuesday, 11 May 2021, 17:21:40 GMT+10, Spencer Dawkins at IETF <[email-masked]> wrote: 

Hi, Lloyd,

> If you want to read an RFC or internet draft, you need to fire up git to go get it. Or rather, its up to date authoritative main revision repo tree.

I'm most familiar with WHATWG, and their "living standards", but I think that's reasonably close to what you're describing.

https://whatwg.org/

So I wouldn't dare argue that what you sketched out is unlikely. ISTM we're just trying to figure out how close, or how far away, that's going to be to what the IETF is doing in ten years. 

Best,

Spencer

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 11 May 2021 09:30:50 +0200

On 11. May 2021, at 09:21, Spencer Dawkins at IETF <[email-masked]> wrote:
> https://whatwg.org/ <https://whatwg.org/>
> 

(1) This is completely unrelated to the subject of errata reports and their processing.

(2) The WHATWG process works great if you have a couple of people sitting around a table and deciding what will be implemented in their products next.
Many IETF standards have more complex audiences and more complex interoperability requirements.
I don’t think the WHATWG style will replace the style with defined milestones (“versions”) completely for the IETF, probably not even for a large subset.
We could get more agile churning out those milestones, which is where the focus should be.

Grüße, Carsten

---

From: Julian Reschke <[email-masked]>
Date: Tue, 11 May 2021 09:41:47 +0200

> If you want to read an RFC or internet draft, you need to fire up git to go get it. Or rather, its up to date authoritative main revision repo tree.

FWIW; even if all of the previous points were true: you wouldn't need to
fire up Git, but just open a specific HTTP(s) URL in a browser.

Best regards, Julian