Subject: Re: [Last-Call] Six months in: evaluation of the last-call mailing list experiment

From: "Matthew A. Miller" <[email-masked]>
Date: Fri, 1 May 2020 07:43:44 -0600

< 4: I really like it and want to keep it. >

I think the list is a very worthwhile improvement, and the suggested
inclusion of LC announcements would make it better.

- m&m

Matthew A. Miller
On 20/04/30 09:15, Barry Leiba wrote:
> (I'm replying to this and adding the last-call list, which I forgot to
> do when I sent it.  Please reply to *this* message, to keep both lists
> included.)
> 
> On Thu, Apr 30, 2020 at 11:00 AM Barry Leiba <[email-masked]> wrote:
>>
>> It's been around six months since we started the <[email-masked]>
>> list, and we said that we would evaluate the results after six months.
>> To that end, the IESG would like to see comments about how it's been
>> working.
>>
>> Please respond to this thread to comment.  It would be helpful if the
>> first line of your comment gave a succinct view of your opinion on the
>> following 5-to-1 scale:
>>
>> 5: Perfect!  Don't even think about going back!
>> 4: I really like it and want to keep it.
>> 3: Neutral: I don't care either way.
>> 2: I don't like it, but I can live with it if we decide to keep it.
>> 1: It's terrible!  Please, please go back to the old way!
>>
>> And, of course, if you have further comments beyond those numbers,
>> include those as well.  We want to know what you think.  Comments,
>> please, in the next couple of weeks, by 15 May.  Thanks!
>>
>> Barry, for the IESG
>

---

From: Barry Leiba <[email-masked]>
Date: Fri, 1 May 2020 09:54:28 -0400

On Thu, Apr 30, 2020 at 5:40 PM Barry Leiba <[email-masked]> wrote:
>
> > > 4, Please keep it. However, I think it could be improved if you also sent LC *announcements* there.
> >
> > I hope you mean to Cc: them there.
>
> Of course; if we do that, it will be with CC.

We now have ample support for CCing the last-call list on
announcements.  I want to check explicitly for objections.

Please add to your comments if you *object* to the following:
"If we decide to keep the last-call list, we will add a CC to that
list on last call *announcements*, in addition to sending the
announcements to the ietf-announce list (as now)."

Barry

---

From: Bob Hinden <[email-masked]>
Date: Fri, 1 May 2020 08:35:03 -0700

Barry,

Overall, I don’t care.

However, sending this thread to the [email-masked] list, seems to me to defeat the purpose of the last call list.   That is, it is to be used for IETF w.g. last calls.   Other discussions, like this one, should happen on the [email-masked] list.

Bob

> On Apr 30, 2020, at 8:15 AM, Barry Leiba <[email-masked]> wrote:
> 
> --
> last-call mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/last-call

---

From: Keith Moore <[email-masked]>
Date: Fri, 1 May 2020 11:44:37 -0400

On 4/30/20 7:12 PM, Joel M. Halpern wrote:

> Keith, why do you say that last call comments are going all over the 
> place.  The last call announcement sets reply-to to the last-call 
> list. So unless people deliberately change it, that is where those 
> comments go.

I haven't tried to survey MUAs for this behavior in a few years, but in 
practice I don't think you can depend on reply-to to be treated 
consistently by all MUAs.

The behavior of email users, MUAs, list software, etc. are all somewhat 
unpredictable, and I suspect inherently so, for a variety of reasons 
which for the sake of brevity I'll not mention here. So it's difficult 
for the sender of a message to force recipients to behave in the ways 
that the sender specifies.

-----

The Last Call announcements go to lost of places: ietf-announce, the 
working group mailing list, a specific email address for the document 
([email-masked]), [email-masked], [*] and separately, 
the addresses of the chairs of the WG.

The reply-to on the Last Call announcement is indeed set to 
[email-masked].   However the mailing list typically adds a 
List-Post header, and some MUAs encourage replies to go to that header 
when it is present.

IMO it might be desirable for a WG mailing list to be in the loop for 
Last Call comments (and for the WG list to be cc'ed on announcements), 
but I would still like all of those comments to be collected together 
and made publicly visible in one obvious place.  Whether the best place 
for that is the [email-masked] list, or the datatracker, or some 
other place, I'm not entirely sure.   For archive purposes the 
datatracker seems good, for the purpose of having an effective ongoing 
conversation there's an advantage to keeping email messages in the email 
system and using ordinary email tools to deal with them.

The text of the announcement does say "Please send substantive comments 
to the [email-masked] mailing lists [sic] by {date}". So that's good.

-----

I'm tempted to suggest that every Last Call announcement should be From 
and have Reply-To set to [email-masked] (where {number} is 
unique to that Last Call) and for that address to forward the message 
(adding/modifying To/Cc header fields as necessary) to wherever it needs 
to go: perhaps [email-masked], perhaps the WG mailing list, perhaps 
the datatracker, certainly an archive.   The same 
[email-masked] should be specified in the announcement text 
itself.   That way, all of the responses would be funneled through one 
place and processing/archiving/etc. would be consistent.    Searching of 
messages would also be more reliable as one could look for that specific 
address in the to/cc message headers.

Adding {area} and {wg}, e.g. [email-masked], 
where applicable, might facilitate easy searching for Last Call traffic 
in a particular area or WG.

What makes me only "tempted" to suggest this, is that I don't know 
whether people have email filters in place that this would confuse (lots 
of people expect intended recipient addresses to appear in the to/cc 
headers, even though this has never been required or expected), or if 
there are still some WG lists hosted outside of ietf.org that would 
alter the reply-to field, etc.  The email network is a vast and diverse 
place.

For the moment, these are the recommendations that make sense to me as 
being safe changes:

1. Announcements should CC [email-masked] in addition to the long 
list of other CCs.

2. Last Call comments submitted via the datatracker should use the same 
Subject format as replies to the announcement:

Re: Last Call: <{internet-draft-identifier}> ({internet-draft-title}) to 
{intended-status}

(even if the Last Call comment is generated before the announcement - 
the Re: prefix is important to distinguish the responses from the 
announcement itself)

3. The Last Call responses, along with the announcement, should be made 
easily viewable in some way, perhaps via the datatracker, perhaps via an 
email archive that's specific to the Last Call.

(I've not found the IETF email archives to be very usable in the past, 
but improving them might make more sense than fixing the datatracker to 
be a good email archive.)

Keith

[*] notation: I'm using curly braces {} as a substitute for italics, 
because some email processors don't preserve HTML and some of them 
translate HTML in weird ways that make it less readable than plain text.

---

From: Barry Leiba <[email-masked]>
Date: Fri, 1 May 2020 12:01:01 -0400

(Removed last-call from this, as it's meta-discussion.)

Hi, Bob,

> However, sending this thread to the [email-masked] list, seems to me to defeat the purpose of the last
> call list.   That is, it is to be used for IETF w.g. last calls.   Other discussions, like this one, should happen
> on the [email-masked] list.

That's a good point, and I mostly agree.  I decided to send this to
last-call also, because it involves the very existence of that list,
and I wanted to be sure to catch people who follow <last-call> but not
<ietf>, as we need their input as well.  Perhaps I should have just
posted a notice about the discussion and asked people to come here to
respond.  In any case, thanks for keeping me honest.  :-)

Barry

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Fri, 01 May 2020 19:06:41 +0200

On Fri, 2020-05-01 at 08:35 -0700, Bob Hinden wrote:
> Barry,
> 
> Overall, I don’t care.
> 
> However, sending this thread to the [email-masked] list, seems to
> me to defeat the purpose of the last call list.   That is, it is to
> be used for IETF w.g. last calls.   Other discussions, like this one,
> should happen on the [email-masked] list.

well, that would exclude users like me from giving feedback, who are
happy to be on <[email-masked]> (for the record: "4") and are not
following <[email-masked]>.

I don't know how many are in my position, but I noted at least this
response from Alexey Melnikov <[email-masked]>:

> (In fact, I unsubscribed from [email-masked])

-- 
venleg helsing,
Kjetil T.

---

From: Adam Roach <[email-masked]>
Date: Fri, 1 May 2020 13:07:44 -0500

On 5/1/2020 12:06 PM, Kjetil Torgrim Homme wrote:
> happy to be on<[email-masked]>  (for the record: "4") and are not
> following<[email-masked]>.

This is true, and I think it's compatible with what Bob is saying: this 
is the kind of conversation that [email-masked] is specifically set up 
for. It's entirely reasonable for you to opt out of such conversations. 
But by doing so, you're opting out of such conversations.

/a

---

From: "Timothy J. Salo" <[email-masked]>
Date: Fri, 1 May 2020 13:16:06 -0500

I don't know whether it makes the world a better place, but I read the
last call discussions because they were on a separate list.  I
wouldn't have bothered to, and don't bother to, follow these discussions
when they are held on [email-masked].

-tjs

---

From: Rob Sayre <[email-masked]>
Date: Fri, 1 May 2020 11:38:00 -0700

> But by doing so, you're opting out of such conversations.

Yes. But for this specific poll, it is important to have last-call@
included, so people that have tuned out ietf@ are consulted.

fwiw, I am still subscribed to ietf@ but it goes into a folder that is not
displayed by default, and I don't remember to check it very often.

thanks,
Rob