Subject: Guidelines for authors and reviewers

From: Suresh Krishnan <[email-masked]>
Date: Thu, 29 May 2008 18:04:49 -0400

Hi Folks,
   We have written a draft describing some guidelines for authors and 
reviewers of internet drafts. We would really appreciate it if you can 
spend some time to go over it and provide comments and/or suggestions 
for improvement.

Thanks
Suresh, Pasi and Eric

-------- Original Message --------
Subject: I-D Action:draft-krishnan-review-process-00.txt
Date: Wed, 28 May 2008 11:15:01 -0700 (PDT)
From: [email-masked]
Reply-To: [email-masked]
To: [email-masked]

A New Internet-Draft is available from the on-line Internet-Drafts 
directories.

	Title           : Guidelines to authors and reviewers regarding the 
IETF review process
	Author(s)       : S. Krishnan, et al.
	Filename        : draft-krishnan-review-process-00.txt
	Pages           : 10
	Date            : 2008-05-28

This document describes the IETF review process and provides
guidelines to draft authors and reviewers on how to effectively
participate in it.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-krishnan-review-process-00.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

_______________________________________________
IETF mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ietf

---

From: "Frank Ellermann" <[email-masked]>
Date: Fri, 30 May 2008 14:31:52 +0200

Suresh Krishnan wrote:

> We would really appreciate
[...]
In the spirit of your draft:  s /one an author/once an author/ ;-)

As the draft says, the main thing is to get any reviews at all:
A "negative" review means that somebody bothered to read it, and
to note whatever attracted their attention.  I'm not sure if 
that needs a separate "review netiquette RFC", IMO it should be
a part of the "Tao", or the next Tao if it is not already clear.

Please use the term "review team" instead of "directorate".

 Frank

---

From: Ted Hardie <[email-masked]>
Date: Fri, 30 May 2008 10:50:05 -0700

>spend some time to go over it and provide comments and/or suggestions

Hi Suresh, Pasi, Eric,

	I looked at it, and, while I laud any efforts to get folks to review
things effectively, I have to say that I found this to be a pretty drafty draft.
It does not reference the Tao, 2026, or any of the developed educational materials;
its only listed reference, in fact, is 2119 and that does not seem to be referenced
within the text.  That means that this comes across as pretty context free.
It needs anchoring to the rest of our processes.
	One of the things that I believe that anchoring should provide
is a pretty significant change of perspective.  As this reads now,
it implies a lot of power in the hands of reviews to elicit (or even require)
change.   It seems to want document authors to accede to requests for
tutorial material as a matter of course and to significant technical changes
with a modicum of fuss.  That's not the right approach.  The outcome of our
document development should not be a negotiation between the authors
and the assigned reviewers.  It should be a conversation in the working group
among those who will actually develop the implementations, those who
will deploy it, and those who are affected by the system of which the
documented technology  is a part. 
	Reviews that work to relate a particular document's technology
to the larger whole of which it is a part (asking: how does this impact
congestion control in the access network or core, use deployed security systems,
relate to the identifier mechanisms common to URIs, etc.) are very valuable.
But many reviews represent questions about decisions that come down to
design choices that working groups should have the power to make without
extensive second-guessing.  Folks who want to have a say at the level can and
should do so with the simple method of joining the working group list and commenting
as part of the general development.  That's not "review" (of someone else's work)
it is "participation" (in joint work), and it is fundamentally more valuable.
Without the context of how "participation" works, your documents description
of "review" comes off very badly indeed.  I hope that future versions can correct
that and place review within a broader, more productive context.
			good luck,
				Ted

---

From: Suresh Krishnan <[email-masked]>
Date: Fri, 30 May 2008 17:03:41 -0400

Hi Frank,
   Thanks for your comments. Please see responses inline.

> In the spirit of your draft:  s /one an author/once an author/ ;-)

Will fix :-)

> a part of the "Tao", or the next Tao if it is not already clear.

Paul Hoffman is working on the TAObis. Maybe he can chime in on this.

> Please use the term "review team" instead of "directorate".

OK. Will do.

Thanks
Suresh

---

From: Suresh Krishnan <[email-masked]>
Date: Fri, 30 May 2008 17:42:55 -0400

Hi Ted,

> It needs anchoring to the rest of our processes.

I do share your sentiment, but this is not what we set out to do 
originally. We set out to document the review process(es) and provide 
some guidelines to effectively use the received reviews. I think the 
right document to weave the stuff together would be the Tao. But I do 
agree that adding references to the Tao and 2026 make sense. The 
dangling reference to 2119 is an editorial oversight :-).

> tutorial material as a matter of course and to significant technical changes

I am sorry you feel that way. It was not the intent of the document. The 
document just states that a concern raised in the review deserves a 
response (not necessarily a change). One valid response is "Mr. 
reviewer, what you raised is not a problem. This was a design decision 
taken by the WG and the discussion thread about this is at 
http://mlarchive.invalid/msgfoo.html".

> documented technology  is a part. 

Agree. And hence this text in section 3.5

"   After the author and reviewer agree that the issues have been fixed,
    for working group documents, substantial issues need to be confirmed
    on the mailing list.  Once the document has entered IESG processing,
    new versions should not be posted unless the document shepherd or AD
    explicitly says so."

> it is "participation" (in joint work), and it is fundamentally more valuable.

Agree in principle. But a "late" reviewer usually gets documents from 
multiple wgs and subscribing to the MLs just to raise one issue can get 
to be a tiring endeavor. Last year, I reviewed documents from 20 
different WGs (apart from the ones I am usually involved in). I cannot 
handle mailing list traffic from 20 more mailing lists. If this were 
mandated, I would rather not review the document. I would also like to 
point out that most of the reviews received during IETF Last Call are of 
the nature you describe.

> that and place review within a broader, more productive context.

I personally feel that the TAO (4677 and descendants) should be the 
document that sets the various pieces in context, but I am open to 
suggestions on how to go about fitting this document into a broader 
context.

Thanks
Suresh

---

From: Paul Hoffman <[email-masked]>
Date: Fri, 30 May 2008 14:59:01 -0700

At 5:03 PM -0400 5/30/08, Suresh Krishnan wrote:
>I'm not sure if

<ching!>

The past few editions of the Tao do indeed talk about taking reviews 
with an open mind. The Tao doesn't talk much about *giving* reviews, 
mostly because the intended audience (IETF newcomers) are mostly 
interested in learning how to be in the normal IETF structures, like 
WGs.

Having said that, I agree with some of what Ted Hardie said about the 
tone of the document. It sounds like there are instructions to 
document authors on how they are supposed to act when they get 
reviews. That's bordering on a revision to RFC 2026, which I don't 
think is what you intended. "It is polite to" and "some document 
authors like to" are quite different than "are expected to" and 
"needs to".

This document emphasizes reviews going to authors instead of reviews 
going to WGs or, in the case of individual submissions, reviews going 
to mailing lists. In the Tao, we emphasize the value of 
communications to groups so that the group can agree, amplify, show 
disinterest, or disagree. In the WGs I have co-chaired, the WG got 
good value out of some of the GenART and SecDir reviews in that it 
made the whole WG think about the topics brought up. This may be a 
fundamental difference in view between this document's authors and my 
preferences, but I think the discussion of where reviewers should be 
sending their reviews is an important one for the IETF community to 
have.

--Paul Hoffman, Director
--VPN Consortium

---

From: Ted Hardie <[email-masked]>
Date: Fri, 30 May 2008 15:30:01 -0700

>dangling reference to 2119 is an editorial oversight :-).

If the Tao is the right document to do the weaving in, then the right thing
to do may well be incorporate more discussion of the review
process in the next version of the Tao.  A stand-alone document which
says "to get this context go read this first" might work, but it runs
the usual risk.  And statements like:

"reviews also play an important part in how IETF determines
   consensus. "

really need to be related to our process documents.  Personally, I
don't think that's actually either true or the point.  Many solicited
reviews  are meant to discover technical show stoppers or areas where
the impact of technology on other parts of 'net have not been
taken into account.  Determination of consensus is different,
and it has a large amount to do with our participation model.
Changes to the pool of folks from whom consensus is
solicited should not be done lightly in informational documents.

>http://mlarchive.invalid/msgfoo.html".

The tutorial material comment comes from this section:

      If the reviewer has not understood some part of the draft, a very
      common response is to explain the topic in email.  However, often
      that explanation should really be in the draft itself, not just in
      emails to the authors, so that future readers will also understand
      the text.

The key missing piece here is that the effort to get outside reviewers
means that they are missing context that those actually using the
document will likely have.  Asking that it be included as a matter of
course on the query of a reviewer produces changes that then have to
be reviewed and agreed to by the working group.  That's a lot of work
for a lot of people for things which may well be obvious to those
within the group using a technology. 
"
The document's description of politeness in responses also sets
timeframes that sound a lot like deadlines:

"two or three business days might be a
   reasonable amount of time to take for a response.  It is possible
   that the authors may be unavailable to respond to a review within
   this timeline since they may be sick/on vacation/busy with dayjob
   etc.  In this case, the document shepherd (if any) can respond to the
   review and follow up with the authors at a later time."

The ability to force an interrupt on busy people (either authors or shepherds)
is real power, and we need to be careful how we cast that.  A comment on a document
(and that is what reviews are) does deserve to be considered, but the extent
to which a response is required and the timeframe for that response is much
more fluid than this implies.  Bundling all comments received during a
Last Call, considering them as an author team, then issuing a new draft
or set of RFC Editor notes (or, honestly, punting them all) may be the most
effective thing to do.  In that case, the only early response you'll
get is "message received".  If you want that, permit me to suggest
the handy MDN functionality likely available in your mail client.

>    new versions should not be posted unless the document shepherd or AD

You say "agree", but the text you point to says something substantially
different from what I did.  The text you point to says, in essence,
"after the reviewer and document author have finished their negotiation,
the mailing list should confirm".  That's exactly wrong.

>handle mailing list traffic from 20 more mailing lists.

I must not have been clear.  I am not saying that a reviewer must subscribe
to a mailing list to raise an issue.  I am saying there are two classes of
comments: 

1) issues raised about the impact of the technology on the larger 'net
	or the usage of the technology within a larger context;

2) discussion of design decisions. 

Late comments of type 2 by folks who did not participate in the design
discussions should expect fairly short replies; the right time and place
to involve yourself in that is during the participation phase.  If you
deeply care that folks using ZNK use its ChillOut encoding when describing gttp
extensions, you can  join the gttp mailing list and argue the point.  If you
come as a reviewer and find they have chosen  ZNK using its older
BindMeTight encoding, you may ask something like "Were the trade-offs
between BindMeTight and ChillOut considered?" and be satisfied
with "Yep" (or prepared to help if they say "What's ChillOut?").  But if
you push back on the use of BindMeTight, you had better be able to
say how using it causes the protocol to fail to meet its goals.  Anything
less is not an appropriate late-stage review comment.  It comes
across as an attempt to subvert the process for attaining consensus.

>If this were

I tried to describe two different types, so it is not clear to me which you
mean here.

>suggestions on how to go about fitting this document into a broader

I have obviously not been clear here either, so permit this bluntness:
as it stands now,this document is harmful.  It fails to place the process
in sufficient context.  The result is a description that over-emphasizes
late stage review and satisfying those who conduct those reviews.
It needs to be re-written using a full context, folded into a document
that establishes that context, or abandoned.  Having it go forward
alone, without that context, is not appropriate.

To end on a high note, it is beautifully formatted.

				regards,
					Ted Hardie

---

From: "Joel M. Halpern" <[email-masked]>
Date: Fri, 30 May 2008 19:08:37 -0400

I have to strongly disagree with the tone of this review of the review 
document.
(Which is not to say that Ted isn't entitled to provide the review, or 
that he is not entitled to a reasonable response.)

There are several issues that get mixed up together in defining a late 
external review process.  By definition, this is an external review.  So 
it is not reasonable to say that the reviewer should think like a WG 
member, or have the full email conversations available.

So, the first question is what should happen when a reviewer says "X 
does not make sense" or "Y seems to be assumed without explanation." 
Some of the time, that is because that context is provided by other 
documents which are reasonable to have read.  Someone needs to say that. 
  (If a reviewer has become confused, likely some AD sill also become 
confused.  Maybe the sponsoring AD will realize that the solution is 
some other document that is already referenced.  But then again, maybe 
he won't.)
An awful lot of the time though, the issue is one of "common knowledge". 
  Sometimes bluntly stated as "all the participants understand that Z, 
which means ..."  Well, the fact that working group members know Z does 
not mean taht readers know it if it is not part of some other document 
that is a reasonable context.  (The OSPF base document is reasonable 
assumed context for an OSPF extension.  But the full set of all 
published BGP extensions that are not mentioned in the document is not 
suitable context for a BGP extensions.  Fortunately, the recent 
extension I reviewed did state what document was needed for context.)
If the reviewer is confused, it is reasoanble to assume that other 
readers will be confused.

On design decisions, there is an evenr more complex tradeoff.  I have 
reviewed several documents which had questionable design decisions.  In 
one review I recently wrote that I did not expect the following comment 
to change the WG consensus, but that I considered the specific mechanism 
used by the document a bad idea.  If I had not known that the particular 
mechanism had been discussed, I might have put it more forcefully.
On the other hand, a while back I reviewed a document which had a 
mechanism which, although the working group had indeed agreed on it, 
fundamentally didn't work.  I don't care how much they agreed.  It 
needed to be changed.  And they changed it.  (It was incumbent upon me 
to provide a clear and coherent explanation of why it was broken.)

The problem I see is that working groups make all sorts of interesting 
mistakes.  Sometimes deliberately, sometimes by accident.  The job of 
the external reviewer is to look for such things.
Given that we have a policy that all comments deserve responses, it 
seems particularly appropriate that review team comments better get 
responses.

Also, on the time frames, it is worth noting that these are late 
reviews.  If you are lucky, they are produced early in the LC cycle. 
But even reviewers are human and take some time.  But if the review is 
not responded to in a timely fashion, then other folks are likely to 
assume that there is a real problem that needs to be addressed, rather 
than finding out that the reviewer simply misread the document 
(reviewers make mistakes too.)

The whole tone of your comment seems to be "these should have been made 
during and as part of the WG process.  But late external reviews benefit 
from being external.  (When I write a paper, I like outside review 
because I get used to my own writing and assume that things flow. 
Sometimes they don't   Working groups make the same mistakes sometimes.)

Yours,
Joel M. Halpern

---

From: Suresh Krishnan <[email-masked]>
Date: Fri, 30 May 2008 19:31:44 -0400

Hi Ted,
   Please find responses inline

> solicited should not be done lightly in informational documents.

I think there is some context missing here. This is what we talk about 
in section 4.1.

>       emails to the authors, so that future readers will also understand

This is from my personal experience. I have been asked by reviewers 
(including IESG members) to add such text in the past. For the sake of 
clarity, I complied. I do not see the downside of this.

> within the group using a technology. 

If someone reads a document and the normative references, and does not 
understand something, it needs to be fixed. RFCs are not only for people 
who worked on creating the technology.

> the handy MDN functionality likely available in your mail client.

I can remove this from the document. This is the amount of time that I 
take on average to respond to a review. But, let's say somebody writes a 
draft for the first time ever in her life. She gets a review from you on 
her draft. She currently has no idea how much time she has to respond to 
you. The options being

* Reasonable amount of time (specified in an IETF process document: not 
this one)
* Reasonable amount of time (as deemed appropriate by the authors)

Since we limit the amount of time people have to send in the reviews we 
need to limit the time people take to respond to reviews. Otherwise, 
reviews just fall through the cracks and end up being useless.

> the mailing list should confirm".  That's exactly wrong.

I do see the difference, but I quoted the text to show that I agree with 
your text that "The outcome of our document development should not be a 
negotiation between the authors and the assigned reviewers". The WG has 
the final say. When the WG gets involved is where we disagree.

> across as an attempt to subvert the process for attaining consensus.

I think a late stage reviewer should be able to ask for the rationale of 
choosing BindMeTight over ChillOut. The rationale should be available 
either in the document itself or on a publicly accessible archive. 
Section 4.1 provides guidelines for reviewers about the level of review. 
  This is the text about "late" reviews

"The reviews that occur after the
    draft has been progressed from the working group have a very
    different perspective on the contents of the draft.  They explore
    very different aspects of the draft such as security aspects,
    operational aspects, readability etc. that are not usually on the top
    of the list of priorities of the authors and the early reviewers.
    They also explore how the mechanisms proposed in the document fit
    into the Internet architecture as a whole and what detrimental
    effects they will have, if any."

What do you disagree with here?

> I tried to describe two different types, so it is not clear to me which you

I meant 2).

> alone, without that context, is not appropriate.

I am open to suggestions on how to go forward with this document. This 
is just version -00 of the document. It is pretty much table of contents 
and preliminary thoughts. Give it some time to mature.

> To end on a high note, it is beautifully formatted.

:-)

Thanks
Suresh

---

From: Suresh Krishnan <[email-masked]>
Date: Fri, 30 May 2008 19:43:24 -0400

Hi Paul,
   Thanks for the comments. Please see responses inline.

> authors like to" are quite different than "are expected to" and 

I agree that tone might be a bit strong but this can be easily fixed in 
the document. e.g. Replace

"The authors are expected to respond to the reviews within a
  reasonable amount of time."

with

"It is considered polite to respond to the reviews within a
  reasonable amount of time.

but it might be more tricky to define "reasonable amount of time". Do 
you feel that it is out of the scope of this document to define this? If 
so, we can take it out of the document. But doing so diminishes the 
guiding value of the document.

> sending their reviews is an important one for the IETF community to 

Agree. And this topic (the recipient list of the review) is something I 
think hard about before I send out any review.

Thanks
Suresh

---

From: Ted Hardie <[email-masked]>
Date: Fri, 30 May 2008 16:54:07 -0700

>that he is not entitled to a reasonable response.)

Tone is notoriously hard to get right, in email especially.  As an
early reviewer here, I am trying to make a forceful statement that
this is off on the wrong foot and needs to start over to make much
progress.  If that force comes off as rude, forgive me.  But this
document lacks context that you have at this point absorbed into
your bones, and I believe that without it, it will be harmful.

>member, or have the full email conversations available.

No, it is not.  But it is also not reasonable to assume that each document
carry the full context of an area with it.  There is a balance there,
and assuming that contextualizing information should be added to
each document on the basis of reviewer request is a problem.  In addition
to making each document very large, they run the great risk of getting
out of synch, with the usual hilarity resulting.

>readers will be confused.

But we're not writing our documents for reviewers; we're writing them for
implementors, operators, and our successors, toiling in these protocol farms,
right?  So there is some level of common knowledge among those working
within a protocol area we can and should assume in our documents.
We can and should provide citations (that's what those nifty normative
and informative references are for), but that doesn't translate into providing
tutorial text.   I was once a big fan of it, but I see people go wrong on the basis
of thrown-in tutorial text often enough now that I think it is often
less than helpful and sometimes actively harmful (by causing the reader
to believe that it is sufficient context).

>On design decisions, there is an even more complex tradeoff.  I have

These both sound like excellent reviews:  you expressed your personal
design preferences in the first instance but did not try to force it over
the consensus of the working group, and pointed out a showstopper
in the second. 

Now, show me in this draft how these two cases are distinguished,
which is critical to getting a review done right? 

>the external reviewer is to look for such things.

Mistakes are not "does not agree with my prejudices", and that is a key
problem in many of the reviews we actually see. 

>seems particularly appropriate that review team comments better get

Uh, why?  If you up and decide to read every 17th Last Call for your
own edification and respond with comments  that are reasonable reviews,
why should someone else's comments get better responses than yours?

>Sometimes they don't   Working groups make the same mistakes sometimes.)

My comment is not simply that late reviews should be made earlier.  It is that
we need to distinguish between issues that a reviewer can and should raise
as showstoppers "Will not work| Will break Internet | Uses component parts in
ways will cause whole to fall apart" and those that question the decisions
of the working group when none of those conditions are fulfilled.  That kind
of issue has a time and a place; document development (and, if necessary, the
bis or ter versions) is where that should happen.  If you push for ZNK
ChillOut over BindMeTight in a late review, in other words, you need to show
that BindMeTight doesn't work.  If you push it over the consensus of the working
group at late stage, you should be satisfied with a "considered; choice made"
and nothing else; anything else puts the power for determining the shape of
the document in the hands of those doing the review instead of doing the work.
That's wrong, that's dangerous, and any document that seems to push that
point of view needs to be extensively revised or dropped.

To help with the tone issue, permit me to wish you a happy weekend,
				regards,
					Ted

---

From: Paul Hoffman <[email-masked]>
Date: Fri, 30 May 2008 17:07:45 -0700

At 7:43 PM -0400 5/30/08, Suresh Krishnan wrote:
>I agree that tone might be a bit strong but this can be easily fixed 
>in the document. e.g. Replace
>
>but it might be more tricky to define "reasonable amount of time". 
>Do you feel that it is out of the scope of this document to define 
>this? If so, we can take it out of the document. But doing so 
>diminishes the guiding value of the document.

If you want the document to have "guiding value", it really is an 
update to RFC 2026. The PUFI BoF experience at the last IETF might 
dissuade you from trying to do that; maybe not.

In the Tao, Susan and I tried hard to keep focus on helping the 
readers remember that everyone in the IETF is a person, most are 
volunteers, and we come from very different backgrounds with very 
different expectations about how each other will (and should!) act. 
That's quite different than RFC 2026, for very good reason, I believe.

>>This document emphasizes reviews going to authors instead of 
>>reviews going to WGs or, in the case of individual submissions, 
>>reviews going to mailing lists. In the Tao, we emphasize the value 
>>of communications to groups so that the group can agree, amplify, 
>>show disinterest, or disagree. In the WGs I have co-chaired, the WG 
>>got good value out of some of the GenART and SecDir reviews in that 
>>it made the whole WG think about the topics brought up. This may be 
>>a fundamental difference in view between this document's authors 
>>and my preferences, but I think the discussion of where reviewers 
>>should be sending their reviews is an important one for the IETF 
>>community to have.
>
>Agree. And this topic (the recipient list of the review) is 
>something I think hard about before I send out any review.

That's good to hear, but I didn't see it reflected in the document; 
maybe your co-authors had a different slant. Regardless, my 
preference is to encourage group communication during reviews for 
anything other than editorial nits and "I was told to read this; I 
did; it was fine" reviews. Group communication, in both directions 
for a review, helps everyone. It also helps prevent a WG hearing that 
"I changed this thing we had all agreed to because I was told to by { 
a security person | an IAB member | an ex-AD | ... }". Those kinds of 
changes tend to make a document weaker if they aren't agreed to and 
possibly modified by the WG who worked on the document.

--Paul Hoffman, Director
--VPN Consortium

---

From: "Joel M. Halpern" <[email-masked]>
Date: Fri, 30 May 2008 20:12:23 -0400

Comment inline, with most of the discussion elided.  I believe that one 
particular question gets to the heart of what is bothering me.

> At 4:08 PM -0700 5/30/08, Joel M. Halpern wrote:
...
> which is critical to getting a review done right? 

The problem I have is that I do not know how to write text in a draft 
that distinguishes those two.  The line between them is very tricky, and 
possibly subjective.
And part of the problem is to avoid turning it into a fight.  If all 
review comments get clear, reasonably timely responses, there is room 
for the discussion without acrimony.

Would it address your concern if the document said something like:
    "Reviewers should be sensitive to the difference between
    their personal opinions (and preferences) and issues
    which will affect the correct operation or interoperation
    of the documents under review"
?

I have no problem with pointing out that there are two different 
categories.  I have real problems with trying to define a hard line 
which distinguishes them.

Yours,
Joel

PS: While there are other differences in our views, they seem to be 
questions on which reasonable folk may differ and we can let the 
community sort out how to write the wording.
_______________________________________________
IETF mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ietf

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 31 May 2008 14:12:41 +1200

Joel,

<snip>
> which distinguishes them.

Really? Imagine the undiplomatic version of a review:

 I thought that the IESG was crazy to approve this WG; its
 basic premise is stupid, the basic idea is undeployable, and
 if deployed, it will seriously harm Internet congestion control.
 Now, here are my comments on the MIB for this stupid protocol:

 The MIB is well constructed, consistent, and in fact it's
 the easiest MIB to read that I've ever seen. Apart from one
 minor typo on page 97, it's ready for publication.

When you look at it like that, the solution seems obvious.
A reviewer who has fundamental disagreement with the work
under review should IMHO separate that disagreement out from
the review, and send it as a distinct last call comment,
so that document quality control is not mixed up with
fundamental criticisms of the work.

I agree there may not always be a hard line, but if in doubt,
separating the disagreement out seems best.

    Brian