Subject: RE: Resume command for ESMTP?

From: Franck Martin <[email-masked]>
Date: Mon, 6 Aug 2001 10:11:33 +1200

Vladis,

You won't get me I though about this one ;-)

As I said in my previous e-mail there are several cases to handle error and
security related to such command...

First of all I feel it is important to implement such mechanism to avoid to
increase the digital divide between countries which have bandwidth and
countries which have not. I see so many sites which disregard servers
because the latency is too big or the downloads are too slow. Basically they
send the message: we don't want to talk with you because you are inferior. I
won't give names but they exist and are not the small companies...

Basically, I would like your cooperation in trying to make this thing work.
If you think there is an issue try to imagine the RESUME command exists and
how it should respond to the problem...

Enough rant, let's go back to technical issues.

The mail server needs obviously to become more intelligent, than store,
forward and forget.

First to put the RESUME command just before DATA ensure that most of the
sanity checks have been passed (RBL, fake domain, ESMTP size too big,...).
So, so far nothing really has happened...

Now let see how someone could do a DOS and how the server should avoid it.

The first principle, is not to store a mail part indifinitively, but to
timeout it in the queue. The parameter should be left configurable. Let's
say that the mail sender has 5mn to resume the session or it has to start
from the beginning again. This would allow that the queue doesn't grow out
of proportion because of errors.

The second principle, is that most servers do a content analysis based on
the line received. The mail server, needs to know where it is in the
session. Are we still in the headers or already in the content? Based on
content anylysis of the mail being received the server may send an error
back, like "Error content rejected", "Malformed error", "Message to big".
The server trash the message and clear the session. This would take care of
the sircam virus, like on my system where I block attachments with .exe or
.lnk extensions. This is done easily on the fly while receiving the e-mails.
Here the e-mail does not need to be fully received but as soon as a
condition is met the session is closed.

The thrid issue, is that some mail system, get the whole e-mail then pass it
to an antivirus software before re-injecting the e-mail back into the
system. There is no possibility than to wait the completion of the whole
session, but with the timeout above, you can't keep a session open too
long...

Finally, there could be a separate resume queue with a fixed size limit so
that after receiving the RESUME command the server may answer "NOT AVAILABLE
NOW". It is then up to the sender to try to send or try later. Later the
messages queued will expire and the RESUME queue will be available again...

The RESUME capability is useful only for big mail in poor network
conditions. The SIZE limit is for queue space and processing capabilities,
it should'nt be used as a poor network configuration parameter...

I understand that a mail server with RESUME capability may need a bigger
queue than one without, but HD space is cheap, and it saves huge amount of
expensive bandwidth...

So What do you think? Anybody that support the idea and want to develop it
further?

Franck Martin
Network and Database Development Officer
SOPAC South Pacific Applied Geoscience Commission
Fiji
E-mail: [email-masked] <mailto:[email-masked]> 
Web site: http://www.sopac.org/
<http://www.sopac.org/> Support FMaps: http://fmaps.sourceforge.net/
<http://fmaps.sourceforge.net/> 

This e-mail is intended for its addresses only. Do not forward this e-mail
without approval. The views expressed in this e-mail may not be necessarily
the views of SOPAC.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: Monday, 6 August 2001 7:45 
To: Franck Martin
Cc: [email-masked]
Subject: Re: Resume command for ESMTP? 

On Sun, 05 Aug 2001 19:39:21 +1200, Franck Martin <[email-masked]>  said:

> So ESMTP could have an extension called RESUME
> 
> The protocol would be standard, except that before DATA and after RCPT
> TO, the sending server who has identified the RESUME keyword could ask a
> RESUME session...

On the other hand, SMTP is a store-and-forward system where the files are
kept in essentially *temporary* storage.  As such, the general design
philosophy is to throw away the temporary files if a transfer fails, so
as to free up queue space.

This in and of itself is not a problem, until you realize that quite
possibly
more than one file may need to be saved in case the person will *someday*
try to RESUME the transfer.  Now.. a real-life case in point.

And that's why there isn't a RESUME.

-- 
				Valdis Kletnieks
				Operating Systems Analyst
				Virginia Tech

---

From: [email-masked]
Date: Sun, 05 Aug 2001 20:09:52 -0400

Am taking further followups to [email-masked] where it belongs...

> of proportion because of errors.

Is there any evidence that it is common enough to have a network outage severe
enough that the TCP connection gets broken, but that within 5 minutes it's
resumable?

> condition is met the session is closed.

Closing the session on the fly is *very* anti-standard.  Also, note that if
you close it on the fly, a standard conforming MTA will *retry* it.  Over and
over, as you keep closing the session on the fly.  A much better solution is
to wait until the final '.' is received, and THEN do something interesting.

See RFC2821, section 4.2.5 for details.

> queue than one without, but HD space is cheap, and it saves huge amount of

The problem is that it may be cheap to say "add another 500M to the spool"
for a *small* system.  However, when you start talking about *large* mail
systems, scaling becomes an important factor.  I spent some time at a recent
Sendmail workshop, and some of the people there were working on systems that
had to do a million deliveries an hour.  At these levels, even saying "just move
it to another queue for later" becomes problematic - it takes a lot of I/O to
sync everything to disk...

And remember - these "big systems" are the ones you're probably having the
trouble talking to.

> So What do you think? Anybody that support the idea and want to develop it

Well.. I still think that the *proper* solution is to fix the TCP infrastructure
so that there isn't a NEED for a RESUME....

---

From: Gaurang Pandya <[email-masked]>
Date: Mon, 6 Aug 2001 07:02:29 -0700 (PDT)

Hi Frank,

I liked all these ideas very much.
But I may be of no use to you since i am newbie.

Gaurang.
> And that's why there isn't a RESUME.

__________________________________________________
Do You Yahoo!?
Make international calls for as low as $.04/minute with Yahoo! Messenger
http://phonecard.yahoo.com/

---

From: Einar Stefferud <[email-masked]>
Date: Mon, 6 Aug 2001 13:22:22 -0400

Does anyone remember the trick of breaking a single huge message into 
smaller parts, to be reassembled by he final recipient?

Its called fragmentation, as I recall.

Marshall Rose implemented in his version of the RAND MH mail system.

I think this is a better solution to the whole problem;-)...\Stef