Subject: Re: ISMS working group

From: Margaret Wasserman <[email-masked]>
Date: Wed, 7 Sep 2005 13:44:17 -0400

Hi Ken,

The call home solution doesn't help with the problem of the _manager_ 
being behind a NAT.  It only applies to situations where the manager 
is at a fixed location on a globally-addressable network and the 
managed device is behind a NAT or firewall.

In those cases, the choices would be for the administrator to 
configure the NAT/firewall to admit his management traffic  so that 
the manager can establish a connection to the device, or for the 
device to perform a "call home" function so that the device can 
establish a connection to the manager.

BTW, the solution where you configure the NAT/firewall to admit the 
appropriate traffic could also be set-up to work properly when the 
manager is mobile and/or when the manager is gaining Internet access 
through a NAT.  "Call home" would not work in these situations.

The only cases that I can see where "call home" is a better solution are when:

(1) The NAT/firewall is not configurable to admit specific traffic, 
do static port mapping and/or to handle SSH tunneling.

(2) The NAT/firewall (nearest to the agent) is not under the 
administrative control of the people who want to do the management.

I'm not sure how often either of these cases really occurs on 
SNMP-managed networks in practice, but I think that is an interesting 
question.

Margaret

At 4:46 PM -0400 9/6/05, Ken Arnold wrote:
>Firewalls are a ubiquitous feature of my everyday life, as is NAT.
>My home is behind a NAT-ing firewall; my laptop has its own 
>firewall; every time I travel I rely on that firewall in hot spots 
>(such as starbucks); I run VirtualPC on my Mac, which has a firewall 
>and NAT translation; my parent's system is similar to mine: NAT-ing 
>firewall and on-system firewall for double protection (as are my 
>in-laws' and brothers' systems); my previous company had a NAT-ing 
>firewall for each ISP connection, and their product (a linux 
>cluster) had a firewall to protect itself from the customer's own 
>internal network.
>Many small network appliances have firewalls (sometimes NAT-ing 
>ones) that protect themselves the same way.  I could go on and on.
>
>Which means that any network solution -- let alone something as 
>potentially central as SNMP -- that ignores either NAT or firewall 
>-- is dead on arrival as far as I'm concerned, and as far as any IT 
>group I've ever seen is concerned.  If there are non-firewalled 
>networks of consequence anymore I am unaware of them.  Ignoring 
>these relegates any solution to theoretical situations or very small 
>in-home or in-group solutions.  Then someone else will have to 
>figure out some way to manage anything larger scale, which will be 
>able to also handle small scale and so will overwhelm the 
>non-firewalling, non-NAT-ing designs.  But only after such a 
>relatively impotent design confuses the world by adding yet one more 
>standard to chose from.
>
>Please get this right this time by including firewalls and NAT in 
>the solution space so it really *is* a solution.
>
>         Ken
>
>P.S.  Please also note that a non-firewalled SNMP creates some 
>pressure to bring down firewalls, to trade off SNMP for security.
>Things that pressure networks to be less secure are pretty 
>antisocial in our current security environment of zombie spam farms, 
>etc.

_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf

---

From: Ken Arnold <[email-masked]>
Date: Tue, 6 Sep 2005 16:46:33 -0400

Firewalls are a ubiquitous feature of my everyday life, as is NAT.   
My home is behind a NAT-ing firewall; my laptop has its own firewall;  
every time I travel I rely on that firewall in hot spots (such as  
starbucks); I run VirtualPC on my Mac, which has a firewall and NAT  
translation; my parent's system is similar to mine: NAT-ing firewall  
and on-system firewall for double protection (as are my in-laws' and  
brothers' systems); my previous company had a NAT-ing firewall for  
each ISP connection, and their product (a linux cluster) had a  
firewall to protect itself from the customer's own internal network.   
Many small network appliances have firewalls (sometimes NAT-ing ones)  
that protect themselves the same way.  I could go on and on.

Which means that any network solution -- let alone something as  
potentially central as SNMP -- that ignores either NAT or firewall --  
is dead on arrival as far as I'm concerned, and as far as any IT  
group I've ever seen is concerned.  If there are non-firewalled  
networks of consequence anymore I am unaware of them.  Ignoring these  
relegates any solution to theoretical situations or very small in- 
home or in-group solutions.  Then someone else will have to figure  
out some way to manage anything larger scale, which will be able to  
also handle small scale and so will overwhelm the non-firewalling,  
non-NAT-ing designs.  But only after such a relatively impotent  
design confuses the world by adding yet one more standard to chose from.

Please get this right this time by including firewalls and NAT in the  
solution space so it really *is* a solution.

         Ken

P.S.  Please also note that a non-firewalled SNMP creates some  
pressure to bring down firewalls, to trade off SNMP for security.   
Things that pressure networks to be less secure are pretty antisocial  
in our current security environment of zombie spam farms, etc.

---

From: Jim Thompson <[email-masked]>
Date: Tue, 6 Sep 2005 13:42:01 -1000

I agree with Eliot on this subject.

> From: Eliot Lear <[email-masked]>
> Date: September 6, 2005 7:37:01 AM HST
> To: IETF Discussion <[email-masked]>,  [email-masked],  [email-masked]
> Subject: ISMS working group and charter problems
>
>
> Dear Communities,
>
> I need your help to correct for an impending mistake by the ISMS
> working group in the IETF.
>
> Short Version
>
> The ISMS working group is chartered to find a way for SNMP to make use
> of existing authentication mechanisms.  The current proposed
> approaches will make use of TCP.
>
> I seek a change to the proposed ISMS charter that requests the working
> group pay attention to firewall and NAT concerns.  The current
> envisioned approach will not work through firewalls and NATs.  I
> specifically request that the working group be directed to consider
> "Call Home" functionality as a non-exclusive alternative, where the
> managed device contacts the manager much the same way as your PC
> contacts Microsoft, Apple, etc for updates.
>
> The addition of call home functionality won't represent a major
> architectural change to SNMP.  The major architectural change (if  
> there
> is one) will be the use of SSH at all and the use of TCP.
>
> If you agree with me, I ask that you respond to this note including
> the [email-masked] and [email-masked] so indicating.
>
> Reasoning
>
> Long gone are the days when the IETF can simply ignore firewalls, as
> this working group is currently planning.  An approach that
> demonstrates robustness in the face of firewalls is required.
>
> Long Version
>
> SNMP version 3 has a unique authentication mechanism that does not
> easily integrate with other AAA systems such as radius or kerberos.
> After many years of complaints and lack of deployment of SNMPv3, a
> working group called ISMS was chartered last year to address this
> problem.  At IETF 63 the working group decided to move forward with an
> approach based on SSH.
>
> As you all know, SSH is TCP-based.  This represents a substantial
> change for SNMP.  It also represents a substantial opportunity to
> extend manageability through firewalls.  Currently, if you want to
> manage devices with SNMP through firewalls you must either have the
> firewall run an application layer gateway or you must poke a hole in
> the firewall based on UDP ports.  If you do not control the network
> element, the network manager, AND the firewall, you assuredly have no
> hope of getting SNMP through.  Even if you DO control the firewall,
> configuration of authorized address mappings may make it prohibitively
> difficult to allow management protocols through, especially in the
> face of dynamic address assignment mechanisms such as DHCP and PPP.
> Because we now are considering a TCP-based approach we have the
> opportunity to fix this huge problem.
>
> Why is this important?
>
> Firewalls exist today throughout our infrastructure.  There's a good
> chance you have a simple one at home.  Indeed firewalls exist between
> departments within enterprises.  As companies add services and
> functions onto the Internet their ability to manage these services
> with SNMP deteriorates, requiring the need for expensive custom
> solutions and out of band management.
>
> Furthermore, many telcos today offer managed services, where they
> manage enterprise and consumer devices (routers, switches, etc).  The
> whole concept of an enterprise network has, if you will, become
> virtualized.  Today SNMP does not offer any joy to those who want to
> build a unified management system.
>
> More and more voice over ip (VoIP) has gained acceptance in the market
> place.  However, the ability to debug end points real time is limited.
> Wouldn't it be nice for a manager to query a phone to determine how
> many data packets it thinks it has sent to a far end and then follow
> that stream to determine who is dropping?  In order to accomplish this
> task, the manager has to have access to a phone which, if remote, may
> well be sitting behind a firewall such as the one you have at home.
> Furthermore, if the phone wants to send a notification to a  
> manager, it
> too is likely to reside behind a firewall.
>
> Networks are certainly not the only functions to be managed.  One
> could easily imagine power management services making use of standard
> MIBs as well as enterprise MIBs to handle capacity planning as well as
> dispatch in the face of live problems.
>
> What is currently envisioned by ISMS?
>
> What is currently being discussed is the traditional model, where if
> you want to request information you open up an SSH connection and make
> an SNMP query on top of it once you've authenticated.  If the managed
> device resides behind a firewall, you lose.
>
> Worse, the currently envisioned solution calls for a separate
> connection to be opened to send notifications (traps).  This time, if
> the network management station resides behind you lose again.
>
> What this means is that if there exists a firewall anywhere between
> the firewall and the management station, the currently proposed
> solution will fail.  The astute will note that this approach looks a
> lot like old fashion FTP and will break just in just the same way.
>
> What is needed?
>
> I propose a flexible standard mechanism where either the device or the
> manager can be configured to initiate a connection, and that
> notifications occur either across the same TCP stream, when it
> exists.  For instance, if in the classic case of a manager  
> connecting to
> the element the manager requests the "snmp-request" ssh service, we  
> also
> simply allow for the device to also initiate a connection but instead
> ask for the "snmp-turn" service (with all due credit to the authors of
> SMTP who first anticipated this problem over 20 years ago!).  The same
> for notifications.  This "-turn" approach is sometimes referred to as
> "Call Home" (CH) functionality.
>
> CH has an added advantage as well.  Many devices come and go from the
> network, and it is not reasonable, scalable, or cost effective to poll
> such devices if they are not there.  CH provides a natural discovery
> mechanism for such devices because they initiate request for  
> management.
>
> This proposal does NOT represent a dramatic architectural change to
> SNMP.  The dramatic architectural change will be the use of SSH at  
> all,
> and not who initiates the connection.
>
> What I'm asking you to do
>
> The good news here is that all you have to do is drop the IETF and the
> IESG a note, saying you want to a solution.  This is one of those
> times when you don't even need to fix it yourself.
>
> Please reply to this message, CCing the [email-masked] and
> [email-masked] indicating that you agree that ISMS should not miss the
> opportunity now take into account firewalls.
>
> If you do nothing, the problem will probably be ignored by ISMS.  SNMP
> will continue to serve the needs it serves today, but likely no more.
> Proprietary and competing standards approaches will continue to be
> developed.  Multiple standards in this space would be a waste of
> effort on the part of implementors and operators alike.  Don't let
> this happen!
>
> Eliot
>
>
>
>

---

From: Ken Arnold <[email-masked]>
Date: Wed, 7 Sep 2005 15:29:35 -0400

On Sep 7, 2005, at 1:44 PM, Margaret Wasserman wrote:

> The call home solution doesn't help with the problem of the  
> _manager_ being behind a NAT.  It only applies to situations where  
> the manager is at a fixed location on a globally-addressable  
> network and the managed device is behind a NAT or firewall.

Thanks, Margaret.  I know that some think CH is good and some think  
it isn't, etc.  I'm not taking a stand on that, I'm just saying that  
the standard needs to *handle* these cases.  I actually have no horse  
in that race; I'm sitting at the "deliver SNMP solutions for our  
products to the customer" end of the world, and I want the standard  
-- whatever it is -- to work in these cases.  If CH is a partial  
solution that needs augmenting with another approach, itself also  
incomplete, fine.  If the best that can be done, no matter what  
protocol is defined, is to handle the case you describe, well, nobody  
can do the theoretically impossible and it will have to do.  At least  
it's an important case in this world, and better to include it than not.

But honestly, I didn't mention Call Home or any other approach in my  
letter.  I'm implementation-neutral on the subject.  I'm solution- 
centric -- I want the problems addressed so they are solved.

         Ken

---

From: Eliot Lear <[email-masked]>
Date: Mon, 12 Sep 2005 09:44:11 +0200

Margaret,
> being behind a NAT.  It only applies to situations where the manager is
> at a fixed location on a globally-addressable network and the managed
> device is behind a NAT or firewall.

Actually, depending on how the solution is developed it certainly CAN
help the problem with the manager being outside a NAT.  But we are now
being somewhat loose with terms, so let me be more specific.

Consider the case where a manager is behind a firewall or NAT and the
device sits out in the open.  This could easily be an enterprise where
the network management station itself sits behind a firewall in order to
prevent/detect/respond to control attacks.

Now consider notifications.  The device will have no means to get a
notification to a manager unless a hole is specifically poked for the
purpose and the management station is in the same address space as the
managed device.

If the management station has initiated an SSH connection to the agent,
the agent can then initiate notifications over the SSH connection,
either by using the "snmp" application as a simple two way stream as
Harald suggests, or as a "turned" connection as I had suggested in the
first message on this thread (I think both approaches should be debated).

> through a NAT.  "Call home" would not work in these situations.

And so your premise above not holding, the remainder of your argument
also does not hold.

Eliot

---

From: Margaret Wasserman <[email-masked]>
Date: Mon, 12 Sep 2005 08:44:42 -0400

Hi Eliot,

>being somewhat loose with terms, so let me be more specific.

I am sorry that I attempted to state any properties of the call-home 
solution, because it is clear that the term is not well-enough 
defined or specified to be discussed clearly.  I thought that I 
understood what it was, but apparently I am mistaken.

If you really believe that this solution is needed, I think you would 
do best to write a draft and _then_ try to get it adopted by an 
appropriate WG.

Margaret

---

From: Eliot Lear <[email-masked]>
Date: Mon, 12 Sep 2005 14:52:39 +0200

Margaret Wasserman wrote:
> If you really believe that this solution is needed, I think you would do
> best to write a draft and _then_ try to get it adopted by an appropriate
> WG.

I (amongst others) *did*.  draft-kaushik-isms-btsm-01.txt.  What had
been missing up until this point was an SSH draft.  And the working
group developed consensus on this non-existent draft.  You've got to be
impressed.

Eliot

---

From: Margaret Wasserman <[email-masked]>
Date: Mon, 12 Sep 2005 10:05:17 -0400

Hi Eliot,

I have, of course, read the draft that you cited below, but the term 
"call home" is not defined or used in that draft...

The document does discuss the concept that either end of the SNMP 
exchange could initiate the BEEP connection at the transport level, 
but I don't see that it explains anywhere how/when/why a command 
responder would _decide_ (or even know how) to contact a command 
requestor and/or how a command responder could find a command 
requestor if it were not at a fixed, globally addressable location.

IMO, there is a lot more to building a system that is capable of SNMP 
initiation in both directions than simply having a mechanism to 
set-up the transport connection from the command responder to the 
command generator.  It would also be possible to set-up an SSH 
connection from either end, but I don't see how that even begins to 
offer the benefits that you've attributed to "call home".

None of this seems very material to the ISMS discussion, though...

Today SNMP (whether it is running over UDP or TCP) doesn't have the 
call home feature.  Do you really think it is reasonable to tie the 
addition of that feature to the definition of a new security 
mechanism for the existing SNMP protocol?  If so, why?

IMO, we need to try to do our work in manageable chunks in the right 
groups/areas.  A security area working group working on a new 
security mechanism for the existing SNMP model is one chunk.  Perhaps 
an OPS area WG working on an optional SNMP call home mechanism is 
another...?  I don't see how the level of change/disruption to the 
vendor community is substantially affected by whether these two 
separate mechanisms are defined in one IETF working group or two.

Margaret

---

From: Daniel Senie <[email-masked]>
Date: Mon, 12 Sep 2005 10:06:43 -0400

At 08:44 AM 9/12/2005, Margaret Wasserman wrote:

>If you really believe that this solution is needed, I think you 
>would do best to write a draft and _then_ try to get it adopted by 
>an appropriate WG.

Stepping back a bit from this discussion and thinking about what has 
been said to date, it occurs to me that part of the problem with how 
the IETF presently does business comes from a reliance on 
pigeon-holing any particular work into one of several areas. The 
Security area is working on fixing the security issues of SNMP, but 
O&M is the "home" of SNMP. Is there enough involvement of folks from 
both sides? Should there be working groups that are co-owned by 
multiple areas/ADs? Would this result in a more balanced set of conclusions?

Eliot rightly asks the question in one of his emails whether SNMP 
changes should be considered more widely so as to develop one new 
protocol revision for implementers to work toward, rather than a 
series of incremental changes. If indeed we're talking about SNMPv4, 
then perhaps a series of drafts should work toward that, and be 
unified into one or a few proposed standard documents that are 
released at one time, with all of the new work.

---

From: Eliot Lear <[email-masked]>
Date: Mon, 12 Sep 2005 18:15:28 +0200

Margaret,

I find myself repeating statements I've made recently, and so I'd
propose that you catch up on my other messages.  But for the record...

> "call home" is not defined or used in that draft...

Because it wasn't necessary to do so, as you allude below.  The draft
you haven't yet read is SNMP/SSH.  The working group decision was made
absent a draft.  Doesn't that bother you at all?

> exchange could initiate the BEEP connection at the transport level, but
> I don't see that it explains anywhere how/when/why a command responder
> would _decide_ (or even know how) to contact a command requestor and/or
> how a command responder could find a command requestor if it were not at
> a fixed, globally addressable location.

As I wrote previously this is a matter for configuration and policy.
There are some obvious configurations (some are obviously good and
others are obviously bad ;-) and some non-obvious ones.  It is also a
matter for a working group to discuss.  The conversation within the WG
has thus far been cut short by AD decree.

> initiation in both directions than simply having a mechanism to set-up
> the transport connection from the command responder to the command
> generator. 

Of course there is.  Nobody said otherwise.  There are matters of end
point authentication, for instance.  But these problems are not
insurmountable.  Perhaps you would like to participate in those
discussions, assuming we can find a place where they are not ruled out
of scope?

> It would also be possible to set-up an SSH connection from
> either end, but I don't see how that even begins to offer the benefits
> that you've attributed to "call home".

You asked me for a concrete proposal.  I responded with one using BEEP
as an example.  BEEP / SSH are bits on a wire.  Either can be used, but
as I've written previously the horse I have in the game is whether CH is
supported and whether trap-based polling will properly function through
firewalls (something very much in doubt at this point), and not whether
we're talking about BEEP or SSH.

> None of this seems very material to the ISMS discussion, though...

As I've written, The reason it is relevant at all is that the ISMS
working group has decided to make use of a transport where firewall
traversal becomes a real possibility, and I (and others) have needs for it.

> 
> Today SNMP (whether it is running over UDP or TCP) doesn't have the call
> home feature.  Do you really think it is reasonable to tie the addition
> of that feature to the definition of a new security mechanism for the
> existing SNMP protocol?  If so, why?

Margaret, if I only thought it was reasonable to tie these two functions
together we wouldn't be having this discussion amongst a the IETF and
IESG.  I think it is UNREASONABLE and a huge waste of resources to
consider an SSH/TCP solution without taking into account connection
direction.  Here's why:

 - in its current direction the protocol will fail to be useful in
   the face of firewalls.  In this day and age that is a foolhardy
   approach.  [It would be excusable failure if SNMP were to
   remain UDP-based since firewall/NAT traversal is already
   problematic.]  Under the current approach the basic concept of
   trap-based polling will fail in the face of firewalls.

 - why now? because as I wrote earlier we really don't want to churn
   SNMP continuously.  Put yourself in the position of a tools
   developer.  What sort of investment am I going to put into
   development, q/a, release engineering, documentation now when I
   know this is hanging over my head?

In short, if we're going to turn the crank at all on SNMP, let's make
sure the resultant will work and will be useful for a while without
having to go do it again six months or a year down the road.
> groups/areas.  A security area working group working on a new security
> mechanism for the existing SNMP model is one chunk.  Perhaps an OPS area
> WG working on an optional SNMP call home mechanism is another...?

Yes, you've repeatedly said this.  And I've repeatedly responded with
two reasons why this work should continue in a single working group:

1.  The ISMS working group has had the correct set of people with eyes
on the problem, including security experts who ought to see their
concerns (such as those raised by Steve Bellovin and others) addressed
as well as network management people.

2.  And as has been pointed out repeatedly (first by Steve Bellovin and
then by others), there are security implications here.  We currently
need and have the correct the mix of security and management people in
the working group to both develop and properly vet the work being done.

>  I
> don't see how the level of change/disruption to the vendor community is
> substantially affected by whether these two separate mechanisms are
> defined in one IETF working group or two.

Because you view the mechanisms as separate.  As currently envisioned
the approach proposed will fail in the general case in the face of
firewalls and NATs(*).

Eliot
--
ps: I agree with Melinda that there are differences between the two, but
for common cases the results are the same.

---

From: Eliot Lear <[email-masked]>
Date: Mon, 12 Sep 2005 18:47:24 +0200

Uch... I wrote:

Eliot Lear wrote:
> And I've repeatedly responded with
> 1. [security considerations]
> 
> 2. [security considerations].
> 

The second reason is the reason mentioned elsewhere in the message, that
turning the crank should be done only once on this...

Sorry for the confusion.

Eliot

---

From: Wes Hardaker <[email-masked]>
Date: Mon, 12 Sep 2005 11:12:17 -0700

>>>>> On Mon, 12 Sep 2005 14:52:39 +0200, Eliot Lear <[email-masked]> said:

>> best to write a draft and _then_ try to get it adopted by an appropriate

Eliot> I (amongst others) *did*.  draft-kaushik-isms-btsm-01.txt.
Eliot> What had been missing up until this point was an SSH draft.
Eliot> And the working group developed consensus on this non-existent
Eliot> draft.  You've got to be impressed.

Elliot, sorry to hear that you wrote a draft that the WG didn't want.
It happens.  In fact, it happened to me in this WG, no less.  In fact,
if you want to play the game of "I had a draft written first and thus
I should win", then I think mine would trump yours ;-)

Wouldn't the best thing to do be to help the SSH draft authors ensure
your needs are met rather than complain about your draft not being the
choice the WG went with?  Just because CH functionality isn't in the
charter doesn't mean you can't analyze the resulting SSH draft for
problems with (future or now) CH functionality.  That would mean
either CH would be possible now or it would leave room for an easy
addition for support for it in the future.  Obviously, the WG has
chosen not to make it their top priority but I doubt anyone would
complain about leaving room for it unless the suggestions dramatically
affected the agreed upon architecture or solution.

-- 
Wes Hardaker
Sparta, Inc.

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 13 Sep 2005 14:37:45 +0200

Eliot Lear wrote:

<snip>

> we're talking about BEEP or SSH.

Let me be clear about what you mean here. You mean that getting through
firewalls and NATs is equally possible with SSH or BEEP (or TLS or HTTP
for that matter)?

I'd also observe in terms of charter language that the ability to
have agents and managers indifferently either side of NATs and
firewalls, and the ability to support traps in such scenarios,
are potential charter requirements. But "call home" is a solution,
and there might be others.

    Brian

---

From: Eliot Lear <[email-masked]>
Date: Tue, 13 Sep 2005 14:50:43 +0200

Brian,
> firewalls and NATs is equally possible with SSH or BEEP (or TLS or HTTP

Yes, pretty much.  The key useful function is an substrate that allows
for multiple channels.

> and there might be others.

Brian, I would be very happy with the language you mentioned:

  Solution must support {pick your favorite SNMP terminology: command
generators/responders/.../agents/managers} on either side of
  NATs and firewalls

Whether it's CH or some other Thing, as long as it's not incredibly
contortive, I'm okay with it, and I'm perfectly fine with a WG
evaluating such solutions and making a considered decision.

Eliot