Softpanorama (slightly skeptical) Open Source Software Educational Society

May the source be with you, but remember the KISS principle ;-)

Tivoli Endpoint and Gateway Proxy

Tivoli Security Toolbox introduces the Endpoint and Gateway Proxy functions.

On the secure side of the firewall, there is an endpoint proxy that connects to the gateway as if it were the endpoint itself. On the less secure side of the firewall, the endpoints are connected to a gateway proxy, which acts as if it were the real gateway. The gateway proxy and the endpoint proxy then communicate with each other through the firewall.

Just as multiple endpoints can connect to a single gateway and multiple gateways to a single server, multiple endpoints can connect to a single gateway proxy and multiple gateway proxies can connect to a single endpoint proxy. The endpoint proxy emulates all the endpoints to the gateway that manages them.

The endpoint and gateway proxies consolidate all communications between multiple endpoints and gateways over a single port. All such communications are encapsulated in a connection-based TCP protocol.

Another benefit to the endpoint and gateway proxies is that they allow customers to keep all of their Tivoli Management infrastructure (servers and gateways) inside their most secure zones, and only the endpoints and their gateway proxies outside.

The endpoint and gateway proxy solution effectively addresses the issues of communicating over a firewall between the gateway and the endpoint. But what if there is more than one firewall between these systems? In most installations, there will be multiple security zones or DMZs, each separated by a firewall layer, between a gateway and its endpoints. For these environments, the next component, relays, provides a solution.

Old News ;-)

IBM - IY55176 Epproxy-Gwproxy could have many connections in CLOSE_WAIT and than the process may run out of threads

Tivoli Firewall Security Toolbox Interim Fix 1.3.1-TFS-0009
Tivoli Firewall Security Toolbox Interim Fix 1.3.1-TFS-0011
 
APAR status
Closed as program error.

Error description

Env: Tivoli Firewall Solution Toolbox 1.3-TFS-0003A

Problem: In this customers environment we saw Epproxy leaves tons of connections in CLOSE_WAIT state. These connections are between gateway and endpoint proxy emulating the endpoint.

This ultimately causes the Endpoint proxy to run out of threads. Epp.log shows

   03/09/09 16:41:01  1 1779102 upcallSender [line=3505]:
   accept() failed (e=24 - Too many open files)
   03/09/09 16:41:02  1 1779106 LEAVE open_db (abdb_open
   failed)
   03/09/09 16:41:02  1 1779106 find_record: wrong parameter

We ran pfiles on customers Solaris Endpoint proxy and saw that the files utilization increased over a period of time

Recreate: Havent been able to recreate in house.

Summary: After talking to L3 here is what we think is happening. There can be instances where one TFST peer (e.g. the gateway proxy) is for some reason unable to communicate to its peer (e,g, the endpoint proxy) that a routed session needs to be closed. When this occurs, TFST fails to close its end of a TFST<->TMF connection. We see this at this particular customer site epproxy<->gateway connections stuck in CLOSE_WAIT state.

This same problem can theoretically happen on the gateway proxy, leaving gwproxy<->endpoint connections stuck in CLOSE_WAIT state.

Problem summary

The endpoint proxy closes its end of a connection with the gateway after receiving a command from the gateway proxy that the proxied endpoint using this connection has no further data to send to the gateway. Likewise, the gateway proxy closes its end of a connection with an endpoint after receiving a command from the endpoint proxy that the gateway has no further data to send to the proxied endpoint.

If for some reason the command to close the connection is not received, the connection between the endpoint proxy and gateway (or between the gateway proxy and an endpoint) persists in CLOSE_WAIT state.

Over time, the number of connections in CLOSE_WAIT state may increase to the point where all file descriptors available to the process are consumed. This results in failed upcalls and downcalls.

Problem conclusion

This is fixed in interim fix 1.3.1-TFS-0003.

Temporary fix

Comments

APAR information
APAR number	IY55176
Reported component name	MGMT. FRAMEWORK
Reported component ID	5698FRA00
Reported release	371
Status	CLOSED PER
PE	NoPE
HIPER	NoHIPER
Special Attention	NoSpecatt
Submitted date	2004-03-25
Closed date	2004-05-06
Last modified date	2004-05-06

APAR is sysrouted FROM one or more of the following:

APAR is sysrouted TO one or more of the following:

Modules/Macros

Publications Referenced
 

Fix information
Fixed component name	MGMT. FRAMEWORK
Fixed component ID	5698FRA00

 

Applicable component levels
R371 PSN	UP

IBM - IY55176 Epproxy-Gwproxy could have many connections in CLOSE_WAIT and than the process may run out of threads

Summary: After talking to L3 here is what we think is
   happening. There can be instances where one TFST peer
   (e.g. the gateway proxy) is for some reason unable to
   communicate to its peer (e,g, the endpoint proxy) that a
   routed session needs to be closed.  When this occurs, TFST
   fails to close its end of a TFST<->TMF connection. We see
   this at this particular customer site epproxy<->gateway
   connections stuck in CLOSE_WAIT state.  This same problem
   can theoretically happen on the gateway proxy, leaving
   gwproxy<->endpoint connections stuck in CLOSE_WAIT state

Documentation

Recommended Links

[PDF] Tivoli Enterprise Management Across Firewalls

IBM - Tivoli Security Firewall Toolkit gateway proxy failover

Tivoli Firewall Security Toolbox changes

Installing the Tivoli Remote Control Proxy Using a Response File

Last modified: June 02, 2008