I am currently developing a tool using Python 2.4.2 which will be used
as a sink to pump TCP messages. During which i have observed that the
TCP close interface provided by Python is not closing the connection.
This i confirmed by looking at the ethereal logs, which show proper 3
way FIN ACK Handshake. But the netstat reports TIME_WAIT state for the
TCP connection, which is hindering the messages to be pumped later. I
would like to know the problem cause. Is it because of the application
or due to operating system?

On May 5, 4:31 am, Madhur <madhurr...@gmail.com> wrote:
> I am currently developing a tool using Python 2.4.2 which will be used
> as a sink to pump TCP messages. During which i have observed that the
> TCP close interface provided by Python is not closing the connection.
> This i confirmed by looking at the ethereal logs, which show proper 3
> way FIN ACK Handshake. But the netstat reports TIME_WAIT state for the
> TCP connection, which is hindering the messages to be pumped later. I
> would like to know the problem cause. Is it because of the application
> or due to operating system?

Umm, huh? How can you pump messages after the connection is closed?

The TIME_WAIT state is just a relic from connections past. It is
needed to allow the TCP stack to do the right thing (which is nothing
at all) if it receives a duplicate packet from the session. Even
though it closed the connection down, there could still be an old lost
or duplicate packet in the network somewhere and if the stack didn't
remember recent connections, it wouldn't know to ignore those packets.

What problem are you having *precisely*? "Hindering the messages to be
pumped later" is both not precise (hindering how?) and makes no sense
(how can there be later messages after you close the connection?).
It's possible you problem, whatever it is, has nothing to do with
connections in TIME_WAIT state.

DS

On May 6, 3:09 am, David Schwartz <dav...@webmaster.com> wrote:
> On May 5, 4:31 am, Madhur <madhurr...@gmail.com> wrote:
>
> > I am currently developing a tool using Python 2.4.2 which will be used
> > as a sink to pump TCP messages. During which i have observed that the
> > TCP close interface provided by Python is not closing the connection.
> > This i confirmed by looking at the ethereal logs, which show proper 3
> > way FIN ACK Handshake. But the netstat reports TIME_WAIT state for the
> > TCP connection, which is hindering the messages to be pumped later. I
> > would like to know the problem cause. Is it because of the application
> > or due to operating system?
>
> Umm, huh? How can you pump messages after the connection is closed?
>
> The TIME_WAIT state is just a relic from connections past. It is
> needed to allow the TCP stack to do the right thing (which is nothing
> at all) if it receives a duplicate packet from the session. Even
> though it closed the connection down, there could still be an old lost
> or duplicate packet in the network somewhere and if the stack didn't
> remember recent connections, it wouldn't know to ignore those packets.
>
> What problem are you having *precisely*? "Hindering the messages to be
> pumped later" is both not precise (hindering how?) and makes no sense
> (how can there be later messages after you close the connection?).
> It's possible you problem, whatever it is, has nothing to do with
> connections in TIME_WAIT state.
>
>

Sorry for lack of clarity in my statements. Consider a simulated
environment where i have one client and one server. The client is
opening a NEW CONNECTION each time it wants to send message to server.
After receiving the message from client, the server responds with a
response to client over the same connection. The client upon accept of
the server response initiates shutdown of the Connection opened. Each
time I call a shutdown of the connection the connection goes to
TIME_WAIT state. The connections are created at such a rate that the
ephemeral port to be used are getting exhausted. I can increase the
port range but it is not the appropriate turn around. My concern was
even though i do a proper shutdown upon the connection and underlying
FIN,ACK 3 way handshake even though happens,, the netstat reports
TIME_WAIT state for the connection. Therefore I would like to know is
there anything i need to do whenever I disconnect the connection,
other than calling shutdown and close on the socket descriptor?

In article <1178429316.017120.239710@e65g2000hsc.googlegroups .com>,
Madhur <madhurrajn@gmail.com> wrote:

> My concern was
> even though i do a proper shutdown upon the connection and underlying
> FIN,ACK 3 way handshake even though happens,, the netstat reports
> TIME_WAIT state for the connection.

This is normal. The side that initiates the connection close always
goes into TIME-WAIT for 2*MSL seconds after receiving the FIN from the
other end. Read RFC 793.

Is there some way you can redesign your application so that it keeps
connections open longer, rather than opening and closing them so rapidly?

--
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
*** PLEASE don't copy me on replies, I'll read them in the group ***

On May 5, 10:28 pm, Madhur <madhurr...@gmail.com> wrote:

> Sorry for lack of clarity in my statements. Consider a simulated
> environment where i have one client and one server. The client is
> opening a NEW CONNECTION each time it wants to send message to server.

This is only reasonable if messages are infrequent.

> After receiving the message from client, the server responds with a
> response to client over the same connection. The client upon accept of
> the server response initiates shutdown of the Connection opened. Each
> time I call a shutdown of the connection the connection goes to
> TIME_WAIT state. The connections are created at such a rate that the
> ephemeral port to be used are getting exhausted.

TCP is not suitable for this use. You can try to make it work, but
you've already tied one hand behind your back.

> I can increase the
> port range but it is not the appropriate turn around. My concern was
> even though i do a proper shutdown upon the connection and underlying
> FIN,ACK 3 way handshake even though happens,, the netstat reports
> TIME_WAIT state for the connection. Therefore I would like to know is
> there anything i need to do whenever I disconnect the connection,
> other than calling shutdown and close on the socket descriptor?

Unfortunately, TCP takes resources and times to start up and shut down
a connection. There's no a whole lot you can do about this. There are
known serious hazards with trying to reduce the TIME_WAIT interval.

DS

David Schwartz <davids@webmaster.com> writes:
> Unfortunately, TCP takes resources and times to start up and shut down
> a connection. There's no a whole lot you can do about this. There are
> known serious hazards with trying to reduce the TIME_WAIT interval.

aka TCP session has a minimum seven packet exchange ... plus FINWAIT
tail-end on close.

mid-90s HTTP webservers encountered significant TCP scale-up problem
with common implementations that had linear scan of FINWAIT list. the
problem was that some loaded webservers were cycling TCP sessions so
fast (resulting in quite lengthy FINWAIT lists) that several of the
major webservers were hitting 100% processor busy and spending nearly
all of that scanning the FINWAIT list. Several vendors relatively
quickly put together new releases that significantly reworked how
FINWAIT list was handled.

there has also been work on reliable transactions protocols that had
fewer minimum packet exchange ... i.e. like VMTP/RFC1045 with 5 minimum
packet exchange.

On Sun, 06 May 2007 15:09:41 -0600, Anne & Lynn Wheeler
<lynn@garlic.com> wrote:

>mid-90s HTTP webservers encountered significant TCP scale-up problem
>with common implementations that had linear scan of FINWAIT list.

The problem was not only the FINWAIT list, it was the general way timers
where processed in most TCP implementations. It was just a sequential
list that was scanned at a specific interval to look if a timer has
expired on a specific connection. If the list was to long the processing
took ages to finish processing. Problematic was especially the TIME-WAIT
processing when thousands of connections finished within a few seconds.

On Sun, 06 May 2007 15:09:41 -0600, Anne & Lynn Wheeler wrote:
> aka TCP session has a minimum seven packet exchange


SYN, data, FIN ->
<- SYN, data, FIN, ACK(FIN)
ACK(FIN) ->


Legal, AFAIK, within the protocol spec.
Unwise in the face of SYN attacks.
Not supported by most OS'.

- Jeremy