In article <1189278847.226956.237620@19g2000hsx.googlegroups. com>,
reuven <wreuven@gmail.com> wrote:
>First, thanks to all the responders.
>
>As I mentioned, my problems are both in CPU and MEMORY and in the fact
>that the customer has put significant restraints on changing the
>hardware & protocol. Cascading the send is not an option because of
>the network topology.
>
>I suspect that you are correct and the CPU issue is mostly a red-
>herring. (the hardware does support offloaded checksum) but the memory
>issue is very real. Also, I was a bit misleading in mentioning 1000
>connections. The number is actually closer to 2000 with a total output
>of around 1.2 gigabit per second (using 2 x 1 gigabit NICs) per server
>(this is the number I need to improve on).

CPU usage, especially when sending data, is rarely a bottleneck. If you
have real-time constraints, it might be, but look elsewhere first. You
will want to collect some statistics on memory usage and correlate that
with the number of active connections. The biggest win is generally in
a good round-robin algorithm. Keep a table of all open connections and
order your sends with the least-recently sent connection, limited by
how many connections can concurrently be sending data without seriously
impacting overall performance.

Your bandwidth numbers and NIC configuration are vital pieces of
information. See if you can find a way to bond the two NICs into a
single virtual port. Or, if you can find a natural division in your
topology, attach this server to two subnets and attempt to balance the
load between them.

>IOCP sounds line an interesting direction. I'll look into it.

IO completion ports are a big win because you don't require a thread for
each open connection. This often alleviates CPU-related issues.