Hi!

Short version:

How do I change the TCP window field of packets as they are bridged
through a Linux box?

Longer version:

I'd like to limit the download speed of TCP to reduce the effect it has
on VoIP streams.

As I understand it part of the solution is to limit the TCP window
size. This will avoid the situation where the sender is clogging up the
receiver's connection.

I can do it if the TCP download is initiated from a box I have control
over (http://proj.sunet.se/E2E/tcptune.html). And it works great.

But I want to share the link with machines I have no control over. I
can place them behind one I can control. But how do I mangle the TCP
packet headers as they come through? I assume using the netfilter
tools, but I don't know where to start.

Thanks!
Janto

On 21 Nov 2006, in the Usenet newsgroup comp.protocols.tcp-ip, in article
<1164102312.350682.229120@m7g2000cwm.googlegroups. com>, Janto wrote:

>How do I change the TCP window field of packets as they are bridged
>through a Linux box?

I don't believe that you do.

>I'd like to limit the download speed of TCP to reduce the effect it has
>on VoIP streams.
>
>As I understand it part of the solution is to limit the TCP window
>size. This will avoid the situation where the sender is clogging up the
>receiver's connection.
>
>I can do it if the TCP download is initiated from a box I have control
>over (http://proj.sunet.se/E2E/tcptune.html). And it works great.

-rw-rw-r-- 1 gferg ldp 297491 Sep 4 2003 Adv-Routing-HOWTO

The term you are looking for is 'throttling'. A google search might also
be productive.

>But I want to share the link with machines I have no control over. I
>can place them behind one I can control. But how do I mangle the TCP
>packet headers as they come through? I assume using the netfilter
>tools, but I don't know where to start.

http://www.iptables.org/documentation/HOWTO/

There are seven HOWTOs there, but the only mention of 'TCP window' is in
the 'netfilter-extensions-HOWTO' and that's not doing what you'd like. You
might be able to get creative with a rate limit and drop rule, but I'm not
aware of any simple/direct method.

Old guy

On Nov 21, 10:11 pm, ibupro...@painkiller.example.tld (Moe Trin) wrote:
> On 21 Nov 2006, in the Usenet newsgroup comp.protocols.tcp-ip, in article
>
> <1164102312.350682.229...@m7g2000cwm.googlegroups. com>, Janto wrote:
> >How do I change theTCPwindowfield of packets as they are bridged
> >through a Linux box?
>
> I don't believe that you do.

That's surprising. Why do you think it's not possible?

I just ran into "route add ... window W" which "Set the TCP window size
for connections over this route to W bytes". I assume it's not
applicable if the session is established from a different PC?

> >I'd like to limit the download speed ofTCPto reduce the effect it has
> >on VoIP streams.
>
> >As I understand it part of the solution is to limit theTCPwindow
> >size. This will avoid the situation where the sender is clogging up the
> >receiver's connection.
>
> >I can do it if theTCPdownload is initiated from a box I have control
> >over (http://proj.sunet.se/E2E/tcptune.html). And it works great.-rw-rw-r-- 1 gferg ldp 297491 Sep 4 2003 Adv-Routing-HOWTO
>
> The term you are looking for is 'throttling'. A google search might also
> be productive.
>
> >But I want to share the link with machines I have no control over. I
> >can place them behind one I can control. But how do I mangle theTCP
> >packet headers as they come through? I assume using the netfilter
> >tools, but I don't know where to start.http://www.iptables.org/documentation/HOWTO/
>
> There are seven HOWTOs there, but the only mention of 'TCPwindow' is in
> the 'netfilter-extensions-HOWTO' and that's not doing what you'd like. You
> might be able to get creative with a rate limit and drop rule, but I'm not
> aware of any simple/direct method.
>
> Old guy

I don't think I can use a rate limit on its own. There's an initial
burst from the sender side that saturates the link and interferes with
the voice traffic.

Thanks.

Janto

Janto wrote:
> On Nov 21, 10:11 pm, ibupro...@painkiller.example.tld (Moe Trin) wrote:
> > On 21 Nov 2006, in the Usenet newsgroup comp.protocols.tcp-ip, in article
> >
> > <1164102312.350682.229...@m7g2000cwm.googlegroups. com>, Janto wrote:
> > >How do I change theTCPwindowfield of packets as they are bridged
> > >through a Linux box?
> >
> > I don't believe that you do.
>
> That's surprising. Why do you think it's not possible?
>
> I just ran into "route add ... window W" which "Set the TCP window size
> for connections over this route to W bytes". I assume it's not
> applicable if the session is established from a different PC?

That sounds like setting the TCP window size for *your*own*
connections. It doesn't apply to this case.

> > >I'd like to limit the download speed ofTCPto reduce the effect it has
> > >on VoIP streams.
> >
> > >As I understand it part of the solution is to limit theTCPwindow
> > >size. This will avoid the situation where the sender is clogging up the
> > >receiver's connection.

limiting the TCP window size may reduce throughput (depending on
end-to-end latency) to less than the overall link capacity. But
that's not how you want to do it.

> > >I can do it if theTCPdownload is initiated from a box I have control
> > >over (http://proj.sunet.se/E2E/tcptune.html). And it works great.-rw-rw-r-- 1 gferg ldp 297491 Sep 4 2003 Adv-Routing-HOWTO
> >
> > The term you are looking for is 'throttling'. A google search might also
> > be productive.

> I don't think I can use a rate limit on its own. There's an initial
> burst from the sender side that saturates the link and interferes with
> the voice traffic.

A well behaved TCP will not send a 'burst' of data unless its packets
are being delivered reliably. Your job as the throttle is to deliver
the packets UNreliably. Throw away any packets that you consider to be
'too much'. The sending TCP will interpret the packet loss as network
congestion and will slow down. For TCP traffic this strategy works
even if you and the sending system are on opposite ends of the link you
want to protect.

/chris

On 22 Nov 2006, in the Usenet newsgroup comp.protocols.tcp-ip, in article
<1164183938.210619.265380@h48g2000cwc.googlegroups .com>, Janto wrote:
> (Moe Trin) wrote:
>>
>> Janto wrote:
>> >How do I change theTCPwindowfield of packets as they are bridged
>> >through a Linux box?
>>
>> I don't believe that you do.
>
>That's surprising. Why do you think it's not possible?

The ideal situation would be to set the window size on a specific
connection, either source/destination IP, or port. That would normally
be a firewall type function, rather than a bridging function (which is
more concerned with IP than TCP).

>I just ran into "route add ... window W" which "Set the TCP window size
>for connections over this route to W bytes". I assume it's not
>applicable if the session is established from a different PC?

I've never used it, but that appears to set the maximum window size on
any traffic using that route from "this" computer. Traffic being
forwarded by this computer over "that" route should be effected.

>I don't think I can use a rate limit on its own. There's an initial
>burst from the sender side that saturates the link and interferes with
>the voice traffic.

There would still be the burst, unless you set the window to something
around the data size of a single (or "few") packets. The normal solution
is either throttling, or getting a bigger pipe.

Old guy

In article <1164183938.210619.265380@h48g2000cwc.googlegroups .com>,
Janto <jantod@gmail.com> wrote:

>> >How do I change theTCPwindowfield of packets as they are bridged
>> >through a Linux box?
>>
>> I don't believe that you do.
>
>That's surprising. Why do you think it's not possible?

Because a bridge or router just forwards packets. It should not
change any bits except IP options and the IP TTL field. (yes, I've
heard of NAT; I wrote "should not.")

>I just ran into "route add ... window W" which "Set the TCP window size
>for connections over this route to W bytes". I assume it's not
>applicable if the session is established from a different PC?

yes.

>> >I'd like to limit the download speed ofTCPto reduce the effect it has
>> >on VoIP streams.

>I don't think I can use a rate limit on its own. There's an initial
>burst from the sender side that saturates the link and interferes with
>the voice traffic.

There is more than one thing wrong there.

Having a router or bridge in the middle of the path fiddle with the
window fields of passing TCP segments is not a reliable way to get the
desired effect. It is often possible to staturate intermediate links
even with small TCP windows. Perhaps that is why very few "traffic
shapers", "policing" and so forth fiddle with the window fields of
passing TCP packets.

Next, there is no big burst of TCP segments with a TCP connection is
made. "Slow start" is part of the TCP congestion avoidance machinery.
It is applied even at the start of a connection (modulo a constant of
some controversy. See RFC 3390). "Slow start" means about what it
sounds like. A TCP sender only slowly starts increasing speed of its
output while it continues to receive ACKs from peer showing that all
of the data is getting throught. The sender very quickly slows down
when there is evidence that the data is not getting through and then
slowly starts to speed up again.

Perhaps the best as well as easiest way to avoid congesting your slow
link is to ensure that it has as little buffering as possible. Dropping
a single voice packet tends to do little harm. Dropping a TCP segment
radically slows down TCP senders. Besides, there are zillions of ways
to prefer dropping bulk TCP segments to voice packets.

This issue is decades old. The same bad, non-solutions such as trashing
passing TCP packets or replacing the whole TCP protocol keep getting
re-invented. Ask Google about "congestion avoidance", because what you
are trying to do is avoid congesting that slow link. Read a good TCP
textbook such as Stevens. See RFC 2581.


Vernon Schryver vjs@rhyolite.com

I haven't heard about network devices manipulate TCP window size (I am
not a TCP or Linux expert though).

I had some experience working on networking devices. And for the case,
some special traffic needs to be protected (voip in your case) we can
use CB(W)Q - Class Based (Weighted) Queueing. In general you define
traffic(s) into a category (Class) and apply that into an entity
(Queueing) within an interface.
You can assign, for example, certain bandwidth for voip traffics and
give the highest priority for that.
Some devices have a RED feature; you can combine this with Class. What
this feature does is if congestion is detected it randomly drop tcp
segments (within that class if class is used) and that causees tcp end
points slow down (since TCPs see packet drops). You can use this for
other traffics if Linux has one.
Seems Linux has CBQ? (when I do google, see some hint but at a glance
it doesn't look very straight forward).

Not sure if this you.

Janto wrote:

>
> I don't think I can use a rate limit on its own. There's an initial
> burst from the sender side that saturates the link and interferes with
> the voice traffic.

Shaping from the wrong end of a bottleneck is a pain, and I agree
slowstart is slower than getting the whole window, but it is somewhat
exponential and will soon bork up the far buffer (esp. as browsers/web
servers may use several concurrent TCPs and start new ones as soon as
they finish).

Depends on how fast your link is as to what you can acheive -
sacrificing bandwidth will be requiired.

Just throttling (ie. having a queue after the ISP queue, but going a bit
slower) doesn't save you from the bursts).

Policing (not buffering but using a virtual buffer and then dropping
when it would be full) is better in someways as at least when you drop
the other end gets to know sooner - ie if you you tail drop from a
queue, the other end doesn't know for a while and if in slowstart will
still be ramping up.

In some ways using queues is better - ie in Linux you get more choice of
queue type/classification. You must keep the queues short, though and
drop packets, wastefull as it may seem.

When I had a 512kbit link I used to use iptables connbytes to send new
connections to an extra short sfq to drop early and get them out of
slowstart (small packets were exempted so the connections got started
OK). Then after XKB they went to slightly longer queues. I also used to
have a hacked esfq that (I think - I am crap at reading code) head
dropped - only bulk packets from established connections were sent to it.

As for messing with windows - I think packeteer does that (but then I've
read it didn't handle scaling well) If you want to do it you will need
to code something that keeps alot of state about every connection (and
possibly not by chance use their patented algorithm what ever that is -
maybe only worry if you live in the usa).

Even if you do create something that dynamically messes with window
sizes you still would be screwed if by chance alot of connections with
different rtts all sent their one packet at the same time etc. There are
alot of things just not in your control when you are at the wrong end of
the bottleneck. I think you may also hurt yourself when in competition
for others maxed egress bandwidth by using a smaller window - TCP is not
very fair.

Saying all that (trying to) shape downloads is still alot better than
doing nothing.

Andy.