Hi,

I am unable to maintain a TCP connection across the internet to a
couple of servers for more than a few KB data sent.

After googling the problem, it appears to be very similar to reports
of 'mtu' issues, however reducing the mtu setting of my local machine,
the router, and the remote machine seems to make no difference. I'd
really appreciate any suggestions people can offer...

My setup:
The local host: Linux 2.4.27
ADSL router: USR 9003
The remote hosts: OpenVZ VPS, running Linux 2.6.x

The problem (most obvious symptoms):
With ssh I can connect to either remote host. Everything seems fine
until at some point the connection hangs (usually, it seems, while
typing fast). With scp I can copy small files from local host to
remote host, but for files >=4k (ish) the connection stalls.

To diagnose the problem, I enabled TCP chargen, echo & discard on one
of the remote boxes and did a certain amount of testing with netcat.

I found that for data sent from the remote box to the local box I was
able to transfer several MB without problem. However, for data sent
from local box to the remote, the connection hung at around 4k-6k data
sent.

A friend opened TCP echo service on his box for me, and I was able to
echo several MB through it without issue (i.e. both send & receive).

He also repeated some tests and found no issue with sending from his
box data to the remote box.

I have two servers (both VPS's), on different networks but both in the
USA, and both exhibit the same problem. I am in the UK, as is the
friend who ed with testing.

I do not have any other (noticable) problems with the connection on my
local machine. It is online 24/7 (via the ADSL router), serves a
'hobby' web-site, and I ssh to it from elsewhere on a regular basis.

Based on a google search for similar problems,I have tried reducing
the 'mtu' setting for eth1 on the local box, as well as the eth
interface on the router, as well as all eth interfaces on the remote
VPS. None of this seemed to make any difference, although tweaking
was done with a certain amount of ignorance so I could still be
persuaded it is an mtu issue...

Hi,

This sounds like a TCP receive buffer issue with OpenVZ.

What is the advertised window size by your VPS ? You can run tcpdump /
ethereal to get this information.




news-03@foobar.clara.co.uk wrote
> Hi,
>
> I am unable to maintain a TCP connection across the internet to a
> couple of servers for more than a few KB data sent.
>
> After googling the problem, it appears to be very similar to reports
> of 'mtu' issues, however reducing the mtu setting of my local machine,
> the router, and the remote machine seems to make no difference. I'd
> really appreciate any suggestions people can offer...
>
> My setup:
> The local host: Linux 2.4.27
> ADSL router: USR 9003
> The remote hosts: OpenVZ VPS, running Linux 2.6.x
>
> The problem (most obvious symptoms):
> With ssh I can connect to either remote host. Everything seems fine
> until at some point the connection hangs (usually, it seems, while
> typing fast). With scp I can copy small files from local host to
> remote host, but for files >=4k (ish) the connection stalls.
>
> To diagnose the problem, I enabled TCP chargen, echo & discard on one
> of the remote boxes and did a certain amount of testing with netcat.
>
> I found that for data sent from the remote box to the local box I was
> able to transfer several MB without problem. However, for data sent
> from local box to the remote, the connection hung at around 4k-6k data
> sent.
>
> A friend opened TCP echo service on his box for me, and I was able to
> echo several MB through it without issue (i.e. both send & receive).
>
> He also repeated some tests and found no issue with sending from his
> box data to the remote box.
>
> I have two servers (both VPS's), on different networks but both in the
> USA, and both exhibit the same problem. I am in the UK, as is the
> friend who ed with testing.
>
> I do not have any other (noticable) problems with the connection on my
> local machine. It is online 24/7 (via the ADSL router), serves a
> 'hobby' web-site, and I ssh to it from elsewhere on a regular basis.
>
> Based on a google search for similar problems,I have tried reducing
> the 'mtu' setting for eth1 on the local box, as well as the eth
> interface on the router, as well as all eth interfaces on the remote
> VPS. None of this seemed to make any difference, although tweaking
> was done with a certain amount of ignorance so I could still be
> persuaded it is an mtu issue...

On Mon, 05 Jun 2006 00:09:03 +0100, news-03 wrote:

....

> I found that for data sent from the remote box to the local box I was
> able to transfer several MB without problem. However, for data sent
> from local box to the remote, the connection hung at around 4k-6k data
> sent.

....

> Based on a google search for similar problems,I have tried reducing
> the 'mtu' setting for eth1 on the local box, as well as the eth
> interface on the router, as well as all eth interfaces on the remote
> VPS. None of this seemed to make any difference, although tweaking
> was done with a certain amount of ignorance so I could still be
> persuaded it is an mtu issue...

Ethernet has a MTU of 1500. If you have MTU problems, you would not be
able to get 4K across. I think you should look elsewhere.

But first thing to do is to get a packet sniffer installed. As most
(all) of your systems use linux, install ethereal and have a look what
happens on the packet level.

If the box doesn't have Xm just use tcpdump (with -s 1500!!!) and -w the
trace to a file. Then use ethereal on any other box (can even be Windows)
to look at the trace.

M4
--
Redundancy is a great way to introduce more single points of failure.

Martijn Lievaart <m@remove.this.part.rtij.nl> wrote:
> On Mon, 05 Jun 2006 00:09:03 +0100, news-03 wrote:
> ...
>> I found that for data sent from the remote box to the local box I was
>> able to transfer several MB without problem. However, for data sent
>> from local box to the remote, the connection hung at around 4k-6k data
>> sent.
> ...
>> Based on a google search for similar problems,I have tried reducing
>> the 'mtu' setting for eth1 on the local box, as well as the eth
>> interface on the router, as well as all eth interfaces on the remote
>> VPS. None of this seemed to make any difference, although tweaking
>> was done with a certain amount of ignorance so I could still be
>> persuaded it is an mtu issue...

> Ethernet has a MTU of 1500. If you have MTU problems, you would not be
> able to get 4K across. I think you should look elsewhere.

Well, _IP_ over Ethernet has an MTU of 1500 bytes Indeed they
probably need to look elsewhere. It is interesting (to me anyway)
that the limit on the initial congestion window over a 1500 byte MTU
IP netowrk is also very close to 4K - at 4380 bytes.

> If the box doesn't have Xm just use tcpdump (with -s 1500!!!) and -w
> the trace to a file. Then use ethereal on any other box (can even be
> Windows) to look at the trace.

Why the snapsize of 1500? I would think that anything beyond the TCP
headers wuldn't be all that interesting here.

rick jones
--
The glass is neither half-empty nor half-full. The glass has a leak.
The real question is "Can it be patched?"
these opinions are mine, all mine; HP might not want them anyway...
feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH...

In article <mgCjg.1822$PF7.872@news.cpqcorp.net>,
Rick Jones <rick.jones2@hp.com> wrote:
>Martijn Lievaart <m@remove.this.part.rtij.nl> wrote:
>> On Mon, 05 Jun 2006 00:09:03 +0100, news-03 wrote:

>> Ethernet has a MTU of 1500. If you have MTU problems, you would not be
>> able to get 4K across. I think you should look elsewhere.
>
>Well, _IP_ over Ethernet has an MTU of 1500 bytes Indeed they
>probably need to look elsewhere.

What about jumbo frames?

Despite the reservations of most contributors to the IETF pppext
mailing list, there is talk about using jumbo frames to patch the
minor braindamage of the PPPoE 1492 MTU. (The major braindamage
of PPPoE is that PPPoE was ever written down, not to mention allowed
to escape into the wild.)


Vernon Schryver vjs@rhyolite.com

Vernon Schryver <vjs@calcite.rhyolite.com> wrote:
> In article <mgCjg.1822$PF7.872@news.cpqcorp.net>,
> Rick Jones <rick.jones2@hp.com> wrote:
>>Martijn Lievaart <m@remove.this.part.rtij.nl> wrote:
>>> On Mon, 05 Jun 2006 00:09:03 +0100, news-03 wrote:

>>> Ethernet has a MTU of 1500. If you have MTU problems, you would
>>> not be able to get 4K across. I think you should look elsewhere.
>>
>>Well, _IP_ over Ethernet has an MTU of 1500 bytes Indeed they
>>probably need to look elsewhere.

> What about jumbo frames?

What about them?-) I think they are fine and dandy, and consider a
9000 byte IP MTU to be the de facto definition of Jumbo Frames, but
since there isn't any de jure standard - esp the IEEE - it seems that
one can have as many definitions of "Jubmo Frames" as there are
vendors.

And, to harken back to other fun sore points of mine if it
ain't in the IEEE spec it ain't "Ethernet" right?-)

> Despite the reservations of most contributors to the IETF pppext
> mailing list, there is talk about using jumbo frames to patch the
> minor braindamage of the PPPoE 1492 MTU. (The major braindamage of
> PPPoE is that PPPoE was ever written down, not to mention allowed to
> escape into the wild.)

Somehow I wonder if that might not be adding fuel to the fire - as I
doubt they would have "Jumbo Frame => 9000 byte MTU" and so _further_
confuse things wrt "What is the size of a Jumbo Frame."

rick jones
--
oxymoron n, commuter in a gas-guzzling luxury SUV with an American flag
these opinions are mine, all mine; HP might not want them anyway...
feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH...

On Tue, 13 Jun 2006 17:06:26 +0000, Rick Jones wrote:

>> Ethernet has a MTU of 1500. If you have MTU problems, you would not be
>> able to get 4K across. I think you should look elsewhere.
>
> Well, _IP_ over Ethernet has an MTU of 1500 bytes Indeed they

Looking at the name for this group made me assume IP Actually an
interesting question, what's the MTU for other protocols on ethernet?

>> If the box doesn't have Xm just use tcpdump (with -s 1500!!!) and -w
>> the trace to a file. Then use ethereal on any other box (can even be
>> Windows) to look at the trace.
>
> Why the snapsize of 1500? I would think that anything beyond the TCP
> headers wuldn't be all that interesting here.

Probably not, but I've more than once needed to see the data in the stream
(wshich I could not foresee), so I make a habit of capturing the whole
frame. Actually I should just use 0 (captrue whole packet), but I always
forget that.

M4
--
Redundancy is a great way to introduce more single points of failure.

"Martijn Lievaart" <m@remove.this.part.rtij.nl> wrote:

> Looking at the name for this group made me assume IP Actually an
> interesting question, what's the MTU for other protocols on ethernet?

The 1500 byte limit *is* Ethernet's, so any protocol running over
Ethernet, in principle, has that limit.

However, many (not all by any means) Ethernet switches permit the use of
jumbo frames. These are vendor specific enhancements, all non-standard.
I guess you'd have to stick with the Ethernet "type" format to use jumbo
frames, though, else there's no way to indicate "length" beyond 1535
bytes, so LLC/SNAP encapsulation is out, I think.

There ought to be a "type" that indicates LLC/SNAP, but I don't see it
in

http://www.iana.org/assignments/ethernet-numbers

Bert