I installed a gigabit network switch and a gigabit enabled laptop wifi
adapter (with gigabit, obviously, available on ethernet ports only) in
my house.

Two computers in my home are connected to the switch and one (laptop)
to the wifi adaptor.

The highest possible speed of a gigabit connection is about 111
megabytes per second.

Naturally, I did some tests with a noncompressible 1 gigabyte long
file (fragment of some gzipped file exactly one GB long).

My first test was to scp files from one computer on the switch to
another. Here, I was disappointed as the highest speed was only 22
megabytes per second one way and 46 another way. About 20 and 40
percent of maximum.

Then I tried using HTTP to transfer the same file (both computers are
webservers). To my huge surprise, it made a world of difference and
the transfer speed was 111 or so megabytes per second.

So, now I have a dilemma, I have a fast pipe, but scp is not fast
enough (given my CPU) to encrypt/decrypt so much data.

I tried something else, which is doing wc -l on a NFS mounted drive
(same two computers). It was UNBELIEVABLY slow and the load average on
the NFS server shot WAY up. Transferring a 336 MB file took 157
seconds, or about e megabytes per second (vs 111 mbps that I achieved
with HTTP).

So, the conclusion is, HTTP is fast (no wonder), SSH is "medium",
and NFS is "slow, very bad".

The connection to laptop is a disappointment in its own right, since
even with all-ethernet connection, I get about 3 megabytes per
second. I think that I need to pull a new wire in the wall.

So, the short of it is that there is much work to be done.

i

Ignoramus26973 wrote:
> I installed a gigabit network switch and a gigabit enabled laptop wifi
> adapter (with gigabit, obviously, available on ethernet ports only) in
> my house.
>
> Two computers in my home are connected to the switch and one (laptop)
> to the wifi adaptor.
>
> The highest possible speed of a gigabit connection is about 111
> megabytes per second.
>
> Naturally, I did some tests with a noncompressible 1 gigabyte long
> file (fragment of some gzipped file exactly one GB long).
>
> My first test was to scp files from one computer on the switch to
> another. Here, I was disappointed as the highest speed was only 22
> megabytes per second one way and 46 another way. About 20 and 40
> percent of maximum.
>
> Then I tried using HTTP to transfer the same file (both computers are
> webservers). To my huge surprise, it made a world of difference and
> the transfer speed was 111 or so megabytes per second.
>
> So, now I have a dilemma, I have a fast pipe, but scp is not fast
> enough (given my CPU) to encrypt/decrypt so much data.
>
> I tried something else, which is doing wc -l on a NFS mounted drive
> (same two computers). It was UNBELIEVABLY slow and the load average on
> the NFS server shot WAY up. Transferring a 336 MB file took 157
> seconds, or about e megabytes per second (vs 111 mbps that I achieved
> with HTTP).

So, about 2.7 MB/s? :-)

>
> So, the conclusion is, HTTP is fast (no wonder), SSH is "medium",
> and NFS is "slow, very bad".

I ran into a similar problem with NFS ages back. Turns out you can
largely fix it by configuring NFS to increase the packet size. Been a
few years since I did anything with NFS, though, so you'll have to look
through the docs. I hear they use it in computer clusters, so it can't
be slow in *all* cases.

Around the same time, I also noticed that rsync can have similar speed
issues to SCP for a first copy, owing to it insisting on using SSH for RSH.

On Wed, 12 Sep 2007 01:14:07 -0400, Michael Mol wrote:

> Ignoramus26973 wrote:
>> I installed a gigabit network switch and a gigabit enabled laptop wifi
>> adapter (with gigabit, obviously, available on ethernet ports only) in
>> my house.
>>
>> Two computers in my home are connected to the switch and one (laptop)
>> to the wifi adaptor.
>>
>> The highest possible speed of a gigabit connection is about 111
>> megabytes per second.
>>
>> Naturally, I did some tests with a noncompressible 1 gigabyte long file
>> (fragment of some gzipped file exactly one GB long).
>>
>> My first test was to scp files from one computer on the switch to
>> another. Here, I was disappointed as the highest speed was only 22
>> megabytes per second one way and 46 another way. About 20 and 40
>> percent of maximum.
>>
>> Then I tried using HTTP to transfer the same file (both computers are
>> webservers). To my huge surprise, it made a world of difference and the
>> transfer speed was 111 or so megabytes per second.
>>
>> So, now I have a dilemma, I have a fast pipe, but scp is not fast
>> enough (given my CPU) to encrypt/decrypt so much data.
>>
>> I tried something else, which is doing wc -l on a NFS mounted drive
>> (same two computers). It was UNBELIEVABLY slow and the load average on
>> the NFS server shot WAY up. Transferring a 336 MB file took 157
>> seconds, or about e megabytes per second (vs 111 mbps that I achieved
>> with HTTP).
>
> So, about 2.7 MB/s? :-)
>
>
>> So, the conclusion is, HTTP is fast (no wonder), SSH is "medium", and
>> NFS is "slow, very bad".
>
> I ran into a similar problem with NFS ages back. Turns out you can
> largely fix it by configuring NFS to increase the packet size. Been a
> few years since I did anything with NFS, though, so you'll have to look
> through the docs. I hear they use it in computer clusters, so it can't
> be slow in *all* cases.

How do you configure NFS to increase the packet size? Also are you
talking about NFS V4 or V3?

On 12 Sep 2007 12:52:09 GMT, General Schvantzkoph <schvantzkoph@yahoo.com> wrote:


>On Wed, 12 Sep 2007 01:14:07 -0400, Michael Mol wrote:

>> Ignoramus26973 wrote:
>>> I installed a gigabit network switch and a gigabit enabled laptop wifi
>>> adapter (with gigabit, obviously, available on ethernet ports only) in
>>> my house.
>>>
>>> Two computers in my home are connected to the switch and one (laptop)
>>> to the wifi adaptor.
>>>
>>> The highest possible speed of a gigabit connection is about 111
>>> megabytes per second.
>>>
>>> Naturally, I did some tests with a noncompressible 1 gigabyte long file
>>> (fragment of some gzipped file exactly one GB long).
>>>
>>> My first test was to scp files from one computer on the switch to
>>> another. Here, I was disappointed as the highest speed was only 22
>>> megabytes per second one way and 46 another way. About 20 and 40
>>> percent of maximum.
>>>
>>> Then I tried using HTTP to transfer the same file (both computers are
>>> webservers). To my huge surprise, it made a world of difference and the
>>> transfer speed was 111 or so megabytes per second.
>>>
>>> So, now I have a dilemma, I have a fast pipe, but scp is not fast
>>> enough (given my CPU) to encrypt/decrypt so much data.
>>>
>>> I tried something else, which is doing wc -l on a NFS mounted drive
>>> (same two computers). It was UNBELIEVABLY slow and the load average on
>>> the NFS server shot WAY up. Transferring a 336 MB file took 157
>>> seconds, or about e megabytes per second (vs 111 mbps that I achieved
>>> with HTTP).
>>
>> So, about 2.7 MB/s? :-)
>>
>>
>>> So, the conclusion is, HTTP is fast (no wonder), SSH is "medium", and
>>> NFS is "slow, very bad".
>>
>> I ran into a similar problem with NFS ages back. Turns out you can
>> largely fix it by configuring NFS to increase the packet size. Been a
>> few years since I did anything with NFS, though, so you'll have to look
>> through the docs. I hear they use it in computer clusters, so it can't
>> be slow in *all* cases.

>How do you configure NFS to increase the packet size? Also are you
>talking about NFS V4 or V3?

when you mount it. Why didn't you just google for "nfs packet size"?

On Tue, 11 Sep 2007 23:02:37 -0500, Ignoramus26973 wrote:

> I installed a gigabit network switch and a gigabit enabled laptop wifi
> adapter (with gigabit, obviously, available on ethernet ports only) in
> my house.
>
> Two computers in my home are connected to the switch and one (laptop)
> to the wifi adaptor.
>
> The highest possible speed of a gigabit connection is about 111
> megabytes per second.
>
> Naturally, I did some tests with a noncompressible 1 gigabyte long
> file (fragment of some gzipped file exactly one GB long).
>
> My first test was to scp files from one computer on the switch to
> another. Here, I was disappointed as the highest speed was only 22
> megabytes per second one way and 46 another way. About 20 and 40
> percent of maximum.
>
> Then I tried using HTTP to transfer the same file (both computers are
> webservers). To my huge surprise, it made a world of difference and
> the transfer speed was 111 or so megabytes per second.
>
> So, now I have a dilemma, I have a fast pipe, but scp is not fast
> enough (given my CPU) to encrypt/decrypt so much data.
>
> I tried something else, which is doing wc -l on a NFS mounted drive
> (same two computers). It was UNBELIEVABLY slow and the load average on
> the NFS server shot WAY up. Transferring a 336 MB file took 157
> seconds, or about e megabytes per second (vs 111 mbps that I achieved
> with HTTP).
>
> So, the conclusion is, HTTP is fast (no wonder), SSH is "medium",
> and NFS is "slow, very bad".
>
> The connection to laptop is a disappointment in its own right, since
> even with all-ethernet connection, I get about 3 megabytes per
> second. I think that I need to pull a new wire in the wall.
>
> So, the short of it is that there is much work to be done.
>
> i

you didn't mention Jumbo Frames.

IF ( BIG IF ) your Gigabit hardware supports 9K MTUs, you can get
a big boost if you set your MTU to 9000 on your nics.

I'll post some numbers from my setup in the next couple of days.

jack


--
D.A.M. - Mothers Against Dyslexia

see http://www.jacksnodgrass.com for my contact info.

jack - Grapevine/Richardson

On Wed, 12 Sep 2007 13:32:12 GMT, Jack Snodgrass <jacks_temp_id_bf2142@verizon.net> wrote:
> On Tue, 11 Sep 2007 23:02:37 -0500, Ignoramus26973 wrote:
>
>> I installed a gigabit network switch and a gigabit enabled laptop wifi
>> adapter (with gigabit, obviously, available on ethernet ports only) in
>> my house.
>>
>> Two computers in my home are connected to the switch and one (laptop)
>> to the wifi adaptor.
>>
>> The highest possible speed of a gigabit connection is about 111
>> megabytes per second.
>>
>> Naturally, I did some tests with a noncompressible 1 gigabyte long
>> file (fragment of some gzipped file exactly one GB long).
>>
>> My first test was to scp files from one computer on the switch to
>> another. Here, I was disappointed as the highest speed was only 22
>> megabytes per second one way and 46 another way. About 20 and 40
>> percent of maximum.
>>
>> Then I tried using HTTP to transfer the same file (both computers are
>> webservers). To my huge surprise, it made a world of difference and
>> the transfer speed was 111 or so megabytes per second.
>>
>> So, now I have a dilemma, I have a fast pipe, but scp is not fast
>> enough (given my CPU) to encrypt/decrypt so much data.
>>
>> I tried something else, which is doing wc -l on a NFS mounted drive
>> (same two computers). It was UNBELIEVABLY slow and the load average on
>> the NFS server shot WAY up. Transferring a 336 MB file took 157
>> seconds, or about e megabytes per second (vs 111 mbps that I achieved
>> with HTTP).
>>
>> So, the conclusion is, HTTP is fast (no wonder), SSH is "medium",
>> and NFS is "slow, very bad".
>>
>> The connection to laptop is a disappointment in its own right, since
>> even with all-ethernet connection, I get about 3 megabytes per
>> second. I think that I need to pull a new wire in the wall.
>>
>> So, the short of it is that there is much work to be done.
>>
>> i
>
> you didn't mention Jumbo Frames.
>
> IF ( BIG IF ) your Gigabit hardware supports 9K MTUs, you can get
> a big boost if you set your MTU to 9000 on your nics.

I think that it does support jumbo frames.

> I'll post some numbers from my setup in the next couple of days.

How do you set MTU? That would only work for local destinations,
right? It would not work for connections outside of my home LAN?

i

On Wed, 12 Sep 2007 13:08:06 GMT, AZ Nomad <aznomad.2@PremoveOBthisOX.COM> wrote:
> On 12 Sep 2007 12:52:09 GMT, General Schvantzkoph <schvantzkoph@yahoo.com> wrote:
>
>
>>On Wed, 12 Sep 2007 01:14:07 -0400, Michael Mol wrote:
>
>>> Ignoramus26973 wrote:
>>>> I installed a gigabit network switch and a gigabit enabled laptop wifi
>>>> adapter (with gigabit, obviously, available on ethernet ports only) in
>>>> my house.
>>>>
>>>> Two computers in my home are connected to the switch and one (laptop)
>>>> to the wifi adaptor.
>>>>
>>>> The highest possible speed of a gigabit connection is about 111
>>>> megabytes per second.
>>>>
>>>> Naturally, I did some tests with a noncompressible 1 gigabyte long file
>>>> (fragment of some gzipped file exactly one GB long).
>>>>
>>>> My first test was to scp files from one computer on the switch to
>>>> another. Here, I was disappointed as the highest speed was only 22
>>>> megabytes per second one way and 46 another way. About 20 and 40
>>>> percent of maximum.
>>>>
>>>> Then I tried using HTTP to transfer the same file (both computers are
>>>> webservers). To my huge surprise, it made a world of difference and the
>>>> transfer speed was 111 or so megabytes per second.
>>>>
>>>> So, now I have a dilemma, I have a fast pipe, but scp is not fast
>>>> enough (given my CPU) to encrypt/decrypt so much data.
>>>>
>>>> I tried something else, which is doing wc -l on a NFS mounted drive
>>>> (same two computers). It was UNBELIEVABLY slow and the load average on
>>>> the NFS server shot WAY up. Transferring a 336 MB file took 157
>>>> seconds, or about e megabytes per second (vs 111 mbps that I achieved
>>>> with HTTP).
>>>
>>> So, about 2.7 MB/s? :-)
>>>
>>>
>>>> So, the conclusion is, HTTP is fast (no wonder), SSH is "medium", and
>>>> NFS is "slow, very bad".
>>>
>>> I ran into a similar problem with NFS ages back. Turns out you can
>>> largely fix it by configuring NFS to increase the packet size. Been a
>>> few years since I did anything with NFS, though, so you'll have to look
>>> through the docs. I hear they use it in computer clusters, so it can't
>>> be slow in *all* cases.
>
>>How do you configure NFS to increase the packet size? Also are you
>>talking about NFS V4 or V3?
>
> when you mount it. Why didn't you just google for "nfs packet size"?

When I configure rsize and wsize, the mount fails for some reason, it
does not like these options, even if set at 4192. Client is Fedore 7,
server is Fedora Core 6.

i

Ignoramus26973 <ignoramus26973@NOSPAM.26973.invalid> writes:
>So, now I have a dilemma, I have a fast pipe, but scp is not fast
>enough (given my CPU) to encrypt/decrypt so much data.

Using a different cipher like blowfish will speed this up a bit but http
or ftp will still be faster.

Later

Mark Hittinger
bugs@pu.net