I have a mildly busy server that handles about 20k emails per
day. Most are spams. It runs Fedora 3.

I was plagued by awful performance and 200+ load averages when
sendmail was running. I had to run many sendmail children to deal with
queues.

It went on for months.

Investigation involving top -n 1 -b > top.txt showed a lot of
sendmails in uninterruptible sleep state. This means usually waiting
on disk.

I suspected that this behavior is due to sendmail's wanting to commit
files to disk too much. Some google research brought up the SuperSafe
option which commits files to disk a lot in case of a computer crash.

Guess what, this computer does not crash. And Linux is very good about
finally committing files to disks, WHEN IT HAS TIME.

I turned off SuperSafe option by saying O SuperSafe=False in all .cf
files.

After this, load averages went way down and are mostly staying in
single digits.

i

On Thu, 20 Mar 2008, Ignoramus547 wrote:

>
> I have a mildly busy server that handles about 20k emails per
> day. Most are spams. It runs Fedora 3.
>
> I was plagued by awful performance and 200+ load averages when
> sendmail was running. I had to run many sendmail children to deal with
> queues.
>


I think oyu have bigger problems then that, we have sendmail boxes doing
several million msgs each a day, using MailScanner with file type,
phishing, virus and spam scanning, the load averages 1-3 and has never
exceeded 5 (and that's all attributed to spamassassin, turn it off and
the load doesnt even hit single figures)

Mind you we use nothing but Slackware for servers, we did try fedora 1 and
even 2 just as a test, we installed a second test news server, it would
bail every 1 to 4 hours, it did not like the SATA, even Slackware ran the
SATA perfectly. Fedora/RH is all too bloated and they mess with things so
much is actually IS a joke, Slackware is as close to true source as they
get and with vendor security updates lasting longer than RH/CentOS so
called commercial support period, we have not looked back, and I used to
be an avid RH fan, but that ended at the end of RH9, the last decent
rockhard stable OS they ever released IMHO.

> After this, load averages went way down and are mostly staying in
> single digits.

You've not told us what version your runnnig what your hardware is, memory
is etc etc etc, also youve not told us if your using a fedora version or
a source version, or whatever else that machine runs, I have in a very
emergency instance used a simple single cpu p4 desktop with 512 ram (hard
to get spare sparts at 230am lol) and although it did the job perfectly
for the 9 hours it had to, the load still never exceeded 15 at worse case.


--
Cheers
Res

mysql> update auth set Framed-IP-Address='127.0.0.127' where user= 'troll';

On 2008-03-20, Res <res@ausics.net> wrote:
> On Thu, 20 Mar 2008, Ignoramus547 wrote:
>
>>
>> I have a mildly busy server that handles about 20k emails per
>> day. Most are spams. It runs Fedora 3.
>>
>> I was plagued by awful performance and 200+ load averages when
>> sendmail was running. I had to run many sendmail children to deal with
>> queues.
>>
>
>
> I think oyu have bigger problems then that, we have sendmail boxes doing
> several million msgs each a day, using MailScanner with file type,
> phishing, virus and spam scanning, the load averages 1-3 and has never
> exceeded 5 (and that's all attributed to spamassassin, turn it off and
> the load doesnt even hit single figures)

In my case, the load went down to low single digits after I made the
change I described (disabled SuperSafe). I run spamassassin and a
webserver (up to 15 HTTP queries per second).

> Mind you we use nothing but Slackware for servers, we did try fedora 1 and
> even 2 just as a test, we installed a second test news server, it would
> bail every 1 to 4 hours, it did not like the SATA, even Slackware ran the
> SATA perfectly. Fedora/RH is all too bloated and they mess with things so
> much is actually IS a joke, Slackware is as close to true source as they
> get and with vendor security updates lasting longer than RH/CentOS so
> called commercial support period, we have not looked back, and I used to
> be an avid RH fan, but that ended at the end of RH9, the last decent
> rockhard stable OS they ever released IMHO.

The OS itself is stable as a rock in my case, year long uptimes etc.

>> After this, load averages went way down and are mostly staying in
>> single digits.
>
> You've not told us what version your runnnig what your hardware is, memory

Fedora 3 for OS, dual CPU, 4 gigs of RAM. A single partition spanning
two IDE hard drives (I know that it is dumb).

> is etc etc etc, also youve not told us if your using a fedora version or
> a source version,

Fedora version.

> or whatever else that machine runs, I have in a very emergency
> instance used a simple single cpu p4 desktop with 512 ram (hard to
> get spare sparts at 230am lol) and although it did the job perfectly
> for the 9 hours it had to, the load still never exceeded 15 at worse
> case.

How many jmessages per day, did you run spamassassin and supersafe?>

What may make my situation different, is that a lot of my accounts are
autoresponders and so I send a lot of emails to defunct addresses. So
my outgoing queue always has quite a bit of stuff.

Anyhow, I am certain that my problems are behind me.

i

On Thu, 20 Mar 2008, Ignoramus547 wrote:

> Fedora 3 for OS, dual CPU, 4 gigs of RAM. A single partition spanning
> two IDE hard drives (I know that it is dumb).

gawd, thats running an 18 wheeler truck with family sedan 3 litre engine.
I also hope that its firewalled off tightly being fedora and an old
unsupported version.

> Fedora version.

yes well who knows what they do it.

> How many jmessages per day, did you run spamassassin and supersafe?>

as in the original posts each smtp box does a couple million each, and yes
supersafe is on.

These front MTA boxes are HP DL360 g4 and g5's 4gig ram, a 1 gig ramdisk
for speed with S.A, we run teh dostech rules as well but do not run dcc
it we found has serious speed implications, razor is fine though, and we
use f-protd anti virus, we run 2 disks (OS) in raid1 and the queues on
their own partition. the only difference is these are sent on to qmail
based backend mail routers with vpopmail with the users dirs on a
netapp filer rather than putting into another directory, but would be
about the same workload I/O wise. Being fedroa it would also use the
slllllllloooooooooooowwwwwwww ext3 FS, thats fine for say the OS, but on
queues we use reiser, incredibly fast.


> What may make my situation different, is that a lot of my accounts are
> autoresponders and so I send a lot of emails to defunct addresses. So
> my outgoing queue always has quite a bit of stuff.

If you have no checks on this, you can be generating backscatter, so you
need a better policy to prevent these problems, not to mention returns to
real forged addresses risking you into RBL's.

> Anyhow, I am certain that my problems are behind me.

Tbhats the main thing I suppose

--
Cheers
Res

mysql> update auth set Framed-IP-Address='127.0.0.127' where user= 'troll';

On 2008-03-21, Res <res@ausics.net> wrote:
> On Thu, 20 Mar 2008, Ignoramus547 wrote:
>
>> Fedora 3 for OS, dual CPU, 4 gigs of RAM. A single partition spanning
>> two IDE hard drives (I know that it is dumb).
>
> gawd, thats running an 18 wheeler truck with family sedan 3 litre engine.

What is the 3 litre engine in your analogy?

> I also hope that its firewalled off tightly being fedora and an old
> unsupported version.

It is indeed firewalled.

>> Fedora version.
>
> yes well who knows what they do it.
>
>> How many jmessages per day, did you run spamassassin and supersafe?>
>
> as in the original posts each smtp box does a couple million each, and yes
> supersafe is on.
>
> These front MTA boxes are HP DL360 g4 and g5's 4gig ram, a 1 gig ramdisk
> for speed with S.A, we run teh dostech rules as well but do not run
> dcc

So, your SuperSafe commits to ramdisk? Is that right?

> it we found has serious speed implications, razor is fine though, and we
> use f-protd anti virus, we run 2 disks (OS) in raid1 and the queues on
> their own partition. the only difference is these are sent on to qmail
> based backend mail routers with vpopmail with the users dirs on a
> netapp filer rather than putting into another directory, but would be
> about the same workload I/O wise. Being fedroa it would also use the
> slllllllloooooooooooowwwwwwww ext3 FS, thats fine for say the OS, but on
> queues we use reiser, incredibly fast.

I use ext3 everywhere.

>
>> What may make my situation different, is that a lot of my accounts are
>> autoresponders and so I send a lot of emails to defunct addresses. So
>> my outgoing queue always has quite a bit of stuff.
>
> If you have no checks on this, you can be generating backscatter, so you
> need a better policy to prevent these problems, not to mention returns to
> real forged addresses risking you into RBL's.

Well, the emails go through spamassassin prior to being fed to
autoresponders.

>> Anyhow, I am certain that my problems are behind me.
>
> Tbhats the main thing I suppose
>

Yes. I dont care if I lose a few emails if my server ever crashes,
which has not happened to date. 90% chance is that they will be spams
anyway.

i

On Fri, 21 Mar 2008, Ignoramus547 wrote:

>> gawd, thats running an 18 wheeler truck with family sedan 3 litre engine.
>
> What is the 3 litre engine in your analogy?

Maybe a better analogy would be you have a car motor in a big truck
where as if you have scsi you could do far more, and that disk I/O would
not be an issue.

> So, your SuperSafe commits to ramdisk? Is that right?

no, Sendmail doesnt go near it.

sendmail takes mail inbound, it is queued on disk
<copy/paste>
/usr/sbin/sendmail -bd -ODeliveryMode=queueonly -OQueueDirectory=/var/spool/mqueue.in -OPidFile=/var/run/sendmail.in || RETVALUE=" Failed, sm.in"

MailScanner takes a copy, does all its tests in the ram drive, if it
passes, MailScanner moves the msg from the in.queue to another queue where
sendmail is kicked into delivering it.
<copy/paste>
/usr/sbin/sendmail -q10m -O ForkEachJob=true -OPidFile=/var/run/sendmail.out || RETVALUE=" Failed, sm.out"

it doesnt take 10 mins to deliver, as MailScanner has hooks to make it
appear to sendmail that it is just now getting this new message..


>> If you have no checks on this, you can be generating backscatter, so you
>> need a better policy to prevent these problems, not to mention returns to
>> real forged addresses risking you into RBL's.
>
> Well, the emails go through spamassassin prior to being fed to
> autoresponders.

That makes no difference, if one of your auto responder messages sends its
normal message it will be passed anyway as one would assume the msg would
not be spam that they are sending out

--
Cheers
Res

mysql> update auth set Framed-IP-Address='127.0.0.127' where user= 'troll';

In comp.mail.sendmail Res <res@ausics.net>:
> On Thu, 20 Mar 2008, Ignoramus547 wrote:

>> Fedora 3 for OS, dual CPU, 4 gigs of RAM. A single partition spanning
>> two IDE hard drives (I know that it is dumb).

> gawd, thats running an 18 wheeler truck with family sedan 3 litre engine.
> I also hope that its firewalled off tightly being fedora and an old
> unsupported version.

>> Fedora version.

> yes well who knows what they do it.

>> How many jmessages per day, did you run spamassassin and supersafe?>

> as in the original posts each smtp box does a couple million each, and yes
> supersafe is on.

> These front MTA boxes are HP DL360 g4 and g5's 4gig ram, a 1 gig ramdisk
> for speed with S.A, we run teh dostech rules as well but do not run dcc
> it we found has serious speed implications, razor is fine though, and we
> use f-protd anti virus, we run 2 disks (OS) in raid1 and the queues on
> their own partition. the only difference is these are sent on to qmail
> based backend mail routers with vpopmail with the users dirs on a
> netapp filer rather than putting into another directory, but would be
> about the same workload I/O wise. Being fedroa it would also use the
> slllllllloooooooooooowwwwwwww ext3 FS, thats fine for say the OS, but on
> queues we use reiser, incredibly fast.

Zero problems with HP DL360g4/5 and RHEL4 and and 2 GB RAM, and
15k scsi hds in raid1 on built in controller, running SA, with
dcc, razor and almost anything enabled. MTA is exim, very little
load with slightly lesser traffic then yours. Queues also on own
partitions. All ext3, mounted with noatime/etc to speed up
things. Had only problems with reiser and tossed it ages ago.
Zero problems with ext3, though I/O might be the real problem
here. top/iostat and alike should reveal how the load is bound?

AV is running on subsequent MTA for better WAN protection with
mailscanner 2 virus engines and sendmail.

[..]

--
Michael Heiming (X-PGP-Sig > GPG-Key ID: EDD27B94)
mail: echo zvpunry@urvzvat.qr | perl -pe 'y/a-z/n-za-m/'
#bofh excuse 63: not properly grounded, please bury computer

On Fri, 21 Mar 2008, Michael Heiming wrote:

> Zero problems with HP DL360g4/5 and RHEL4 and and 2 GB RAM, and
> 15k scsi hds in raid1 on built in controller, running SA, with
> dcc, razor and almost anything enabled. MTA is exim, very little
> load with slightly lesser traffic then yours. Queues also on own
> partitions. All ext3, mounted with noatime/etc to speed up
> things. Had only problems with reiser and tossed it ages ago.
> Zero problems with ext3, though I/O might be the real problem
> here. top/iostat and alike should reveal how the load is bound?

It clearly is I/O based, and really, even 148G 15K SCSI drives are cheap
enough, heck even the 300G scsi are fairly well priced now days, but
pure overkill on a front relay only type server. The noatime is good
choice, made no difference here when we tried it, but makes the world of
difference on the news server spools

I used to use DCC, but found it performed poorly, this was 2 years ago,
maybe its fixed now days, but you know the saying 'if it aint broke dont
break it'.

> AV is running on subsequent MTA for better WAN protection with
> mailscanner 2 virus engines and sendmail.

What AV? Using f-protd there is no load/performance difference and the
speed is phenomenal, where as the free clam alternative is a complete
dismal performer performance wise, and Sophos is a joke at detecting
anything anyway


--
Cheers
Res

mysql> update auth set Framed-IP-Address='127.0.0.127' where user= 'troll';

bugger dont know how i missed replying to this..

On Fri, 21 Mar 2008, Michael Heiming wrote:

> things. Had only problems with reiser and tossed it ages ago.

Care to share?

I've never found anything that reiserfsck couldnt fix, and only had to
run it a couple of times, and only ever with the same darn box after
hard powerfails (old bios at the time, and it would never shutdown in
time when UPS instructed it to).

Although I am aware of the zero byte file issues some people have
mentioned (the same problem you get with xfs and jfs as well) but I am
unaware if those people ever ran reiserfsck or just said "oh well journal
replay didnt work so its screwed".


--
Cheers
Res

mysql> update auth set Framed-IP-Address='127.0.0.127' where user= 'troll';

In comp.mail.sendmail Res <res@ausics.net>:
> On Fri, 21 Mar 2008, Michael Heiming wrote:
[..]
>> AV is running on subsequent MTA for better WAN protection with
>> mailscanner 2 virus engines and sendmail.

> What AV? Using f-protd there is no load/performance difference and the
> speed is phenomenal, where as the free clam alternative is a complete
> dismal performer performance wise, and Sophos is a joke at detecting
> anything anyway

Clam + McAfee. Clam gave some trouble with performance after some
updates, though after switching to clamd it runs like a charm.

--
Michael Heiming (X-PGP-Sig > GPG-Key ID: EDD27B94)
mail: echo zvpunry@urvzvat.qr | perl -pe 'y/a-z/n-za-m/'
#bofh excuse 327: The POP server is out of Coke

In comp.mail.sendmail Res <res@ausics.net>:
> bugger dont know how i missed replying to this..

> On Fri, 21 Mar 2008, Michael Heiming wrote:

>> things. Had only problems with reiser and tossed it ages ago.

> Care to share?

> I've never found anything that reiserfsck couldnt fix, and only had to
> run it a couple of times, and only ever with the same darn box after
> hard powerfails (old bios at the time, and it would never shutdown in
> time when UPS instructed it to).

> Although I am aware of the zero byte file issues some people have
> mentioned (the same problem you get with xfs and jfs as well) but I am
> unaware if those people ever ran reiserfsck or just said "oh well journal
> replay didnt work so its screwed".

Some time ago with suse, reiserfs broke so badly, there was
no possibility then to recover from backup, nothing could fix it.

After a few of those experiences, switching to ext3 and xfs
where resizing was needed solved all problems. Gladly since RHEL4
update 2 or so one can enlarge ext3 while running. ;-)

--
Michael Heiming (X-PGP-Sig > GPG-Key ID: EDD27B94)
mail: echo zvpunry@urvzvat.qr | perl -pe 'y/a-z/n-za-m/'
#bofh excuse 344: Network failure - call NBC