PHWinfo banniere

Titres
PORTAIL ANNUAIRE ARTICLES COMPARATEUR HÉBERGEURS DEVIS FORUMS RÉDUCTEUR D'URL
Précédent   PHWinfo > Forums Hébergement > Forum Logiciels d'hébergement > comp.mail.sendmail > How the fuck can I force sendmail to efficiently process its queue
S'inscrire FAQ Membres Recherche Messages du jour Marquer les forums comme lus
comp.mail.sendmail Configuring and using the BSD sendmail agent.

How the fuck can I force sendmail to efficiently process its queue

Réponse
 
LinkBack Outils de la discussion
Vieux 21/11/2007, 18h35   #1
Ignoramus4770
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut How the fuck can I force sendmail to efficiently process its queue

I have a machine with quad CPU and 4 gigs of RAM.

And yet for the life of me, I cannot figure out why I have so much
crap for local users that cannot be processed.

My sendmail is 8.13.

All I want is to run, maybe 15 queue runners at the same time, so that
undeliverable and slow junk does not slow down delivery of more
valuable things.

How can I do it?

divert(-1)

include(`/usr/share/sendmail-cf/m4/cf.m4')dnl

divert(0)dnl
OSTYPE(linux)dnl
DOMAIN(generic)dnl

DAEMON_OPTIONS(`Name=MTA, Addr=65.182.171.162')
DAEMON_OPTIONS(`Name=MTA1, Addr=127.0.0.1')

define(`LOCAL_HACK', `PUSHDIVERT(-1)define(`_ARG_', `$2')define(`_ARGS_', `shift($@)')include(check_local-4.2/hack/$1.m4)POPDIVERT`'')

define(`confPRIVACY_FLAGS', `authwarnings,novrfy,noexpn')dnl
define( `confTO_QUEUEWARN_NORMAL', `2d' )
define(`confTO_QUIT',`8m')
define(`confEIGHT_BIT_HANDLING',`pass')
# our numerous local host names

# http://www.moshkow.kulichki.com/SEND...e64fiature.txt
define(`confSMTP_MAILER', `smtp8')dnl

define(confQUEUE_LA, 3)dnl
define(confREFUSE_LA, 30)dnl
define(confMAX_DAEMON_CHILDREN, 62)dnl
define(confCONNECTION_RATE_THROTTLE, 13)dnl

dnl define(conf_MAX_RUNNERS_PER_QUEUE, 35)dnl
O MaxQueueChildren
O MaxRunnersPerQueue=30


dnl define(confDAEMON_OPTIONS, Address=208.233.99.160)dnl

FEATURE(use_cw_file)dnl
FEATURE(mailertable)dnl
FEATURE(domaintable)dnl
FEATURE(access_db)dnl
FEATURE(`blacklist_recipients')dnl
FEATURE(virtusertable)dnl
FEATURE(`no_default_msa')dnl

dnl ################################################## ANTISPAM

dnl FEATURE(`delay_checks')dnl
dnl FEATURE(`enhdnsbl', `relays.osirusoft.com', `DNSBL4: You are a rogue listserver', `t', `127.0.0.7.')
dnl FEATURE(dnsbl, `korea.services.net', `DNSBL5: No Mail from Korea is accepted.')dnl
dnl FEATURE(dnsbl, `relays.visi.com', `DNSBL6: visi.com: you are an open relay.')dnl
dnl FEATURE(dnsbl, `relays.ordb.org', `DNSBL7: relays.ordb.org: you are an open relay.')dnl
dnl FEATURE(dnsbl, `dynablock.wirehub.net', `DNSBL8: dynablock.wirehub.net says you are a dynamic IP.')dnl
dnl FEATURE(dnsbl, `nigeria.blackholes.us', `DNSBL9: We do not talk to Nigeria.')dnl
dnl FEATURE(dnsbl, `japan.blackholes.us', `DNSBL10: We do not talk to Japan.')dnl
dnl FEATURE(dnsbl, `argentina.blackholes.us ', `DNSBL11: We do not talk to argentina.')dnl
dnl FEATURE(dnsbl, `china.blackholes.us ', `DNSBL12: We do not talk to China.')dnl
dnl FEATURE(dnsbl, `brazil.blackholes.us ', `DNSBL13: We do not talk to Brazil.')dnl
dnl FEATURE(dnsbl, `malaysia.blackholes.us ', `DNSBL14: We do not talk to Malaysia.')dnl
dnl FEATURE(dnsbl, `taiwan.blackholes.us ', `DNSBL15: We do not talk to Taiwan.')dnl
dnl FEATURE(dnsbl, `hongkong.blackholes.us ', `DNSBL16: We do not talk to hongkong.')dnl


define(`PROCMAIL_MAILER_ARGS',`procmail -m $h $f $u -a $@x')dnl

Tichudov
#Dmak47.algebra.com
#Djak47.algebra.com
Cwstump.algebra.com

dnl FEATURE(local_procmail)dnl

MAILER(smtp)dnl
MAILER(procmail)dnl

LOCAL_CONFIG

LOCAL_RULESETS
SLocal_check_mail_misha
# check address against various regex checks
R$* $: $>Parse0 $>3 $1

#HMessage-Id: $>CheckMessageId
HFrom: $>CheckFrom
HTo: $>CheckTo
HSubject: $>Check_Subject

SCheckMessageId
R< $+ @ $+ > $@ OK
R$* $#error $: 553 Illegal Message-ID

### dnl define(`_READ_X_SPAM_FILT_',`dnl')dnl
### dnl LOCAL_HACK(`check_local')dnl
### dnl LOCAL_HACK(`check_header',`Message-Id',`',`',`',`',`',`',`1')dnl
### dnl LOCAL_HACK(`check_header_end')dnl



SCheckFrom
R $+ @ xxx . net $#error $: 553 xxx.net does not send mail
R $+ @ homebiz . com $#error $: 553 homebiz.com does not exist
R $+ @ something . net $#error $: 553 something.net does not send mail
R $+ @ bar $#error $: 553 and which bar might that be?
R petlover @ $#error $: 553 No e-mails from petlovers (try petlover+real)

SCheckTo
R Friend @ public . com $#error $: 553 no friends at Public.com


D{MPat}Important Message From
D{MMsg}This message may contain the Melissa virus.
D{UDPat}UNIVERSITY DIPLOMAS FAST
SCheck_Subject
R${MPat} $* $#error $: 553 ${MMsg}
RRe: ${MPat} $* $#error $: 553 ${MMsg}
R${UDPat} $* $#error $: 551 Keep your fake diplomas, spammer

#H?M?X-Relay-IP: ${client_addr}


# @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @@@@@@@@@@@@@@@@@@@@
# Igor Chudov's rules for algebra.com to video-collage.com
# conversion for certain addreses (cypherpunks, scrm etc).

LOCAL_RULE_0

################################################## ########## scrm
R scrm < @ algebra.com . > scrm < @ localhost . >
R scrm-board < @ algebra.com . > scrm < @ localhost . >
R scrm-mods < @ algebra.com . > scrm < @ localhost . >
R scrm-approved < @ algebra.com . > scrm < @ localhost . >
R scrm-rejected < @ algebra.com . > scrm < @ localhost . >
R scrm-approval-key < @ algebra.com . > scrm < @ localhost . >
R scrm-admin < @ algebra.com . > scrm < @ localhost . >
R devnull < @ algebra.com . > devnull < @ localhost . >
R passat-approval < @ algebra.com . > klm < @ cs.jhu.edu . >
R ichudov < @ algebra.com . > ichudov-both < @ localhost . >
R dasha < @ algebra.com . > dasha-both < @ localhost . >
#R dasha < @ algebra.com . > pavlovd < @ ics.uci.edu . >

################################################## ########## Cypher Punks
R cypherpunks < @ algebra.com . > cypherpunks < @ localhost . >
R owner-cypherpunks < @ algebra.com . > devnull < @ localhost . >

R cypherpunks-hosts < @ algebra.com . > cypherpunks-hosts < @ localhost . >
R stump-users < @ algebra.com . > stump-users < @ localhost . >
R jobs-discussion < @ algebra.com . > jobs-discussion < @ localhost . >
R majordomo < @ algebra.com . > majordomo < @ localhost . >

LOCAL_RULESETS
SLocal_check_mail
#R < > $# error $@ 5.7.0 $: "554 Temporary rejecting error messages. My domain is forged in thousands of spams."
  Réponse avec citation
Vieux 21/11/2007, 22h45   #2
Bill Cole
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: How the fuck can I force sendmail to efficiently process its queue

In article <3eednYYYNNf-8dnanZ2dnUVZ_tOtnZ2d@giganews.com>,
Ignoramus4770 <ignoramus4770@NOSPAM.4770.invalid> wrote:

> I have a machine with quad CPU and 4 gigs of RAM.
>
> And yet for the life of me, I cannot figure out why I have so much
> crap for local users that cannot be processed.
>
> My sendmail is 8.13.
>
> All I want is to run, maybe 15 queue runners at the same time, so that
> undeliverable and slow junk does not slow down delivery of more
> valuable things.
>
> How can I do it?


Read the documentation? Provide relevant information? Local delivery
slowness is usually not a sendmail config problem. If the problem you
have really is slow local delivery, you need to look at things like your
global or individual procmail rules (since you are using procmail for
local delivery) and at the performance of whatever storage you are using
for local delivery. One thing that does look wrong below is your QueueLA
setting. Unless you are trying to protect something else on the box from
Sendmail, 12 or even 20 would make a lot more sense than 3.

FWIW, I suspect that your bigger problem is that you have not paying
attention to your use of third-party DNSBL's for a long time. Some of
those have been dead for YEARS and that means you are doing a blocking
DNS lookup on every connection that will fail only by timing out. In
addition to doing your own mail server performance damage, this also
makes you one of the many people who take an active part in an ongoing
DDoS of the people who own domains that formerly ran DNSBL's. I expect
that EasyNet can handle the traffic, but I know that Joe Jared has said
that he is essentially unable to do anything with osirusoft.com because
of the continued blind flood of DNS queries from people who do not pay
attention to their own systems. Of the ones that are still theoretically
functional in your config, the bulk are part of a chronically unreliable
operation that has a history of collapsing frequently for hours to days
at a time.


> divert(-1)
>
> include(`/usr/share/sendmail-cf/m4/cf.m4')dnl
>
> divert(0)dnl
> OSTYPE(linux)dnl
> DOMAIN(generic)dnl
>
> DAEMON_OPTIONS(`Name=MTA, Addr=65.182.171.162')
> DAEMON_OPTIONS(`Name=MTA1, Addr=127.0.0.1')
>
> define(`LOCAL_HACK', `PUSHDIVERT(-1)define(`_ARG_', `$2')define(`_ARGS_',
> `shift($@)')include(check_local-4.2/hack/$1.m4)POPDIVERT`'')
>
> define(`confPRIVACY_FLAGS', `authwarnings,novrfy,noexpn')dnl
> define( `confTO_QUEUEWARN_NORMAL', `2d' )
> define(`confTO_QUIT',`8m')
> define(`confEIGHT_BIT_HANDLING',`pass')
> # our numerous local host names
>
> # http://www.moshkow.kulichki.com/SEND...e64fiature.txt
> define(`confSMTP_MAILER', `smtp8')dnl
>
> define(confQUEUE_LA, 3)dnl
> define(confREFUSE_LA, 30)dnl
> define(confMAX_DAEMON_CHILDREN, 62)dnl
> define(confCONNECTION_RATE_THROTTLE, 13)dnl
>
> dnl define(conf_MAX_RUNNERS_PER_QUEUE, 35)dnl
> O MaxQueueChildren
> O MaxRunnersPerQueue=30
>
>
> dnl define(confDAEMON_OPTIONS, Address=208.233.99.160)dnl
>
> FEATURE(use_cw_file)dnl
> FEATURE(mailertable)dnl
> FEATURE(domaintable)dnl
> FEATURE(access_db)dnl
> FEATURE(`blacklist_recipients')dnl
> FEATURE(virtusertable)dnl
> FEATURE(`no_default_msa')dnl
>
> dnl ################################################## ANTISPAM
>
> dnl FEATURE(`delay_checks')dnl
> dnl FEATURE(`enhdnsbl', `relays.osirusoft.com', `DNSBL4: You are a rogue
> listserver', `t', `127.0.0.7.')
> dnl FEATURE(dnsbl, `korea.services.net', `DNSBL5: No Mail from Korea is
> accepted.')dnl
> dnl FEATURE(dnsbl, `relays.visi.com', `DNSBL6: visi.com: you are an open
> relay.')dnl
> dnl FEATURE(dnsbl, `relays.ordb.org', `DNSBL7: relays.ordb.org: you are an
> open relay.')dnl
> dnl FEATURE(dnsbl, `dynablock.wirehub.net', `DNSBL8: dynablock.wirehub.net
> says you are a dynamic IP.')dnl
> dnl FEATURE(dnsbl, `nigeria.blackholes.us', `DNSBL9: We do not talk to
> Nigeria.')dnl
> dnl FEATURE(dnsbl, `japan.blackholes.us', `DNSBL10: We do not talk to
> Japan.')dnl
> dnl FEATURE(dnsbl, `argentina.blackholes.us ', `DNSBL11: We do not talk to
> argentina.')dnl
> dnl FEATURE(dnsbl, `china.blackholes.us ', `DNSBL12: We do not talk to
> China.')dnl
> dnl FEATURE(dnsbl, `brazil.blackholes.us ', `DNSBL13: We do not talk to
> Brazil.')dnl
> dnl FEATURE(dnsbl, `malaysia.blackholes.us ', `DNSBL14: We do not talk to
> Malaysia.')dnl
> dnl FEATURE(dnsbl, `taiwan.blackholes.us ', `DNSBL15: We do not talk to
> Taiwan.')dnl
> dnl FEATURE(dnsbl, `hongkong.blackholes.us ', `DNSBL16: We do not talk to
> hongkong.')dnl
>
>
> define(`PROCMAIL_MAILER_ARGS',`procmail -m $h $f $u -a $@x')dnl
>
> Tichudov
> #Dmak47.algebra.com
> #Djak47.algebra.com
> Cwstump.algebra.com
>
> dnl FEATURE(local_procmail)dnl
>
> MAILER(smtp)dnl
> MAILER(procmail)dnl
>
> LOCAL_CONFIG
>
> LOCAL_RULESETS
> SLocal_check_mail_misha
> # check address against various regex checks
> R$* $: $>Parse0 $>3 $1
>
> #HMessage-Id: $>CheckMessageId
> HFrom: $>CheckFrom
> HTo: $>CheckTo
> HSubject: $>Check_Subject
>
> SCheckMessageId
> R< $+ @ $+ > $@ OK
> R$* $#error $: 553 Illegal Message-ID
>
> ### dnl define(`_READ_X_SPAM_FILT_',`dnl')dnl
> ### dnl LOCAL_HACK(`check_local')dnl
> ### dnl LOCAL_HACK(`check_header',`Message-Id',`',`',`',`',`',`',`1')dnl
> ### dnl LOCAL_HACK(`check_header_end')dnl
>
>
>
> SCheckFrom
> R $+ @ xxx . net $#error $: 553 xxx.net does not send mail
> R $+ @ homebiz . com $#error $: 553 homebiz.com does not exist
> R $+ @ something . net $#error $: 553 something.net does not send mail
> R $+ @ bar $#error $: 553 and which bar might that be?
> R petlover @ $#error $: 553 No e-mails from petlovers (try petlover+real)
>
> SCheckTo
> R Friend @ public . com $#error $: 553 no friends at Public.com
>
>
> D{MPat}Important Message From
> D{MMsg}This message may contain the Melissa virus.
> D{UDPat}UNIVERSITY DIPLOMAS FAST
> SCheck_Subject
> R${MPat} $* $#error $: 553 ${MMsg}
> RRe: ${MPat} $* $#error $: 553 ${MMsg}
> R${UDPat} $* $#error $: 551 Keep your fake diplomas, spammer
>
> #H?M?X-Relay-IP: ${client_addr}
>
>
> # @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @@@@@@@@@@@@@@@@@@@@
> # Igor Chudov's rules for algebra.com to video-collage.com
> # conversion for certain addreses (cypherpunks, scrm etc).
>
> LOCAL_RULE_0
>
> ################################################## ########## scrm
> R scrm < @ algebra.com . > scrm < @ localhost . >
> R scrm-board < @ algebra.com . > scrm < @ localhost . >
> R scrm-mods < @ algebra.com . > scrm < @ localhost . >
> R scrm-approved < @ algebra.com . > scrm < @ localhost . >
> R scrm-rejected < @ algebra.com . > scrm < @ localhost . >
> R scrm-approval-key < @ algebra.com . > scrm < @ localhost . >
> R scrm-admin < @ algebra.com . > scrm < @ localhost . >
> R devnull < @ algebra.com . > devnull < @ localhost . >
> R passat-approval < @ algebra.com . > klm < @ cs.jhu.edu . >
> R ichudov < @ algebra.com . > ichudov-both < @ localhost . >
> R dasha < @ algebra.com . > dasha-both < @ localhost . >
> #R dasha < @ algebra.com . > pavlovd < @ ics.uci.edu . >
>
> ################################################## ########## Cypher Punks
> R cypherpunks < @ algebra.com . > cypherpunks < @ localhost . >
> R owner-cypherpunks < @ algebra.com . > devnull < @ localhost . >
>
> R cypherpunks-hosts < @ algebra.com . > cypherpunks-hosts < @ localhost . >
> R stump-users < @ algebra.com . > stump-users < @ localhost . >
> R jobs-discussion < @ algebra.com . > jobs-discussion < @ localhost . >
> R majordomo < @ algebra.com . > majordomo < @ localhost . >
>
> LOCAL_RULESETS
> SLocal_check_mail
> #R < > $# error $@ 5.7.0 $: "554 Temporary rejecting error messages. My
> domain is forged in thousands of spams."


--
Now where did I hide that website...
  Réponse avec citation
Vieux 22/11/2007, 04h45   #3
Scott Grayban
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: How the fuck can I force sendmail to efficiently process itsqueue

Ignoramus4770 wrote:
> I have a machine with quad CPU and 4 gigs of RAM.
>
> And yet for the life of me, I cannot figure out why I have so much
> crap for local users that cannot be processed.
>
> My sendmail is 8.13.
>
> All I want is to run, maybe 15 queue runners at the same time, so that
> undeliverable and slow junk does not slow down delivery of more
> valuable things.
>
> How can I do it?
>


Anyone that uses fowl language in the subject is much to immature to
understand anything we say.
  Réponse avec citation
Vieux 22/11/2007, 18h59   #4
Ignoramus689
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: How the fuck can I force sendmail to efficiently process its queue

On 2007-11-22, Scott Grayban <sgrayban@NOSPAM-gmail.com> wrote:
> Ignoramus4770 wrote:
>> I have a machine with quad CPU and 4 gigs of RAM.
>>
>> And yet for the life of me, I cannot figure out why I have so much
>> crap for local users that cannot be processed.
>>
>> My sendmail is 8.13.
>>
>> All I want is to run, maybe 15 queue runners at the same time, so that
>> undeliverable and slow junk does not slow down delivery of more
>> valuable things.
>>
>> How can I do it?
>>

>
> Anyone that uses fowl language in the subject is much to immature to
> understand anything we say.


That's "foul". Fowl means certain kinds of birds.

i
  Réponse avec citation
Vieux 22/11/2007, 19h03   #5
Ignoramus689
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: How the fuck can I force sendmail to efficiently process its queue

On 2007-11-21, Bill Cole <bill@scconsult.com> wrote:
> In article <3eednYYYNNf-8dnanZ2dnUVZ_tOtnZ2d@giganews.com>,
> Ignoramus4770 <ignoramus4770@NOSPAM.4770.invalid> wrote:
>
>> I have a machine with quad CPU and 4 gigs of RAM.
>>
>> And yet for the life of me, I cannot figure out why I have so much
>> crap for local users that cannot be processed.
>>
>> My sendmail is 8.13.
>>
>> All I want is to run, maybe 15 queue runners at the same time, so that
>> undeliverable and slow junk does not slow down delivery of more
>> valuable things.
>>
>> How can I do it?

>
> Read the documentation? Provide relevant information? Local delivery
> slowness is usually not a sendmail config problem. If the problem you
> have really is slow local delivery, you need to look at things like your
> global or individual procmail rules (since you are using procmail for
> local delivery) and at the performance of whatever storage you are using
> for local delivery. One thing that does look wrong below is your QueueLA
> setting. Unless you are trying to protect something else on the box from
> Sendmail, 12 or even 20 would make a lot more sense than 3.


I increased my QueueLA setting. Thanks. I also set up spamd
(spamassassin's daemon program) to maximum of 10 children instead of
5. Spamassassin makes a number od DNS queries and generally waits a
lot, so it makes sense to allow for more of its children.

> FWIW, I suspect that your bigger problem is that you have not paying
> attention to your use of third-party DNSBL's for a long time.


OK, maybe I am missing something, but I thought that I had them all
dnl'ed , kind of like commented out? (see quoted below)

> Some of those have been dead for YEARS and that means you are doing
> a blocking DNS lookup on every connection that will fail only by
> timing out. In addition to doing your own mail server performance
> damage, this also makes you one of the many people who take an
> active part in an ongoing DDoS of the people who own domains that
> formerly ran DNSBL's. I expect that EasyNet can handle the traffic,
> but I know that Joe Jared has said that he is essentially unable to
> do anything with osirusoft.com because of the continued blind flood
> of DNS queries from people who do not pay attention to their own
> systems. Of the ones that are still theoretically functional in your
> config, the bulk are part of a chronically unreliable operation that
> has a history of collapsing frequently for hours to days at a time.


I agree, but I thought that I took al lot them out. I will play some
more. Thanks.

i

>
>> divert(-1)
>>
>> include(`/usr/share/sendmail-cf/m4/cf.m4')dnl
>>
>> divert(0)dnl
>> OSTYPE(linux)dnl
>> DOMAIN(generic)dnl
>>
>> DAEMON_OPTIONS(`Name=MTA, Addr=65.182.171.162')
>> DAEMON_OPTIONS(`Name=MTA1, Addr=127.0.0.1')
>>
>> define(`LOCAL_HACK', `PUSHDIVERT(-1)define(`_ARG_', `$2')define(`_ARGS_',
>> `shift($@)')include(check_local-4.2/hack/$1.m4)POPDIVERT`'')
>>
>> define(`confPRIVACY_FLAGS', `authwarnings,novrfy,noexpn')dnl
>> define( `confTO_QUEUEWARN_NORMAL', `2d' )
>> define(`confTO_QUIT',`8m')
>> define(`confEIGHT_BIT_HANDLING',`pass')
>> # our numerous local host names
>>
>> # http://www.moshkow.kulichki.com/SEND...e64fiature.txt
>> define(`confSMTP_MAILER', `smtp8')dnl
>>
>> define(confQUEUE_LA, 3)dnl
>> define(confREFUSE_LA, 30)dnl
>> define(confMAX_DAEMON_CHILDREN, 62)dnl
>> define(confCONNECTION_RATE_THROTTLE, 13)dnl
>>
>> dnl define(conf_MAX_RUNNERS_PER_QUEUE, 35)dnl
>> O MaxQueueChildren
>> O MaxRunnersPerQueue=30
>>
>>
>> dnl define(confDAEMON_OPTIONS, Address=208.233.99.160)dnl
>>
>> FEATURE(use_cw_file)dnl
>> FEATURE(mailertable)dnl
>> FEATURE(domaintable)dnl
>> FEATURE(access_db)dnl
>> FEATURE(`blacklist_recipients')dnl
>> FEATURE(virtusertable)dnl
>> FEATURE(`no_default_msa')dnl
>>
>> dnl ################################################## ANTISPAM
>>
>> dnl FEATURE(`delay_checks')dnl
>> dnl FEATURE(`enhdnsbl', `relays.osirusoft.com', `DNSBL4: You are a rogue
>> listserver', `t', `127.0.0.7.')
>> dnl FEATURE(dnsbl, `korea.services.net', `DNSBL5: No Mail from Korea is
>> accepted.')dnl
>> dnl FEATURE(dnsbl, `relays.visi.com', `DNSBL6: visi.com: you are an open
>> relay.')dnl
>> dnl FEATURE(dnsbl, `relays.ordb.org', `DNSBL7: relays.ordb.org: you are an
>> open relay.')dnl
>> dnl FEATURE(dnsbl, `dynablock.wirehub.net', `DNSBL8: dynablock.wirehub.net
>> says you are a dynamic IP.')dnl
>> dnl FEATURE(dnsbl, `nigeria.blackholes.us', `DNSBL9: We do not talk to
>> Nigeria.')dnl
>> dnl FEATURE(dnsbl, `japan.blackholes.us', `DNSBL10: We do not talk to
>> Japan.')dnl
>> dnl FEATURE(dnsbl, `argentina.blackholes.us ', `DNSBL11: We do not talk to
>> argentina.')dnl
>> dnl FEATURE(dnsbl, `china.blackholes.us ', `DNSBL12: We do not talk to
>> China.')dnl
>> dnl FEATURE(dnsbl, `brazil.blackholes.us ', `DNSBL13: We do not talk to
>> Brazil.')dnl
>> dnl FEATURE(dnsbl, `malaysia.blackholes.us ', `DNSBL14: We do not talk to
>> Malaysia.')dnl
>> dnl FEATURE(dnsbl, `taiwan.blackholes.us ', `DNSBL15: We do not talk to
>> Taiwan.')dnl
>> dnl FEATURE(dnsbl, `hongkong.blackholes.us ', `DNSBL16: We do not talk to
>> hongkong.')dnl
>>
>>
>> define(`PROCMAIL_MAILER_ARGS',`procmail -m $h $f $u -a $@x')dnl
>>
>> Tichudov
>> #Dmak47.algebra.com
>> #Djak47.algebra.com
>> Cwstump.algebra.com
>>
>> dnl FEATURE(local_procmail)dnl
>>
>> MAILER(smtp)dnl
>> MAILER(procmail)dnl
>>
>> LOCAL_CONFIG
>>
>> LOCAL_RULESETS
>> SLocal_check_mail_misha
>> # check address against various regex checks
>> R$* $: $>Parse0 $>3 $1
>>
>> #HMessage-Id: $>CheckMessageId
>> HFrom: $>CheckFrom
>> HTo: $>CheckTo
>> HSubject: $>Check_Subject
>>
>> SCheckMessageId
>> R< $+ @ $+ > $@ OK
>> R$* $#error $: 553 Illegal Message-ID
>>
>> ### dnl define(`_READ_X_SPAM_FILT_',`dnl')dnl
>> ### dnl LOCAL_HACK(`check_local')dnl
>> ### dnl LOCAL_HACK(`check_header',`Message-Id',`',`',`',`',`',`',`1')dnl
>> ### dnl LOCAL_HACK(`check_header_end')dnl
>>
>>
>>
>> SCheckFrom
>> R $+ @ xxx . net $#error $: 553 xxx.net does not send mail
>> R $+ @ homebiz . com $#error $: 553 homebiz.com does not exist
>> R $+ @ something . net $#error $: 553 something.net does not send mail
>> R $+ @ bar $#error $: 553 and which bar might that be?
>> R petlover @ $#error $: 553 No e-mails from petlovers (try petlover+real)
>>
>> SCheckTo
>> R Friend @ public . com $#error $: 553 no friends at Public.com
>>
>>
>> D{MPat}Important Message From
>> D{MMsg}This message may contain the Melissa virus.
>> D{UDPat}UNIVERSITY DIPLOMAS FAST
>> SCheck_Subject
>> R${MPat} $* $#error $: 553 ${MMsg}
>> RRe: ${MPat} $* $#error $: 553 ${MMsg}
>> R${UDPat} $* $#error $: 551 Keep your fake diplomas, spammer
>>
>> #H?M?X-Relay-IP: ${client_addr}
>>
>>
>> # @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @@@@@@@@@@@@@@@@@@@@
>> # Igor Chudov's rules for algebra.com to video-collage.com
>> # conversion for certain addreses (cypherpunks, scrm etc).
>>
>> LOCAL_RULE_0
>>
>> ################################################## ########## scrm
>> R scrm < @ algebra.com . > scrm < @ localhost . >
>> R scrm-board < @ algebra.com . > scrm < @ localhost . >
>> R scrm-mods < @ algebra.com . > scrm < @ localhost . >
>> R scrm-approved < @ algebra.com . > scrm < @ localhost . >
>> R scrm-rejected < @ algebra.com . > scrm < @ localhost . >
>> R scrm-approval-key < @ algebra.com . > scrm < @ localhost . >
>> R scrm-admin < @ algebra.com . > scrm < @ localhost . >
>> R devnull < @ algebra.com . > devnull < @ localhost . >
>> R passat-approval < @ algebra.com . > klm < @ cs.jhu.edu . >
>> R ichudov < @ algebra.com . > ichudov-both < @ localhost . >
>> R dasha < @ algebra.com . > dasha-both < @ localhost . >
>> #R dasha < @ algebra.com . > pavlovd < @ ics.uci.edu . >
>>
>> ################################################## ########## Cypher Punks
>> R cypherpunks < @ algebra.com . > cypherpunks < @ localhost . >
>> R owner-cypherpunks < @ algebra.com . > devnull < @ localhost . >
>>
>> R cypherpunks-hosts < @ algebra.com . > cypherpunks-hosts < @ localhost . >
>> R stump-users < @ algebra.com . > stump-users < @ localhost . >
>> R jobs-discussion < @ algebra.com . > jobs-discussion < @ localhost . >
>> R majordomo < @ algebra.com . > majordomo < @ localhost . >
>>
>> LOCAL_RULESETS
>> SLocal_check_mail
>> #R < > $# error $@ 5.7.0 $: "554 Temporary rejecting error messages. My
>> domain is forged in thousands of spams."

>

  Réponse avec citation
Vieux 22/11/2007, 19h38   #6
Ignoramus689
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: How the fsck can I force sendmail to efficiently process its queue

OK, I have changed my config. Set Queue_LA to higher value, and
removed mentions of all DNSBLs. I also went through my /etc/procmailrc
to see if there is a global recipe that has one lockfile, but no,
calls to spamd are not lockfiled.

Spamd is now being started with -m10 argument, which allows up to 10
children to run.

However, it does not much. Here's the ps output

oot 29112 0.0 0.0 4080 1976 ? Ss 12:30 0:00 sendmail: accepting connections
root 29113 0.0 0.0 5460 2664 ? S 12:30 0:00 sendmail: ./lAMIOEhZ027950 mxpool01.netaddress.usa.net.: client DATA 354
root 30019 0.0 0.0 4400 2372 ? S 12:34 0:00 sendmail: server dsl88-226-55470.ttnet.net.tr [88.226.216.174] (may be forged) cmd read
root 30465 0.0 0.0 5368 2736 ? S 12:36 0:00 sendmail: ./lAMIaeHW030365 from queue
root 30513 0.1 0.0 5472 2684 ? Ss 12:37 0:00 sendmail: ./lAMG8xpu030414 gateway.mailrover.net.: user open
root 30517 0.0 0.0 5472 2644 ? Ss 12:37 0:00 sendmail: ./lAMF8LVv017617 apperception.com.: user open
root 30519 0.0 0.0 4780 2464 ? Ss 12:37 0:00 sendmail: ./lAMHOLIS015189 mail-kr5.bigfoot.com.: user open
root 30531 0.0 0.0 4800 2428 ? Ss 12:37 0:00 sendmail: ./lAMEInNB008825 mail-kr5.bigfoot.com.: user open
ichudov 30575 0.0 0.0 1660 472 pts/12 S+ 12:37 0:00 grep sendmail

root 23473 0.1 1.0 45020 42716 ? Ss 12:00 0:03 /usr/bin/spamd -d -c -m10 -H -r /var/run/spamassassin/spamd.pid
root 23486 4.2 1.1 49908 47728 ? S 12:00 1:35 spamd child
root 23487 2.5 1.1 49276 47064 ? S 12:00 0:56 spamd child



Here's my new sendmail.mc.

divert(-1)

include(`/usr/share/sendmail-cf/m4/cf.m4')dnl

divert(0)dnl
OSTYPE(linux)dnl
DOMAIN(generic)dnl

DAEMON_OPTIONS(`Name=MTA, Addr=65.182.171.162')
DAEMON_OPTIONS(`Name=MTA1, Addr=127.0.0.1')

define(`LOCAL_HACK', `PUSHDIVERT(-1)define(`_ARG_', `$2')define(`_ARGS_', `shift($@)')include(check_local-4.2/hack/$1.m4)POPDIVERT`'')

define(`confPRIVACY_FLAGS', `authwarnings,novrfy,noexpn')dnl
define( `confTO_QUEUEWARN_NORMAL', `2d' )
define(`confTO_QUIT',`8m')
define(`confEIGHT_BIT_HANDLING',`pass')
# our numerous local host names

# http://www.moshkow.kulichki.com/SEND...e64fiature.txt
define(`confSMTP_MAILER', `smtp8')dnl

define(confQUEUE_LA, 6)dnl
define(confREFUSE_LA, 20)dnl
define(confMAX_DAEMON_CHILDREN, 62)dnl
define(confCONNECTION_RATE_THROTTLE, 13)dnl

dnl define(conf_MAX_RUNNERS_PER_QUEUE, 35)dnl
O MaxQueueChildren
O MaxRunnersPerQueue=10


FEATURE(use_cw_file)dnl
FEATURE(mailertable)dnl
FEATURE(domaintable)dnl
FEATURE(access_db)dnl
FEATURE(`blacklist_recipients')dnl
FEATURE(virtusertable)dnl
FEATURE(`no_default_msa')dnl


define(`PROCMAIL_MAILER_ARGS',`procmail -m $h $f $u -a $@x')dnl

Tichudov
#Dmak47.algebra.com
#Djak47.algebra.com
Cwstump.algebra.com

MAILER(smtp)dnl
MAILER(procmail)dnl

LOCAL_CONFIG

LOCAL_RULESETS
SLocal_check_mail_misha
# check address against various regex checks
R$* $: $>Parse0 $>3 $1

#HMessage-Id: $>CheckMessageId
HFrom: $>CheckFrom
HTo: $>CheckTo
HSubject: $>Check_Subject

SCheckMessageId
R< $+ @ $+ > $@ OK
R$* $#error $: 553 Illegal Message-ID

### dnl define(`_READ_X_SPAM_FILT_',`dnl')dnl
### dnl LOCAL_HACK(`check_local')dnl
### dnl LOCAL_HACK(`check_header',`Message-Id',`',`',`',`',`',`',`1')dnl
### dnl LOCAL_HACK(`check_header_end')dnl



# @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @@@@@@@@@@@@@@@@@@@@
# Igor Chudov's rules for algebra.com to video-collage.com
# conversion for certain addreses (cypherpunks, scrm etc).

LOCAL_RULE_0

################################################## ########## scrm
R scrm < @ algebra.com . > scrm < @ localhost . >
R scrm-board < @ algebra.com . > scrm < @ localhost . >
R scrm-mods < @ algebra.com . > scrm < @ localhost . >
R scrm-approved < @ algebra.com . > scrm < @ localhost . >
R scrm-rejected < @ algebra.com . > scrm < @ localhost . >
R scrm-approval-key < @ algebra.com . > scrm < @ localhost . >
R scrm-admin < @ algebra.com . > scrm < @ localhost . >
R devnull < @ algebra.com . > devnull < @ localhost . >
R passat-approval < @ algebra.com . > klm < @ cs.jhu.edu . >
R ichudov < @ algebra.com . > ichudov-both < @ localhost . >
R dasha < @ algebra.com . > dasha-both < @ localhost . >
#R dasha < @ algebra.com . > pavlovd < @ ics.uci.edu . >

################################################## ########## Cypher Punks
R cypherpunks < @ algebra.com . > cypherpunks < @ localhost . >
R owner-cypherpunks < @ algebra.com . > devnull < @ localhost . >

R cypherpunks-hosts < @ algebra.com . > cypherpunks-hosts < @ localhost . >
R stump-users < @ algebra.com . > stump-users < @ localhost . >
R jobs-discussion < @ algebra.com . > jobs-discussion < @ localhost . >
R majordomo < @ algebra.com . > majordomo < @ localhost . >

LOCAL_RULESETS
SLocal_check_mail
#R < > $# error $@ 5.7.0 $: "554 Temporary rejecting error messages. My domain is forged in thousands of spams."

  Réponse avec citation
Vieux 23/11/2007, 18h19   #7
Bill Cole
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: How the fuck can I force sendmail to efficiently process its queue

In article <_-udnXxLgYXxWdjanZ2dnUVZ_s7inZ2d@giganews.com>,
Ignoramus689 <ignoramus689@NOSPAM.689.invalid> wrote:

> On 2007-11-21, Bill Cole <bill@scconsult.com> wrote:
> > In article <3eednYYYNNf-8dnanZ2dnUVZ_tOtnZ2d@giganews.com>,
> > Ignoramus4770 <ignoramus4770@NOSPAM.4770.invalid> wrote:
> >
> >> I have a machine with quad CPU and 4 gigs of RAM.
> >>
> >> And yet for the life of me, I cannot figure out why I have so much
> >> crap for local users that cannot be processed.
> >>
> >> My sendmail is 8.13.
> >>
> >> All I want is to run, maybe 15 queue runners at the same time, so that
> >> undeliverable and slow junk does not slow down delivery of more
> >> valuable things.
> >>
> >> How can I do it?

> >
> > Read the documentation? Provide relevant information? Local delivery
> > slowness is usually not a sendmail config problem. If the problem you
> > have really is slow local delivery, you need to look at things like your
> > global or individual procmail rules (since you are using procmail for
> > local delivery) and at the performance of whatever storage you are using
> > for local delivery. One thing that does look wrong below is your QueueLA
> > setting. Unless you are trying to protect something else on the box from
> > Sendmail, 12 or even 20 would make a lot more sense than 3.

>
> I increased my QueueLA setting. Thanks. I also set up spamd
> (spamassassin's daemon program) to maximum of 10 children instead of
> 5. Spamassassin makes a number od DNS queries and generally waits a
> lot, so it makes sense to allow for more of its children.


*Now* you mention it....

You really need to be looking from the physical mailstore backwards:
disk, filesystem, mailbox access (e.g. locking), filtering, delivery
agent, MTA. Starting with the MTA is not really the best troubleshooting
approach.


> > FWIW, I suspect that your bigger problem is that you have not paying
> > attention to your use of third-party DNSBL's for a long time.

>
> OK, maybe I am missing something, but I thought that I had them all
> dnl'ed , kind of like commented out? (see quoted below)


Yep. Sorry, visual oops on my part.

--
Now where did I hide that website...
  Réponse avec citation
Vieux 23/11/2007, 22h30   #8
Res
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: How the fuck can I force sendmail to efficiently process itsqueue

On Fri, 23 Nov 2007, Bill Cole wrote:

>> I increased my QueueLA setting. Thanks. I also set up spamd
>> (spamassassin's daemon program) to maximum of 10 children instead of
>> 5. Spamassassin makes a number od DNS queries and generally waits a
>> lot, so it makes sense to allow for more of its children.

>
> *Now* you mention it....
>
> You really need to be looking from the physical mailstore backwards:
> disk, filesystem, mailbox access (e.g. locking), filtering, delivery
> agent, MTA. Starting with the MTA is not really the best troubleshooting
> approach.


He also needs to be using a far more efficient method that does not need
spamd .

OP: try www.mailscanner.info



--
Cheers
Res


  Réponse avec citation
Vieux 24/11/2007, 04h53   #9
Ignoramus24248
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: How the fuck can I force sendmail to efficiently process its queue

On 2007-11-23, Bill Cole <bill@scconsult.com> wrote:
> In article <_-udnXxLgYXxWdjanZ2dnUVZ_s7inZ2d@giganews.com>,
> Ignoramus689 <ignoramus689@NOSPAM.689.invalid> wrote:
>
>> On 2007-11-21, Bill Cole <bill@scconsult.com> wrote:
>> > In article <3eednYYYNNf-8dnanZ2dnUVZ_tOtnZ2d@giganews.com>,
>> > Ignoramus4770 <ignoramus4770@NOSPAM.4770.invalid> wrote:
>> >
>> >> I have a machine with quad CPU and 4 gigs of RAM.
>> >>
>> >> And yet for the life of me, I cannot figure out why I have so much
>> >> crap for local users that cannot be processed.
>> >>
>> >> My sendmail is 8.13.
>> >>
>> >> All I want is to run, maybe 15 queue runners at the same time, so that
>> >> undeliverable and slow junk does not slow down delivery of more
>> >> valuable things.
>> >>
>> >> How can I do it?
>> >
>> > Read the documentation? Provide relevant information? Local delivery
>> > slowness is usually not a sendmail config problem. If the problem you
>> > have really is slow local delivery, you need to look at things like your
>> > global or individual procmail rules (since you are using procmail for
>> > local delivery) and at the performance of whatever storage you are using
>> > for local delivery. One thing that does look wrong below is your QueueLA
>> > setting. Unless you are trying to protect something else on the box from
>> > Sendmail, 12 or even 20 would make a lot more sense than 3.

>>
>> I increased my QueueLA setting. Thanks. I also set up spamd
>> (spamassassin's daemon program) to maximum of 10 children instead of
>> 5. Spamassassin makes a number od DNS queries and generally waits a
>> lot, so it makes sense to allow for more of its children.

>
> *Now* you mention it....
>
> You really need to be looking from the physical mailstore backwards:
> disk, filesystem, mailbox access (e.g. locking), filtering, delivery
> agent, MTA. Starting with the MTA is not really the best troubleshooting
> approach.


Right.

Anyway, the changes that I made, along with sendmail -qR... started
REPEATEDLY, did seem to finally bring the count of local queued
messages to zero.

>
>> > FWIW, I suspect that your bigger problem is that you have not paying
>> > attention to your use of third-party DNSBL's for a long time.

>>
>> OK, maybe I am missing something, but I thought that I had them all
>> dnl'ed , kind of like commented out? (see quoted below)

>
> Yep. Sorry, visual oops on my part.
>


No problem. I took them out a year or two ago, when I set up
spamassassin, as its docs told me to stop using blacklists other than
through spamassassin.

i
  Réponse avec citation
Vieux 24/11/2007, 05h52   #10
Bill Cole
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: How the fuck can I force sendmail to efficiently process its queue

In article <TN2dnakDrpqlPdranZ2dnUVZ_oPinZ2d@giganews.com>,
Ignoramus24248 <ignoramus24248@NOSPAM.24248.invalid> wrote:

> On 2007-11-23, Bill Cole <bill@scconsult.com> wrote:
> > In article <_-udnXxLgYXxWdjanZ2dnUVZ_s7inZ2d@giganews.com>,
> > Ignoramus689 <ignoramus689@NOSPAM.689.invalid> wrote:
> >
> >> On 2007-11-21, Bill Cole <bill@scconsult.com> wrote:
> >> > In article <3eednYYYNNf-8dnanZ2dnUVZ_tOtnZ2d@giganews.com>,
> >> > Ignoramus4770 <ignoramus4770@NOSPAM.4770.invalid> wrote:
> >> >
> >> >> I have a machine with quad CPU and 4 gigs of RAM.
> >> >>
> >> >> And yet for the life of me, I cannot figure out why I have so much
> >> >> crap for local users that cannot be processed.
> >> >>
> >> >> My sendmail is 8.13.
> >> >>
> >> >> All I want is to run, maybe 15 queue runners at the same time, so that
> >> >> undeliverable and slow junk does not slow down delivery of more
> >> >> valuable things.
> >> >>
> >> >> How can I do it?
> >> >
> >> > Read the documentation? Provide relevant information? Local delivery
> >> > slowness is usually not a sendmail config problem. If the problem you
> >> > have really is slow local delivery, you need to look at things like your
> >> > global or individual procmail rules (since you are using procmail for
> >> > local delivery) and at the performance of whatever storage you are using
> >> > for local delivery. One thing that does look wrong below is your QueueLA
> >> > setting. Unless you are trying to protect something else on the box from
> >> > Sendmail, 12 or even 20 would make a lot more sense than 3.
> >>
> >> I increased my QueueLA setting. Thanks. I also set up spamd
> >> (spamassassin's daemon program) to maximum of 10 children instead of
> >> 5. Spamassassin makes a number od DNS queries and generally waits a
> >> lot, so it makes sense to allow for more of its children.

> >
> > *Now* you mention it....
> >
> > You really need to be looking from the physical mailstore backwards:
> > disk, filesystem, mailbox access (e.g. locking), filtering, delivery
> > agent, MTA. Starting with the MTA is not really the best troubleshooting
> > approach.

>
> Right.
>
> Anyway, the changes that I made, along with sendmail -qR... started
> REPEATEDLY, did seem to finally bring the count of local queued
> messages to zero.


That implies that you were running in queue-only mode a lot and were
never spawning queue runners.

The queue-only mode is a function of the QueueLA setting. When you had
it set to 3, hitting that should not have been uncommon. The other bit
begs the question: does your main sendmail daemon have a -q<time period>
option in its command line?

--
Now where did I hide that website...
  Réponse avec citation
Vieux 24/11/2007, 05h57   #11
Res
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: How the fuck can I force sendmail to efficiently process itsqueue

On Fri, 23 Nov 2007, Ignoramus24248 wrote:

<SNIP - some people ought to learn how>


> No problem. I took them out a year or two ago, when I set up
> spamassassin, as its docs told me to stop using blacklists other than
> through spamassassin.


I think id rather stop the scum at MTA than accept the message and
then run it through spamassassin

If you have a busy network, you'd want to reverse that, enable at
MTA, and DISABLE all blacklist lookups in SA

I use these and have very little trouble

dnl #cn kr tw hk
FEATURE(`enhdnsbl', `zz.countries.nerd.dk', `"553 rejected"',`',`127.0.0.156.',`127.0.1.154.',`127.0 .0.158.',`127.0.1.88.')dnl
FEATURE(`enhdnsbl', `dnsbl.sorbs.net', `"553 rejected - see http://www.sorbs.net/lookup.shtml?"$&{client_addr}', `')dnl
FEATURE(`enhdnsbl', `bl.spamcop.net', `"553 rejected - see http://spamcop.net/bl.shtml?"$&{client_addr}', `')dnl
FEATURE(`enhdnsbl', `combined.njabl.org',`"553 rejected - see http://njabl.org/lookup?"$&{client_addr}', `')dnl

(yes I'm aware combined.njabl will soon cease to exist probably and we
should use zen, but I disagree with spamhaus's policy on 'we'll block you
from requests, pay us and you can rsync the DB', because so many people
are blocked LONG before spamhaus reckons they should be, and some are not,
so, either someone deliberately mucks with the big red button cause they
are bored or they are technically incompetant in setting up a working
automation of acl's.

Very few hits make it to njabl as SORBS grabs most of it and
Spamcop the few left overs

Some blacklists being looked up in SA *may* be outdated, you'd need to
check that as well if you insist on using SA doing the scumbag lookups


--
Cheers
Res


  Réponse avec citation
Vieux 24/11/2007, 06h27   #12
Ignoramus24248
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: How the fuck can I force sendmail to efficiently process its queue

On 2007-11-24, Bill Cole <bill@scconsult.com> wrote:
> In article <TN2dnakDrpqlPdranZ2dnUVZ_oPinZ2d@giganews.com>,
> Ignoramus24248 <ignoramus24248@NOSPAM.24248.invalid> wrote:
>
>> On 2007-11-23, Bill Cole <bill@scconsult.com> wrote:
>> > In article <_-udnXxLgYXxWdjanZ2dnUVZ_s7inZ2d@giganews.com>,
>> > Ignoramus689 <ignoramus689@NOSPAM.689.invalid> wrote:
>> >
>> >> On 2007-11-21, Bill Cole <bill@scconsult.com> wrote:
>> >> > In article <3eednYYYNNf-8dnanZ2dnUVZ_tOtnZ2d@giganews.com>,
>> >> > Ignoramus4770 <ignoramus4770@NOSPAM.4770.invalid> wrote:
>> >> >
>> >> >> I have a machine with quad CPU and 4 gigs of RAM.
>> >> >>
>> >> >> And yet for the life of me, I cannot figure out why I have so much
>> >> >> crap for local users that cannot be processed.
>> >> >>
>> >> >> My sendmail is 8.13.
>> >> >>
>> >> >> All I want is to run, maybe 15 queue runners at the same time, so that
>> >> >> undeliverable and slow junk does not slow down delivery of more
>> >> >> valuable things.
>> >> >>
>> >> >> How can I do it?
>> >> >
>> >> > Read the documentation? Provide relevant information? Local delivery
>> >> > slowness is usually not a sendmail config problem. If the problem you
>> >> > have really is slow local delivery, you need to look at things like your
>> >> > global or individual procmail rules (since you are using procmail for
>> >> > local delivery) and at the performance of whatever storage you are using
>> >> > for local delivery. One thing that does look wrong below is your QueueLA
>> >> > setting. Unless you are trying to protect something else on the box from
>> >> > Sendmail, 12 or even 20 would make a lot more sense than 3.
>> >>
>> >> I increased my QueueLA setting. Thanks. I also set up spamd
>> >> (spamassassin's daemon program) to maximum of 10 children instead of
>> >> 5. Spamassassin makes a number od DNS queries and generally waits a
>> >> lot, so it makes sense to allow for more of its children.
>> >
>> > *Now* you mention it....
>> >
>> > You really need to be looking from the physical mailstore backwards:
>> > disk, filesystem, mailbox access (e.g. locking), filtering, delivery
>> > agent, MTA. Starting with the MTA is not really the best troubleshooting
>> > approach.

>>
>> Right.
>>
>> Anyway, the changes that I made, along with sendmail -qR... started
>> REPEATEDLY, did seem to finally bring the count of local queued
>> messages to zero.

>
> That implies that you were running in queue-only mode a lot and were
> never spawning queue runners.


Quite possibly (though I am not sure of the exact meaning of what you
said).

> The queue-only mode is a function of the QueueLA setting. When you had
> it set to 3, hitting that should not have been uncommon. The other bit
> begs the question: does your main sendmail daemon have a -q<time period>
> option in its command line?


Yes, I run it with -q20m.

How can I make sendmail spawn many queue runners?

i
  Réponse avec citation
Vieux 24/11/2007, 06h30   #13
Ignoramus24248
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: How the fuck can I force sendmail to efficiently process itsqueue

On 2007-11-24, Res <res@ausics.net> wrote:
> On Fri, 23 Nov 2007, Ignoramus24248 wrote:
>
><SNIP - some people ought to learn how>
>
>
>> No problem. I took them out a year or two ago, when I set up
>> spamassassin, as its docs told me to stop using blacklists other than
>> through spamassassin.

>
> I think id rather stop the scum at MTA than accept the message and
> then run it through spamassassin


True, but the blacklists are usually transient and most go away after
their owners get fed up or move on to other things.


> If you have a busy network, you'd want to reverse that, enable at
> MTA, and DISABLE all blacklist lookups in SA
>
> I use these and have very little trouble
>
> dnl #cn kr tw hk
> FEATURE(`enhdnsbl', `zz.countries.nerd.dk', `"553 rejected"',`',`127.0.0.156.',`127.0.1.154.',`127.0 .0.158.',`127.0.1.88.')dnl
> FEATURE(`enhdnsbl', `dnsbl.sorbs.net', `"553 rejected - see http://www.sorbs.net/lookup.shtml?"$&{client_addr}', `')dnl
> FEATURE(`enhdnsbl', `bl.spamcop.net', `"553 rejected - see http://spamcop.net/bl.shtml?"$&{client_addr}', `')dnl
> FEATURE(`enhdnsbl', `combined.njabl.org',`"553 rejected - see http://njabl.org/lookup?"$&{client_addr}', `')dnl
>
> (yes I'm aware combined.njabl will soon cease to exist probably and we
> should use zen, but I disagree with spamhaus's policy on 'we'll block you
> from requests, pay us and you can rsync the DB', because so many people
> are blocked LONG before spamhaus reckons they should be, and some are not,
> so, either someone deliberately mucks with the big red button cause they
> are bored or they are technically incompetant in setting up a working
> automation of acl's.
>
> Very few hits make it to njabl as SORBS grabs most of it and
> Spamcop the few left overs


I do like spamassassing scoring policy, where a DNS lookup usually is
only advisory and affects the score. Some of my email contacts,
unfortunately, have been blocklisted by some blacklists, usually due
to either being clueless and running viruses, or due to inheriting a
bad IP. In any case, missing their emails would mean loss of $$ and
goodwill.


> Some blacklists being looked up in SA *may* be outdated, you'd need to
> check that as well if you insist on using SA doing the scumbag lookups


I update SA every night (SA rules).

i
  Réponse avec citation
Vieux 24/11/2007, 14h46   #14
Per Hedeland
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: How the fuck can I force sendmail to efficiently process itsqueue

In article <Pine.LNX.4.64.0711241443420.31802@ebfjryy.nhfvpf. arg> Res
<res@ausics.net> writes:
>On Fri, 23 Nov 2007, Ignoramus24248 wrote:
>
>> No problem. I took them out a year or two ago, when I set up
>> spamassassin, as its docs told me to stop using blacklists other than
>> through spamassassin.

>
>I think id rather stop the scum at MTA than accept the message and
>then run it through spamassassin
>
>If you have a busy network, you'd want to reverse that, enable at
>MTA, and DISABLE all blacklist lookups in SA


If you run SA from a milter (e.g. MimeDefang), you don't need to
*accept* the message first - though you do need to receive it. So I
guess the ideal for someone that has a lot of time to spend babysitting
the anti-spam setup is to use a combination - sendmail rule check for
blacklists that you trust/value enough to let them make a final
judgement about your mail on their own (if any), SA for the others where
you want to apply scoring. Though of course if you really have a *lot*
of time, you could implement blacklist scoring in sendmail rules...

--Per Hedeland
per@hedeland.org
  Réponse avec citation
Vieux 24/11/2007, 15h06   #15
Per Hedeland
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: How the fuck can I force sendmail to efficiently process its queue

In article <GqWdnfF3w8irK9ranZ2dnUVZ_qXinZ2d@giganews.com>
Ignoramus24248 <ignoramus24248@NOSPAM.24248.invalid> writes:
>On 2007-11-24, Bill Cole <bill@scconsult.com> wrote:
>>
>> That implies that you were running in queue-only mode a lot and were
>> never spawning queue runners.

>
>Quite possibly (though I am not sure of the exact meaning of what you
>said).


When your load average is above QueueLA, sendmail will only queue
messages, not deliver, even locally - i.e. "queue-only mode" - on the
assumption that delivery will raise the load further. Which is fine,
except that if your RefuseLA is *higher* and you have a busy server, you
will keep accepting more mail and queue it, which will keep the load
high (it will hover around RefuseLA), and you never get around to
delivering the queued mail, and the queue just grows and grows, which
makes queueing more and more expensive...

In short the default of having RefuseLA higher than QueueLA is basically
only appropriate for a server where mail processing is a low-intensity,
low-priority task - you really want to receive it if at all possible,
but other tasks are more important, so it's OK to let the received mail
sit in the queue until they're finished. This was probably the most
common use case back when those values were chosen, but for a dedicated,
busy mail server they are a disaster.

>> The queue-only mode is a function of the QueueLA setting. When you had
>> it set to 3, hitting that should not have been uncommon. The other bit
>> begs the question: does your main sendmail daemon have a -q<time period>
>> option in its command line?

>
>Yes, I run it with -q20m.
>
>How can I make sendmail spawn many queue runners?


There are lots of different knobs to turn, and that particular one may
not even be the optimal one for you. In many/most cases it will be self-
regulating once you have the QueueLA/RefuseLA set up right - with your
-q20m, sendmail will spawn a queue runner every 20 minutes (i.e. *not*
"a new runner 20 minutes after the preceding one finished"), which means
that if the queue is big enough that a runner can't run through it in 20
minutes, the number of runners will keep increasing until they *can* do
that. So wait and see how things develop before you change more stuff.

--Per Hedeland
per@hedeland.org



  Réponse avec citation
Vieux 24/11/2007, 17h53   #16
Bill Cole
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: How the fsck can I force sendmail to efficiently process its queue

In article <GqWdnfF3w8irK9ranZ2dnUVZ_qXinZ2d@giganews.com>,
Ignoramus24248 <ignoramus24248@NOSPAM.24248.invalid> wrote:

> On 2007-11-24, Bill Cole <bill@scconsult.com> wrote:

[...]
> > That implies that you were running in queue-only mode a lot and were
> > never spawning queue runners.

>
> Quite possibly (though I am not sure of the exact meaning of what you
> said).


You said this is on a 4-cpu machine, and originally your QueueLA setting
was 3. Whenever the load average went over 3 (roughly equivalent to a
load average of 0.75 on a 1-cpu machine, i.e. significantly idle)
Sendmail went into queue-only mode: rather than delivering mail as it
comes in (normal operation) Sendmail was dropping mail into the queue
rather than delivering it.

Why you had that and why you only increased the QueueLA to 6 after I
suggested much higher numbers is something only you can explain to
yourself. If you want to understand that setting, I suggest the Bat
Book. If you want a rule of thumb, I suggest 3-5 times the CPU count on
a modern system where your delivery stack (e.g. procmail+spamc/spamd in
your case) is hoggy. Setting it to 3 or 6 on a machine where you want
speedy delivery is self-defeating. The default is 8, and I'm pretty sure
that's chosen to be reasonable for single-cpu machines with very
lightweight local delivery.

> > The queue-only mode is a function of the QueueLA setting. When you had
> > it set to 3, hitting that should not have been uncommon. The other bit
> > begs the question: does your main sendmail daemon have a -q<time period>
> > option in its command line?

>
> Yes, I run it with -q20m.


That's a bit high, particularly with pathologically low QueueLA setting.
You probably want to try spawning a queue runner at least once every 5
minutes if you are choosing to go into queue-only mode so easily.


> How can I make sendmail spawn many queue runners?


Why do you think that's the fix for your problem?

It is almost surely NOT the best fix. The best fix is probably to not
have Sendmail queueing mail and waiting for queue runners to clean up
later. You have not explained why you are choosing that mode of
operation, and it is mystifying to me.

However, from looking at your posted .mc I see this:

> O MaxQueueChildren
> O MaxRunnersPerQueue=10


You have to actually set MaxQueueChildren to a value for
MaxRunnersPerQueue to work. You could unset both to have Sendmail spawn
a new runner periodically and not limit how many are currently active.

I don't believe there's any way to make the Sendmail parent daemon spawn
multiple queue runners periodically on one queue, as that's not really a
great strategy for handling a normal queue. If you are desperate for
multiple truly simultaneous queue runners I think you'll have to look
into setting up queue groups and/or having a cron job spawning your
runners.

--
Now where did I hide that website...
  Réponse avec citation
Vieux 24/11/2007, 23h21   #17
Res
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: How the fuck can I force sendmail to efficiently process itsqueue

On Sat, 24 Nov 2007, Per Hedeland wrote: