George Adams <g_adams27@hotmail.SPAMBGONE.com> writes:

> Our church has a 2.8Ghz Gentoo Linux (2.6.10) server w/1Gig RAM,
> running Apache 2.0.52 . One of the features we offer is the ability
> to download MP3 sermons off the site.
>
>
> The other day I couldn't log on remotely to the server. Logging in at
> the console, I discovered that the machine was in a death spiral of
> "Out of memory - killing process 12842 (apache2)" errors. Since all
> activity on the console was taking a painfully long time to process, I
> eventually just rebooted.
>
>
> Looking at the logs, I saw that a single client had tried downloading
> over 140 MP3 sermons within the span of a few minutes. Undoubtedly
> that was causing the "out of memory" errors.
>
>
> So now I have two goals. First, I want to understand why the problem
> happened. Second, I want to make sure it doesn't happen again.

They may have used a utility that allows multiple simultaneous
connections - can be hard to stop that. I'm assuming they are just
downloading the file which should be that memory intensive for Apache.
You may just be hurt by the overhead of more Apache processes running.

You don't mention it, but make sure you have some swap space
allocated for your machine, at least a Gigabyte -- that will give you
more breathing room.
>
>
> 1) I don't really quite understand how memory usage occurs with
> Apache2. For instance, here's the current output of "ps aux" related
> to Apache:
>
>
> USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
> root 14071 0.0 0.3 7364 2896 ? Ss Mar06 0:00
> /usr/sbin/apache2
>
> apache 14078 0.0 0.3 7304 2824 ? S Mar06 0:00

>
>
> Can I determine the total amount of memory used by Apache by adding up
> the %MEM columns? (i.e. 0.3 + 0.3 + 0.3 + 6.7 etc. = 21.7% of total
> memory?

NO, you should either try 'top' and sort my Memory - OR,
for command line use ps -elfm m Os

Print out all processes and threads, show memory used,
sorted by memory.

>
>
> I'd like to understand what actually happens in terms of memory usage
> when someone starts downloading an MP3 file. Does Linux simply spawn
> another child process of Apache (additional memory requirement = 1
> Apache process), or does Apache have to read the entire MP3 file into
> memory as it delivers it? (additional memory requirement = 16M, the
> size of the MP3 file). I'm guessing that's NOT the case, but when I
> experimented by watching "top" while running a script that downloaded
> 5 MP3s at once, I was surprised to watch the memory usage climb... and
> climb... and climb... and continue to climb until I finally aborted
> the script.

Apache itself will start another child process to service a
request if a spare child is not available, that is the Max/MinSpareServers
stuff in the config file.

You said you watched a 'script' execute. Above I was assuming
you are just giving direct links to the MP3 files? If you are using
a script, then something bizarre could be going on if the script attempt
to read the whole file into memory.
>
>
> 2) So how can I stop the system from running out of memory? I thought
> of maybe writing a script that will stop any given IP address from
> downloading more than X files in a given period of time (say, no more
> than 2 files every 10 minutes), but is there a better way? How do
> larger file servers that could be serving hundreds or thousands of
> files at any given time do it, without having a terabyte of RAM?
>
>
> Thanks to anyone who can me get a handle on this!


Add the SWAP space, it is very easy to do. Here is a little
cheat sheet that works on redhat. If you are using a script, you get
put some tracking in there to block simul. connections from the same
IP.

-------------------
1. Create swap file:

dd if=/dev/zero of=thebookswap bs=1024 count=200000

-- this will create a 200 Meg area!


2. Convert to swap format

mkswap thebookswap


3. Activate as swap, do a 'free' before and after each command

swapon thebookswap

-- you should see the new space


-- To Turn it OFF

swapoff thebookswap

5. To make it permanent add to /etc/fstab -- will be picked
up when system gives swapon -a at boot automatically:

/dev/sda9 swap swap defaults,noatime 0 0
/export/tools2/system/swap/thebookswap swap swap defaults,noatime 0 0

---------------------------
--
John
__________________________________________________ _________________
John Murtari Software Workshop Inc.
jmurtari@following domain 315.635-1968(x-211) "TheBook.Com" (TM)
http://thebook.com/