PHWinfo - Afficher un message

25/02/2008, 16h50

On Mon, Feb 25, 2008 at 4:29 PM, pluskid <pluskid@gmail.com> wrote:
> Hi, all!
>
> I'm having memory leak problem with my program, but can't find where's
> the leak.
> I have a program written in pure-Ruby. It implemented a Chinese word
> segment
> algorithm. I call `segment(text)' to get the result. But each time I
> call `segment',
> the memory usage of my program (Ruby) increased several handred K-
> bytes -- the
> number of bytes of increment is roughly equal during each call.
>
> I can see from outside (using `ps' or `top') that the memory is
> leaking. But I can't
> find where the leaking goes. I tried to use ObjectSpace.each_object to
> find out
> what leaks. I followed this article
>
> http://scottstuff.net/blog/articles/...ing-with-rails
>
> and compare the objects after each call to `segment'. What surprise me
> is that
> there's no object leaked between each call. The number of objects
> after each
> call is roughly equal.
>
> I also tried bleak_house:
>
> http://blog.evanweaver.com/files/doc...es/README.html
>
> I don't precisely how to use it. I just dump a snapshot after each
> call to `segment'
> and use `bleak' to analyze the result. However, it also shows no leak.
> But if
> there's no leak, where goes the memory?
>
> Here's the result of using the `pmap' tool to inspect the running Ruby
> process:
>
> 933: /usr/bin/ruby1.8 /usr/bin/rake PROFILE=false test:many
> 08048000 4K r-x-- /usr/bin/ruby1.8
> 08049000 4K rw--- /usr/bin/ruby1.8
> 0804a000 33412K rw--- [ anon ]
> b74a8000 7016K rw--- [ anon ]
> b7b82000 16K r-x-- /usr/lib/ruby/1.8/i486-linux/strscan.so
> b7b86000 4K rw--- /usr/lib/ruby/1.8/i486-linux/strscan.so
> b7b87000 16K r-x-- /var/lib/gems/1.8/gems/ruby-prof-0.6.0/lib/
> ruby_prof.so
> b7b8b000 4K rw--- /var/lib/gems/1.8/gems/ruby-prof-0.6.0/lib/
> ruby_prof.so
> b7b8c000 1192K rw--- [ anon ]
> b7cb6000 1308K r-x-- /lib/i686/cmov/libc-2.7.so
> b7dfd000 4K r---- /lib/i686/cmov/libc-2.7.so
> b7dfe000 8K rw--- /lib/i686/cmov/libc-2.7.so
> b7e00000 12K rw--- [ anon ]
> b7e03000 140K r-x-- /lib/i686/cmov/libm-2.7.so
> b7e26000 8K rw--- /lib/i686/cmov/libm-2.7.so
> b7e28000 36K r-x-- /lib/i686/cmov/libcrypt-2.7.so
> b7e31000 8K rw--- /lib/i686/cmov/libcrypt-2.7.so
> b7e33000 156K rw--- [ anon ]
> b7e5a000 8K r-x-- /lib/i686/cmov/libdl-2.7.so
> b7e5c000 8K rw--- /lib/i686/cmov/libdl-2.7.so
> b7e5e000 4K rw--- [ anon ]
> b7e5f000 80K r-x-- /lib/i686/cmov/libpthread-2.7.so
> b7e73000 8K rw--- /lib/i686/cmov/libpthread-2.7.so
> b7e75000 8K rw--- [ anon ]
> b7e77000 756K r-x-- /usr/lib/libruby1.8.so.1.8.6
> b7f34000 8K rw--- /usr/lib/libruby1.8.so.1.8.6
> b7f36000 64K rw--- [ anon ]
> b7f48000 8K rw--- [ anon ]
> b7f4a000 8K r-x-- /usr/lib/ruby/1.8/i486-linux/etc.so
> b7f4c000 4K rw--- /usr/lib/ruby/1.8/i486-linux/etc.so
> b7f4d000 12K r-x-- /usr/lib/ruby/1.8/i486-linux/thread.so
> b7f50000 4K rw--- /usr/lib/ruby/1.8/i486-linux/thread.so
> b7f51000 8K rw--- [ anon ]
> b7f53000 104K r-x-- /lib/ld-2.7.so
> b7f6d000 8K rw--- /lib/ld-2.7.so
> bfdea000 116K rw--- [ stack ]
> ffffe000 4K r-x-- [ anon ]
> total 44568K
>
> The section "0804a000 33412K rw--- [ anon ]" increased after each
> call. I've no idea now. Do you have any suggestion for me?

This is what I have read here or somewhere else, and I an not
knowledgeable enough to confirm it, anyway:
Ruby allocs space from the OS in large chunks (initially 8 MB, and
each other chunk is bigger by a predefined coefficient).
Any chunk can be returned to the OS only if it is empty - so if your
memory is fragmented, ruby will have to alloc new chunks
in order to satisfy ruby script being run.

If this is true, it means: the order and size of your allocations is important.
Especially order of the allocations that you later release relative to
those that you don't.

25/02/2008, 16h50	#2
Jano Svitok Messages: n/a Hébergeur:	Re: How to find out memory leak? On Mon, Feb 25, 2008 at 4:29 PM, pluskid <pluskid@gmail.com> wrote: > Hi, all! > > I'm having memory leak problem with my program, but can't find where's > the leak. > I have a program written in pure-Ruby. It implemented a Chinese word > segment > algorithm. I call `segment(text)' to get the result. But each time I > call `segment', > the memory usage of my program (Ruby) increased several handred K- > bytes -- the > number of bytes of increment is roughly equal during each call. > > I can see from outside (using `ps' or `top') that the memory is > leaking. But I can't > find where the leaking goes. I tried to use ObjectSpace.each_object to > find out > what leaks. I followed this article > > http://scottstuff.net/blog/articles/...ing-with-rails > > and compare the objects after each call to `segment'. What surprise me > is that > there's no object leaked between each call. The number of objects > after each > call is roughly equal. > > I also tried bleak_house: > > http://blog.evanweaver.com/files/doc...es/README.html > > I don't precisely how to use it. I just dump a snapshot after each > call to `segment' > and use `bleak' to analyze the result. However, it also shows no leak. > But if > there's no leak, where goes the memory? > > Here's the result of using the `pmap' tool to inspect the running Ruby > process: > > 933: /usr/bin/ruby1.8 /usr/bin/rake PROFILE=false test:many > 08048000 4K r-x-- /usr/bin/ruby1.8 > 08049000 4K rw--- /usr/bin/ruby1.8 > 0804a000 33412K rw--- [ anon ] > b74a8000 7016K rw--- [ anon ] > b7b82000 16K r-x-- /usr/lib/ruby/1.8/i486-linux/strscan.so > b7b86000 4K rw--- /usr/lib/ruby/1.8/i486-linux/strscan.so > b7b87000 16K r-x-- /var/lib/gems/1.8/gems/ruby-prof-0.6.0/lib/ > ruby_prof.so > b7b8b000 4K rw--- /var/lib/gems/1.8/gems/ruby-prof-0.6.0/lib/ > ruby_prof.so > b7b8c000 1192K rw--- [ anon ] > b7cb6000 1308K r-x-- /lib/i686/cmov/libc-2.7.so > b7dfd000 4K r---- /lib/i686/cmov/libc-2.7.so > b7dfe000 8K rw--- /lib/i686/cmov/libc-2.7.so > b7e00000 12K rw--- [ anon ] > b7e03000 140K r-x-- /lib/i686/cmov/libm-2.7.so > b7e26000 8K rw--- /lib/i686/cmov/libm-2.7.so > b7e28000 36K r-x-- /lib/i686/cmov/libcrypt-2.7.so > b7e31000 8K rw--- /lib/i686/cmov/libcrypt-2.7.so > b7e33000 156K rw--- [ anon ] > b7e5a000 8K r-x-- /lib/i686/cmov/libdl-2.7.so > b7e5c000 8K rw--- /lib/i686/cmov/libdl-2.7.so > b7e5e000 4K rw--- [ anon ] > b7e5f000 80K r-x-- /lib/i686/cmov/libpthread-2.7.so > b7e73000 8K rw--- /lib/i686/cmov/libpthread-2.7.so > b7e75000 8K rw--- [ anon ] > b7e77000 756K r-x-- /usr/lib/libruby1.8.so.1.8.6 > b7f34000 8K rw--- /usr/lib/libruby1.8.so.1.8.6 > b7f36000 64K rw--- [ anon ] > b7f48000 8K rw--- [ anon ] > b7f4a000 8K r-x-- /usr/lib/ruby/1.8/i486-linux/etc.so > b7f4c000 4K rw--- /usr/lib/ruby/1.8/i486-linux/etc.so > b7f4d000 12K r-x-- /usr/lib/ruby/1.8/i486-linux/thread.so > b7f50000 4K rw--- /usr/lib/ruby/1.8/i486-linux/thread.so > b7f51000 8K rw--- [ anon ] > b7f53000 104K r-x-- /lib/ld-2.7.so > b7f6d000 8K rw--- /lib/ld-2.7.so > bfdea000 116K rw--- [ stack ] > ffffe000 4K r-x-- [ anon ] > total 44568K > > The section "0804a000 33412K rw--- [ anon ]" increased after each > call. I've no idea now. Do you have any suggestion for me? This is what I have read here or somewhere else, and I an not knowledgeable enough to confirm it, anyway: Ruby allocs space from the OS in large chunks (initially 8 MB, and each other chunk is bigger by a predefined coefficient). Any chunk can be returned to the OS only if it is empty - so if your memory is fragmented, ruby will have to alloc new chunks in order to satisfy ruby script being run. If this is true, it means: the order and size of your allocations is important. Especially order of the allocations that you later release relative to those that you don't.