Erlang Mailing Lists

Author Message

<  Erlang questions mailing list  ~  Erlang - Question on memory usage reported

Guest
Posted: Wed Jun 16, 2010 12:56 pm Reply with quote
Guest
Hi All,

Apologies for the long email...

I am running an application that starts up with a memory footprint of around ~725MB when started (Approx 9000 processes on startup).. Application spawns processes as new requests are processed and grows to about 50K processes and memory usage of 5.1GB during the day. Output of memory(). is as follows -

memory().
[{total,5196843904},
{processes,3676103336},
{processes_used,3670262752},
{system,1520740568},
{atom,768433},
{atom_used,743978},
{binary,199344664},
{code,8016451},
{ets,1304839200}]

At the end of the day, a cleanup job reviews the process state and kills the pids that are not required and cleans up the ets table. Essentially returning it to start up stage of around 9K processes. Running the memory command in erlang shell reports -

memory().
[{total,721747840},
{processes,584000792},
{processes_used,552084576},
{system,137747048},
{atom,768433},
{atom_used,744177},
{binary,44468264},
{code,8016451},
{ets,80471216}]

Which is pretty consistent with what I am expecting, however running 'htop' on the system still indicates Erlang beam process with a resident memory usage of around 5.1GB. Any pointers that explains this behavior is greatly appreciated. Is there a way to release the memory? There was a suggestion in one of the Erlang forums that bouncing the Supervisor process as part of the clean up could be one of the options? Anyone with similar experience?

I am running the version - Erlang R13B02 (erts-5.7.3)

Thanks much
-- Rajesh Bhat.

______________________________________________

See http://www.peak6.com/email_disclaimer.php
for terms and conditions related to this email


Post received from mailinglist
Guest
Posted: Mon Jun 28, 2010 11:24 am Reply with quote
Guest
Sorry for replying to a message a week and a half old...

Rajesh Bhat <rbhat@optionshouse.com> wrote:

rb> I am running an application that starts up with a memory footprint
rb> of around ~725MB when started (Approx 9000 processes on
rb> startup).. Application spawns processes as new requests are
rb> processed and grows to about 50K processes and memory usage of 5.1GB
rb> during the day. [...]

rb> At the end of the day, a cleanup job reviews the process state and
rb> kills the pids that are not required and cleans up the ets
rb> table. [...]

rb> [...] however running 'htop' on the system still indicates Erlang
rb> beam process with a resident memory usage of around 5.1GB. Any
rb> pointers that explains this behavior is greatly appreciated. Is
rb> there a way to release the memory?

Not easily, no. When a UNIX process calls free(), there's no guarantee
that the memory free'd by that call will be returned to the OS now,
later, or ever. It depends on how it's allocated. If the allocator
used mmap(), then it can't munmap() the memory until (at least) all
other chunks in the mmap()'ed region are also freed. If it's allocated
using sbrk() or related, you can't move the break back down (to
deallocate memory) if there are still live chunks of memory above the
break.

The Erlang VM uses its own set of allocators instead of using malloc
directly, so they add another layer of when-to-release-to-the-OS that I
can't comment directly on, sorry.

-Scott

________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-questions-unsubscribe@erlang.org

Post received from mailinglist
Guest
Posted: Mon Jun 28, 2010 1:34 pm Reply with quote
Guest
Thank you.

Yes, it is the release memory back to OS option that I am looking for. I have run garbage collector on the pid, however that did not do the trick.

Thanks
-- Rajesh Bhat.


-----Original Message-----
From: Scott Lystig Fritchie [mailto:fritchie@snookles.com]
Sent: Friday, June 25, 2010 5:00 PM
To: Rajesh Bhat
Cc: 'erlang-questions@erlang.org'
Subject: Re: [erlang-questions] Erlang - Question on memory usage reported

Sorry for replying to a message a week and a half old...

Rajesh Bhat <rbhat@optionshouse.com> wrote:

rb> I am running an application that starts up with a memory footprint
rb> of around ~725MB when started (Approx 9000 processes on
rb> startup).. Application spawns processes as new requests are
rb> processed and grows to about 50K processes and memory usage of 5.1GB
rb> during the day. [...]

rb> At the end of the day, a cleanup job reviews the process state and
rb> kills the pids that are not required and cleans up the ets
rb> table. [...]

rb> [...] however running 'htop' on the system still indicates Erlang
rb> beam process with a resident memory usage of around 5.1GB. Any
rb> pointers that explains this behavior is greatly appreciated. Is
rb> there a way to release the memory?

Not easily, no. When a UNIX process calls free(), there's no guarantee
that the memory free'd by that call will be returned to the OS now,
later, or ever. It depends on how it's allocated. If the allocator
used mmap(), then it can't munmap() the memory until (at least) all
other chunks in the mmap()'ed region are also freed. If it's allocated
using sbrk() or related, you can't move the break back down (to
deallocate memory) if there are still live chunks of memory above the
break.

The Erlang VM uses its own set of allocators instead of using malloc
directly, so they add another layer of when-to-release-to-the-OS that I
can't comment directly on, sorry.

-Scott

______________________________________________

See http://www.peak6.com/email_disclaimer.php
for terms and conditions related to this email

________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-questions-unsubscribe@erlang.org

Post received from mailinglist
wuji
Posted: Mon Aug 20, 2012 8:27 am Reply with quote
User Joined: 10 Aug 2012 Posts: 654
age the way of her mother, Judy Loe, 65."I much much replica designer *beep* much prefer how my mother looks to the people I
here [in Los Angeles] with wind-tunnel face," Beckinsale, who turns turns imitation designer *beep* turns 39 at the end of this month, tells the
edition of Glamour magazine."I feel like beauty is a gift gift jordan 6 olympic gift that you have for a while, and you enjoy
hell out of it while you have it," she tells tells cheap designer *beep* tells the August issue. "And if you're lucky enough to
a daughter and you give it to her, you enjoy enjoy cheap Ralph Lauren enjoy the fact she has it. My mother was always
very beautiful -- she still is, in her sixties."Beckinsale also also [h3]jordan 6[/h3] also lashed out against the pressure put on new celebrity
to bounce back after their pregnancies."There's an obsessional hatred of of cheap polo ralph lauren of normal human processes. Pregnancy changes a woman's body and
It isn't normal to not look like you've had a a cheap designer *beep* a baby immediately after you've had a baby."Fortunately for Beckinsale,
View user's profile Send private message

Display posts from previous:  

All times are GMT
Page 1 of 1
This forum is locked: you cannot post, reply to, or edit topics.

Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum