Erlang/OTP Forums

Author Message

<  Yaws mailing list  ~  Overloaded yaws!

Guest
Posted: Fri Feb 05, 2010 11:08 am Reply with quote
Guest
Hi there,

I have a production yaws server, that works perfectly - for a while, then over time slowly uses up ~70% of CPU time, with the rest spread between idle and system processes.


This happens with only a few simultaneous users (2-3!), and once it has reached that point, never returns, even once incoming traffic has been removed.


I don't seem to be able to replicate the conditions in testing, so I really need a way to see what is taking the time on the live server.
Guest
Posted: Fri Feb 05, 2010 4:44 pm Reply with quote
Guest
andrew mmc wrote:
> Hi there,
>
> I have a production yaws server, that works perfectly - for a while,
> then over time slowly uses up ~70% of CPU time, with the rest spread
> between idle and system processes.
>
> Any ideas, anyone?!


Absolutely, been there - done that ..

First off - there is no way to attach and get a shell into a
running yaws daemon. This has been up for discussions a couple of times,
but I seem to recall that there was no sufficiently secure way to do it.
OTOH, some time ago, all security issues with

$YAWSHOME/.yaws/yaws/default/CTL

were resolved, so I think it would be ok to add that feature to yaws now.
I.e

# yaws --shell

(Any takers ??)

However, that doesn't even begin to address your problem , a running
production server that leaks memory.
You have two options.

1.

# man yaws
....
--debug-dump [--id id]
Produce a debug dump on stdout. In particular this code lists
what we refer to as suspicious processes. I.e. processes that
might be hanging or processes that are "large" - hardcoded to
40k words.


2.

Run the daemon with -i in /usr/bin/screen


One there - you can start to try to inspect different
processes and find the perpetrator.


/klacke



------------------------------------------------------------------------------
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com
_______________________________________________
Erlyaws-list mailing list
Erlyaws-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/erlyaws-list
Post received from mailinglist
Guest
Posted: Fri Feb 05, 2010 5:44 pm Reply with quote
Guest
Hi...

Thanks for this.
Guest
Posted: Fri Feb 05, 2010 8:05 pm Reply with quote
Guest
andrew mmc wrote:
> Hi...
>
> Thanks for this. I get the attached output. I haven't quite figured
> out what it all means just yet, but I'm guessing all those
> ets:select_trap are the symptom of the problem, but how can I find out
> what is the related issue?

Those processes sure look suspicious. There maybe other issues, but
the select_trap procs would certainly be my first pick here.

Next step is to figure out who is calling ets:select_trap .. whatever
that is - I've never even heard of the function.

Two options -

- some code inside yaws that use ets, end up in some
internal unheard of function ets:select_trap and spends cycles.
Not very likely.

- Your code does that, more likely.

My guess here is that you have app code that abuses ets/mnesia some
way and leaks memory. You need to debug. If you can't reproduce
offline, run production server inside screen with -i and be careful.

Good luck,

/klacke



------------------------------------------------------------------------------
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com
_______________________________________________
Erlyaws-list mailing list
Erlyaws-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/erlyaws-list
Post received from mailinglist
Guest
Posted: Fri Feb 05, 2010 8:18 pm Reply with quote
Guest
andrew mmc wrote:

>
> etop for some reason doesn't work - I get badmatch,{error,nxdomain}} on
> startup.


Here's a tty based top that I usually keep handy on production
servers.

/klacke


Post received from mailinglist
wuji
Posted: Sat Aug 11, 2012 2:13 am Reply with quote
User Joined: 10 Aug 2012 Posts: 654
than half a century, and a football team's worth of of Cheap Ralph Lauren Shirts of children and grandchildren."He died as he lived. He
hard until the end, stayed positive, thought only of others others replica Christian Louboutin others and constantly reminded everyone of how blessed his life
been. His ambitions were far reaching, but he never never discount designer *beep* never believed he had to leave this Happy Valley to
them. He was a man devoted to his family, family, [h4]designer replica *beep*[/h4] family, his university, his players and his community," Paterno's family
in a statement.While at Penn State's helm, Paterno, who was was [h2]replica designer bags for sale[/h2] was born in Brooklyn, N.Y., led the Nittany Lions to
undefeated seasons and two NCAA championships, had only five losing losing cheap Ralph Lauren Polo losing seasons, was inducted into the College Football Hall of
View user's profile Send private message

Display posts from previous:  

All times are GMT
Page 1 of 1
This forum is locked: you cannot post, reply to, or edit topics.

Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum