| Author |
Message |
|
| Guest |
 Posted: Wed Dec 30, 2009 11:26 pm |
 |
|
|
Guest
|
| I want to set up 3 queues, High, Medium, and Low where I always pull messages from the higher priority queues first. |
|
|
| Back to top |
|
| Guest |
| Posted: Wed Dec 30, 2009 11:58 pm |
|
|
|
Guest
|
Weird.
In any case: Have you tried using basic.consume? While not directly
addressing your question, it may help.
alexis
On Wed, Dec 30, 2009 at 11:25 PM, Bryan Murphy <bmurphy1976@gmail.com> wrote:
> I want to set up 3 queues, High, Medium, and Low where I always pull
> messages from the higher priority queues first. |
|
|
| Back to top |
|
| Guest |
| Posted: Thu Dec 31, 2009 12:01 am |
|
|
|
Guest
|
|
| Back to top |
|
| Guest |
| Posted: Wed Jan 06, 2010 11:17 am |
|
|
|
Guest
|
Hi Bryan,
On Wed, Dec 30, 2009 at 05:25:57PM -0600, Bryan Murphy wrote:
> I want to set up 3 queues, High, Medium, and Low where I always pull
> messages from the higher priority queues first. Right now, I more or less
> do the following:
>
> while (true)
> {
> result = null;
>
> foreach (var queue in queues)
> {
> result = model.BasicGet(queue, false);
>
> if (result != null)
> break;
> }
>
> if (result != null)
> DoSomething(result);
>
> Thread.Sleep(15); // 15ms
> }
>
> Functionally this achieves what I'm trying to accomplish, however, the CPU
> usage of the RabbitMQ server goes through the roof once I spin up a few
> listeners.
Hmmm. Which version of Rabbit are you using. I've just done some maths
and this loop may be doing about 143 calls to basic.get per second
(assuming 2ms for each basic.get). I'm just wondering whether you're
using a version before 1.7.0 in which the queues are hibernating and
thawing too frequently, thus burning up a lot of CPU. On the other hand,
if this was the case, then I'd expect adding more listeners would
improve the situation, not make it worse.
> Is there another way I can accomplish the same thing that doesn't hit the
> RabbitMQ server so hard?
Yes. 3 channels, each one consumes from 1 queue. Then (assuming you're
using the Java client), use the QueueingConsumer, and translate your
foreach to the nextDelivery with a timeout of 0. Also set qos.prefetch
to 1.
Thus the logic is effectively pushed client side.
However, this is not quite the same. With this version, each listener
would buffer up to 1 message from the high queue, 1 from the medium
queue and 1 from the low queue. Thus some other listener could come
along, find that there are no messages at all to process, because
they're currently being buffered by the other listeners. Those listeners
will eventually get to those buffered messages when they finish
processing they're current message, get back to their loop, and find the
lower priority messages. So it could simply mean that lower priority
messages are ignored for longer than they otherwise would be.
Oh, and you'll probably want some logic so that if all three queues are
empty, you don't sit spinning.
Matthew
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post received from mailinglist |
|
|
| Back to top |
|
|
|
All times are GMT
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum
|
|
|
Digg It
Del.icio.us
Reddit
Facebook
Stumble Upon
Technorati
