| Author |
Message |
|
| Guest |
 Posted: Wed Feb 20, 2008 7:44 am |
 |
|
|
Guest
|
I see that RabbitMQ does not yet have replication for message queues. I need a high-available message queue that never loses messages. I'm not terribly concerned about performance, as I just need to process several thousand messages per day. It is just hard to repair consistency if any are lost.
I thought about using DRDB to replicate RabbitMQ's on-disk Mnesia store to a standby node. I've used DRDB successfully for MySQL, and it works well. And then use Hearbeat to start RabbitMQ on the standby node, which would then recover the Mnesia tables, and start running.
Anyone tried DRDB, or any other block mirroring system to create a cluster layer beneath RabbitMQ?
Tom
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post recived from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Wed Feb 20, 2008 2:39 pm |
|
|
|
Guest
|
Hi Tom,
Tom Samplonius wrote:
> I see that RabbitMQ does not yet have replication for message queues. I need a high-available message queue that never loses messages. I'm not terribly concerned about performance, as I just need to process several thousand messages per day. It is just hard to repair consistency if any are lost.
I am also in a very similar situation - HA is more important for my project than
throughput. It is my understanding that if you define your queues as durable and send
messages as persistent, your messages will be saved on disk until consumed. A bigger
question is how long it will take between a message is sent and the message is consumed.
Out of the box, I guess it depends on how fast you can restore the original broker (app,
system and/or network connectivity, depending on what failed).
> I thought about using DRDB to replicate RabbitMQ's on-disk Mnesia store to a standby node. I've used DRDB successfully for MySQL, and it works well. And then use Hearbeat to start RabbitMQ on the standby node, which would then recover the Mnesia tables, and start running.
So you are planning to produce to a broker and consume from the same broker. And once that
broker fails, you will have both producers and consumers shift to another broker, which
you will start on DRBD-cloned disk after the first broker failed?
I am currently leaning towards a cluster of rabbitmq brokers. Producers will send messages
to one broker, consumers will connect to *all* brokers in a cluster and will have
application logic to identify duplicate messages (just in case). Producers will get an
exception if their broker failed to properly receive their message and will reconnect and
resend (tested with QPid python, planning to switch py-amqplib shortly). Consumers might
not get an exception about broker failure soon enough, therefore for now I will have them
consuming from all brokers in the cluster.
I hope that latter approach will work better for me, since in my case I might have trouble
convincing all consumers at the same time that one particular broker is not available
(some of my consumers are over LAN, some are over WAN; if I lose WAN access, consumers on
the WAN will think the broker is gone, while consumers on LAN will still be able to talk
to it).
Please let us know if you get to deploy rabbitmq with DRBD-backed storage and how it works
for you.
Thanks,
Dmitriy
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post recived from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Fri Feb 22, 2008 3:30 am |
|
|
|
Guest
|
----- "Dmitriy Samovskiy" <dmitriy.samovskiy@cohesiveft.com> wrote:
> ... A bigger
> question is how long it will take between a message is sent and the
> message is consumed.
> Out of the box, I guess it depends on how fast you can restore the
> original broker (app,
> system and/or network connectivity, depending on what failed).
Yes, only you will be able to answer that. In my case, the time to fix a unresponsive server is about an hour.
> > I thought about using DRDB to replicate RabbitMQ's on-disk Mnesia
> store to a standby node. I've used DRDB successfully for MySQL, and
> it works well. And then use Hearbeat to start RabbitMQ on the standby
> node, which would then recover the Mnesia tables, and start running.
> So you are planning to produce to a broker and consume from the same
> broker. And once that
> broker fails, you will have both producers and consumers shift to
> another broker, which
> you will start on DRBD-cloned disk after the first broker failed?
Yes. DRDB keeps the master disk synced to the slave disk. Hearbeat determines if the master fails, and switches the direction of the syncing, and can also move the virtual IP from the master to the slave. Everything just connects to the virtual IP. Obviously clients (producers and consumers) need to be robust, and have the ability to recovery from timeouts, and reconnect.
...
> application logic to identify duplicate messages (just in case).
...
I've never come across a good way of doing this. I can attach a unique id to each message, but then the clients have to keep state on which ids they've processed. And if you multiple clients, they need to talk to each other too. I don't want consumers to have to keep state.
Now depending on the message, of course, processing it twice may not matter that much. For example, if the message is "Turn switch 2 On", then if you do that twice, it is harmless. But obviously you don't want to processing every message twice just for high-availability.
...
> Please let us know if you get to deploy rabbitmq with DRBD-backed
> storage and how it works
> for you.
I have to do a lot of testing of this configuration first.
> Thanks,
> Dmitriy
Tom
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post recived from mailinglist |
|
|
| Back to top |
|
| tonyg |
| Posted: Thu Feb 28, 2008 8:16 am |
|
|
|
User
Joined: 07 Nov 2006
Posts: 199
|
Hi Tom,
Tom Samplonius wrote:
> I see that RabbitMQ does not yet have replication for message queues.
You're right. One of the items we're discussing informally at the moment
here at LShift is the best way to refactor the codebase to support
pluggable queue backends. Once that's done, it'll be easy to support the
existing fast journal-based implementation alongside such as
- a mnesia-backed queue
- an ODBC-backed queue
- other?
> I need a high-available message queue that never loses messages. I'm not
> terribly concerned about performance, as I just need to process several
> thousand messages per day. It is just hard to repair consistency if any
> are lost.
Understood.
> Anyone tried DRDB, or any other block mirroring system to create a
> cluster layer beneath RabbitMQ?
That's a *really* interesting idea. Rabbit's message store is in fact
not in Mnesia (otherwise it'd already be replicated!  ), but is in a
separate disk_log-based journal called "rabbit_persister.LOG". From
casual investigation of DRBD it seems that your approach could work well.
The more I think about it the more I like the idea. If you try it,
please share your results with us. If I get time (!) I'll have a stab at
it myself, but I'm afraid that's not likely for the next few weeks.
Regards,
Tony
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post recived from mailinglist |
|
|
| Back to top |
  
|
| wuji |
| Posted: Wed Sep 19, 2012 3:55 am |
|
|
|
User
Joined: 10 Aug 2012
Posts: 654
|
Washington D.C., State Department spokesman Victoria Nuland acknowledged the case case cheap designer *beep* case of Cao Ruyi."We've seen the reports that a Chinese
is being detained and possibly pressured into a forced abortion abortion [h3]cheap jordan shoes[/h3] abortion by Chinese family planning authorities after purportedly violating China's
policy," she told reporters during a press briefing. "We have have [h3]cheap replica *beep*[/h3] have reached out to the authorities in Beijing to ask
this issue."Nuland reiterated that the U.S. strongly opposes "all aspects aspects cheap replica designer *beep* aspects of China's coercive birth limitation policies," which they have
a serious human rights abuse.Groups from around the world became became replica designer *beep* became involved, including organization such as All Girls Allowed and
Rights in China.Rep. Chris Smith, R-N.J., sent a letter to to jordan 11 to local Changsha officials. The Texas-based group ChinaAid published his |
|
|
| Back to top |
|
| mbtshoes88 |
| Posted: Wed Sep 19, 2012 7:50 am |
|
|
|
User
Joined: 18 Sep 2012
Posts: 30
|
| Combine it with black or brown Cheap Armani Watches shoes, even white if you want to look different. Replica Armani Watches Standard blue generally means navy blue and Emporio Armani not any other paler shades of blue that will look odd. Classic Gray: Can be worn for different occasions and this will make even ladies turn their heads to notice you. Grays also look imposing Cheap Armani Shoes with different stripes and patterns classic gray suits are the hallmark of a gentleman. Cheap Georgio Armani Basic Black: The favorite suit of many well-dressed men and goes well Armani Online with a good white shirt. If you Armani Outlet can afford only one suit, Armani Suits then prefer this versatile Basis Black. As a fourth choice, you can select any of the above three with pinstripes. Cheap Armani Online Men Cheap Armani of average physical build Cheap Emporio Armani can choose either single-breasted or double-breasted Georgio Armani jackets. Double-breasted jackets are intended for slim men so that their leanness is not too visible and single-breasted jackets are meant for the stout to somewhat hide their obesity. Choose the jacket style Armani Sale that best suits your build. fmzds120823 The jackets must be of perfect fit and cause no wrinkles when you button them up. As far as possible, shoulder pads should be avoided and your tailor should guide you in Cheap Armani Suits this regard. You must now choose a trouser style although most men do not attach much Cheap Amarni EA7 importance to trousers and accept any pants that come with a jacket. When you discuss trousers, you need to worry about waist, drape, belt-loops, posterior-hugging, crotch-dangling, and whether or not you need a watch pocket. The best way to judge a pair of pants is to ask “Will I be able to wear these pants without the jacket and still look fine?” If the answer is Cheap Armani Outlet yes, then the pant is of perfect style and fit. Sometimes it is good to heed to the advice of a salesman and buy two pants for the same jacket. The idea is you can alternate pants with the jacket so they wear evenly over time. There may be occasions you may wear the pants minus Armani Shirts the jacket, it Cheap Armani Sale is always prudent to have a back-up pair. |
|
|
| Back to top |
|
|
|
All times are GMT
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum
|
|
|
Digg It
Del.icio.us
Reddit
Facebook
Stumble Upon
Technorati
