Erlang Mailing Lists

Author Message

<  RabbitMQ mailing list  ~  Persisting Data

Guest
Posted: Fri Dec 14, 2007 10:17 pm Reply with quote
Guest
Hi,

Sorry, I'm a little new to this mailing list thing. I posted
previously about durable queues, and I will
try to make my problem more precise.

I create a durable exchange/queue and send the queue persistent
messages. Then I restart my broker. How would I retrieve
those messages? I am trying to read the messages with
channel.basicGet, but they aren't in the queue. Is there
anything I'm missing? Mnesia or administrative stuff? I am connecting
to the same channel, I try passively declaring
the queue and exchange, but still there is nothing to read. I hope I
am not missing anything basic.

Thanks,
Varun

_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post recived from mailinglist
tonyg
Posted: Mon Dec 17, 2007 3:17 pm Reply with quote
User Joined: 07 Nov 2006 Posts: 199
Hi Varun,

Varun Samuel wrote:
> Sorry, I'm a little new to this mailing list thing. I posted
> previously about durable queues, and I will
> try to make my problem more precise.

Sorry for the delay in responding. I got half-way through replying to
your other mail on Friday and arrived this morning to find an open,
half-full mail composition window Smile

> I create a durable exchange/queue and send the queue persistent
> messages. Then I restart my broker. How would I retrieve
> those messages? I am trying to read the messages with
> channel.basicGet, but they aren't in the queue.

This is very strange - strange enough for me to double-check that the
scenario works for me.

1. create "durable" queue
2. post messages to it with deliveryMode = 2
3. restart broker
4. consume or basicGet from queue
5. messages (should, and do for me) arrive

If that basic scenario isn't working for you, there's something wrong -
perhaps you could email your code to me privately? I'd be happy to cast
an eye over it.

I'll reply to your other email separately - please do feel free to
forward my reply to the mailing list if you feel that's appropriate.

Regards,
Tony
--
[][][] Tony Garnock-Jones | Mob: +44 (0)7905 974 211
[][] LShift Ltd | Tel: +44 (0)20 7729 7060
[] [] http://www.lshift.net/ | Email: tonyg@lshift.net

_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post recived from mailinglist
View user's profile Send private message MSN Messenger
Guest
Posted: Tue Dec 25, 2007 5:25 am Reply with quote
Guest
Tony,

I posted a little while back about persisting data with durable
queues, which was giving me some problems.
I've got it working now, and I think the basic problem lay in my
approach.

I wrote a small program to create and send persistent messages to a
durable queue. Once it had sent the
messages it would try to read them immediately using basicGet. This
seems to be the problem. Once I had a separate
client try to read the messages, everything was fine. But one program
sending messages to the queue then
immediately trying to read them doesn't work. Just thought I'd post
in case anyone else was trying the same thing.
Thanks very much for your offer to read over my code though. If you'd
still like to see it, I can send it. Sorry it took a while
to post the resolution of my problem- me and my partner had to change
direction a bit for our project.

With regards to the other issue- reading the last n messages in a
queue, you wrote:

---

I see - it looks like you're interested in something more like a "last
value cache" than a queue.

The way durable queues are designed in AMQP is simply that the messages
in the queues are stored to disk, and so survive server restarts. They
are still consumed in the same way as non-persistent/non-durable
messages: that is, for each *queue*, a single message is delivered to a
single consumer, and if that consumer acknowledges the message, it will
not be delivered to other consumers.

To simulate a last-value cache, you could write a simple "replay"
service that keeps a log of recent chat messages, and sends them out
when asked by a new chat participant.

---

I wasn't sure if you meant this was done inside rabbitmq. I just
settled on sticking every sent message in a LIFO data
structure, outside of rabbitmq. Any client connecting to rabbitmq
would first get the messages from that

This is a very basic way to do it. I read a bit of the AMQP spec and
didn't see anything about doing it in there. Does the
method you describe make use of rabbitmq to log recent chat messages?

Thanks, and Happy Holidays!
Varun





> Hi Varun,
>
> Varun Samuel wrote:
>> Sorry, I'm a little new to this mailing list thing. I posted
>> previously about durable queues, and I will
>> try to make my problem more precise.
>
> Sorry for the delay in responding. I got half-way through replying to
> your other mail on Friday and arrived this morning to find an open,
> half-full mail composition window Smile
>
>> I create a durable exchange/queue and send the queue persistent
>> messages. Then I restart my broker. How would I retrieve
>> those messages? I am trying to read the messages with
>> channel.basicGet, but they aren't in the queue.
>
> This is very strange - strange enough for me to double-check that the
> scenario works for me.
>
> 1. create "durable" queue
> 2. post messages to it with deliveryMode = 2
> 3. restart broker
> 4. consume or basicGet from queue
> 5. messages (should, and do for me) arrive
>
> If that basic scenario isn't working for you, there's something
> wrong -
> perhaps you could email your code to me privately? I'd be happy to
> cast
> an eye over it.
>
> I'll reply to your other email separately - please do feel free to
> forward my reply to the mailing list if you feel that's appropriate.
>
> Regards,
> Tony
> --
> [][][] Tony Garnock-Jones | Mob: +44 (0)7905 974 211
> [][] LShift Ltd | Tel: +44 (0)20 7729 7060
> [] [] http://www.lshift.net/ | Email: tonyg@lshift.net


_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post recived from mailinglist
Guest
Posted: Tue Dec 25, 2007 10:54 am Reply with quote
Guest
Varun,

Varun Samuel wrote:
> I wrote a small program to create and send persistent messages to a
> durable queue. Once it had sent the messages it would try to read
> them immediately using basicGet. This seems to be the problem. Once I
> had a separate client try to read the messages, everything was fine.
> But one program sending messages to the queue then immediately trying
> to read them doesn't work.

basic.publish is an asynchronous operation. There is no way to tell when
the messages thus published will be available via basic.get. Whether you
perform the publish and get on the same or different connections does
not change that, but it does alter the timing, which means you can (but
are not guaranteed to) get different results.

> With regards to the other issue- reading the last n messages in a
> queue, you wrote:
>
> ---
> To simulate a last-value cache, you could write a simple "replay"
> service that keeps a log of recent chat messages, and sends them out
> when asked by a new chat participant.
> ---
>
> I wasn't sure if you meant this was done inside rabbitmq. I just
> settled on sticking every sent message in a LIFO data
> structure, outside of rabbitmq. Any client connecting to rabbitmq
> would first get the messages from that
>
> This is a very basic way to do it. I read a bit of the AMQP spec and
> didn't see anything about doing it in there. Does the
> method you describe make use of rabbitmq to log recent chat messages?

What you implemented is exactly what Tony suggested. There is no way to
do this inside AMQP 0-8/9. AMQP 0-10 will offer queue browsing, allowing
clients to read messages from a queue without consuming them. There have
also been some discussions in the working group about supporting last
value caches more directly.

> Thanks, and Happy Holidays!

Thank you, and the same to you.


Matthias.

_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss@lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Post recived from mailinglist

Display posts from previous:  

All times are GMT
Page 1 of 1
This forum is locked: you cannot post, reply to, or edit topics.

Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum