Reliable Queue Pattern

messaging server is often used to implement processing of background jobs or other kinds of messaging tasks. A simple form of queue is often obtained pushing values into a queue in the producer side, and waiting for this values in the consumer side using POP (using polling), or BLOCKING_POP if the client is better served by a blocking operation.

However in this context the obtained queue is not reliable as messages can be lost, for example in the case there is a network problem or if the consumer crashes just after the message is received but it is still to process.

an atomic operation POP_AND_PUSH (or BLOCKING version for the blocking variant) offers a way to avoid this problem: the consumer fetches the message and at the same time pushes it into a processingqueue (note this is an atomic operation). It will use the REMOVE command in order to remove the message from the processing queue once the message has been processed.

An additional client may monitor the processing queue for items that remain there for too much time, and will push those timed out items into the queue again if needed.

posted @ 2015-07-29 18:21  ScottGu  阅读(157)  评论(0编辑  收藏  举报