Working with Queues

From Zenoss Wiki
This is the approved revision of this page, as well as being the most recent.
Jump to: navigation, search

Zenoss has an event processing pipeline that uses AMQP messaging, and RabbitMQ in particular for the pipeline. This allows for quite a bit of transparency when it comes to identifying problems in the pipeline. This page describes some of tools that can be used to inspect and otherwise manage the queues.

Note: This only applies to Zenoss 4 and later.

rabbitmqctl

The rabbitmqctl command must be run as the root user. See the following examples for common usages.

Show all Zenoss queues. The -p parameter is used to specify the /zenoss vhost. The desired output is below, all these queues should exist, and all should be zero.

# rabbitmqctl -p /zenoss list_queues
Listing queues ...
celery  0
zenoss.queues.zep.signal        0
zenoss.queues.zep.modelchange   0
zenoss.queues.zep.migrated.summary      0
zenoss.queues.zep.rawevents     0
zenoss.queues.zep.heartbeats    0
zenoss.queues.zep.zenevents     0
zenoss424test.celeryd.pidbox    0
zenoss.queues.zep.migrated.archive      0
...done.

The following steps can be used to recreate all Zenoss exchanges and queues. The entire vhost is deleted and recreated. This can often be the fastest way to recover from seriously runaway queues when the data contained within the queues is not valuable.

$ zenoss stop
$ sudo su - # become root
# export VHOST="/zenoss"
# export USER="zenoss"
# export PASS="zenoss" # replace it with $amqppassword from $ZENHOME/etc/global.conf, if needed
# rabbitmqctl stop_app
# rabbitmqctl reset
# rabbitmqctl start_app
# rabbitmqctl add_vhost $VHOST
# rabbitmqctl add_user "$USER" "$PASS"
# rabbitmqctl set_permissions -p "$VHOST" "$USER" '.*' '.*' '.*'
# rabbitmqctl -p $VHOST list_queues # check if all queues exist
# exit # back to zenoss user
$ zenoss start

Old script version: https://github.com/Hackman238/Random/blob/master/resetRabbit.sh

zenqdump

zenqdump can be used to print the contents of messages in a queue. This tool automatically handles conversion from the native message format which is usually protobuf to something human-readable. This command, like all zen* commands must be run as the zenoss user.

$ zenqdump <queue_name>

zenq

zenq can be used for high-level management of Zenoss queues without resorting to using rabbitmqctl. It must be run as the zenoss user.

Count, purge or delete names queue(s). Note that it is safe to count or purge queues while Zenoss is running. It is only safe to delete queues when Zenoss is not running. The appropriate Zenoss processes will recreate the queues they need on startup.

$ zenq [count|purge|delete] <queue_name> [...]