Here is the Zabbix Agent configuration I use to keep track of the number of messages pending delivery and the total number of queues (this second parameter may not make sense for you if you don't create a lot of dynamic queues):
As you can see, these user parameters are parameterized: they take a single parameter being the virtual host path that you want to monitor. Note also that the zabbix group must be added to the non-password sudoers for rabbitmqctl.
With these parameters in place, you'll be able to build graphs and set alarms for your favorite RabbitMQ virtual hosts!
UPDATE 10-FEB-2010: Alexis Richardson has been kind enough to point towards an SNMP plug-in for RabbitMQ that has been very recently released on GitHub. I have added a few features to it, so be sure to check my fork too.
UPDATE 04-MAR-2010: I'm now using the SNMP plug-in for RabbitMQ in production instead of the above solution, which is way more efficient. The use case for the above would then be only when SNMP is not an option for you.