A RabbitMQ node is inactive

Description

This alert monitors the availability of a RabbitMQ node and triggers when a node becomes inactive, indicating potential messaging disruptions.

Resolution Guidance

Impact When Active

  • Document conversion jobs will fail, preventing users from viewing documents in the Relativity Viewer.
  • A backlog of unprocessed messages will build up in the RabbitMQ queue, increasing system load and delaying job processing.
  • Downstream systems and workflows that depend on conversion results (e.g., document previewing or review workflows) may also be affected.

How To Resolve

Complete the following steps if Agent is unable to connect to RabbitMQ:

  • Verify the RabbitMQ server and/or cluster is running by logging into management UI and verifying each expected server is present and displaying as green under nodes. For more information, see RabbitMQ management UI.
  • On the Overview tab in the management UI, ensure RabbitMQ is listening on the expected ports:
  • Verify RabbitMQ can be reached by navigating to the management page from the affected server(s).
  • Additionally, ensure the settings do not contain any trailing or leading whitespace, and ensure the casing matches for RabbitMQ's SharedAccessKey, SharedAccessKeyName, and ServiceNamespace settings.
  • Ensure the RabbitMQ user being used for authentication has full permission for the configured virtual host, see Configuring RabbitMQ

Alert Details

Alert Condition Details

Name Value Description
Rule Type Elastic Query
Data View metrics-*
Filter Query labels.rabbitmq_node_name : * If the RabbitMQ node exist in the last 45 seconds
Group Count Number of RabbitMQ node exist
Threshold <= 0 If the count is lest than or equal to 0, alert triggers
Time Window Last 45 sec Verified data for last 45 seconds
Frequency 30 sec Checks every 30 sec

Alert Metric Details

Metric Name: labels.rabbitmq_node_name

Metric Description: RabbitMQ node name

Metric Attributes:

Attribute Name Description Value
labels.rabbitmq_queue_name The name of the RabbitMQ queue.
labels.rabbitmq_vhost_name The name of the RabbitMQ vHost.