2026-03-05 02:05:29,390 - INFO - Using existing project ['default-domain', 'admin'](2ddec3c9-d075-47ef-bbd7-70ab863c6a6f)
2026-03-05 02:05:29,398 - INFO - ================================================================================
2026-03-05 02:05:29,398 - INFO - STARTING TEST    : test_contrail_status
2026-03-05 02:05:29,399 - INFO - TEST DESCRIPTION :  Test to verify that all services are running and active

        
2026-03-05 02:05:30,038 - DEBUG - Nothing to compare xmpp stats {'10.0.0.141': {'10.20.0.254': '0', '10.20.0.14': '0'}, '10.0.0.73': {'10.20.0.14': '0', '10.20.0.254': '0'}} with
2026-03-05 02:05:30,038 - INFO - Initial checks done. Running the testcase now
2026-03-05 02:05:30,038 - INFO - 
2026-03-05 02:05:30,130 - INFO - 10.0.0.65
2026-03-05 02:05:30,130 - INFO -   config
2026-03-05 02:05:30,135 - INFO -     config-nodemgr:active
2026-03-05 02:05:30,140 - INFO -     api-server:initializing (Generic Connection:Keystone[Error: Unable to establish connection to http://10.0.0.65:5000/v3/domains? at UTC 2026-03-05 02:04:55.957495] connection down)
2026-03-05 02:05:30,144 - INFO -     schema:active
2026-03-05 02:05:30,149 - INFO -     svc-monitor:active
2026-03-05 02:05:30,154 - INFO -     device-manager:active
2026-03-05 02:05:30,154 - INFO -   config-database
2026-03-05 02:05:30,154 - INFO -     config-cassandra:active
2026-03-05 02:05:30,154 - INFO -     config-zookeeper:active
2026-03-05 02:05:30,154 - INFO -     config-rabbitmq:active
2026-03-05 02:05:30,154 - INFO -   control
2026-03-05 02:05:30,161 - INFO -     control-nodemgr:active
2026-03-05 02:05:30,168 - INFO -     control:active
2026-03-05 02:05:30,168 - INFO -     named:active
2026-03-05 02:05:30,175 - INFO -     dns:active
2026-03-05 02:05:30,176 - INFO -   analytics
2026-03-05 02:05:30,183 - INFO -     analytics-nodemgr:active
2026-03-05 02:05:30,189 - INFO -     analytics-api:active
2026-03-05 02:05:30,194 - INFO -     collector:active
2026-03-05 02:05:30,194 - INFO -   analytics-database
2026-03-05 02:05:30,195 - INFO -     analytics-cassandra:active
2026-03-05 02:05:30,202 - INFO -     analyticsdb-nodemgr:active
2026-03-05 02:05:30,209 - INFO -     query-engine:active
2026-03-05 02:05:30,209 - INFO -   webui
2026-03-05 02:05:30,209 - INFO -     webui:active
2026-03-05 02:05:30,209 - INFO -     webui-middleware:active
2026-03-05 02:05:30,209 - INFO -     redis:active
2026-03-05 02:05:30,209 - INFO -   analytics_snmp
2026-03-05 02:05:30,214 - INFO -     snmp-collector:active
2026-03-05 02:05:30,214 - INFO -     snmp-topology:active
2026-03-05 02:05:30,214 - INFO -   analytics_alarm
2026-03-05 02:05:30,214 - INFO -     alarmgen:active
2026-03-05 02:05:30,260 - INFO - 10.0.0.33
2026-03-05 02:05:30,261 - INFO -   config
2026-03-05 02:05:30,268 - INFO -     config-nodemgr:active
2026-03-05 02:05:30,276 - INFO -     api-server:active
2026-03-05 02:05:30,282 - INFO -     schema:backup
2026-03-05 02:05:30,286 - INFO -     svc-monitor:backup
2026-03-05 02:05:30,289 - INFO -     device-manager:backup
2026-03-05 02:05:30,289 - INFO -   config-database
2026-03-05 02:05:30,290 - INFO -     config-cassandra:active
2026-03-05 02:05:30,290 - INFO -     config-zookeeper:active
2026-03-05 02:05:30,290 - INFO -     config-rabbitmq:active
2026-03-05 02:05:30,290 - INFO -   control
2026-03-05 02:05:30,295 - INFO -     control-nodemgr:active
2026-03-05 02:05:30,300 - INFO -     control:active
2026-03-05 02:05:30,300 - INFO -     named:active
2026-03-05 02:05:30,305 - INFO -     dns:active
2026-03-05 02:05:30,306 - INFO -   analytics
2026-03-05 02:05:30,310 - INFO -     analytics-nodemgr:active
2026-03-05 02:05:30,314 - INFO -     analytics-api:active
2026-03-05 02:05:30,319 - INFO -     collector:active
2026-03-05 02:05:30,319 - INFO -   analytics-database
2026-03-05 02:05:30,319 - INFO -     analytics-cassandra:active
2026-03-05 02:05:30,324 - INFO -     analyticsdb-nodemgr:active
2026-03-05 02:05:30,330 - INFO -     query-engine:active
2026-03-05 02:05:30,330 - INFO -   webui
2026-03-05 02:05:30,331 - INFO -     webui:active
2026-03-05 02:05:30,331 - INFO -     webui-middleware:active
2026-03-05 02:05:30,331 - INFO -     redis:active
2026-03-05 02:05:30,332 - INFO -   analytics_snmp
2026-03-05 02:05:30,336 - INFO -     snmp-collector:active
2026-03-05 02:05:30,337 - INFO -     snmp-topology:active
2026-03-05 02:05:30,337 - INFO -   analytics_alarm
2026-03-05 02:05:30,338 - INFO -     alarmgen:active
2026-03-05 02:05:30,393 - INFO - 10.0.0.133
2026-03-05 02:05:30,394 - INFO -   config
2026-03-05 02:05:30,401 - INFO -     config-nodemgr:active
2026-03-05 02:05:30,407 - INFO -     api-server:active
2026-03-05 02:05:30,412 - INFO -     schema:backup
2026-03-05 02:05:30,416 - INFO -     svc-monitor:backup
2026-03-05 02:05:30,419 - INFO -     device-manager:backup
2026-03-05 02:05:30,419 - INFO -   config-database
2026-03-05 02:05:30,420 - INFO -     config-cassandra:active
2026-03-05 02:05:30,420 - INFO -     config-zookeeper:active
2026-03-05 02:05:30,420 - INFO -     config-rabbitmq:active
2026-03-05 02:05:30,421 - INFO -   control
2026-03-05 02:05:30,426 - INFO -     control-nodemgr:active
2026-03-05 02:05:30,431 - INFO -     control:active
2026-03-05 02:05:30,431 - INFO -     named:active
2026-03-05 02:05:30,436 - INFO -     dns:active
2026-03-05 02:05:30,437 - INFO -   analytics
2026-03-05 02:05:30,442 - INFO -     analytics-nodemgr:active
2026-03-05 02:05:30,446 - INFO -     analytics-api:active
2026-03-05 02:05:30,451 - INFO -     collector:active
2026-03-05 02:05:30,451 - INFO -   analytics-database
2026-03-05 02:05:30,451 - INFO -     analytics-cassandra:active
2026-03-05 02:05:30,456 - INFO -     analyticsdb-nodemgr:active
2026-03-05 02:05:30,462 - INFO -     query-engine:active
2026-03-05 02:05:30,462 - INFO -   webui
2026-03-05 02:05:30,462 - INFO -     webui:active
2026-03-05 02:05:30,462 - INFO -     webui-middleware:active
2026-03-05 02:05:30,462 - INFO -     redis:active
2026-03-05 02:05:30,462 - INFO -   analytics_snmp
2026-03-05 02:05:30,466 - INFO -     snmp-collector:active
2026-03-05 02:05:30,466 - INFO -     snmp-topology:active
2026-03-05 02:05:30,466 - INFO -   analytics_alarm
2026-03-05 02:05:30,466 - INFO -     alarmgen:active
2026-03-05 02:05:30,522 - INFO - 10.0.0.141
2026-03-05 02:05:30,522 - INFO -   vrouter
2026-03-05 02:05:30,529 - INFO -     vrouter-nodemgr:active
2026-03-05 02:05:30,534 - INFO -     agent:active
2026-03-05 02:05:30,589 - INFO - 10.0.0.73
2026-03-05 02:05:30,589 - INFO -   vrouter
2026-03-05 02:05:30,594 - INFO -     vrouter-nodemgr:active
2026-03-05 02:05:30,599 - INFO -     agent:active
2026-03-05 02:05:30,600 - DEBUG - defaultdict(<class 'dict'>, {'10.0.0.65': {'api-server': {'status': 'initializing', 'description': 'Generic Connection:Keystone[Error: Unable to establish connection to http://10.0.0.65:5000/v3/domains? at UTC 2026-03-05 02:04:55.957495] connection down'}}})
2026-03-05 02:05:30,600 - DEBUG - Not all services up. Sleeping for 10 seconds. iteration: 0
2026-03-05 02:05:40,666 - INFO - 10.0.0.65
2026-03-05 02:05:40,667 - INFO -   config
2026-03-05 02:05:40,672 - INFO -     config-nodemgr:active
2026-03-05 02:05:40,677 - INFO -     api-server:initializing (Generic Connection:Keystone[Error: Unable to establish connection to http://10.0.0.65:5000/v3/domains? at UTC 2026-03-05 02:04:55.957495] connection down)
2026-03-05 02:05:40,681 - INFO -     schema:active
2026-03-05 02:05:40,685 - INFO -     svc-monitor:active
2026-03-05 02:05:40,689 - INFO -     device-manager:active
2026-03-05 02:05:40,689 - INFO -   config-database
2026-03-05 02:05:40,689 - INFO -     config-cassandra:active
2026-03-05 02:05:40,689 - INFO -     config-zookeeper:active
2026-03-05 02:05:40,689 - INFO -     config-rabbitmq:active
2026-03-05 02:05:40,689 - INFO -   control
2026-03-05 02:05:40,693 - INFO -     control-nodemgr:active
2026-03-05 02:05:40,700 - INFO -     control:active
2026-03-05 02:05:40,700 - INFO -     named:active
2026-03-05 02:05:40,706 - INFO -     dns:active
2026-03-05 02:05:40,706 - INFO -   analytics
2026-03-05 02:05:40,711 - INFO -     analytics-nodemgr:active
2026-03-05 02:05:40,715 - INFO -     analytics-api:active
2026-03-05 02:05:40,720 - INFO -     collector:active
2026-03-05 02:05:40,720 - INFO -   analytics-database
2026-03-05 02:05:40,720 - INFO -     analytics-cassandra:active
2026-03-05 02:05:40,725 - INFO -     analyticsdb-nodemgr:active
2026-03-05 02:05:40,731 - INFO -     query-engine:active
2026-03-05 02:05:40,731 - INFO -   webui
2026-03-05 02:05:40,731 - INFO -     webui:active
2026-03-05 02:05:40,731 - INFO -     webui-middleware:active
2026-03-05 02:05:40,731 - INFO -     redis:active
2026-03-05 02:05:40,731 - INFO -   analytics_snmp
2026-03-05 02:05:40,735 - INFO -     snmp-collector:active
2026-03-05 02:05:40,735 - INFO -     snmp-topology:active
2026-03-05 02:05:40,735 - INFO -   analytics_alarm
2026-03-05 02:05:40,735 - INFO -     alarmgen:active
2026-03-05 02:05:40,791 - INFO - 10.0.0.33
2026-03-05 02:05:40,791 - INFO -   config
2026-03-05 02:05:40,797 - INFO -     config-nodemgr:active
2026-03-05 02:05:40,801 - INFO -     api-server:active
2026-03-05 02:05:40,805 - INFO -     schema:backup
2026-03-05 02:05:40,808 - INFO -     svc-monitor:backup
2026-03-05 02:05:40,816 - INFO -     device-manager:backup
2026-03-05 02:05:40,816 - INFO -   config-database
2026-03-05 02:05:40,816 - INFO -     config-cassandra:active
2026-03-05 02:05:40,816 - INFO -     config-zookeeper:active
2026-03-05 02:05:40,816 - INFO -     config-rabbitmq:active
2026-03-05 02:05:40,816 - INFO -   control
2026-03-05 02:05:40,821 - INFO -     control-nodemgr:active
2026-03-05 02:05:40,826 - INFO -     control:active
2026-03-05 02:05:40,826 - INFO -     named:active
2026-03-05 02:05:40,831 - INFO -     dns:active
2026-03-05 02:05:40,831 - INFO -   analytics
2026-03-05 02:05:40,835 - INFO -     analytics-nodemgr:active
2026-03-05 02:05:40,840 - INFO -     analytics-api:active
2026-03-05 02:05:40,845 - INFO -     collector:active
2026-03-05 02:05:40,845 - INFO -   analytics-database
2026-03-05 02:05:40,845 - INFO -     analytics-cassandra:active
2026-03-05 02:05:40,850 - INFO -     analyticsdb-nodemgr:active
2026-03-05 02:05:40,855 - INFO -     query-engine:active
2026-03-05 02:05:40,855 - INFO -   webui
2026-03-05 02:05:40,856 - INFO -     webui:active
2026-03-05 02:05:40,856 - INFO -     webui-middleware:active
2026-03-05 02:05:40,856 - INFO -     redis:active
2026-03-05 02:05:40,856 - INFO -   analytics_snmp
2026-03-05 02:05:40,861 - INFO -     snmp-collector:active
2026-03-05 02:05:40,861 - INFO -     snmp-topology:active
2026-03-05 02:05:40,861 - INFO -   analytics_alarm
2026-03-05 02:05:40,861 - INFO -     alarmgen:active
2026-03-05 02:05:40,917 - INFO - 10.0.0.133
2026-03-05 02:05:40,917 - INFO -   config
2026-03-05 02:05:40,922 - INFO -     config-nodemgr:active
2026-03-05 02:05:40,926 - INFO -     api-server:active
2026-03-05 02:05:40,930 - INFO -     schema:backup
2026-03-05 02:05:40,933 - INFO -     svc-monitor:backup
2026-03-05 02:05:40,936 - INFO -     device-manager:backup
2026-03-05 02:05:40,936 - INFO -   config-database
2026-03-05 02:05:40,936 - INFO -     config-cassandra:active
2026-03-05 02:05:40,936 - INFO -     config-zookeeper:active
2026-03-05 02:05:40,936 - INFO -     config-rabbitmq:active
2026-03-05 02:05:40,936 - INFO -   control
2026-03-05 02:05:40,941 - INFO -     control-nodemgr:active
2026-03-05 02:05:40,946 - INFO -     control:active
2026-03-05 02:05:40,946 - INFO -     named:active
2026-03-05 02:05:40,951 - INFO -     dns:active
2026-03-05 02:05:40,951 - INFO -   analytics
2026-03-05 02:05:40,956 - INFO -     analytics-nodemgr:active
2026-03-05 02:05:40,960 - INFO -     analytics-api:active
2026-03-05 02:05:40,965 - INFO -     collector:active
2026-03-05 02:05:40,966 - INFO -   analytics-database
2026-03-05 02:05:40,966 - INFO -     analytics-cassandra:active
2026-03-05 02:05:40,971 - INFO -     analyticsdb-nodemgr:active
2026-03-05 02:05:40,976 - INFO -     query-engine:active
2026-03-05 02:05:40,976 - INFO -   webui
2026-03-05 02:05:40,977 - INFO -     webui:active
2026-03-05 02:05:40,977 - INFO -     webui-middleware:active
2026-03-05 02:05:40,977 - INFO -     redis:active
2026-03-05 02:05:40,977 - INFO -   analytics_snmp
2026-03-05 02:05:40,981 - INFO -     snmp-collector:active
2026-03-05 02:05:40,982 - INFO -     snmp-topology:active
2026-03-05 02:05:40,982 - INFO -   analytics_alarm
2026-03-05 02:05:40,982 - INFO -     alarmgen:active
2026-03-05 02:05:41,039 - INFO - 10.0.0.141
2026-03-05 02:05:41,039 - INFO -   vrouter
2026-03-05 02:05:41,045 - INFO -     vrouter-nodemgr:active
2026-03-05 02:05:41,051 - INFO -     agent:active
2026-03-05 02:05:41,107 - INFO - 10.0.0.73
2026-03-05 02:05:41,107 - INFO -   vrouter
2026-03-05 02:05:41,112 - INFO -     vrouter-nodemgr:active
2026-03-05 02:05:41,117 - INFO -     agent:active
2026-03-05 02:05:41,118 - DEBUG - defaultdict(<class 'dict'>, {'10.0.0.65': {'api-server': {'status': 'initializing', 'description': 'Generic Connection:Keystone[Error: Unable to establish connection to http://10.0.0.65:5000/v3/domains? at UTC 2026-03-05 02:04:55.957495] connection down'}}})
2026-03-05 02:05:41,118 - DEBUG - Not all services up. Sleeping for 10 seconds. iteration: 1
2026-03-05 02:05:51,187 - INFO - 10.0.0.65
2026-03-05 02:05:51,187 - INFO -   config
2026-03-05 02:05:51,191 - INFO -     config-nodemgr:active
2026-03-05 02:05:51,195 - INFO -     api-server:initializing (Generic Connection:Keystone[Error: Unable to establish connection to http://10.0.0.65:5000/v3/domains? at UTC 2026-03-05 02:04:55.957495] connection down)
2026-03-05 02:05:51,199 - INFO -     schema:active
2026-03-05 02:05:51,203 - INFO -     svc-monitor:active
2026-03-05 02:05:51,206 - INFO -     device-manager:active
2026-03-05 02:05:51,206 - INFO -   config-database
2026-03-05 02:05:51,207 - INFO -     config-cassandra:active
2026-03-05 02:05:51,207 - INFO -     config-zookeeper:active
2026-03-05 02:05:51,207 - INFO -     config-rabbitmq:active
2026-03-05 02:05:51,207 - INFO -   control
2026-03-05 02:05:51,212 - INFO -     control-nodemgr:active
2026-03-05 02:05:51,218 - INFO -     control:active
2026-03-05 02:05:51,218 - INFO -     named:active
2026-03-05 02:05:51,223 - INFO -     dns:active
2026-03-05 02:05:51,224 - INFO -   analytics
2026-03-05 02:05:51,228 - INFO -     analytics-nodemgr:active
2026-03-05 02:05:51,232 - INFO -     analytics-api:active
2026-03-05 02:05:51,237 - INFO -     collector:active
2026-03-05 02:05:51,238 - INFO -   analytics-database
2026-03-05 02:05:51,238 - INFO -     analytics-cassandra:active
2026-03-05 02:05:51,242 - INFO -     analyticsdb-nodemgr:active
2026-03-05 02:05:51,250 - INFO -     query-engine:active
2026-03-05 02:05:51,250 - INFO -   webui
2026-03-05 02:05:51,251 - INFO -     webui:active
2026-03-05 02:05:51,251 - INFO -     webui-middleware:active
2026-03-05 02:05:51,251 - INFO -     redis:active
2026-03-05 02:05:51,251 - INFO -   analytics_snmp
2026-03-05 02:05:51,255 - INFO -     snmp-collector:active
2026-03-05 02:05:51,255 - INFO -     snmp-topology:active
2026-03-05 02:05:51,256 - INFO -   analytics_alarm
2026-03-05 02:05:51,256 - INFO -     alarmgen:active
2026-03-05 02:05:51,307 - INFO - 10.0.0.33
2026-03-05 02:05:51,307 - INFO -   config
2026-03-05 02:05:51,312 - INFO -     config-nodemgr:active
2026-03-05 02:05:51,316 - INFO -     api-server:active
2026-03-05 02:05:51,319 - INFO -     schema:backup
2026-03-05 02:05:51,323 - INFO -     svc-monitor:backup
2026-03-05 02:05:51,326 - INFO -     device-manager:backup
2026-03-05 02:05:51,326 - INFO -   config-database
2026-03-05 02:05:51,326 - INFO -     config-cassandra:active
2026-03-05 02:05:51,326 - INFO -     config-zookeeper:active
2026-03-05 02:05:51,326 - INFO -     config-rabbitmq:active
2026-03-05 02:05:51,326 - INFO -   control
2026-03-05 02:05:51,331 - INFO -     control-nodemgr:active
2026-03-05 02:05:51,335 - INFO -     control:active
2026-03-05 02:05:51,336 - INFO -     named:active
2026-03-05 02:05:51,340 - INFO -     dns:active
2026-03-05 02:05:51,340 - INFO -   analytics
2026-03-05 02:05:51,345 - INFO -     analytics-nodemgr:active
2026-03-05 02:05:51,349 - INFO -     analytics-api:active
2026-03-05 02:05:51,353 - INFO -     collector:active
2026-03-05 02:05:51,353 - INFO -   analytics-database
2026-03-05 02:05:51,353 - INFO -     analytics-cassandra:active
2026-03-05 02:05:51,357 - INFO -     analyticsdb-nodemgr:active
2026-03-05 02:05:51,362 - INFO -     query-engine:active
2026-03-05 02:05:51,362 - INFO -   webui
2026-03-05 02:05:51,362 - INFO -     webui:active
2026-03-05 02:05:51,362 - INFO -     webui-middleware:active
2026-03-05 02:05:51,362 - INFO -     redis:active
2026-03-05 02:05:51,362 - INFO -   analytics_snmp
2026-03-05 02:05:51,366 - INFO -     snmp-collector:active
2026-03-05 02:05:51,366 - INFO -     snmp-topology:active
2026-03-05 02:05:51,366 - INFO -   analytics_alarm
2026-03-05 02:05:51,366 - INFO -     alarmgen:active
2026-03-05 02:05:51,411 - INFO - 10.0.0.133
2026-03-05 02:05:51,411 - INFO -   config
2026-03-05 02:05:51,417 - INFO -     config-nodemgr:active
2026-03-05 02:05:51,421 - INFO -     api-server:active
2026-03-05 02:05:51,425 - INFO -     schema:backup
2026-03-05 02:05:51,429 - INFO -     svc-monitor:backup
2026-03-05 02:05:51,432 - INFO -     device-manager:backup
2026-03-05 02:05:51,432 - INFO -   config-database
2026-03-05 02:05:51,432 - INFO -     config-cassandra:active
2026-03-05 02:05:51,432 - INFO -     config-zookeeper:active
2026-03-05 02:05:51,432 - INFO -     config-rabbitmq:active
2026-03-05 02:05:51,432 - INFO -   control
2026-03-05 02:05:51,437 - INFO -     control-nodemgr:active
2026-03-05 02:05:51,442 - INFO -     control:active
2026-03-05 02:05:51,442 - INFO -     named:active
2026-03-05 02:05:51,447 - INFO -     dns:active
2026-03-05 02:05:51,447 - INFO -   analytics
2026-03-05 02:05:51,451 - INFO -     analytics-nodemgr:active
2026-03-05 02:05:51,455 - INFO -     analytics-api:active
2026-03-05 02:05:51,460 - INFO -     collector:active
2026-03-05 02:05:51,460 - INFO -   analytics-database
2026-03-05 02:05:51,460 - INFO -     analytics-cassandra:active
2026-03-05 02:05:51,464 - INFO -     analyticsdb-nodemgr:active
2026-03-05 02:05:51,471 - INFO -     query-engine:active
2026-03-05 02:05:51,471 - INFO -   webui
2026-03-05 02:05:51,471 - INFO -     webui:active
2026-03-05 02:05:51,471 - INFO -     webui-middleware:active
2026-03-05 02:05:51,471 - INFO -     redis:active
2026-03-05 02:05:51,471 - INFO -   analytics_snmp
2026-03-05 02:05:51,475 - INFO -     snmp-collector:active
2026-03-05 02:05:51,476 - INFO -     snmp-topology:active
2026-03-05 02:05:51,476 - INFO -   analytics_alarm
2026-03-05 02:05:51,476 - INFO -     alarmgen:active
2026-03-05 02:05:51,534 - INFO - 10.0.0.141
2026-03-05 02:05:51,535 - INFO -   vrouter
2026-03-05 02:05:51,540 - INFO -     vrouter-nodemgr:active
2026-03-05 02:05:51,547 - INFO -     agent:active
2026-03-05 02:05:51,601 - INFO - 10.0.0.73
2026-03-05 02:05:51,601 - INFO -   vrouter
2026-03-05 02:05:51,607 - INFO -     vrouter-nodemgr:active
2026-03-05 02:05:51,611 - INFO -     agent:active
2026-03-05 02:05:51,612 - DEBUG - defaultdict(<class 'dict'>, {'10.0.0.65': {'api-server': {'status': 'initializing', 'description': 'Generic Connection:Keystone[Error: Unable to establish connection to http://10.0.0.65:5000/v3/domains? at UTC 2026-03-05 02:04:55.957495] connection down'}}})
2026-03-05 02:05:51,612 - DEBUG - Not all services up. Sleeping for 10 seconds. iteration: 2
2026-03-05 02:06:01,679 - INFO - 10.0.0.65
2026-03-05 02:06:01,679 - INFO -   config
2026-03-05 02:06:01,684 - INFO -     config-nodemgr:active
2026-03-05 02:06:01,687 - INFO -     api-server:active
2026-03-05 02:06:01,691 - INFO -     schema:active
2026-03-05 02:06:01,695 - INFO -     svc-monitor:active
2026-03-05 02:06:01,700 - INFO -     device-manager:active
2026-03-05 02:06:01,700 - INFO -   config-database
2026-03-05 02:06:01,700 - INFO -     config-cassandra:active
2026-03-05 02:06:01,700 - INFO -     config-zookeeper:active
2026-03-05 02:06:01,700 - INFO -     config-rabbitmq:active
2026-03-05 02:06:01,700 - INFO -   control
2026-03-05 02:06:01,704 - INFO -     control-nodemgr:active
2026-03-05 02:06:01,711 - INFO -     control:active
2026-03-05 02:06:01,711 - INFO -     named:active
2026-03-05 02:06:01,717 - INFO -     dns:active
2026-03-05 02:06:01,717 - INFO -   analytics
2026-03-05 02:06:01,722 - INFO -     analytics-nodemgr:active
2026-03-05 02:06:01,726 - INFO -     analytics-api:active
2026-03-05 02:06:01,731 - INFO -     collector:active
2026-03-05 02:06:01,731 - INFO -   analytics-database
2026-03-05 02:06:01,731 - INFO -     analytics-cassandra:active
2026-03-05 02:06:01,737 - INFO -     analyticsdb-nodemgr:active
2026-03-05 02:06:01,744 - INFO -     query-engine:active
2026-03-05 02:06:01,744 - INFO -   webui
2026-03-05 02:06:01,744 - INFO -     webui:active
2026-03-05 02:06:01,744 - INFO -     webui-middleware:active
2026-03-05 02:06:01,744 - INFO -     redis:active
2026-03-05 02:06:01,744 - INFO -   analytics_snmp
2026-03-05 02:06:01,748 - INFO -     snmp-collector:active
2026-03-05 02:06:01,748 - INFO -     snmp-topology:active
2026-03-05 02:06:01,749 - INFO -   analytics_alarm
2026-03-05 02:06:01,749 - INFO -     alarmgen:active
2026-03-05 02:06:01,805 - INFO - 10.0.0.33
2026-03-05 02:06:01,806 - INFO -   config
2026-03-05 02:06:01,812 - INFO -     config-nodemgr:active
2026-03-05 02:06:01,817 - INFO -     api-server:active
2026-03-05 02:06:01,821 - INFO -     schema:backup
2026-03-05 02:06:01,825 - INFO -     svc-monitor:backup
2026-03-05 02:06:01,830 - INFO -     device-manager:backup
2026-03-05 02:06:01,831 - INFO -   config-database
2026-03-05 02:06:01,831 - INFO -     config-cassandra:active
2026-03-05 02:06:01,831 - INFO -     config-zookeeper:active
2026-03-05 02:06:01,831 - INFO -     config-rabbitmq:active
2026-03-05 02:06:01,831 - INFO -   control
2026-03-05 02:06:01,836 - INFO -     control-nodemgr:active
2026-03-05 02:06:01,841 - INFO -     control:active
2026-03-05 02:06:01,841 - INFO -     named:active
2026-03-05 02:06:01,845 - INFO -     dns:active
2026-03-05 02:06:01,845 - INFO -   analytics
2026-03-05 02:06:01,851 - INFO -     analytics-nodemgr:active
2026-03-05 02:06:01,856 - INFO -     analytics-api:active
2026-03-05 02:06:01,860 - INFO -     collector:active
2026-03-05 02:06:01,861 - INFO -   analytics-database
2026-03-05 02:06:01,861 - INFO -     analytics-cassandra:active
2026-03-05 02:06:01,866 - INFO -     analyticsdb-nodemgr:active
2026-03-05 02:06:01,871 - INFO -     query-engine:active
2026-03-05 02:06:01,871 - INFO -   webui
2026-03-05 02:06:01,871 - INFO -     webui:active
2026-03-05 02:06:01,871 - INFO -     webui-middleware:active
2026-03-05 02:06:01,871 - INFO -     redis:active
2026-03-05 02:06:01,871 - INFO -   analytics_snmp
2026-03-05 02:06:01,876 - INFO -     snmp-collector:active
2026-03-05 02:06:01,876 - INFO -     snmp-topology:active
2026-03-05 02:06:01,876 - INFO -   analytics_alarm
2026-03-05 02:06:01,876 - INFO -     alarmgen:active
2026-03-05 02:06:01,933 - INFO - 10.0.0.133
2026-03-05 02:06:01,934 - INFO -   config
2026-03-05 02:06:01,939 - INFO -     config-nodemgr:active
2026-03-05 02:06:01,944 - INFO -     api-server:active
2026-03-05 02:06:01,948 - INFO -     schema:backup
2026-03-05 02:06:01,952 - INFO -     svc-monitor:backup
2026-03-05 02:06:01,956 - INFO -     device-manager:backup
2026-03-05 02:06:01,956 - INFO -   config-database
2026-03-05 02:06:01,956 - INFO -     config-cassandra:active
2026-03-05 02:06:01,956 - INFO -     config-zookeeper:active
2026-03-05 02:06:01,957 - INFO -     config-rabbitmq:active
2026-03-05 02:06:01,957 - INFO -   control
2026-03-05 02:06:01,962 - INFO -     control-nodemgr:active
2026-03-05 02:06:01,967 - INFO -     control:active
2026-03-05 02:06:01,967 - INFO -     named:active
2026-03-05 02:06:01,972 - INFO -     dns:active
2026-03-05 02:06:01,972 - INFO -   analytics
2026-03-05 02:06:01,977 - INFO -     analytics-nodemgr:active
2026-03-05 02:06:01,981 - INFO -     analytics-api:active
2026-03-05 02:06:01,986 - INFO -     collector:active
2026-03-05 02:06:01,986 - INFO -   analytics-database
2026-03-05 02:06:01,986 - INFO -     analytics-cassandra:active
2026-03-05 02:06:01,990 - INFO -     analyticsdb-nodemgr:active
2026-03-05 02:06:01,995 - INFO -     query-engine:active
2026-03-05 02:06:01,995 - INFO -   webui
2026-03-05 02:06:01,995 - INFO -     webui:active
2026-03-05 02:06:01,995 - INFO -     webui-middleware:active
2026-03-05 02:06:01,995 - INFO -     redis:active
2026-03-05 02:06:01,995 - INFO -   analytics_snmp
2026-03-05 02:06:01,999 - INFO -     snmp-collector:active
2026-03-05 02:06:01,999 - INFO -     snmp-topology:active
2026-03-05 02:06:01,999 - INFO -   analytics_alarm
2026-03-05 02:06:01,999 - INFO -     alarmgen:active
2026-03-05 02:06:02,055 - INFO - 10.0.0.141
2026-03-05 02:06:02,055 - INFO -   vrouter
2026-03-05 02:06:02,061 - INFO -     vrouter-nodemgr:active
2026-03-05 02:06:02,067 - INFO -     agent:active
2026-03-05 02:06:02,111 - INFO - 10.0.0.73
2026-03-05 02:06:02,112 - INFO -   vrouter
2026-03-05 02:06:02,117 - INFO -     vrouter-nodemgr:active
2026-03-05 02:06:02,123 - INFO -     agent:active
2026-03-05 02:06:02,123 - INFO - All the contrail services are up on all nodes
2026-03-05 02:06:02,123 - INFO - contrail-status passed
2026-03-05 02:06:02,750 - DEBUG - No XMPP flaps were noticed during the test
2026-03-05 02:06:02,750 - INFO - END TEST : test_contrail_status : PASSED[0:00:33]
2026-03-05 02:06:02,750 - INFO - --------------------------------------------------------------------------------
2026-03-05 04:05:21,929 - INFO - Using existing project ['default-domain', 'admin'](2ddec3c9-d075-47ef-bbd7-70ab863c6a6f)
2026-03-05 04:05:21,944 - INFO - ================================================================================
2026-03-05 04:05:21,944 - INFO - STARTING TEST    : test_contrail_status
2026-03-05 04:05:21,945 - INFO - TEST DESCRIPTION :  Test to verify that all services are running and active

        
2026-03-05 04:05:23,249 - DEBUG - Skipping xmpp flap check
2026-03-05 04:05:23,249 - INFO - Initial checks done. Running the testcase now
2026-03-05 04:05:23,249 - INFO - 
2026-03-05 04:05:23,525 - INFO - 10.0.0.65
2026-03-05 04:05:23,525 - INFO -   config
2026-03-05 04:05:23,533 - INFO -     config-nodemgr:active
2026-03-05 04:05:23,537 - INFO -     api-server:active
2026-03-05 04:05:23,543 - INFO -     schema:active
2026-03-05 04:05:23,547 - INFO -     svc-monitor:active
2026-03-05 04:05:23,551 - INFO -     device-manager:active
2026-03-05 04:05:23,551 - INFO -   config-database
2026-03-05 04:05:23,551 - INFO -     config-cassandra:active
2026-03-05 04:05:23,552 - INFO -     config-zookeeper:active
2026-03-05 04:05:23,552 - INFO -     config-rabbitmq:active
2026-03-05 04:05:23,552 - INFO -   control
2026-03-05 04:05:23,556 - INFO -     control-nodemgr:active
2026-03-05 04:05:23,561 - INFO -     control:active
2026-03-05 04:05:23,561 - INFO -     named:active
2026-03-05 04:05:23,574 - INFO -     dns:active
2026-03-05 04:05:23,574 - INFO -   analytics
2026-03-05 04:05:23,579 - INFO -     analytics-nodemgr:active
2026-03-05 04:05:23,583 - INFO -     analytics-api:active
2026-03-05 04:05:23,597 - INFO -     collector:active
2026-03-05 04:05:23,598 - INFO -   analytics-database
2026-03-05 04:05:23,598 - INFO -     analytics-cassandra:active
2026-03-05 04:05:23,603 - INFO -     analyticsdb-nodemgr:active
2026-03-05 04:05:23,623 - INFO -     query-engine:active
2026-03-05 04:05:23,623 - INFO -   webui
2026-03-05 04:05:23,623 - INFO -     webui:active
2026-03-05 04:05:23,623 - INFO -     webui-middleware:active
2026-03-05 04:05:23,623 - INFO -     redis:active
2026-03-05 04:05:23,623 - INFO -   analytics_snmp
2026-03-05 04:05:23,627 - INFO -     snmp-collector:active
2026-03-05 04:05:23,627 - INFO -     snmp-topology:active
2026-03-05 04:05:23,627 - INFO -   analytics_alarm
2026-03-05 04:05:23,627 - INFO -     alarmgen:active
2026-03-05 04:05:23,728 - INFO - 10.0.0.33
2026-03-05 04:05:23,728 - INFO -   config
2026-03-05 04:05:23,734 - INFO -     config-nodemgr:active
2026-03-05 04:05:23,738 - INFO -     api-server:active
2026-03-05 04:05:23,742 - INFO -     schema:backup
2026-03-05 04:05:23,746 - INFO -     svc-monitor:backup
2026-03-05 04:05:23,751 - INFO -     device-manager:backup
2026-03-05 04:05:23,751 - INFO -   config-database
2026-03-05 04:05:23,751 - INFO -     config-cassandra:active
2026-03-05 04:05:23,751 - INFO -     config-zookeeper:active
2026-03-05 04:05:23,751 - INFO -     config-rabbitmq:active
2026-03-05 04:05:23,751 - INFO -   control
2026-03-05 04:05:23,756 - INFO -     control-nodemgr:active
2026-03-05 04:05:23,762 - INFO -     control:active
2026-03-05 04:05:23,762 - INFO -     named:active
2026-03-05 04:05:23,770 - INFO -     dns:active
2026-03-05 04:05:23,771 - INFO -   analytics
2026-03-05 04:05:23,780 - INFO -     analytics-nodemgr:active
2026-03-05 04:05:23,785 - INFO -     analytics-api:active
2026-03-05 04:05:23,791 - INFO -     collector:active
2026-03-05 04:05:23,791 - INFO -   analytics-database
2026-03-05 04:05:23,791 - INFO -     analytics-cassandra:active
2026-03-05 04:05:23,798 - INFO -     analyticsdb-nodemgr:active
2026-03-05 04:05:23,804 - INFO -     query-engine:active
2026-03-05 04:05:23,804 - INFO -   webui
2026-03-05 04:05:23,804 - INFO -     webui:active
2026-03-05 04:05:23,805 - INFO -     webui-middleware:active
2026-03-05 04:05:23,805 - INFO -     redis:active
2026-03-05 04:05:23,805 - INFO -   analytics_snmp
2026-03-05 04:05:23,810 - INFO -     snmp-collector:active
2026-03-05 04:05:23,810 - INFO -     snmp-topology:active
2026-03-05 04:05:23,811 - INFO -   analytics_alarm
2026-03-05 04:05:23,811 - INFO -     alarmgen:active
2026-03-05 04:05:23,919 - INFO - 10.0.0.133
2026-03-05 04:05:23,920 - INFO -   config
2026-03-05 04:05:23,926 - INFO -     config-nodemgr:active
2026-03-05 04:05:23,932 - INFO -     api-server:active
2026-03-05 04:05:23,937 - INFO -     schema:backup
2026-03-05 04:05:23,941 - INFO -     svc-monitor:backup
2026-03-05 04:05:23,944 - INFO -     device-manager:backup
2026-03-05 04:05:23,944 - INFO -   config-database
2026-03-05 04:05:23,944 - INFO -     config-cassandra:active
2026-03-05 04:05:23,944 - INFO -     config-zookeeper:active
2026-03-05 04:05:23,944 - INFO -     config-rabbitmq:active
2026-03-05 04:05:23,944 - INFO -   control
2026-03-05 04:05:23,950 - INFO -     control-nodemgr:active
2026-03-05 04:05:23,955 - INFO -     control:active
2026-03-05 04:05:23,955 - INFO -     named:active
2026-03-05 04:05:23,960 - INFO -     dns:active
2026-03-05 04:05:23,961 - INFO -   analytics
2026-03-05 04:05:23,966 - INFO -     analytics-nodemgr:active
2026-03-05 04:05:23,970 - INFO -     analytics-api:active
2026-03-05 04:05:23,975 - INFO -     collector:active
2026-03-05 04:05:23,975 - INFO -   analytics-database
2026-03-05 04:05:23,975 - INFO -     analytics-cassandra:active
2026-03-05 04:05:23,982 - INFO -     analyticsdb-nodemgr:active
2026-03-05 04:05:23,989 - INFO -     query-engine:active
2026-03-05 04:05:23,989 - INFO -   webui
2026-03-05 04:05:23,989 - INFO -     webui:active
2026-03-05 04:05:23,989 - INFO -     webui-middleware:active
2026-03-05 04:05:23,989 - INFO -     redis:active
2026-03-05 04:05:23,989 - INFO -   analytics_snmp
2026-03-05 04:05:23,993 - INFO -     snmp-collector:active
2026-03-05 04:05:23,994 - INFO -     snmp-topology:active
2026-03-05 04:05:23,994 - INFO -   analytics_alarm
2026-03-05 04:05:23,994 - INFO -     alarmgen:active
2026-03-05 04:05:24,096 - INFO - 10.0.0.141
2026-03-05 04:05:24,096 - INFO -   vrouter
2026-03-05 04:05:24,104 - INFO -     vrouter-nodemgr:active
2026-03-05 04:05:24,111 - INFO -     agent:active
2026-03-05 04:05:24,209 - INFO - 10.0.0.73
2026-03-05 04:05:24,210 - INFO -   vrouter
2026-03-05 04:05:24,219 - INFO -     vrouter-nodemgr:active
2026-03-05 04:05:24,229 - INFO -     agent:active
2026-03-05 04:05:24,230 - INFO - All the contrail services are up on all nodes
2026-03-05 04:05:24,230 - INFO - contrail-status passed
2026-03-05 04:05:25,516 - DEBUG - Skipping xmpp flap check
2026-03-05 04:05:25,516 - INFO - END TEST : test_contrail_status : PASSED[0:00:04]
2026-03-05 04:05:25,516 - INFO - --------------------------------------------------------------------------------