2025-08-08 07:02:48,935 - INFO - Using existing project ['default-domain', 'admin'](250e3217-c8cb-45a3-a16c-e74f96b2674e) 2025-08-08 07:02:48,946 - INFO - ================================================================================ 2025-08-08 07:02:48,947 - INFO - STARTING TEST : test_contrail_status 2025-08-08 07:02:48,947 - INFO - TEST DESCRIPTION : Test to verify that all services are running and active 2025-08-08 07:02:49,642 - DEBUG - Nothing to compare xmpp stats {'10.0.0.41': {'10.20.0.254': '0', '10.20.0.25': '0'}, '10.0.0.27': {'10.20.0.254': '0', '10.20.0.25': '0'}} with 2025-08-08 07:02:49,642 - INFO - Initial checks done. Running the testcase now 2025-08-08 07:02:49,642 - INFO - 2025-08-08 07:02:49,750 - INFO - 10.0.0.50 2025-08-08 07:02:49,750 - INFO - config 2025-08-08 07:02:49,759 - INFO - config-nodemgr:active 2025-08-08 07:02:49,768 - INFO - api-server:active 2025-08-08 07:02:49,775 - INFO - schema:backup 2025-08-08 07:02:49,781 - INFO - svc-monitor:backup 2025-08-08 07:02:49,789 - INFO - device-manager:backup 2025-08-08 07:02:49,789 - INFO - config-database 2025-08-08 07:02:49,789 - INFO - config-cassandra:active 2025-08-08 07:02:49,789 - INFO - config-zookeeper:active 2025-08-08 07:02:49,789 - INFO - config-rabbitmq:active 2025-08-08 07:02:49,789 - INFO - control 2025-08-08 07:02:49,798 - INFO - control-nodemgr:active 2025-08-08 07:02:49,808 - INFO - control:active 2025-08-08 07:02:49,808 - INFO - named:active 2025-08-08 07:02:49,822 - INFO - dns:active 2025-08-08 07:02:49,823 - INFO - analytics 2025-08-08 07:02:49,830 - INFO - analytics-nodemgr:active 2025-08-08 07:02:49,841 - INFO - analytics-api:active 2025-08-08 07:02:49,856 - INFO - collector:active 2025-08-08 07:02:49,856 - INFO - analytics-database 2025-08-08 07:02:49,857 - INFO - analytics-cassandra:active 2025-08-08 07:02:49,868 - INFO - analyticsdb-nodemgr:active 2025-08-08 07:02:49,880 - INFO - query-engine:active 2025-08-08 07:02:49,881 - INFO - webui 2025-08-08 07:02:49,881 - INFO - webui:active 2025-08-08 07:02:49,881 - INFO - webui-middleware:active 2025-08-08 07:02:49,881 - INFO - redis:active 2025-08-08 07:02:49,881 - INFO - analytics_snmp 2025-08-08 07:02:49,886 - INFO - snmp-collector:active 2025-08-08 07:02:49,886 - INFO - snmp-topology:active 2025-08-08 07:02:49,886 - INFO - analytics_alarm 2025-08-08 07:02:49,886 - INFO - alarmgen:active 2025-08-08 07:02:49,943 - INFO - 10.0.0.254 2025-08-08 07:02:49,943 - INFO - config 2025-08-08 07:02:49,954 - INFO - config-nodemgr:active 2025-08-08 07:02:49,963 - INFO - api-server:initializing (Generic Connection:Keystone[Error: Unable to establish connection to http://10.0.0.50:5000/v3/domains? at UTC 2025-08-08 07:01:55.575358] connection down) 2025-08-08 07:02:49,981 - INFO - schema:backup 2025-08-08 07:02:49,989 - INFO - svc-monitor:backup 2025-08-08 07:02:49,996 - INFO - device-manager:backup 2025-08-08 07:02:49,996 - INFO - config-database 2025-08-08 07:02:49,997 - INFO - config-cassandra:active 2025-08-08 07:02:49,997 - INFO - config-zookeeper:active 2025-08-08 07:02:49,997 - INFO - config-rabbitmq:active 2025-08-08 07:02:49,997 - INFO - control 2025-08-08 07:02:50,007 - INFO - control-nodemgr:active 2025-08-08 07:02:50,017 - INFO - control:active 2025-08-08 07:02:50,018 - INFO - named:active 2025-08-08 07:02:50,029 - INFO - dns:active 2025-08-08 07:02:50,029 - INFO - analytics 2025-08-08 07:02:50,039 - INFO - analytics-nodemgr:active 2025-08-08 07:02:50,048 - INFO - analytics-api:active 2025-08-08 07:02:50,059 - INFO - collector:active 2025-08-08 07:02:50,059 - INFO - analytics-database 2025-08-08 07:02:50,060 - INFO - analytics-cassandra:active 2025-08-08 07:02:50,074 - INFO - analyticsdb-nodemgr:active 2025-08-08 07:02:50,082 - INFO - query-engine:active 2025-08-08 07:02:50,082 - INFO - webui 2025-08-08 07:02:50,082 - INFO - webui:active 2025-08-08 07:02:50,082 - INFO - webui-middleware:active 2025-08-08 07:02:50,083 - INFO - redis:active 2025-08-08 07:02:50,083 - INFO - analytics_snmp 2025-08-08 07:02:50,091 - INFO - snmp-collector:active 2025-08-08 07:02:50,092 - INFO - snmp-topology:active 2025-08-08 07:02:50,092 - INFO - analytics_alarm 2025-08-08 07:02:50,092 - INFO - alarmgen:active 2025-08-08 07:02:50,160 - INFO - 10.0.0.65 2025-08-08 07:02:50,160 - INFO - config 2025-08-08 07:02:50,167 - INFO - config-nodemgr:active 2025-08-08 07:02:50,174 - INFO - api-server:active 2025-08-08 07:02:50,180 - INFO - schema:active 2025-08-08 07:02:50,186 - INFO - svc-monitor:active 2025-08-08 07:02:50,193 - INFO - device-manager:active 2025-08-08 07:02:50,195 - INFO - config-database 2025-08-08 07:02:50,196 - INFO - config-cassandra:active 2025-08-08 07:02:50,196 - INFO - config-zookeeper:active 2025-08-08 07:02:50,197 - INFO - config-rabbitmq:active 2025-08-08 07:02:50,197 - INFO - control 2025-08-08 07:02:50,206 - INFO - control-nodemgr:active 2025-08-08 07:02:50,217 - INFO - control:active 2025-08-08 07:02:50,218 - INFO - named:active 2025-08-08 07:02:50,227 - INFO - dns:active 2025-08-08 07:02:50,228 - INFO - analytics 2025-08-08 07:02:50,238 - INFO - analytics-nodemgr:active 2025-08-08 07:02:50,246 - INFO - analytics-api:active 2025-08-08 07:02:50,256 - INFO - collector:active 2025-08-08 07:02:50,257 - INFO - analytics-database 2025-08-08 07:02:50,258 - INFO - analytics-cassandra:active 2025-08-08 07:02:50,269 - INFO - analyticsdb-nodemgr:active 2025-08-08 07:02:50,280 - INFO - query-engine:active 2025-08-08 07:02:50,280 - INFO - webui 2025-08-08 07:02:50,280 - INFO - webui:active 2025-08-08 07:02:50,280 - INFO - webui-middleware:active 2025-08-08 07:02:50,280 - INFO - redis:active 2025-08-08 07:02:50,281 - INFO - analytics_snmp 2025-08-08 07:02:50,292 - INFO - snmp-collector:active 2025-08-08 07:02:50,292 - INFO - snmp-topology:active 2025-08-08 07:02:50,293 - INFO - analytics_alarm 2025-08-08 07:02:50,293 - INFO - alarmgen:active 2025-08-08 07:02:50,342 - INFO - 10.0.0.41 2025-08-08 07:02:50,343 - INFO - vrouter 2025-08-08 07:02:50,355 - INFO - vrouter-nodemgr:active 2025-08-08 07:02:50,366 - INFO - agent:active 2025-08-08 07:02:50,415 - INFO - 10.0.0.27 2025-08-08 07:02:50,416 - INFO - vrouter 2025-08-08 07:02:50,426 - INFO - vrouter-nodemgr:active 2025-08-08 07:02:50,449 - INFO - agent:active 2025-08-08 07:02:50,449 - DEBUG - defaultdict(, {'10.0.0.254': {'api-server': {'status': 'initializing', 'description': 'Generic Connection:Keystone[Error: Unable to establish connection to http://10.0.0.50:5000/v3/domains? at UTC 2025-08-08 07:01:55.575358] connection down'}}}) 2025-08-08 07:02:50,449 - DEBUG - Not all services up. Sleeping for 10 seconds. iteration: 0 2025-08-08 07:03:00,531 - INFO - 10.0.0.50 2025-08-08 07:03:00,531 - INFO - config 2025-08-08 07:03:00,539 - INFO - config-nodemgr:active 2025-08-08 07:03:00,547 - INFO - api-server:active 2025-08-08 07:03:00,553 - INFO - schema:backup 2025-08-08 07:03:00,557 - INFO - svc-monitor:backup 2025-08-08 07:03:00,562 - INFO - device-manager:backup 2025-08-08 07:03:00,562 - INFO - config-database 2025-08-08 07:03:00,562 - INFO - config-cassandra:active 2025-08-08 07:03:00,562 - INFO - config-zookeeper:active 2025-08-08 07:03:00,562 - INFO - config-rabbitmq:active 2025-08-08 07:03:00,563 - INFO - control 2025-08-08 07:03:00,569 - INFO - control-nodemgr:active 2025-08-08 07:03:00,579 - INFO - control:active 2025-08-08 07:03:00,579 - INFO - named:active 2025-08-08 07:03:00,590 - INFO - dns:active 2025-08-08 07:03:00,590 - INFO - analytics 2025-08-08 07:03:00,597 - INFO - analytics-nodemgr:active 2025-08-08 07:03:00,603 - INFO - analytics-api:active 2025-08-08 07:03:00,610 - INFO - collector:active 2025-08-08 07:03:00,610 - INFO - analytics-database 2025-08-08 07:03:00,610 - INFO - analytics-cassandra:active 2025-08-08 07:03:00,619 - INFO - analyticsdb-nodemgr:active 2025-08-08 07:03:00,631 - INFO - query-engine:active 2025-08-08 07:03:00,631 - INFO - webui 2025-08-08 07:03:00,631 - INFO - webui:active 2025-08-08 07:03:00,631 - INFO - webui-middleware:active 2025-08-08 07:03:00,632 - INFO - redis:active 2025-08-08 07:03:00,632 - INFO - analytics_snmp 2025-08-08 07:03:00,637 - INFO - snmp-collector:active 2025-08-08 07:03:00,637 - INFO - snmp-topology:active 2025-08-08 07:03:00,637 - INFO - analytics_alarm 2025-08-08 07:03:00,637 - INFO - alarmgen:active 2025-08-08 07:03:00,706 - INFO - 10.0.0.254 2025-08-08 07:03:00,707 - INFO - config 2025-08-08 07:03:00,724 - INFO - config-nodemgr:active 2025-08-08 07:03:00,734 - INFO - api-server:active 2025-08-08 07:03:00,740 - INFO - schema:backup 2025-08-08 07:03:00,746 - INFO - svc-monitor:backup 2025-08-08 07:03:00,752 - INFO - device-manager:backup 2025-08-08 07:03:00,752 - INFO - config-database 2025-08-08 07:03:00,753 - INFO - config-cassandra:active 2025-08-08 07:03:00,753 - INFO - config-zookeeper:active 2025-08-08 07:03:00,753 - INFO - config-rabbitmq:active 2025-08-08 07:03:00,753 - INFO - control 2025-08-08 07:03:00,762 - INFO - control-nodemgr:active 2025-08-08 07:03:00,771 - INFO - control:active 2025-08-08 07:03:00,771 - INFO - named:active 2025-08-08 07:03:00,779 - INFO - dns:active 2025-08-08 07:03:00,779 - INFO - analytics 2025-08-08 07:03:00,786 - INFO - analytics-nodemgr:active 2025-08-08 07:03:00,793 - INFO - analytics-api:active 2025-08-08 07:03:00,800 - INFO - collector:active 2025-08-08 07:03:00,801 - INFO - analytics-database 2025-08-08 07:03:00,801 - INFO - analytics-cassandra:active 2025-08-08 07:03:00,809 - INFO - analyticsdb-nodemgr:active 2025-08-08 07:03:00,817 - INFO - query-engine:active 2025-08-08 07:03:00,817 - INFO - webui 2025-08-08 07:03:00,817 - INFO - webui:active 2025-08-08 07:03:00,817 - INFO - webui-middleware:active 2025-08-08 07:03:00,818 - INFO - redis:active 2025-08-08 07:03:00,818 - INFO - analytics_snmp 2025-08-08 07:03:00,825 - INFO - snmp-collector:active 2025-08-08 07:03:00,825 - INFO - snmp-topology:active 2025-08-08 07:03:00,826 - INFO - analytics_alarm 2025-08-08 07:03:00,826 - INFO - alarmgen:active 2025-08-08 07:03:00,871 - INFO - 10.0.0.65 2025-08-08 07:03:00,871 - INFO - config 2025-08-08 07:03:00,878 - INFO - config-nodemgr:active 2025-08-08 07:03:00,884 - INFO - api-server:active 2025-08-08 07:03:00,892 - INFO - schema:active 2025-08-08 07:03:00,901 - INFO - svc-monitor:active 2025-08-08 07:03:00,907 - INFO - device-manager:active 2025-08-08 07:03:00,907 - INFO - config-database 2025-08-08 07:03:00,907 - INFO - config-cassandra:active 2025-08-08 07:03:00,907 - INFO - config-zookeeper:active 2025-08-08 07:03:00,908 - INFO - config-rabbitmq:active 2025-08-08 07:03:00,908 - INFO - control 2025-08-08 07:03:00,915 - INFO - control-nodemgr:active 2025-08-08 07:03:00,922 - INFO - control:active 2025-08-08 07:03:00,922 - INFO - named:active 2025-08-08 07:03:00,929 - INFO - dns:active 2025-08-08 07:03:00,929 - INFO - analytics 2025-08-08 07:03:00,935 - INFO - analytics-nodemgr:active 2025-08-08 07:03:00,941 - INFO - analytics-api:active 2025-08-08 07:03:00,949 - INFO - collector:active 2025-08-08 07:03:00,949 - INFO - analytics-database 2025-08-08 07:03:00,949 - INFO - analytics-cassandra:active 2025-08-08 07:03:00,957 - INFO - analyticsdb-nodemgr:active 2025-08-08 07:03:00,964 - INFO - query-engine:active 2025-08-08 07:03:00,964 - INFO - webui 2025-08-08 07:03:00,964 - INFO - webui:active 2025-08-08 07:03:00,964 - INFO - webui-middleware:active 2025-08-08 07:03:00,964 - INFO - redis:active 2025-08-08 07:03:00,964 - INFO - analytics_snmp 2025-08-08 07:03:00,971 - INFO - snmp-collector:active 2025-08-08 07:03:00,972 - INFO - snmp-topology:active 2025-08-08 07:03:00,972 - INFO - analytics_alarm 2025-08-08 07:03:00,972 - INFO - alarmgen:active 2025-08-08 07:03:01,020 - INFO - 10.0.0.41 2025-08-08 07:03:01,021 - INFO - vrouter 2025-08-08 07:03:01,030 - INFO - vrouter-nodemgr:active 2025-08-08 07:03:01,038 - INFO - agent:active 2025-08-08 07:03:01,086 - INFO - 10.0.0.27 2025-08-08 07:03:01,086 - INFO - vrouter 2025-08-08 07:03:01,095 - INFO - vrouter-nodemgr:active 2025-08-08 07:03:01,105 - INFO - agent:active 2025-08-08 07:03:01,105 - INFO - All the contrail services are up on all nodes 2025-08-08 07:03:01,105 - INFO - contrail-status passed 2025-08-08 07:03:01,804 - DEBUG - No XMPP flaps were noticed during the test 2025-08-08 07:03:01,804 - INFO - END TEST : test_contrail_status : PASSED[0:00:13] 2025-08-08 07:03:01,804 - INFO - -------------------------------------------------------------------------------- 2025-08-08 09:37:06,060 - INFO - Using existing project ['default-domain', 'admin'](250e3217-c8cb-45a3-a16c-e74f96b2674e) 2025-08-08 09:37:06,077 - INFO - ================================================================================ 2025-08-08 09:37:06,078 - INFO - STARTING TEST : test_contrail_status 2025-08-08 09:37:06,078 - INFO - TEST DESCRIPTION : Test to verify that all services are running and active 2025-08-08 09:37:07,364 - DEBUG - Skipping xmpp flap check 2025-08-08 09:37:07,365 - INFO - Initial checks done. Running the testcase now 2025-08-08 09:37:07,365 - INFO - 2025-08-08 09:37:07,675 - INFO - 10.0.0.50 2025-08-08 09:37:07,675 - INFO - config 2025-08-08 09:37:07,684 - INFO - config-nodemgr:active 2025-08-08 09:37:07,688 - INFO - api-server:active 2025-08-08 09:37:07,693 - INFO - schema:backup 2025-08-08 09:37:07,696 - INFO - svc-monitor:backup 2025-08-08 09:37:07,701 - INFO - device-manager:backup 2025-08-08 09:37:07,701 - INFO - config-database 2025-08-08 09:37:07,701 - INFO - config-cassandra:active 2025-08-08 09:37:07,701 - INFO - config-zookeeper:active 2025-08-08 09:37:07,701 - INFO - config-rabbitmq:active 2025-08-08 09:37:07,701 - INFO - control 2025-08-08 09:37:07,707 - INFO - control-nodemgr:active 2025-08-08 09:37:07,713 - INFO - control:active 2025-08-08 09:37:07,714 - INFO - named:active 2025-08-08 09:37:07,726 - INFO - dns:active 2025-08-08 09:37:07,726 - INFO - analytics 2025-08-08 09:37:07,732 - INFO - analytics-nodemgr:active 2025-08-08 09:37:07,736 - INFO - analytics-api:active 2025-08-08 09:37:07,754 - INFO - collector:active 2025-08-08 09:37:07,754 - INFO - analytics-database 2025-08-08 09:37:07,754 - INFO - analytics-cassandra:active 2025-08-08 09:37:07,760 - INFO - analyticsdb-nodemgr:active 2025-08-08 09:37:07,779 - INFO - query-engine:active 2025-08-08 09:37:07,780 - INFO - webui 2025-08-08 09:37:07,780 - INFO - webui:active 2025-08-08 09:37:07,780 - INFO - webui-middleware:active 2025-08-08 09:37:07,780 - INFO - redis:active 2025-08-08 09:37:07,780 - INFO - analytics_snmp 2025-08-08 09:37:07,785 - INFO - snmp-collector:active 2025-08-08 09:37:07,786 - INFO - snmp-topology:active 2025-08-08 09:37:07,786 - INFO - analytics_alarm 2025-08-08 09:37:07,786 - INFO - alarmgen:active 2025-08-08 09:37:07,885 - INFO - 10.0.0.254 2025-08-08 09:37:07,886 - INFO - config 2025-08-08 09:37:07,895 - INFO - config-nodemgr:active 2025-08-08 09:37:07,903 - INFO - api-server:initializing (Database:Cassandra[] connection down) 2025-08-08 09:37:07,910 - INFO - schema:backup 2025-08-08 09:37:07,917 - INFO - svc-monitor:backup 2025-08-08 09:37:07,924 - INFO - device-manager:backup 2025-08-08 09:37:07,924 - INFO - config-database 2025-08-08 09:37:07,924 - INFO - config-cassandra:active 2025-08-08 09:37:07,924 - INFO - config-zookeeper:active 2025-08-08 09:37:07,924 - INFO - config-rabbitmq:active 2025-08-08 09:37:07,924 - INFO - control 2025-08-08 09:37:07,933 - INFO - control-nodemgr:active 2025-08-08 09:37:07,941 - INFO - control:active 2025-08-08 09:37:07,942 - INFO - named:active 2025-08-08 09:37:07,950 - INFO - dns:active 2025-08-08 09:37:07,950 - INFO - analytics 2025-08-08 09:37:07,957 - INFO - analytics-nodemgr:active 2025-08-08 09:37:07,963 - INFO - analytics-api:active 2025-08-08 09:37:07,970 - INFO - collector:active 2025-08-08 09:37:07,970 - INFO - analytics-database 2025-08-08 09:37:07,970 - INFO - analytics-cassandra:active 2025-08-08 09:37:07,977 - INFO - analyticsdb-nodemgr:active 2025-08-08 09:37:07,983 - INFO - query-engine:active 2025-08-08 09:37:07,983 - INFO - webui 2025-08-08 09:37:07,984 - INFO - webui:active 2025-08-08 09:37:07,984 - INFO - webui-middleware:active 2025-08-08 09:37:07,984 - INFO - redis:active 2025-08-08 09:37:07,984 - INFO - analytics_snmp 2025-08-08 09:37:07,989 - INFO - snmp-collector:active 2025-08-08 09:37:07,990 - INFO - snmp-topology:active 2025-08-08 09:37:07,990 - INFO - analytics_alarm 2025-08-08 09:37:07,990 - INFO - alarmgen:active 2025-08-08 09:37:08,078 - INFO - 10.0.0.65 2025-08-08 09:37:08,078 - INFO - config 2025-08-08 09:37:08,085 - INFO - config-nodemgr:active 2025-08-08 09:37:08,091 - INFO - api-server:active 2025-08-08 09:37:08,098 - INFO - schema:active 2025-08-08 09:37:08,106 - INFO - svc-monitor:active 2025-08-08 09:37:08,112 - INFO - device-manager:active 2025-08-08 09:37:08,112 - INFO - config-database 2025-08-08 09:37:08,112 - INFO - config-cassandra:active 2025-08-08 09:37:08,113 - INFO - config-zookeeper:active 2025-08-08 09:37:08,113 - INFO - config-rabbitmq:active 2025-08-08 09:37:08,113 - INFO - control 2025-08-08 09:37:08,119 - INFO - control-nodemgr:active 2025-08-08 09:37:08,125 - INFO - control:active 2025-08-08 09:37:08,126 - INFO - named:active 2025-08-08 09:37:08,131 - INFO - dns:active 2025-08-08 09:37:08,132 - INFO - analytics 2025-08-08 09:37:08,138 - INFO - analytics-nodemgr:active 2025-08-08 09:37:08,144 - INFO - analytics-api:active 2025-08-08 09:37:08,152 - INFO - collector:active 2025-08-08 09:37:08,152 - INFO - analytics-database 2025-08-08 09:37:08,152 - INFO - analytics-cassandra:active 2025-08-08 09:37:08,166 - INFO - analyticsdb-nodemgr:active 2025-08-08 09:37:08,179 - INFO - query-engine:active 2025-08-08 09:37:08,179 - INFO - webui 2025-08-08 09:37:08,179 - INFO - webui:active 2025-08-08 09:37:08,179 - INFO - webui-middleware:active 2025-08-08 09:37:08,179 - INFO - redis:active 2025-08-08 09:37:08,179 - INFO - analytics_snmp 2025-08-08 09:37:08,196 - INFO - snmp-collector:active 2025-08-08 09:37:08,196 - INFO - snmp-topology:active 2025-08-08 09:37:08,196 - INFO - analytics_alarm 2025-08-08 09:37:08,196 - INFO - alarmgen:active 2025-08-08 09:37:08,285 - INFO - 10.0.0.41 2025-08-08 09:37:08,285 - INFO - vrouter 2025-08-08 09:37:08,293 - INFO - vrouter-nodemgr:active 2025-08-08 09:37:08,301 - INFO - agent:active 2025-08-08 09:37:08,394 - INFO - 10.0.0.27 2025-08-08 09:37:08,394 - INFO - vrouter 2025-08-08 09:37:08,403 - INFO - vrouter-nodemgr:active 2025-08-08 09:37:08,413 - INFO - agent:active 2025-08-08 09:37:08,413 - DEBUG - defaultdict(, {'10.0.0.254': {'api-server': {'status': 'initializing', 'description': 'Database:Cassandra[] connection down'}}}) 2025-08-08 09:37:08,413 - DEBUG - Not all services up. Sleeping for 10 seconds. iteration: 0 2025-08-08 09:37:18,536 - INFO - 10.0.0.50 2025-08-08 09:37:18,536 - INFO - config 2025-08-08 09:37:18,542 - INFO - config-nodemgr:active 2025-08-08 09:37:18,547 - INFO - api-server:active 2025-08-08 09:37:18,551 - INFO - schema:backup 2025-08-08 09:37:18,555 - INFO - svc-monitor:backup 2025-08-08 09:37:18,559 - INFO - device-manager:backup 2025-08-08 09:37:18,559 - INFO - config-database 2025-08-08 09:37:18,559 - INFO - config-cassandra:active 2025-08-08 09:37:18,559 - INFO - config-zookeeper:active 2025-08-08 09:37:18,559 - INFO - config-rabbitmq:active 2025-08-08 09:37:18,559 - INFO - control 2025-08-08 09:37:18,564 - INFO - control-nodemgr:active 2025-08-08 09:37:18,570 - INFO - control:active 2025-08-08 09:37:18,570 - INFO - named:active 2025-08-08 09:37:18,576 - INFO - dns:active 2025-08-08 09:37:18,576 - INFO - analytics 2025-08-08 09:37:18,581 - INFO - analytics-nodemgr:active 2025-08-08 09:37:18,586 - INFO - analytics-api:active 2025-08-08 09:37:18,591 - INFO - collector:active 2025-08-08 09:37:18,592 - INFO - analytics-database 2025-08-08 09:37:18,592 - INFO - analytics-cassandra:active 2025-08-08 09:37:18,597 - INFO - analyticsdb-nodemgr:active 2025-08-08 09:37:18,602 - INFO - query-engine:active 2025-08-08 09:37:18,603 - INFO - webui 2025-08-08 09:37:18,603 - INFO - webui:active 2025-08-08 09:37:18,603 - INFO - webui-middleware:active 2025-08-08 09:37:18,603 - INFO - redis:active 2025-08-08 09:37:18,603 - INFO - analytics_snmp 2025-08-08 09:37:18,607 - INFO - snmp-collector:active 2025-08-08 09:37:18,607 - INFO - snmp-topology:active 2025-08-08 09:37:18,608 - INFO - analytics_alarm 2025-08-08 09:37:18,608 - INFO - alarmgen:active 2025-08-08 09:37:18,712 - INFO - 10.0.0.254 2025-08-08 09:37:18,712 - INFO - config 2025-08-08 09:37:18,719 - INFO - config-nodemgr:active 2025-08-08 09:37:18,724 - INFO - api-server:initializing (Database:Cassandra[] connection down) 2025-08-08 09:37:18,729 - INFO - schema:backup 2025-08-08 09:37:18,733 - INFO - svc-monitor:backup 2025-08-08 09:37:18,737 - INFO - device-manager:backup 2025-08-08 09:37:18,737 - INFO - config-database 2025-08-08 09:37:18,737 - INFO - config-cassandra:active 2025-08-08 09:37:18,737 - INFO - config-zookeeper:active 2025-08-08 09:37:18,737 - INFO - config-rabbitmq:active 2025-08-08 09:37:18,737 - INFO - control 2025-08-08 09:37:18,743 - INFO - control-nodemgr:active 2025-08-08 09:37:18,751 - INFO - control:active 2025-08-08 09:37:18,751 - INFO - named:active 2025-08-08 09:37:18,757 - INFO - dns:active 2025-08-08 09:37:18,757 - INFO - analytics 2025-08-08 09:37:18,763 - INFO - analytics-nodemgr:active 2025-08-08 09:37:18,768 - INFO - analytics-api:active 2025-08-08 09:37:18,774 - INFO - collector:active 2025-08-08 09:37:18,774 - INFO - analytics-database 2025-08-08 09:37:18,774 - INFO - analytics-cassandra:active 2025-08-08 09:37:18,779 - INFO - analyticsdb-nodemgr:active 2025-08-08 09:37:18,786 - INFO - query-engine:active 2025-08-08 09:37:18,786 - INFO - webui 2025-08-08 09:37:18,787 - INFO - webui:active 2025-08-08 09:37:18,787 - INFO - webui-middleware:active 2025-08-08 09:37:18,787 - INFO - redis:active 2025-08-08 09:37:18,787 - INFO - analytics_snmp 2025-08-08 09:37:18,794 - INFO - snmp-collector:active 2025-08-08 09:37:18,794 - INFO - snmp-topology:active 2025-08-08 09:37:18,794 - INFO - analytics_alarm 2025-08-08 09:37:18,794 - INFO - alarmgen:active 2025-08-08 09:37:18,900 - INFO - 10.0.0.65 2025-08-08 09:37:18,900 - INFO - config 2025-08-08 09:37:18,909 - INFO - config-nodemgr:active 2025-08-08 09:37:18,914 - INFO - api-server:active 2025-08-08 09:37:18,920 - INFO - schema:active 2025-08-08 09:37:18,926 - INFO - svc-monitor:active 2025-08-08 09:37:18,931 - INFO - device-manager:active 2025-08-08 09:37:18,932 - INFO - config-database 2025-08-08 09:37:18,932 - INFO - config-cassandra:active 2025-08-08 09:37:18,932 - INFO - config-zookeeper:active 2025-08-08 09:37:18,932 - INFO - config-rabbitmq:active 2025-08-08 09:37:18,932 - INFO - control 2025-08-08 09:37:18,939 - INFO - control-nodemgr:active 2025-08-08 09:37:18,946 - INFO - control:active 2025-08-08 09:37:18,946 - INFO - named:active 2025-08-08 09:37:18,954 - INFO - dns:active 2025-08-08 09:37:18,955 - INFO - analytics 2025-08-08 09:37:18,962 - INFO - analytics-nodemgr:active 2025-08-08 09:37:18,969 - INFO - analytics-api:active 2025-08-08 09:37:18,978 - INFO - collector:active 2025-08-08 09:37:18,979 - INFO - analytics-database 2025-08-08 09:37:18,979 - INFO - analytics-cassandra:active 2025-08-08 09:37:18,987 - INFO - analyticsdb-nodemgr:active 2025-08-08 09:37:18,994 - INFO - query-engine:active 2025-08-08 09:37:18,995 - INFO - webui 2025-08-08 09:37:18,995 - INFO - webui:active 2025-08-08 09:37:18,995 - INFO - webui-middleware:active 2025-08-08 09:37:18,995 - INFO - redis:active 2025-08-08 09:37:18,995 - INFO - analytics_snmp 2025-08-08 09:37:19,001 - INFO - snmp-collector:active 2025-08-08 09:37:19,001 - INFO - snmp-topology:active 2025-08-08 09:37:19,001 - INFO - analytics_alarm 2025-08-08 09:37:19,001 - INFO - alarmgen:active 2025-08-08 09:37:19,092 - INFO - 10.0.0.41 2025-08-08 09:37:19,093 - INFO - vrouter 2025-08-08 09:37:19,102 - INFO - vrouter-nodemgr:active 2025-08-08 09:37:19,110 - INFO - agent:active 2025-08-08 09:37:19,201 - INFO - 10.0.0.27 2025-08-08 09:37:19,201 - INFO - vrouter 2025-08-08 09:37:19,208 - INFO - vrouter-nodemgr:active 2025-08-08 09:37:19,216 - INFO - agent:active 2025-08-08 09:37:19,216 - DEBUG - defaultdict(, {'10.0.0.254': {'api-server': {'status': 'initializing', 'description': 'Database:Cassandra[] connection down'}}}) 2025-08-08 09:37:19,216 - DEBUG - Not all services up. Sleeping for 10 seconds. iteration: 1 2025-08-08 09:37:29,343 - INFO - 10.0.0.50 2025-08-08 09:37:29,343 - INFO - config 2025-08-08 09:37:29,350 - INFO - config-nodemgr:active 2025-08-08 09:37:29,354 - INFO - api-server:active 2025-08-08 09:37:29,358 - INFO - schema:backup 2025-08-08 09:37:29,362 - INFO - svc-monitor:backup 2025-08-08 09:37:29,366 - INFO - device-manager:backup 2025-08-08 09:37:29,366 - INFO - config-database 2025-08-08 09:37:29,366 - INFO - config-cassandra:active 2025-08-08 09:37:29,366 - INFO - config-zookeeper:active 2025-08-08 09:37:29,366 - INFO - config-rabbitmq:active 2025-08-08 09:37:29,366 - INFO - control 2025-08-08 09:37:29,371 - INFO - control-nodemgr:active 2025-08-08 09:37:29,378 - INFO - control:active 2025-08-08 09:37:29,378 - INFO - named:active 2025-08-08 09:37:29,385 - INFO - dns:active 2025-08-08 09:37:29,386 - INFO - analytics 2025-08-08 09:37:29,392 - INFO - analytics-nodemgr:active 2025-08-08 09:37:29,398 - INFO - analytics-api:active 2025-08-08 09:37:29,405 - INFO - collector:active 2025-08-08 09:37:29,406 - INFO - analytics-database 2025-08-08 09:37:29,406 - INFO - analytics-cassandra:active 2025-08-08 09:37:29,411 - INFO - analyticsdb-nodemgr:active 2025-08-08 09:37:29,417 - INFO - query-engine:active 2025-08-08 09:37:29,418 - INFO - webui 2025-08-08 09:37:29,418 - INFO - webui:active 2025-08-08 09:37:29,418 - INFO - webui-middleware:active 2025-08-08 09:37:29,418 - INFO - redis:active 2025-08-08 09:37:29,418 - INFO - analytics_snmp 2025-08-08 09:37:29,422 - INFO - snmp-collector:active 2025-08-08 09:37:29,423 - INFO - snmp-topology:active 2025-08-08 09:37:29,423 - INFO - analytics_alarm 2025-08-08 09:37:29,423 - INFO - alarmgen:active 2025-08-08 09:37:29,523 - INFO - 10.0.0.254 2025-08-08 09:37:29,523 - INFO - config 2025-08-08 09:37:29,530 - INFO - config-nodemgr:active 2025-08-08 09:37:29,536 - INFO - api-server:initializing (Database:Cassandra[] connection down) 2025-08-08 09:37:29,541 - INFO - schema:backup 2025-08-08 09:37:29,545 - INFO - svc-monitor:backup 2025-08-08 09:37:29,551 - INFO - device-manager:backup 2025-08-08 09:37:29,551 - INFO - config-database 2025-08-08 09:37:29,551 - INFO - config-cassandra:active 2025-08-08 09:37:29,551 - INFO - config-zookeeper:active 2025-08-08 09:37:29,551 - INFO - config-rabbitmq:active 2025-08-08 09:37:29,551 - INFO - control 2025-08-08 09:37:29,557 - INFO - control-nodemgr:active 2025-08-08 09:37:29,564 - INFO - control:active 2025-08-08 09:37:29,565 - INFO - named:active 2025-08-08 09:37:29,571 - INFO - dns:active 2025-08-08 09:37:29,571 - INFO - analytics 2025-08-08 09:37:29,577 - INFO - analytics-nodemgr:active 2025-08-08 09:37:29,583 - INFO - analytics-api:active 2025-08-08 09:37:29,589 - INFO - collector:active 2025-08-08 09:37:29,589 - INFO - analytics-database 2025-08-08 09:37:29,589 - INFO - analytics-cassandra:active 2025-08-08 09:37:29,595 - INFO - analyticsdb-nodemgr:active 2025-08-08 09:37:29,603 - INFO - query-engine:active 2025-08-08 09:37:29,603 - INFO - webui 2025-08-08 09:37:29,603 - INFO - webui:active 2025-08-08 09:37:29,603 - INFO - webui-middleware:active 2025-08-08 09:37:29,603 - INFO - redis:active 2025-08-08 09:37:29,603 - INFO - analytics_snmp 2025-08-08 09:37:29,610 - INFO - snmp-collector:active 2025-08-08 09:37:29,610 - INFO - snmp-topology:active 2025-08-08 09:37:29,610 - INFO - analytics_alarm 2025-08-08 09:37:29,610 - INFO - alarmgen:active 2025-08-08 09:37:29,702 - INFO - 10.0.0.65 2025-08-08 09:37:29,703 - INFO - config 2025-08-08 09:37:29,712 - INFO - config-nodemgr:active 2025-08-08 09:37:29,722 - INFO - api-server:active 2025-08-08 09:37:29,728 - INFO - schema:active 2025-08-08 09:37:29,734 - INFO - svc-monitor:active 2025-08-08 09:37:29,742 - INFO - device-manager:active 2025-08-08 09:37:29,743 - INFO - config-database 2025-08-08 09:37:29,743 - INFO - config-cassandra:active 2025-08-08 09:37:29,743 - INFO - config-zookeeper:active 2025-08-08 09:37:29,743 - INFO - config-rabbitmq:active 2025-08-08 09:37:29,743 - INFO - control 2025-08-08 09:37:29,750 - INFO - control-nodemgr:active 2025-08-08 09:37:29,756 - INFO - control:active 2025-08-08 09:37:29,757 - INFO - named:active 2025-08-08 09:37:29,764 - INFO - dns:active 2025-08-08 09:37:29,764 - INFO - analytics 2025-08-08 09:37:29,771 - INFO - analytics-nodemgr:active 2025-08-08 09:37:29,777 - INFO - analytics-api:active 2025-08-08 09:37:29,784 - INFO - collector:active 2025-08-08 09:37:29,785 - INFO - analytics-database 2025-08-08 09:37:29,785 - INFO - analytics-cassandra:active 2025-08-08 09:37:29,792 - INFO - analyticsdb-nodemgr:active 2025-08-08 09:37:29,800 - INFO - query-engine:active 2025-08-08 09:37:29,800 - INFO - webui 2025-08-08 09:37:29,800 - INFO - webui:active 2025-08-08 09:37:29,800 - INFO - webui-middleware:active 2025-08-08 09:37:29,800 - INFO - redis:active 2025-08-08 09:37:29,800 - INFO - analytics_snmp 2025-08-08 09:37:29,807 - INFO - snmp-collector:active 2025-08-08 09:37:29,807 - INFO - snmp-topology:active 2025-08-08 09:37:29,809 - INFO - analytics_alarm 2025-08-08 09:37:29,809 - INFO - alarmgen:active 2025-08-08 09:37:29,901 - INFO - 10.0.0.41 2025-08-08 09:37:29,902 - INFO - vrouter 2025-08-08 09:37:29,910 - INFO - vrouter-nodemgr:active 2025-08-08 09:37:29,918 - INFO - agent:active 2025-08-08 09:37:30,020 - INFO - 10.0.0.27 2025-08-08 09:37:30,020 - INFO - vrouter 2025-08-08 09:37:30,029 - INFO - vrouter-nodemgr:active 2025-08-08 09:37:30,036 - INFO - agent:active 2025-08-08 09:37:30,036 - DEBUG - defaultdict(, {'10.0.0.254': {'api-server': {'status': 'initializing', 'description': 'Database:Cassandra[] connection down'}}}) 2025-08-08 09:37:30,036 - DEBUG - Not all services up. Sleeping for 10 seconds. iteration: 2 2025-08-08 09:37:40,149 - INFO - 10.0.0.50 2025-08-08 09:37:40,149 - INFO - config 2025-08-08 09:37:40,160 - INFO - config-nodemgr:active 2025-08-08 09:37:40,165 - INFO - api-server:initializing (Generic Connection:Keystone[Error: Unable to establish connection to http://10.0.0.50:5000/v3/domains? at UTC 2025-08-08 09:37:31.346031] connection down) 2025-08-08 09:37:40,169 - INFO - schema:backup 2025-08-08 09:37:40,173 - INFO - svc-monitor:backup 2025-08-08 09:37:40,177 - INFO - device-manager:backup 2025-08-08 09:37:40,177 - INFO - config-database 2025-08-08 09:37:40,177 - INFO - config-cassandra:active 2025-08-08 09:37:40,177 - INFO - config-zookeeper:active 2025-08-08 09:37:40,177 - INFO - config-rabbitmq:active 2025-08-08 09:37:40,177 - INFO - control 2025-08-08 09:37:40,182 - INFO - control-nodemgr:active 2025-08-08 09:37:40,190 - INFO - control:active 2025-08-08 09:37:40,191 - INFO - named:active 2025-08-08 09:37:40,198 - INFO - dns:active 2025-08-08 09:37:40,198 - INFO - analytics 2025-08-08 09:37:40,204 - INFO - analytics-nodemgr:active 2025-08-08 09:37:40,209 - INFO - analytics-api:active 2025-08-08 09:37:40,215 - INFO - collector:active 2025-08-08 09:37:40,215 - INFO - analytics-database 2025-08-08 09:37:40,216 - INFO - analytics-cassandra:active 2025-08-08 09:37:40,221 - INFO - analyticsdb-nodemgr:active 2025-08-08 09:37:40,227 - INFO - query-engine:active 2025-08-08 09:37:40,227 - INFO - webui 2025-08-08 09:37:40,227 - INFO - webui:active 2025-08-08 09:37:40,227 - INFO - webui-middleware:active 2025-08-08 09:37:40,227 - INFO - redis:active 2025-08-08 09:37:40,227 - INFO - analytics_snmp 2025-08-08 09:37:40,232 - INFO - snmp-collector:active 2025-08-08 09:37:40,232 - INFO - snmp-topology:active 2025-08-08 09:37:40,233 - INFO - analytics_alarm 2025-08-08 09:37:40,233 - INFO - alarmgen:active 2025-08-08 09:37:40,339 - INFO - 10.0.0.254 2025-08-08 09:37:40,339 - INFO - config 2025-08-08 09:37:40,347 - INFO - config-nodemgr:active 2025-08-08 09:37:40,353 - INFO - api-server:initializing (Database:Cassandra[] connection down) 2025-08-08 09:37:40,358 - INFO - schema:backup 2025-08-08 09:37:40,363 - INFO - svc-monitor:backup 2025-08-08 09:37:40,367 - INFO - device-manager:backup 2025-08-08 09:37:40,367 - INFO - config-database 2025-08-08 09:37:40,367 - INFO - config-cassandra:active 2025-08-08 09:37:40,367 - INFO - config-zookeeper:active 2025-08-08 09:37:40,368 - INFO - config-rabbitmq:active 2025-08-08 09:37:40,368 - INFO - control 2025-08-08 09:37:40,374 - INFO - control-nodemgr:active 2025-08-08 09:37:40,380 - INFO - control:active 2025-08-08 09:37:40,381 - INFO - named:active 2025-08-08 09:37:40,393 - INFO - dns:active 2025-08-08 09:37:40,393 - INFO - analytics 2025-08-08 09:37:40,400 - INFO - analytics-nodemgr:active 2025-08-08 09:37:40,406 - INFO - analytics-api:active 2025-08-08 09:37:40,412 - INFO - collector:active 2025-08-08 09:37:40,413 - INFO - analytics-database 2025-08-08 09:37:40,413 - INFO - analytics-cassandra:active 2025-08-08 09:37:40,420 - INFO - analyticsdb-nodemgr:active 2025-08-08 09:37:40,428 - INFO - query-engine:active 2025-08-08 09:37:40,428 - INFO - webui 2025-08-08 09:37:40,429 - INFO - webui:active 2025-08-08 09:37:40,429 - INFO - webui-middleware:active 2025-08-08 09:37:40,429 - INFO - redis:active 2025-08-08 09:37:40,429 - INFO - analytics_snmp 2025-08-08 09:37:40,435 - INFO - snmp-collector:active 2025-08-08 09:37:40,436 - INFO - snmp-topology:active 2025-08-08 09:37:40,436 - INFO - analytics_alarm 2025-08-08 09:37:40,436 - INFO - alarmgen:active 2025-08-08 09:37:40,525 - INFO - 10.0.0.65 2025-08-08 09:37:40,525 - INFO - config 2025-08-08 09:37:40,533 - INFO - config-nodemgr:active 2025-08-08 09:37:40,538 - INFO - api-server:active 2025-08-08 09:37:40,544 - INFO - schema:active 2025-08-08 09:37:40,549 - INFO - svc-monitor:active 2025-08-08 09:37:40,555 - INFO - device-manager:active 2025-08-08 09:37:40,555 - INFO - config-database 2025-08-08 09:37:40,555 - INFO - config-cassandra:active 2025-08-08 09:37:40,555 - INFO - config-zookeeper:active 2025-08-08 09:37:40,556 - INFO - config-rabbitmq:active 2025-08-08 09:37:40,556 - INFO - control 2025-08-08 09:37:40,561 - INFO - control-nodemgr:active 2025-08-08 09:37:40,567 - INFO - control:active 2025-08-08 09:37:40,567 - INFO - named:active 2025-08-08 09:37:40,573 - INFO - dns:active 2025-08-08 09:37:40,573 - INFO - analytics 2025-08-08 09:37:40,579 - INFO - analytics-nodemgr:active 2025-08-08 09:37:40,585 - INFO - analytics-api:active 2025-08-08 09:37:40,591 - INFO - collector:active 2025-08-08 09:37:40,592 - INFO - analytics-database 2025-08-08 09:37:40,592 - INFO - analytics-cassandra:active 2025-08-08 09:37:40,598 - INFO - analyticsdb-nodemgr:active 2025-08-08 09:37:40,607 - INFO - query-engine:active 2025-08-08 09:37:40,607 - INFO - webui 2025-08-08 09:37:40,607 - INFO - webui:active 2025-08-08 09:37:40,607 - INFO - webui-middleware:active 2025-08-08 09:37:40,608 - INFO - redis:active 2025-08-08 09:37:40,608 - INFO - analytics_snmp 2025-08-08 09:37:40,614 - INFO - snmp-collector:active 2025-08-08 09:37:40,614 - INFO - snmp-topology:active 2025-08-08 09:37:40,614 - INFO - analytics_alarm 2025-08-08 09:37:40,614 - INFO - alarmgen:active 2025-08-08 09:37:40,704 - INFO - 10.0.0.41 2025-08-08 09:37:40,704 - INFO - vrouter 2025-08-08 09:37:40,711 - INFO - vrouter-nodemgr:active 2025-08-08 09:37:40,718 - INFO - agent:active 2025-08-08 09:37:40,818 - INFO - 10.0.0.27 2025-08-08 09:37:40,818 - INFO - vrouter 2025-08-08 09:37:40,824 - INFO - vrouter-nodemgr:active 2025-08-08 09:37:40,832 - INFO - agent:active 2025-08-08 09:37:40,832 - DEBUG - defaultdict(, {'10.0.0.50': {'api-server': {'status': 'initializing', 'description': 'Generic Connection:Keystone[Error: Unable to establish connection to http://10.0.0.50:5000/v3/domains? at UTC 2025-08-08 09:37:31.346031] connection down'}}, '10.0.0.254': {'api-server': {'status': 'initializing', 'description': 'Database:Cassandra[] connection down'}}}) 2025-08-08 09:37:40,832 - DEBUG - Not all services up. Sleeping for 10 seconds. iteration: 3 2025-08-08 09:37:50,956 - INFO - 10.0.0.50 2025-08-08 09:37:50,956 - INFO - config 2025-08-08 09:37:50,962 - INFO - config-nodemgr:active 2025-08-08 09:37:50,968 - INFO - api-server:initializing (Generic Connection:Keystone[Error: Unable to establish connection to http://10.0.0.50:5000/v3/domains? at UTC 2025-08-08 09:37:31.346031] connection down) 2025-08-08 09:37:50,972 - INFO - schema:backup 2025-08-08 09:37:50,977 - INFO - svc-monitor:backup 2025-08-08 09:37:50,981 - INFO - device-manager:backup 2025-08-08 09:37:50,981 - INFO - config-database 2025-08-08 09:37:50,981 - INFO - config-cassandra:active 2025-08-08 09:37:50,982 - INFO - config-zookeeper:active 2025-08-08 09:37:50,982 - INFO - config-rabbitmq:active 2025-08-08 09:37:50,982 - INFO - control 2025-08-08 09:37:50,988 - INFO - control-nodemgr:active 2025-08-08 09:37:50,996 - INFO - control:active 2025-08-08 09:37:50,997 - INFO - named:active 2025-08-08 09:37:51,004 - INFO - dns:active 2025-08-08 09:37:51,004 - INFO - analytics 2025-08-08 09:37:51,011 - INFO - analytics-nodemgr:active 2025-08-08 09:37:51,018 - INFO - analytics-api:active 2025-08-08 09:37:51,025 - INFO - collector:active 2025-08-08 09:37:51,026 - INFO - analytics-database 2025-08-08 09:37:51,026 - INFO - analytics-cassandra:active 2025-08-08 09:37:51,035 - INFO - analyticsdb-nodemgr:active 2025-08-08 09:37:51,046 - INFO - query-engine:active 2025-08-08 09:37:51,046 - INFO - webui 2025-08-08 09:37:51,047 - INFO - webui:active 2025-08-08 09:37:51,047 - INFO - webui-middleware:active 2025-08-08 09:37:51,047 - INFO - redis:active 2025-08-08 09:37:51,047 - INFO - analytics_snmp 2025-08-08 09:37:51,055 - INFO - snmp-collector:active 2025-08-08 09:37:51,056 - INFO - snmp-topology:active 2025-08-08 09:37:51,056 - INFO - analytics_alarm 2025-08-08 09:37:51,056 - INFO - alarmgen:active 2025-08-08 09:37:51,159 - INFO - 10.0.0.254 2025-08-08 09:37:51,159 - INFO - config 2025-08-08 09:37:51,166 - INFO - config-nodemgr:active 2025-08-08 09:37:51,172 - INFO - api-server:initializing (Database:Cassandra[] connection down) 2025-08-08 09:37:51,177 - INFO - schema:backup 2025-08-08 09:37:51,182 - INFO - svc-monitor:backup 2025-08-08 09:37:51,186 - INFO - device-manager:backup 2025-08-08 09:37:51,186 - INFO - config-database 2025-08-08 09:37:51,187 - INFO - config-cassandra:active 2025-08-08 09:37:51,187 - INFO - config-zookeeper:active 2025-08-08 09:37:51,187 - INFO - config-rabbitmq:active 2025-08-08 09:37:51,187 - INFO - control 2025-08-08 09:37:51,192 - INFO - control-nodemgr:active 2025-08-08 09:37:51,199 - INFO - control:active 2025-08-08 09:37:51,199 - INFO - named:active 2025-08-08 09:37:51,206 - INFO - dns:active 2025-08-08 09:37:51,206 - INFO - analytics 2025-08-08 09:37:51,212 - INFO - analytics-nodemgr:active 2025-08-08 09:37:51,217 - INFO - analytics-api:active 2025-08-08 09:37:51,224 - INFO - collector:active 2025-08-08 09:37:51,224 - INFO - analytics-database 2025-08-08 09:37:51,224 - INFO - analytics-cassandra:active 2025-08-08 09:37:51,231 - INFO - analyticsdb-nodemgr:active 2025-08-08 09:37:51,238 - INFO - query-engine:active 2025-08-08 09:37:51,239 - INFO - webui 2025-08-08 09:37:51,239 - INFO - webui:active 2025-08-08 09:37:51,239 - INFO - webui-middleware:active 2025-08-08 09:37:51,239 - INFO - redis:active 2025-08-08 09:37:51,239 - INFO - analytics_snmp 2025-08-08 09:37:51,244 - INFO - snmp-collector:active 2025-08-08 09:37:51,244 - INFO - snmp-topology:active 2025-08-08 09:37:51,245 - INFO - analytics_alarm 2025-08-08 09:37:51,245 - INFO - alarmgen:active 2025-08-08 09:37:51,333 - INFO - 10.0.0.65 2025-08-08 09:37:51,333 - INFO - config 2025-08-08 09:37:51,341 - INFO - config-nodemgr:active 2025-08-08 09:37:51,348 - INFO - api-server:active 2025-08-08 09:37:51,355 - INFO - schema:active 2025-08-08 09:37:51,361 - INFO - svc-monitor:active 2025-08-08 09:37:51,367 - INFO - device-manager:active 2025-08-08 09:37:51,367 - INFO - config-database 2025-08-08 09:37:51,367 - INFO - config-cassandra:active 2025-08-08 09:37:51,367 - INFO - config-zookeeper:active 2025-08-08 09:37:51,367 - INFO - config-rabbitmq:active 2025-08-08 09:37:51,367 - INFO - control 2025-08-08 09:37:51,374 - INFO - control-nodemgr:active 2025-08-08 09:37:51,380 - INFO - control:active 2025-08-08 09:37:51,381 - INFO - named:active 2025-08-08 09:37:51,387 - INFO - dns:active 2025-08-08 09:37:51,387 - INFO - analytics 2025-08-08 09:37:51,394 - INFO - analytics-nodemgr:active 2025-08-08 09:37:51,400 - INFO - analytics-api:active 2025-08-08 09:37:51,407 - INFO - collector:active 2025-08-08 09:37:51,407 - INFO - analytics-database 2025-08-08 09:37:51,407 - INFO - analytics-cassandra:active 2025-08-08 09:37:51,413 - INFO - analyticsdb-nodemgr:active 2025-08-08 09:37:51,426 - INFO - query-engine:active 2025-08-08 09:37:51,426 - INFO - webui 2025-08-08 09:37:51,426 - INFO - webui:active 2025-08-08 09:37:51,426 - INFO - webui-middleware:active 2025-08-08 09:37:51,426 - INFO - redis:active 2025-08-08 09:37:51,426 - INFO - analytics_snmp 2025-08-08 09:37:51,433 - INFO - snmp-collector:active 2025-08-08 09:37:51,433 - INFO - snmp-topology:active 2025-08-08 09:37:51,433 - INFO - analytics_alarm 2025-08-08 09:37:51,434 - INFO - alarmgen:active 2025-08-08 09:37:51,535 - INFO - 10.0.0.41 2025-08-08 09:37:51,535 - INFO - vrouter 2025-08-08 09:37:51,543 - INFO - vrouter-nodemgr:active 2025-08-08 09:37:51,551 - INFO - agent:active 2025-08-08 09:37:51,640 - INFO - 10.0.0.27 2025-08-08 09:37:51,640 - INFO - vrouter 2025-08-08 09:37:51,646 - INFO - vrouter-nodemgr:active 2025-08-08 09:37:51,653 - INFO - agent:active 2025-08-08 09:37:51,653 - DEBUG - defaultdict(, {'10.0.0.50': {'api-server': {'status': 'initializing', 'description': 'Generic Connection:Keystone[Error: Unable to establish connection to http://10.0.0.50:5000/v3/domains? at UTC 2025-08-08 09:37:31.346031] connection down'}}, '10.0.0.254': {'api-server': {'status': 'initializing', 'description': 'Database:Cassandra[] connection down'}}}) 2025-08-08 09:37:51,653 - ERROR - Not all services up , Gave up! 2025-08-08 09:37:51,653 - INFO - Failed services are : defaultdict(, {'10.0.0.50': {'api-server': {'status': 'initializing', 'description': 'Generic Connection:Keystone[Error: Unable to establish connection to http://10.0.0.50:5000/v3/domains? at UTC 2025-08-08 09:37:31.346031] connection down'}}, '10.0.0.254': {'api-server': {'status': 'initializing', 'description': 'Database:Cassandra[] connection down'}}}) 2025-08-08 09:37:51,653 - ERROR - contrail-status failed 2025-08-08 09:37:52,938 - DEBUG - Skipping xmpp flap check 2025-08-08 09:37:52,939 - INFO - 2025-08-08 09:37:52,939 - INFO - END TEST : test_contrail_status : FAILED[0:00:46] 2025-08-08 09:37:52,939 - INFO - --------------------------------------------------------------------------------