11/13/2025 01:39:55.731 7fbcec063720 [contrail-topology] [ERROR]: Cannot write http_port 5921 to /tmp/contrail-topology.0.http_port 11/13/2025 01:39:55.731 7fbcec063720 [contrail-topology] [ERROR]: Starting Introspect on HTTP Port 5921 11/13/2025 01:39:55.732 7fbcec086680 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.33:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Zookeeper:Zookeeper[Connection to Zookeeper initialized] connection down >>, ] >> 11/13/2025 01:39:55.732 7fbcec086680 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.33:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Zookeeper:Zookeeper[Connection to Zookeeper initialized], Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/13/2025 01:39:55.732 7fbcec086680 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local:AnalyticsSNMP:contrail-topology:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5921 start_time = 1762997995732300 collector_name = collector_ip = collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 2 >> 11/13/2025 01:39:55.906 7fbcec0869a0 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.33:2181, ] status = Up description = Connection to Zookeeper established >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/13/2025 01:39:56.954 7fbcebfd8040 [contrail-topology] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.33:5673//'] 11/13/2025 01:39:56.962 7fbcebfd8040 [contrail-topology] [WARNING]: RabbitMQ drainer connection down 11/13/2025 01:39:56.962 7fbceb932540 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.33:2181, ] status = Up description = Connection to Zookeeper established >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.33:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/13/2025 01:39:56.963 7fbceb932540 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.33:2181, ] status = Up description = Connection to Zookeeper established >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.33:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/13/2025 01:39:56.975 7fbcebfd8040 [contrail-topology] [WARNING]: RabbitMQ drainer connection ESTABLISHED 11/13/2025 01:39:56.976 7fbcebfd84a0 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.33:2181, ] status = Up description = Connection to Zookeeper established >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.33:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/13/2025 01:39:57.017 7fbcebfd8040 [contrail-topology] [WARNING]: RabbitMQ publish connection down 11/13/2025 01:39:57.019 7fbceb9537c0 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.33:2181, ] status = Up description = Connection to Zookeeper established >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.33:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/13/2025 01:39:57.033 7fbcebfd8040 [contrail-topology] [WARNING]: RabbitMQ publish connection ESTABLISHED 11/13/2025 01:39:57.034 7fbceb9537c0 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.33:2181, ] status = Up description = Connection to Zookeeper established >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.33:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/13/2025 01:39:57.035 7fbcebfd8040 [contrail-topology] [WARNING]: VNCCassandra started with driver cql 11/13/2025 01:39:57.035 7fbcebfd8040 [contrail-topology] [WARNING]: initializing cassandra cluster 11/13/2025 01:39:57.035 7fbcebfd8040 [contrail-topology] [WARNING]: CassandraDriverCQL.__init__Cluster before create 11/13/2025 01:39:57.035 7fbceb953c20 [contrail-topology] [WARNING]: RabbitMQ publish connection down 11/13/2025 01:39:57.036 7fbceb953f40 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.33:2181, ] status = Up description = Connection to Zookeeper established >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.33:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 11/13/2025 01:39:57.036 7fbceb953f40 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.33:2181, ] status = Up description = Connection to Zookeeper established >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.33:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.33:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 11/13/2025 01:39:57.038 7fbcebfd8040 [contrail-topology] [WARNING]: CassandraDriverCQL.__init__Cluster created 11/13/2025 01:39:57.057 7fbceb953c20 [contrail-topology] [WARNING]: RabbitMQ publish connection ESTABLISHED 11/13/2025 01:39:57.057 7fbceb8ff0e0 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.33:2181, ] status = Up description = Connection to Zookeeper established >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.33:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 11/13/2025 01:39:57.150 7fbcebfd8040 [contrail-topology] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 11/13/2025 01:39:57.153 7fbcee654180 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.33:2181, ] status = Initializing description = >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.33:9041, ] status = Initializing description = >>, ] description = Zookeeper:Zookeeper[], Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 11/13/2025 01:39:57.234 7fbcebfd8040 [contrail-topology] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 11/13/2025 01:39:57.235 7fbcee654680 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.33:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.33:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 11/13/2025 01:39:58.248 7fbcebfd8040 [contrail-topology] [WARNING]: Waiting for keyspace config_db_uuid to be created 11/13/2025 01:39:59.262 7fbcebfd8040 [contrail-topology] [WARNING]: Waiting for keyspace config_db_uuid to be created 11/13/2025 01:39:59.733 7fbcee654d60 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.33:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.33:9041, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 11/13/2025 01:40:00.279 7fbcebfd8040 [contrail-topology] [WARNING]: Waiting for keyspace config_db_uuid to be created 11/13/2025 01:40:01.299 7fbcebfd8040 [contrail-topology] [WARNING]: Waiting for keyspace config_db_uuid to be created 11/13/2025 01:40:02.316 7fbcebfd8040 [contrail-topology] [WARNING]: Waiting for keyspace config_db_uuid to be created 11/13/2025 01:40:03.352 7fbcebfd8040 [contrail-topology] [WARNING]: Waiting for keyspace config_db_uuid to be created 11/13/2025 01:40:04.391 7fbcebfd8040 [contrail-topology] [WARNING]: Waiting for keyspace config_db_uuid to be created 11/13/2025 01:40:04.407 7fbcebfd8040 [contrail-topology] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 11/13/2025 01:40:04.429 7fbcebfd8040 [contrail-topology] [ERROR]: Exception NoHostAvailable connecting to Config DB. Arguments: ("Unable to connect to any servers using keyspace 'config_db_uuid'", ['10.0.0.33']): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/tf_topology/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 441, in _Init_Cluster self.create_session(self.keyspace(ks), cf_name) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 183, in create_session return self._Create_Session(keyspace, cf_name, **cf_args) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 454, in _Create_Session self._cf_dict[cf_name] = self._cluster.connect( File "/usr/local/lib64/python3.9/site-packages/cassandra/cluster.py", line 1765, in connect session = self._new_session(keyspace) File "/usr/local/lib64/python3.9/site-packages/cassandra/cluster.py", line 1822, in _new_session session = Session(self, self.metadata.all_hosts(), keyspace) File "/usr/local/lib64/python3.9/site-packages/cassandra/cluster.py", line 2603, in __init__ raise NoHostAvailable(msg, [h.address for h in hosts]) cassandra.cluster.NoHostAvailable: ("Unable to connect to any servers using keyspace 'config_db_uuid'", ['10.0.0.33']) 11/13/2025 01:40:04.432 7fbcef9622c0 [contrail-topology] [ERROR]: stopping everything! 11/13/2025 01:40:04.432 7fbcef9622c0 [contrail-topology] [ERROR]: Stopped http server 11/13/2025 01:40:04.432 7fbcef9622c0 [contrail-topology] [ERROR]: killing 1 of 2 11/13/2025 01:40:04.432 7fbcef9622c0 [contrail-topology] [ERROR]: joining 1 of 2 11/13/2025 01:40:04.432 7fbcef9622c0 [contrail-topology] [ERROR]: stopped 1 of 2 11/13/2025 01:40:04.432 7fbcef9622c0 [contrail-topology] [ERROR]: killing 2 of 2 11/13/2025 01:40:04.432 7fbcef9622c0 [contrail-topology] [ERROR]: joining 2 of 2 11/13/2025 01:40:04.437 7fbcef9622c0 [contrail-topology] [ERROR]: stopped 2 of 2 11/13/2025 01:40:16.242 7f214db02720 [contrail-topology] [ERROR]: Cannot write http_port 5921 to /tmp/contrail-topology.0.http_port 11/13/2025 01:40:16.242 7f214db02720 [contrail-topology] [ERROR]: Starting Introspect on HTTP Port 5921 11/13/2025 01:40:16.243 7f214db25680 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.33:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Zookeeper:Zookeeper[Connection to Zookeeper initialized] connection down >>, ] >> 11/13/2025 01:40:16.243 7f214db25680 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.33:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Zookeeper:Zookeeper[Connection to Zookeeper initialized], Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/13/2025 01:40:16.243 7f214db25680 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local:AnalyticsSNMP:contrail-topology:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5921 start_time = 1762998016242817 collector_name = collector_ip = collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 2 >> 11/13/2025 01:40:16.257 7f214db259a0 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.33:2181, ] status = Up description = Connection to Zookeeper established >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/13/2025 01:40:16.448 7f214da77040 [contrail-topology] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.33:5673//'] 11/13/2025 01:40:16.458 7f214da77040 [contrail-topology] [WARNING]: RabbitMQ drainer connection down 11/13/2025 01:40:16.459 7f214d3d1540 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.33:2181, ] status = Up description = Connection to Zookeeper established >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.33:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/13/2025 01:40:16.459 7f214d3d1540 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.33:2181, ] status = Up description = Connection to Zookeeper established >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.33:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/13/2025 01:40:16.474 7f214da77040 [contrail-topology] [WARNING]: RabbitMQ drainer connection ESTABLISHED 11/13/2025 01:40:16.474 7f214d3d1540 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.33:2181, ] status = Up description = Connection to Zookeeper established >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.33:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/13/2025 01:40:16.526 7f214da77040 [contrail-topology] [WARNING]: RabbitMQ publish connection down 11/13/2025 01:40:16.527 7f214d3f2ae0 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.33:2181, ] status = Up description = Connection to Zookeeper established >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.33:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/13/2025 01:40:16.539 7f214da77040 [contrail-topology] [WARNING]: RabbitMQ publish connection ESTABLISHED 11/13/2025 01:40:16.539 7f214d3f2ae0 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.33:2181, ] status = Up description = Connection to Zookeeper established >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.33:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/13/2025 01:40:16.540 7f214da77040 [contrail-topology] [WARNING]: VNCCassandra started with driver cql 11/13/2025 01:40:16.540 7f214da77040 [contrail-topology] [WARNING]: initializing cassandra cluster 11/13/2025 01:40:16.540 7f214da77040 [contrail-topology] [WARNING]: CassandraDriverCQL.__init__Cluster before create 11/13/2025 01:40:16.540 7f214d3f2f40 [contrail-topology] [WARNING]: RabbitMQ publish connection down 11/13/2025 01:40:16.541 7f214d39e360 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.33:2181, ] status = Up description = Connection to Zookeeper established >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.33:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 11/13/2025 01:40:16.541 7f214d39e360 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.33:2181, ] status = Up description = Connection to Zookeeper established >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.33:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.33:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 11/13/2025 01:40:16.543 7f214da77040 [contrail-topology] [WARNING]: CassandraDriverCQL.__init__Cluster created 11/13/2025 01:40:16.577 7f214d3f2f40 [contrail-topology] [WARNING]: RabbitMQ publish connection ESTABLISHED 11/13/2025 01:40:16.577 7f214d39e540 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.33:2181, ] status = Up description = Connection to Zookeeper established >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.33:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 11/13/2025 01:40:16.724 7f214da77040 [contrail-topology] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 11/13/2025 01:40:16.725 7f214d39e540 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.33:2181, ] status = Initializing description = >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.33:9041, ] status = Initializing description = >>, ] description = Zookeeper:Zookeeper[], Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 11/13/2025 01:40:16.730 7f214da77040 [contrail-topology] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 11/13/2025 01:40:16.730 7f214d39e860 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.33:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.33:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 11/13/2025 01:40:16.757 7f214da77040 [contrail-topology] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 11/13/2025 01:40:16.829 7f214da77040 [contrail-topology] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 11/13/2025 01:40:16.830 7f214d336400 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.33:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.33:9041, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/13/2025 01:40:16.865 7f214d3f2c20 [contrail-topology] [ERROR]: Error in rabbitmq drainer greenlet: 'AnalyticsLogger' object has no attribute 'warn' 11/13/2025 01:40:16.866 7f214d3f2c20 [contrail-topology] [WARNING]: RabbitMQ drainer connection down 11/13/2025 01:40:16.867 7f214d39e0e0 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.33:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.33:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.33:9041, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/13/2025 01:40:16.881 7f214d3f2c20 [contrail-topology] [WARNING]: RabbitMQ drainer connection ESTABLISHED 11/13/2025 01:40:16.882 7f214d39e0e0 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.33:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.33:9041, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/13/2025 01:40:16.901 7f214d3f2c20 [contrail-topology] [ERROR]: Error in rabbitmq drainer greenlet: 'AnalyticsLogger' object has no attribute 'warn' 11/13/2025 01:40:16.902 7f214d3f2c20 [contrail-topology] [WARNING]: RabbitMQ drainer connection down 11/13/2025 01:40:16.902 7f214d39e0e0 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.33:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.33:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.33:9041, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/13/2025 01:40:16.922 7f214d3f2c20 [contrail-topology] [WARNING]: RabbitMQ drainer connection ESTABLISHED 11/13/2025 01:40:16.922 7f214d3364a0 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.33:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.33:9041, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/13/2025 01:40:17.113 7f214d3f2c20 [contrail-topology] [ERROR]: Error in rabbitmq drainer greenlet: 'AnalyticsLogger' object has no attribute 'warn' 11/13/2025 01:40:17.113 7f214d3f2c20 [contrail-topology] [WARNING]: RabbitMQ drainer connection down 11/13/2025 01:40:17.114 7f214d336900 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.33:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.33:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.33:9041, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/13/2025 01:40:17.130 7f214d3f2c20 [contrail-topology] [WARNING]: RabbitMQ drainer connection ESTABLISHED 11/13/2025 01:40:17.130 7f214d336900 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.33:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.33:9041, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/13/2025 01:40:20.245 7f214d39ed60 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.33:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.33:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.33:9041, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 01:40:47.709 7f214db259a0 [contrail-topology] [WARNING]: Failed to retrieve vRouter data for cn-jenkins-deploy-platform-k8s-manifests-270-1.cluster.local: 'VRouterL2IfInfo'