10/23/2025 01:29:12.892 7fce1ed905e0 [contrail-topology] [ERROR]: Cannot write http_port 5921 to /tmp/contrail-topology.0.http_port 10/23/2025 01:29:12.893 7fce1ed905e0 [contrail-topology] [ERROR]: Starting Introspect on HTTP Port 5921 10/23/2025 01:29:12.956 7fce1edab360 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-264-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.23:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Zookeeper:Zookeeper[Connection to Zookeeper initialized] connection down >>, ] >> 10/23/2025 01:29:12.956 7fce1edab360 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-264-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.23:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Zookeeper:Zookeeper[Connection to Zookeeper initialized], Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 10/23/2025 01:29:12.956 7fce1edab360 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-k8s-manifests-264-1.cluster.local:AnalyticsSNMP:contrail-topology:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5921 start_time = 1761182952955977 collector_name = collector_ip = collector_list = [ 10.0.0.23:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 2 >> 10/23/2025 01:29:13.152 7fce1edab360 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-264-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.23:2181, ] status = Up description = Connection to Zookeeper established >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 10/23/2025 01:29:13.822 7fce1ee1a5e0 [contrail-topology] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.23:5673//'] 10/23/2025 01:29:13.835 7fce1ee1a5e0 [contrail-topology] [WARNING]: RabbitMQ drainer connection down 10/23/2025 01:29:13.838 7fce1eca4040 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-264-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.23:2181, ] status = Up description = Connection to Zookeeper established >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.23:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 10/23/2025 01:29:13.839 7fce1eca4040 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-264-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.23:2181, ] status = Up description = Connection to Zookeeper established >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.23:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 10/23/2025 01:29:13.881 7fce1ee1a5e0 [contrail-topology] [WARNING]: RabbitMQ drainer connection ESTABLISHED 10/23/2025 01:29:13.881 7fce1eca4040 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-264-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.23:2181, ] status = Up description = Connection to Zookeeper established >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.23:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 10/23/2025 01:29:13.915 7fce1ee1a5e0 [contrail-topology] [WARNING]: RabbitMQ publish connection down 10/23/2025 01:29:13.945 7fce1ec58180 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-264-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.23:2181, ] status = Up description = Connection to Zookeeper established >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.23:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 10/23/2025 01:29:13.977 7fce1ee1a5e0 [contrail-topology] [WARNING]: RabbitMQ publish connection ESTABLISHED 10/23/2025 01:29:13.977 7fce1ec58860 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-264-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.23:2181, ] status = Up description = Connection to Zookeeper established >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.23:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 10/23/2025 01:29:13.981 7fce1ee1a5e0 [contrail-topology] [WARNING]: VNCCassandra started with driver cql 10/23/2025 01:29:13.981 7fce1ee1a5e0 [contrail-topology] [WARNING]: initializing cassandra cluster 10/23/2025 01:29:13.981 7fce1ee1a5e0 [contrail-topology] [WARNING]: CassandraDriverCQL.__init__Cluster before create 10/23/2025 01:29:13.982 7fce1ec58b80 [contrail-topology] [WARNING]: RabbitMQ publish connection down 10/23/2025 01:29:13.982 7fce1ec78220 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-264-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.23:2181, ] status = Up description = Connection to Zookeeper established >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.23:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 10/23/2025 01:29:13.982 7fce1ec78220 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-264-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.23:2181, ] status = Up description = Connection to Zookeeper established >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.23:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.23:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 10/23/2025 01:29:13.986 7fce1ee1a5e0 [contrail-topology] [WARNING]: CassandraDriverCQL.__init__Cluster created 10/23/2025 01:29:14.027 7fce1ec58b80 [contrail-topology] [WARNING]: RabbitMQ publish connection ESTABLISHED 10/23/2025 01:29:14.028 7fce1ec78720 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-264-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.23:2181, ] status = Up description = Connection to Zookeeper established >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.23:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 10/23/2025 01:29:14.065 7fce1ee1a5e0 [contrail-topology] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 10/23/2025 01:29:14.076 7fce1e5ea7c0 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-264-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.23:2181, ] status = Initializing description = >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.23:9041, ] status = Initializing description = >>, ] description = Zookeeper:Zookeeper[], Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 10/23/2025 01:29:14.101 7fce1ee1a5e0 [contrail-topology] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 10/23/2025 01:29:14.102 7fce1e5ea680 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-264-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.23:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.23:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 10/23/2025 01:29:14.135 7fce1ee1a5e0 [contrail-topology] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 10/23/2025 01:29:14.211 7fce1ee1a5e0 [contrail-topology] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 10/23/2025 01:29:14.212 7fce1e60ad60 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-264-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.23:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.23:9041, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 10/23/2025 01:29:16.896 7fce1e60af40 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-264-1.cluster.local process_status = [ << module_id = contrail-topology instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.23:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.23:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.23:9041, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/23/2025 01:29:45.092 7fce1e5ea680 [contrail-topology] [WARNING]: Failed to retrieve vRouter data for cn-jenkins-deploy-platform-k8s-manifests-264-1.cluster.local: 'VRouterL2IfInfo'