10/12/2026 01:39:50.628 7fdbc03ef220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-435-1.cluster.local process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 10/12/2026 01:39:50.629 7fdbc03ef220 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-k8s-manifests-435-1.cluster.local:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 110 start_time = 1791769190628509 collector_name = collector_ip = collector_list = [ 10.0.0.161:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 10/12/2026 01:39:50.736 7fdbc504df80 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.161:5673//'] 10/12/2026 01:39:50.744 7fdbc504df80 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 10/12/2026 01:39:50.745 7fdbc03ef9a0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-435-1.cluster.local process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.161:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 10/12/2026 01:39:50.745 7fdbc03ef9a0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-435-1.cluster.local process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.161:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 10/12/2026 01:39:50.787 7fdbc504df80 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 10/12/2026 01:39:50.788 7fdbc03efb80 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-435-1.cluster.local process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.161:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 10/12/2026 01:39:50.933 7fdbc504df80 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 10/12/2026 01:39:50.933 7fdbc504df80 [contrail-device-manager] [WARNING]: initializing cassandra cluster 10/12/2026 01:39:50.933 7fdbc504df80 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 10/12/2026 01:39:50.934 7fdbc05f1cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-435-1.cluster.local process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.161:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.161:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 10/12/2026 01:39:50.935 7fdbc504df80 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 10/12/2026 01:39:51.054 7fdbc504df80 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 10/12/2026 01:39:51.056 7fdbc06a8cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-435-1.cluster.local process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.161:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.161:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.161:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 10/12/2026 01:39:51.071 7fdbc504df80 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 10/12/2026 01:39:51.072 7fdbc03ef9a0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-435-1.cluster.local process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.161:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.161:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.161:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 10/12/2026 01:39:52.087 7fdbc504df80 [contrail-device-manager] [WARNING]: Waiting for keyspace config_db_uuid to be created 10/12/2026 01:39:52.096 7fdbc504df80 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 10/12/2026 01:39:52.115 7fdbc504df80 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 10/12/2026 01:39:52.302 7fdbc504df80 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 10/12/2026 01:39:52.471 7fdbc504df80 [contrail-device-manager] [WARNING]: table 'obj_shared_table', created 10/12/2026 01:39:52.477 7fdbc504df80 [contrail-device-manager] [WARNING]: Zookeeper lock released. 10/12/2026 01:39:52.780 7fdbc504df80 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 10/12/2026 01:39:52.836 7fdbc504df80 [contrail-device-manager] [WARNING]: table 'obj_shared_table' fixed with properties {'gc_grace_seconds': 864000} 10/12/2026 01:39:52.839 7fdbc504df80 [contrail-device-manager] [WARNING]: Zookeeper lock released. 10/12/2026 01:39:52.840 7fdbc05f1040 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-435-1.cluster.local process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.161:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.161:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.161:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 10/12/2026 01:39:52.846 7fdbc05f1b80 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-435-1.cluster.local process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.161:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.161:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.161:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:Zookeeper[] connection down >>, ] >> 10/12/2026 01:39:52.873 7fdbc05f1b80 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-435-1.cluster.local process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.161:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.161:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.161:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 10/12/2026 01:39:53.378 7fdbc504df80 [contrail-device-manager] [ERROR]: ZTP manager: Error restarting dnsmasq DockerException("Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))") 10/12/2026 01:39:53.881 7fdbc504df80 [contrail-device-manager] [WARNING]: Process 110 prepared to run in Full mode... 10/12/2026 01:39:53.881 7fdbc504df80 [contrail-device-manager] [WARNING]: Elected master Device Manager node. Initializing... 10/12/2026 01:39:54.169 7fdbc0653680 [contrail-device-manager] [ERROR]: Cannot write http_port 8096 to /tmp/contrail-device-manager.1.http_port 10/12/2026 01:39:54.170 7fdbc0653680 [contrail-device-manager] [ERROR]: Starting Introspect on HTTP Port 8096 10/12/2026 01:39:54.170 7fdbbba21900 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-435-1.cluster.local process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.161:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.161:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.161:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.161,:8082, ] status = Initializing description = ApiServer Connection State updated >>, ] description = Collector:Collector[none to Idle on EvStart], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 10/12/2026 01:39:54.182 7fdbbba219a0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-435-1.cluster.local process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.161:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.161:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.161:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.161,:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[none to Idle on EvStart], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 10/12/2026 01:39:54.630 7fdbbba21b80 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-435-1.cluster.local process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.161:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.161:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.161:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.161:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.161,:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 10/12/2026 01:39:57.235 7fdbc504df80 [contrail-device-manager] [ERROR]: predef payloads file does not exist: /opt/contrail/fabric_ansible_playbooks/conf/predef_payloads.json 10/12/2026 01:39:57.235 7fdbc504df80 [contrail-device-manager] [ERROR]: Unable to load init data 10/12/2026 01:39:57.235 7fdbc504df80 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 10/12/2026 01:39:57.235 7fdbc504df80 [contrail-device-manager] [WARNING]: initializing cassandra cluster 10/12/2026 01:39:57.235 7fdbc504df80 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 10/12/2026 01:39:57.251 7fdbc504df80 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 10/12/2026 01:39:57.469 7fdbc504df80 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 10/12/2026 01:39:57.500 7fdbc504df80 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 10/12/2026 01:39:57.547 7fdbc504df80 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 10/12/2026 01:39:57.637 7fdbc504df80 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace', created 10/12/2026 01:39:57.642 7fdbc504df80 [contrail-device-manager] [WARNING]: Zookeeper lock released. 10/12/2026 01:39:57.655 7fdbc504df80 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 10/12/2026 01:39:57.694 7fdbc504df80 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace' fixed with replication {'class': 'SimpleStrategy', 'replication_factor': None} 10/12/2026 01:39:57.699 7fdbc504df80 [contrail-device-manager] [WARNING]: Zookeeper lock released. 10/12/2026 01:39:57.699 7fdbc504df80 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 10/12/2026 01:39:57.828 7fdbc504df80 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 10/12/2026 01:39:57.842 7fdbc504df80 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 10/12/2026 01:39:57.957 7fdbc504df80 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table', created 10/12/2026 01:39:57.960 7fdbc504df80 [contrail-device-manager] [WARNING]: Zookeeper lock released. 10/12/2026 01:39:57.976 7fdbc504df80 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 10/12/2026 01:39:58.069 7fdbc504df80 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table' fixed with properties {'gc_grace_seconds': 864000} 10/12/2026 01:39:58.074 7fdbc504df80 [contrail-device-manager] [WARNING]: Zookeeper lock released. 10/12/2026 01:39:58.085 7fdbc504df80 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 10/12/2026 01:39:58.242 7fdbc504df80 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table', created 10/12/2026 01:39:58.252 7fdbc504df80 [contrail-device-manager] [WARNING]: Zookeeper lock released. 10/12/2026 01:39:58.298 7fdbc504df80 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 10/12/2026 01:39:58.420 7fdbc504df80 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table' fixed with properties {'gc_grace_seconds': 864000} 10/12/2026 01:39:58.429 7fdbc504df80 [contrail-device-manager] [WARNING]: Zookeeper lock released. 10/12/2026 01:39:58.442 7fdbc504df80 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 10/12/2026 01:39:58.594 7fdbc504df80 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table', created 10/12/2026 01:39:58.604 7fdbc504df80 [contrail-device-manager] [WARNING]: Zookeeper lock released. 10/12/2026 01:39:58.625 7fdbc504df80 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 10/12/2026 01:39:58.808 7fdbc504df80 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table' fixed with properties {'gc_grace_seconds': 864000} 10/12/2026 01:39:58.814 7fdbc504df80 [contrail-device-manager] [WARNING]: Zookeeper lock released. 10/12/2026 01:39:58.821 7fdbc504df80 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 10/12/2026 01:39:58.953 7fdbc504df80 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table', created 10/12/2026 01:39:58.959 7fdbc504df80 [contrail-device-manager] [WARNING]: Zookeeper lock released. 10/12/2026 01:39:58.971 7fdbc504df80 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 10/12/2026 01:39:59.047 7fdbc504df80 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table' fixed with properties {'gc_grace_seconds': 864000} 10/12/2026 01:39:59.052 7fdbc504df80 [contrail-device-manager] [WARNING]: Zookeeper lock released. 10/12/2026 01:39:59.072 7fdbc504df80 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.161:5673//'] 10/12/2026 01:39:59.073 7fdbc504df80 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 10/12/2026 01:39:59.083 7fdbc504df80 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED