11/09/2026 01:47:33.731 7f03aad04180 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-439-1.cluster.local process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/09/2026 01:47:33.731 7f03aad04180 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-k8s-manifests-439-1.cluster.local:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 106 start_time = 1794188853730806 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 11/09/2026 01:47:33.815 7f03af9fc1c0 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.50:5673//'] 11/09/2026 01:47:33.832 7f03af9fc1c0 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 11/09/2026 01:47:33.833 7f03aad04900 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-439-1.cluster.local process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/09/2026 01:47:33.833 7f03aad04900 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-439-1.cluster.local process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 11/09/2026 01:47:33.848 7f03af9fc1c0 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 11/09/2026 01:47:33.849 7f03aad04ae0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-439-1.cluster.local process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/09/2026 01:47:33.899 7f03af9fc1c0 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 11/09/2026 01:47:33.899 7f03af9fc1c0 [contrail-device-manager] [WARNING]: initializing cassandra cluster 11/09/2026 01:47:33.900 7f03af9fc1c0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 11/09/2026 01:47:33.900 7f03aaf8ec20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-439-1.cluster.local process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 11/09/2026 01:47:33.907 7f03af9fc1c0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 11/09/2026 01:47:34.062 7f03af9fc1c0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 11/09/2026 01:47:34.063 7f03aad98c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-439-1.cluster.local process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 11/09/2026 01:47:34.102 7f03af9fc1c0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 11/09/2026 01:47:34.103 7f03aad04900 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-439-1.cluster.local process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 11/09/2026 01:47:34.142 7f03af9fc1c0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 11/09/2026 01:47:34.251 7f03af9fc1c0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 11/09/2026 01:47:34.252 7f03ab0f1860 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-439-1.cluster.local process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/09/2026 01:47:34.303 7f03aaf8eae0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-439-1.cluster.local process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:Zookeeper[] connection down >>, ] >> 11/09/2026 01:47:34.463 7f03aaf8eae0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-439-1.cluster.local process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/09/2026 01:47:34.988 7f03af9fc1c0 [contrail-device-manager] [ERROR]: ZTP manager: Error restarting dnsmasq DockerException("Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))") 11/09/2026 01:47:35.489 7f03af9fc1c0 [contrail-device-manager] [WARNING]: Process 106 prepared to run in Full mode... 11/09/2026 01:47:35.490 7f03af9fc1c0 [contrail-device-manager] [WARNING]: Elected master Device Manager node. Initializing... 11/09/2026 01:47:36.017 7f03ab009360 [contrail-device-manager] [ERROR]: Cannot write http_port 8096 to /tmp/contrail-device-manager.1.http_port 11/09/2026 01:47:36.017 7f03ab009360 [contrail-device-manager] [ERROR]: Starting Introspect on HTTP Port 8096 11/09/2026 01:47:36.017 7f03a9f98680 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-439-1.cluster.local process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,:8082, ] status = Initializing description = ApiServer Connection State updated >>, ] description = Collector:Collector[none to Idle on EvStart], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 11/09/2026 01:47:36.022 7f03a9f98900 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-439-1.cluster.local process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[none to Idle on EvStart], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 11/09/2026 01:47:37.733 7f03a9f98c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-439-1.cluster.local process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 11/09/2026 01:47:39.043 7f03af9fc1c0 [contrail-device-manager] [ERROR]: predef payloads file does not exist: /opt/contrail/fabric_ansible_playbooks/conf/predef_payloads.json 11/09/2026 01:47:39.043 7f03af9fc1c0 [contrail-device-manager] [ERROR]: Unable to load init data 11/09/2026 01:47:39.043 7f03af9fc1c0 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 11/09/2026 01:47:39.044 7f03af9fc1c0 [contrail-device-manager] [WARNING]: initializing cassandra cluster 11/09/2026 01:47:39.044 7f03af9fc1c0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 11/09/2026 01:47:39.052 7f03af9fc1c0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 11/09/2026 01:47:39.131 7f03af9fc1c0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 11/09/2026 01:47:39.164 7f03af9fc1c0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 11/09/2026 01:47:39.325 7f03af9fc1c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/09/2026 01:47:39.418 7f03af9fc1c0 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace', created 11/09/2026 01:47:39.425 7f03af9fc1c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/09/2026 01:47:39.738 7f03af9fc1c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/09/2026 01:47:39.886 7f03af9fc1c0 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace' fixed with replication {'class': 'SimpleStrategy', 'replication_factor': None} 11/09/2026 01:47:39.910 7f03af9fc1c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/09/2026 01:47:39.910 7f03af9fc1c0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 11/09/2026 01:47:40.050 7f03af9fc1c0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 11/09/2026 01:47:40.108 7f03af9fc1c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/09/2026 01:47:40.265 7f03af9fc1c0 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table', created 11/09/2026 01:47:40.277 7f03af9fc1c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/09/2026 01:47:40.557 7f03af9fc1c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/09/2026 01:47:40.677 7f03af9fc1c0 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table' fixed with properties {'gc_grace_seconds': 864000} 11/09/2026 01:47:40.688 7f03af9fc1c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/09/2026 01:47:40.827 7f03af9fc1c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/09/2026 01:47:41.114 7f03af9fc1c0 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table', created 11/09/2026 01:47:41.129 7f03af9fc1c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/09/2026 01:47:41.405 7f03af9fc1c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/09/2026 01:47:41.594 7f03af9fc1c0 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table' fixed with properties {'gc_grace_seconds': 864000} 11/09/2026 01:47:41.606 7f03af9fc1c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/09/2026 01:47:41.832 7f03af9fc1c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/09/2026 01:47:42.183 7f03af9fc1c0 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table', created 11/09/2026 01:47:42.197 7f03af9fc1c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/09/2026 01:47:42.227 7f03af9fc1c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/09/2026 01:47:42.383 7f03af9fc1c0 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table' fixed with properties {'gc_grace_seconds': 864000} 11/09/2026 01:47:42.418 7f03af9fc1c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/09/2026 01:47:42.444 7f03af9fc1c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/09/2026 01:47:42.702 7f03af9fc1c0 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table', created 11/09/2026 01:47:42.715 7f03af9fc1c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/09/2026 01:47:42.743 7f03af9fc1c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 11/09/2026 01:47:42.895 7f03af9fc1c0 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table' fixed with properties {'gc_grace_seconds': 864000} 11/09/2026 01:47:42.904 7f03af9fc1c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 11/09/2026 01:47:42.932 7f03af9fc1c0 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.50:5673//'] 11/09/2026 01:47:42.933 7f03af9fc1c0 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 11/09/2026 01:47:42.958 7f03af9fc1c0 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED