09/29/2025 04:56:32.350 7fc385348b48 [contrail-device-manager] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/29/2025 04:56:32.351 7fc385348b48 [contrail-device-manager] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/29/2025 04:56:32.351 7fc385348b48 [contrail-device-manager] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/29/2025 04:56:32.351 7fc37ca42048 [contrail-device-manager] [INFO]: Processing event[EvStart] in state[none] 09/29/2025 04:56:32.351 7fc37ca42048 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectConfigNode:NodeStatusUVE] map 09/29/2025 04:56:32.351 7fc37ca42048 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/29/2025 04:56:32.352 7fc37ca42048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/29/2025 04:56:32.352 7fc37ca42048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:32.352 7fc37ca42048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/29/2025 04:56:32.352 7fc37ca42048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:32.352 7fc37ca42048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:32.352 7fc37ca42048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 start_time = 1759121792351872 collector_name = collector_ip = collector_list = [ 10.0.0.254:8086, 10.0.0.48:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/29/2025 04:56:32.352 7fc37ca42048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:32.400 7fc385348b48 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.254:5673//', 'pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.48:5673//'] 09/29/2025 04:56:32.410 7fc385348b48 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 09/29/2025 04:56:32.410 7fc37c9b72e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:32.410 7fc37c9b72e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 09/29/2025 04:56:32.411 7fc37c9b72e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:32.411 7fc37c9b72e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:32.411 7fc37c9b72e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 09/29/2025 04:56:32.411 7fc37c9b72e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:32.414 7fc385348b48 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/29/2025 04:56:32.414 7fc37c9b7828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:32.414 7fc37c9b7828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/29/2025 04:56:32.414 7fc37c9b7828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:32.451 7fc385348b48 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 09/29/2025 04:56:32.452 7fc37c9a54e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:32.452 7fc37c9a54e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 09/29/2025 04:56:32.452 7fc37c9a54e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:32.454 7fc385348b48 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/29/2025 04:56:32.455 7fc38b243978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:32.455 7fc38b243978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/29/2025 04:56:32.455 7fc38b243978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:32.646 7fc385348b48 [contrail-device-manager] [DEBUG]: Removed remaining AMQP queue from previous run 09/29/2025 04:56:32.646 7fc385348b48 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 09/29/2025 04:56:32.646 7fc385348b48 [contrail-device-manager] [WARNING]: initializing cassandra cluster 09/29/2025 04:56:32.647 7fc385348b48 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/29/2025 04:56:32.647 7fc3822c96d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting consumer greenlet 09/29/2025 04:56:32.647 7fc3822c96d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 09/29/2025 04:56:32.648 7fc37c9b7ac8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting publisher greenlet 09/29/2025 04:56:32.648 7fc37c9b7ac8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Publisher connection 09/29/2025 04:56:32.648 7fc37c82d048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:32.648 7fc37c82d048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 09/29/2025 04:56:32.648 7fc37c82d048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:32.650 7fc385348b48 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/29/2025 04:56:32.652 7fc37c9b7ac8 [contrail-device-manager] [INFO]: KombuAmqpClient: Publisher connection established 09/29/2025 04:56:32.652 7fc3822c96d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 09/29/2025 04:56:32.652 7fc3822c96d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Waiting for consumer 09/29/2025 04:56:32.692 7fc385348b48 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/29/2025 04:56:32.694 7fc37c7c12e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:32.694 7fc37c7c12e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/29/2025 04:56:32.694 7fc37c7c12e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:32.701 7fc385348b48 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 09/29/2025 04:56:32.702 7fc37c74b978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:32.702 7fc37c74b978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 09/29/2025 04:56:32.702 7fc37c74b978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:33.721 7fc385348b48 [contrail-device-manager] [WARNING]: Waiting for keyspace config_db_uuid to be created 09/29/2025 04:56:33.730 7fc385348b48 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 09/29/2025 04:56:33.757 7fc385348b48 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 09/29/2025 04:56:34.515 7fc385348b48 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/29/2025 04:56:35.614 7fc385348b48 [contrail-device-manager] [WARNING]: table 'obj_uuid_table', created 09/29/2025 04:56:35.621 7fc385348b48 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/29/2025 04:56:36.352 7fc37c7138d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 04:56:36.353 7fc37c7138d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 04:56:36.354 7fc37c7138d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 04:56:36.354 7fc37c7138d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:56:36.354 7fc37c7138d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 09/29/2025 04:56:36.354 7fc37c7138d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:56:36.354 7fc37c7138d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 04:56:36.354 7fc37c7138d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 04:56:36.354 7fc37c7138d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:36.354 7fc37c7138d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/29/2025 04:56:36.354 7fc37c7138d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:36.355 7fc37c7138d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:36.355 7fc37c7138d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 start_time = 1759121792351872 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.48:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 04:56:36.355 7fc37c7138d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:37.671 7fc385348b48 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/29/2025 04:56:39.215 7fc385348b48 [contrail-device-manager] [WARNING]: table 'obj_uuid_table' fixed with properties {'gc_grace_seconds': 864000} 09/29/2025 04:56:39.226 7fc385348b48 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/29/2025 04:56:40.355 7fc37c715240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 04:56:40.356 7fc37c715240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 04:56:40.356 7fc37c715240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 04:56:40.356 7fc37c715240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:56:40.357 7fc37c715240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 09/29/2025 04:56:40.357 7fc37c715240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:56:40.357 7fc37c715240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 04:56:40.357 7fc37c715240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 04:56:40.357 7fc37c715240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:40.357 7fc37c715240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/29/2025 04:56:40.357 7fc37c715240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:40.357 7fc37c715240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:40.357 7fc37c715240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 start_time = 1759121792351872 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.48:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 04:56:40.358 7fc37c715240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:40.610 7fc385348b48 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/29/2025 04:56:41.582 7fc385348b48 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table', created 09/29/2025 04:56:41.587 7fc385348b48 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/29/2025 04:56:43.729 7fc385348b48 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/29/2025 04:56:44.357 7fc37c6a8ac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 04:56:44.358 7fc37c6a8ac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 04:56:44.358 7fc37c6a8ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 04:56:44.358 7fc37c6a8ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:56:44.358 7fc37c6a8ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 09/29/2025 04:56:44.358 7fc37c6a8ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:56:44.359 7fc37c6a8ac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 04:56:44.359 7fc37c6a8ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 04:56:44.359 7fc37c6a8ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:44.359 7fc37c6a8ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/29/2025 04:56:44.359 7fc37c6a8ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:44.359 7fc37c6a8ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:44.359 7fc37c6a8ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 start_time = 1759121792351872 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.48:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 04:56:44.359 7fc37c6a8ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:44.822 7fc385348b48 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table' fixed with properties {'gc_grace_seconds': 864000} 09/29/2025 04:56:44.827 7fc385348b48 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/29/2025 04:56:46.679 7fc385348b48 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/29/2025 04:56:46.684 7fc385348b48 [contrail-device-manager] [WARNING]: table 'obj_shared_table', created 09/29/2025 04:56:46.687 7fc385348b48 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/29/2025 04:56:48.359 7fc37c6a3eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 04:56:48.360 7fc37c6a3eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 04:56:48.360 7fc37c6a3eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 04:56:48.361 7fc37c6a3eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:56:48.361 7fc37c6a3eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 09/29/2025 04:56:48.361 7fc37c6a3eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:56:48.361 7fc37c6a3eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 04:56:48.361 7fc37c6a3eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 04:56:48.361 7fc37c6a3eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:48.361 7fc37c6a3eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/29/2025 04:56:48.361 7fc37c6a3eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:48.361 7fc37c6a3eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:48.362 7fc37c6a3eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 start_time = 1759121792351872 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.48:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 04:56:48.362 7fc37c6a3eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:48.601 7fc385348b48 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/29/2025 04:56:49.713 7fc385348b48 [contrail-device-manager] [WARNING]: table 'obj_shared_table' fixed with properties {'gc_grace_seconds': 864000} 09/29/2025 04:56:49.717 7fc385348b48 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/29/2025 04:56:49.718 7fc37d5e1ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:49.718 7fc37d5e1ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 04:56:49.718 7fc37d5e1ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:49.735 7fc385348b48 [contrail-device-manager] [ERROR]: Error in db walk read 'type' 09/29/2025 04:56:49.736 7fc37c6b5240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:49.736 7fc37c6b5240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Zookeeper:Zookeeper[] connection down >>, ] >> 09/29/2025 04:56:49.737 7fc37c6b5240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:49.750 7fc37c6b5048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:49.750 7fc37c6b5048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 04:56:49.750 7fc37c6b5048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:49.750 7fc385348b48 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_status_consumer.dummy 09/29/2025 04:56:49.750 7fc385348b48 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_request_consumer 09/29/2025 04:56:49.750 7fc3822c96d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 09/29/2025 04:56:49.751 7fc385348b48 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-4204-2. 09/29/2025 04:56:49.754 7fc3822c96d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 09/29/2025 04:56:49.864 7fc3822c96d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-4204-2.'] 09/29/2025 04:56:49.887 7fc3822c96d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 09/29/2025 04:56:49.892 7fc3822c96d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 09/29/2025 04:56:49.936 7fc3822c96d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-4204-2.'] 09/29/2025 04:56:50.262 7fc385348b48 [contrail-device-manager] [DEBUG]: Fetching all containers 09/29/2025 04:56:50.301 7fc385348b48 [contrail-device-manager] [INFO]: Restarting dnsmasq docker: config_dnsmasq_1 09/29/2025 04:56:50.476 7fc385348b48 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4204-2..config_queue 09/29/2025 04:56:50.476 7fc385348b48 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4204-2..tftp_queue 09/29/2025 04:56:50.957 7fc3822c96d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 09/29/2025 04:56:50.961 7fc3822c96d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 09/29/2025 04:56:50.977 7fc385348b48 [contrail-device-manager] [WARNING]: Process 138 prepared to run in Full mode... 09/29/2025 04:56:50.977 7fc385348b48 [contrail-device-manager] [WARNING]: Elected master Device Manager node. Initializing... 09/29/2025 04:56:50.977 7fc385348b48 [contrail-device-manager] [INFO]: SANDESH: INTROSPECT IS ON: 0.0.0.0:8096 09/29/2025 04:56:50.980 7fc385348b48 [contrail-device-manager] [DEBUG]: Add [config_updater.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/config_updater] to home page 09/29/2025 04:56:50.980 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_ConfigApiUpdateReq] 09/29/2025 04:56:50.981 7fc385348b48 [contrail-device-manager] [DEBUG]: Add [greenlets.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/greenlets] to home page 09/29/2025 04:56:50.981 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_GreenletObjectReq] 09/29/2025 04:56:50.982 7fc385348b48 [contrail-device-manager] [DEBUG]: Add [cpuinfo.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/nodeinfo/cpuinfo] to home page 09/29/2025 04:56:50.982 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CpuLoadInfoReq] 09/29/2025 04:56:50.986 7fc385348b48 [contrail-device-manager] [DEBUG]: Add [dm_introspect.xml:/usr/lib/python3.6/site-packages/device_manager/sandesh/dm_introspect] to home page 09/29/2025 04:56:50.986 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_RoutingInstanceList] 09/29/2025 04:56:50.986 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualNetworkList] 09/29/2025 04:56:50.986 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualMachineInterfaceList] 09/29/2025 04:56:50.986 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalInterfaceList] 09/29/2025 04:56:50.986 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalInterfaceList] 09/29/2025 04:56:50.986 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalRouterList] 09/29/2025 04:56:50.986 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_BgpRouterList] 09/29/2025 04:56:50.986 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalRouterList] 09/29/2025 04:56:50.986 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_InstanceIpList] 09/29/2025 04:56:50.986 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FloatingIpList] 09/29/2025 04:56:50.986 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PortTupleList] 09/29/2025 04:56:50.986 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FeatureFlagList] 09/29/2025 04:56:50.988 7fc385348b48 [contrail-device-manager] [DEBUG]: Add [sandesh_alarm.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_alarm] to home page 09/29/2025 04:56:50.988 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmCacheRequest] 09/29/2025 04:56:50.988 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmTypesRequest] 09/29/2025 04:56:50.988 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmAckRequest] 09/29/2025 04:56:50.988 7fc385348b48 [contrail-device-manager] [DEBUG]: Add [sandesh_ctrl.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_ctrl] to home page 09/29/2025 04:56:50.988 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlClientToServer] 09/29/2025 04:56:50.988 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlServerToClient] 09/29/2025 04:56:50.989 7fc385348b48 [contrail-device-manager] [DEBUG]: Add [sandesh_trace.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_trace] to home page 09/29/2025 04:56:50.989 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceRequest] 09/29/2025 04:56:50.989 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferListRequest] 09/29/2025 04:56:50.989 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceEnableDisableReq] 09/29/2025 04:56:50.989 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufStatusReq] 09/29/2025 04:56:50.989 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferEnableDisableReq] 09/29/2025 04:56:50.990 7fc385348b48 [contrail-device-manager] [DEBUG]: Add [sandesh_uve.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_uve] to home page 09/29/2025 04:56:50.990 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshMessageStatsReq] 09/29/2025 04:56:50.990 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsSet] 09/29/2025 04:56:50.990 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsStatus] 09/29/2025 04:56:50.990 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsSet] 09/29/2025 04:56:50.990 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsStatus] 09/29/2025 04:56:50.990 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsSet] 09/29/2025 04:56:50.990 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsReset] 09/29/2025 04:56:50.991 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsStatus] 09/29/2025 04:56:50.991 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueSet] 09/29/2025 04:56:50.991 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueStatus] 09/29/2025 04:56:50.991 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CollectorInfoRequest] 09/29/2025 04:56:50.991 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVEDSConfReq] 09/29/2025 04:56:50.991 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVECacheReq] 09/29/2025 04:56:50.991 7fc385348b48 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVETypesReq] 09/29/2025 04:56:50.994 7fc385348b48 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.ztp_queue 09/29/2025 04:56:51.007 7fc37c5e4828 [contrail-device-manager] [ERROR]: Cannot write http_port 8096 to /tmp/contrail-device-manager.1.http_port 09/29/2025 04:56:51.007 7fc37c5e4828 [contrail-device-manager] [ERROR]: Starting Introspect on HTTP Port 8096 09/29/2025 04:56:51.051 7fc37762db70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:51.052 7fc37762db70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Initializing description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 09/29/2025 04:56:51.052 7fc37762db70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:51.063 7fc37750f2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:51.064 7fc37750f2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 09/29/2025 04:56:51.064 7fc37750f2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:51.178 7fc3822c96d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-4204-2.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4204-2..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4204-2..tftp_queue', 'device_manager_ztp.ztp_queue'] 09/29/2025 04:56:51.202 7fc3822c96d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 09/29/2025 04:56:51.206 7fc3822c96d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 09/29/2025 04:56:51.248 7fc3822c96d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-4204-2.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4204-2..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4204-2..tftp_queue', 'device_manager_ztp.ztp_queue'] 09/29/2025 04:56:52.361 7fc377516f60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 04:56:52.362 7fc377516f60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 04:56:52.362 7fc377516f60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 04:56:52.362 7fc377516f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:56:52.362 7fc377516f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 09/29/2025 04:56:52.362 7fc377516f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:56:52.363 7fc377516f60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 04:56:52.363 7fc377516f60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 04:56:52.363 7fc377516f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:52.363 7fc377516f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 09/29/2025 04:56:52.363 7fc377516f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:52.363 7fc377516f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:52.363 7fc377516f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1759121792351872 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.48:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 04:56:52.363 7fc377516f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:54.197 7fc37c7156d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:54.197 7fc37c7156d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 04:56:54.197 7fc37c7156d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:56.363 7fc3774398d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 04:56:56.364 7fc3774398d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 04:56:56.364 7fc3774398d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 04:56:56.365 7fc3774398d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:56:56.365 7fc3774398d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 04:56:56.365 7fc3774398d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:56:56.365 7fc3774398d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 04:56:56.365 7fc3774398d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 04:56:56.365 7fc3774398d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:56.365 7fc3774398d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 04:56:56.365 7fc3774398d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:56.365 7fc3774398d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:56:56.366 7fc3774398d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1759121792351872 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.48:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 04:56:56.366 7fc3774398d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:00.365 7fc377516c18 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 04:57:00.367 7fc377516c18 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 04:57:00.367 7fc377516c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 04:57:00.367 7fc377516c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:57:00.367 7fc377516c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 04:57:00.367 7fc377516c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:57:00.367 7fc377516c18 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 04:57:00.367 7fc377516c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 04:57:00.367 7fc377516c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:00.368 7fc377516c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 04:57:00.368 7fc377516c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:00.368 7fc377516c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:00.368 7fc377516c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1759121792351872 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.48:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 04:57:00.368 7fc377516c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:04.367 7fc37751d588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 04:57:04.369 7fc37751d588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 04:57:04.369 7fc37751d588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 04:57:04.369 7fc37751d588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:57:04.369 7fc37751d588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 04:57:04.369 7fc37751d588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:57:04.369 7fc37751d588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 04:57:04.370 7fc37751d588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 04:57:04.370 7fc37751d588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:04.370 7fc37751d588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 04:57:04.370 7fc37751d588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:04.370 7fc37751d588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:04.370 7fc37751d588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1759121792351872 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.48:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 04:57:04.370 7fc37751d588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:08.369 7fc3773e4eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 04:57:08.370 7fc3773e4eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 04:57:08.371 7fc3773e4eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 04:57:08.371 7fc3773e4eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:57:08.371 7fc3773e4eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 04:57:08.371 7fc3773e4eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:57:08.371 7fc3773e4eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 04:57:08.371 7fc3773e4eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 04:57:08.371 7fc3773e4eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:08.372 7fc3773e4eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 04:57:08.372 7fc3773e4eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:08.372 7fc3773e4eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:08.372 7fc3773e4eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1759121792351872 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.48:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 04:57:08.372 7fc3773e4eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:12.371 7fc3775d9a20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 04:57:12.372 7fc3775d9a20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 04:57:12.373 7fc3775d9a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 04:57:12.373 7fc3775d9a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:57:12.373 7fc3775d9a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 04:57:12.373 7fc3775d9a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:57:12.373 7fc3775d9a20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 04:57:12.373 7fc3775d9a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 04:57:12.373 7fc3775d9a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:12.373 7fc3775d9a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 04:57:12.374 7fc3775d9a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:12.374 7fc3775d9a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:12.374 7fc3775d9a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1759121792351872 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.48:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 04:57:12.374 7fc3775d9a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:16.374 7fc3775d9a20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 04:57:16.375 7fc3775d9a20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 04:57:16.375 7fc3775d9a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 04:57:16.375 7fc3775d9a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:57:16.375 7fc3775d9a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 04:57:16.375 7fc3775d9a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:57:16.376 7fc3775d9a20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 04:57:16.376 7fc3775d9a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 04:57:16.376 7fc3775d9a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:16.376 7fc3775d9a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 04:57:16.376 7fc3775d9a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:16.376 7fc3775d9a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:16.376 7fc3775d9a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1759121792351872 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.48:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 04:57:16.376 7fc3775d9a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:20.376 7fc3775d9a20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 04:57:20.377 7fc3775d9a20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 04:57:20.377 7fc3775d9a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 04:57:20.377 7fc3775d9a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:57:20.378 7fc3775d9a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 04:57:20.378 7fc3775d9a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:57:20.378 7fc3775d9a20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 04:57:20.378 7fc3775d9a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 04:57:20.378 7fc3775d9a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:20.378 7fc3775d9a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 04:57:20.378 7fc3775d9a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:20.378 7fc3775d9a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:20.378 7fc3775d9a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1759121792351872 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.48:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 04:57:20.378 7fc3775d9a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:24.378 7fc3775d9a20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 04:57:24.379 7fc3775d9a20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 04:57:24.380 7fc3775d9a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 04:57:24.380 7fc3775d9a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:57:24.380 7fc3775d9a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 04:57:24.380 7fc3775d9a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:57:24.380 7fc3775d9a20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 04:57:24.380 7fc3775d9a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 04:57:24.380 7fc3775d9a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:24.380 7fc3775d9a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 04:57:24.380 7fc3775d9a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:24.381 7fc3775d9a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:24.381 7fc3775d9a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1759121792351872 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.48:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 04:57:24.381 7fc3775d9a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:28.380 7fc3775d9a20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 04:57:28.381 7fc3775d9a20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 04:57:28.381 7fc3775d9a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 04:57:28.381 7fc3775d9a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:57:28.382 7fc3775d9a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 04:57:28.389 7fc3775d9a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:57:28.390 7fc3775d9a20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 04:57:28.390 7fc3775d9a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 04:57:28.390 7fc3775d9a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:28.390 7fc3775d9a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 04:57:28.390 7fc3775d9a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:28.390 7fc3775d9a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:28.390 7fc3775d9a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1759121792351872 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.48:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 04:57:28.390 7fc3775d9a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:32.382 7fc3775d9a20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 04:57:32.383 7fc3775d9a20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 04:57:32.383 7fc3775d9a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 04:57:32.383 7fc3775d9a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:57:32.383 7fc3775d9a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 04:57:32.383 7fc3775d9a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:57:32.383 7fc3775d9a20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 04:57:32.384 7fc3775d9a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 04:57:32.384 7fc3775d9a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:32.384 7fc3775d9a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 04:57:32.384 7fc3775d9a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:32.384 7fc3775d9a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:32.384 7fc3775d9a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1759121792351872 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.48:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 04:57:32.384 7fc3775d9a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:36.384 7fc37c6b1d68 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 04:57:36.385 7fc37c6b1d68 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 04:57:36.385 7fc37c6b1d68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 04:57:36.385 7fc37c6b1d68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:57:36.385 7fc37c6b1d68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 04:57:36.385 7fc37c6b1d68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:57:36.385 7fc37c6b1d68 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 04:57:36.386 7fc37c6b1d68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 04:57:36.386 7fc37c6b1d68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:36.386 7fc37c6b1d68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 04:57:36.386 7fc37c6b1d68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:36.386 7fc37c6b1d68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:36.386 7fc37c6b1d68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1759121792351872 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.48:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 04:57:36.386 7fc37c6b1d68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:40.386 7fc37750dc18 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 04:57:40.387 7fc37750dc18 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 04:57:40.387 7fc37750dc18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 04:57:40.387 7fc37750dc18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:57:40.387 7fc37750dc18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 04:57:40.387 7fc37750dc18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:57:40.388 7fc37750dc18 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 04:57:40.388 7fc37750dc18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 04:57:40.388 7fc37750dc18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:40.388 7fc37750dc18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 04:57:40.388 7fc37750dc18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:40.388 7fc37750dc18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:40.388 7fc37750dc18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1759121792351872 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.48:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 04:57:40.388 7fc37750dc18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:44.387 7fc37c774ac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 04:57:44.388 7fc37c774ac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 04:57:44.388 7fc37c774ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 04:57:44.388 7fc37c774ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:57:44.389 7fc37c774ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 04:57:44.389 7fc37c774ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:57:44.389 7fc37c774ac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 04:57:44.389 7fc37c774ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 04:57:44.389 7fc37c774ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:44.389 7fc37c774ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 04:57:44.389 7fc37c774ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:44.389 7fc37c774ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:44.390 7fc37c774ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1759121792351872 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.48:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 04:57:44.390 7fc37c774ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:48.389 7fc377439e10 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 04:57:48.390 7fc377439e10 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 04:57:48.390 7fc377439e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 04:57:48.390 7fc377439e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:57:48.390 7fc377439e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 04:57:48.390 7fc377439e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:57:48.390 7fc377439e10 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 04:57:48.390 7fc377439e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 04:57:48.390 7fc377439e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:48.391 7fc377439e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 04:57:48.391 7fc377439e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:48.391 7fc377439e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:48.391 7fc377439e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1759121792351872 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.48:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 04:57:48.391 7fc377439e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:52.390 7fc37743beb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 04:57:52.392 7fc37743beb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 04:57:52.392 7fc37743beb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 04:57:52.392 7fc37743beb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:57:52.392 7fc37743beb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 04:57:52.392 7fc37743beb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:57:52.392 7fc37743beb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 04:57:52.392 7fc37743beb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 04:57:52.392 7fc37743beb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:52.393 7fc37743beb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 04:57:52.393 7fc37743beb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:52.393 7fc37743beb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:52.393 7fc37743beb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1759121792351872 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.48:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 04:57:52.393 7fc37743beb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:56.393 7fc377439d68 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 04:57:56.393 7fc377439d68 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 04:57:56.394 7fc377439d68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 04:57:56.394 7fc377439d68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:57:56.394 7fc377439d68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 04:57:56.394 7fc377439d68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:57:56.394 7fc377439d68 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 04:57:56.394 7fc377439d68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 04:57:56.394 7fc377439d68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:56.394 7fc377439d68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 04:57:56.394 7fc377439d68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:56.394 7fc377439d68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:57:56.395 7fc377439d68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1759121792351872 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.48:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 04:57:56.395 7fc377439d68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:00.395 7fc3775048d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 04:58:00.396 7fc3775048d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 04:58:00.397 7fc3775048d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 04:58:00.397 7fc3775048d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:58:00.397 7fc3775048d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 04:58:00.397 7fc3775048d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:58:00.397 7fc3775048d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 04:58:00.398 7fc3775048d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 04:58:00.398 7fc3775048d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:00.398 7fc3775048d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 04:58:00.398 7fc3775048d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:00.398 7fc3775048d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:00.398 7fc3775048d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1759121792351872 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.48:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 04:58:00.398 7fc3775048d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:04.397 7fc377439ac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 04:58:04.398 7fc377439ac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 04:58:04.398 7fc377439ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 04:58:04.398 7fc377439ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:58:04.398 7fc377439ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 04:58:04.398 7fc377439ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:58:04.398 7fc377439ac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 04:58:04.399 7fc377439ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 04:58:04.399 7fc377439ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:04.399 7fc377439ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 04:58:04.399 7fc377439ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:04.399 7fc377439ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:04.399 7fc377439ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1759121792351872 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.48:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 04:58:04.399 7fc377439ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:08.399 7fc377439cc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 04:58:08.400 7fc377439cc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 04:58:08.400 7fc377439cc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 04:58:08.400 7fc377439cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:58:08.400 7fc377439cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 04:58:08.400 7fc377439cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:58:08.400 7fc377439cc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 04:58:08.400 7fc377439cc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 04:58:08.400 7fc377439cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:08.400 7fc377439cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 04:58:08.401 7fc377439cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:08.401 7fc377439cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:08.401 7fc377439cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1759121792351872 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.48:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 04:58:08.401 7fc377439cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:12.402 7fc37c60ecc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 04:58:12.403 7fc37c60ecc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 04:58:12.403 7fc37c60ecc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 04:58:12.403 7fc37c60ecc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:58:12.403 7fc37c60ecc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 04:58:12.403 7fc37c60ecc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:58:12.403 7fc37c60ecc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 04:58:12.403 7fc37c60ecc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 04:58:12.403 7fc37c60ecc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:12.404 7fc37c60ecc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 04:58:12.404 7fc37c60ecc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:12.404 7fc37c60ecc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:12.404 7fc37c60ecc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1759121792351872 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.48:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 04:58:12.404 7fc37c60ecc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:13.163 7fc385348b48 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 09/29/2025 04:58:13.164 7fc385348b48 [contrail-device-manager] [WARNING]: initializing cassandra cluster 09/29/2025 04:58:13.164 7fc385348b48 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/29/2025 04:58:13.164 7fc3774b1780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:13.164 7fc3774b1780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/29/2025 04:58:13.164 7fc3774b1780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:13.165 7fc385348b48 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/29/2025 04:58:13.218 7fc385348b48 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/29/2025 04:58:13.220 7fc377592c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:13.220 7fc377592c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Initializing description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/29/2025 04:58:13.220 7fc377592c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:13.227 7fc385348b48 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 09/29/2025 04:58:13.227 7fc377592ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:13.227 7fc377592ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/29/2025 04:58:13.228 7fc377592ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:13.247 7fc385348b48 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/29/2025 04:58:14.538 7fc385348b48 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace', created 09/29/2025 04:58:14.543 7fc385348b48 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/29/2025 04:58:14.549 7fc385348b48 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/29/2025 04:58:15.376 7fc385348b48 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace' fixed with replication {'class': 'SimpleStrategy', 'replication_factor': None} 09/29/2025 04:58:15.385 7fc385348b48 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/29/2025 04:58:15.385 7fc385348b48 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 09/29/2025 04:58:15.456 7fc385348b48 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 09/29/2025 04:58:15.463 7fc385348b48 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/29/2025 04:58:15.864 7fc385348b48 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table', created 09/29/2025 04:58:15.868 7fc385348b48 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/29/2025 04:58:15.874 7fc385348b48 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/29/2025 04:58:16.403 7fc377309c18 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 04:58:16.404 7fc377309c18 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 04:58:16.404 7fc377309c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 04:58:16.404 7fc377309c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:58:16.404 7fc377309c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 09/29/2025 04:58:16.404 7fc377309c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:58:16.405 7fc377309c18 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 04:58:16.405 7fc377309c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 04:58:16.405 7fc377309c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:16.405 7fc377309c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/29/2025 04:58:16.405 7fc377309c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:16.405 7fc377309c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:16.405 7fc377309c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1759121792351872 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.48:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 04:58:16.405 7fc377309c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:16.782 7fc385348b48 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table' fixed with properties {'gc_grace_seconds': 864000} 09/29/2025 04:58:16.787 7fc385348b48 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/29/2025 04:58:16.793 7fc385348b48 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/29/2025 04:58:18.317 7fc385348b48 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table', created 09/29/2025 04:58:18.322 7fc385348b48 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/29/2025 04:58:18.328 7fc385348b48 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/29/2025 04:58:18.849 7fc385348b48 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table' fixed with properties {'gc_grace_seconds': 864000} 09/29/2025 04:58:18.853 7fc385348b48 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/29/2025 04:58:18.859 7fc385348b48 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/29/2025 04:58:19.649 7fc385348b48 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table', created 09/29/2025 04:58:19.654 7fc385348b48 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/29/2025 04:58:19.660 7fc385348b48 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/29/2025 04:58:20.404 7fc377309630 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 04:58:20.405 7fc377309630 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 04:58:20.406 7fc377309630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 04:58:20.406 7fc377309630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:58:20.406 7fc377309630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 09/29/2025 04:58:20.406 7fc377309630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:58:20.406 7fc377309630 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 04:58:20.406 7fc377309630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 04:58:20.406 7fc377309630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:20.406 7fc377309630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/29/2025 04:58:20.406 7fc377309630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:20.406 7fc377309630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:20.407 7fc377309630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1759121792351872 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.48:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 04:58:20.407 7fc377309630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:20.548 7fc385348b48 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table' fixed with properties {'gc_grace_seconds': 864000} 09/29/2025 04:58:20.553 7fc385348b48 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/29/2025 04:58:20.559 7fc385348b48 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/29/2025 04:58:21.734 7fc385348b48 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table', created 09/29/2025 04:58:21.740 7fc385348b48 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/29/2025 04:58:21.746 7fc385348b48 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/29/2025 04:58:22.642 7fc385348b48 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table' fixed with properties {'gc_grace_seconds': 864000} 09/29/2025 04:58:22.648 7fc385348b48 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/29/2025 04:58:22.649 7fc377371780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:22.649 7fc377371780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 04:58:22.649 7fc377371780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:22.709 7fc385348b48 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.254:5673//', 'pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.48:5673//'] 09/29/2025 04:58:22.710 7fc385348b48 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 09/29/2025 04:58:22.710 7fc3775bb4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:22.711 7fc3775bb4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Initializing description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 09/29/2025 04:58:22.711 7fc3775bb4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:22.711 7fc3775bb4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:22.711 7fc3775bb4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 09/29/2025 04:58:22.711 7fc3775bb4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:22.716 7fc385348b48 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/29/2025 04:58:22.716 7fc37731d780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:22.716 7fc37731d780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 04:58:22.716 7fc37731d780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:22.746 7fc385348b48 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 09/29/2025 04:58:22.747 7fc37731d780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:22.747 7fc37731d780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 09/29/2025 04:58:22.747 7fc37731d780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:22.757 7fc385348b48 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/29/2025 04:58:22.757 7fc37731ce10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:22.757 7fc37731ce10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 04:58:22.757 7fc37731ce10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:22.758 7fc37731d198 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 09/29/2025 04:58:22.759 7fc377317cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:22.759 7fc377317cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 09/29/2025 04:58:22.759 7fc377317cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:22.766 7fc37731d198 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/29/2025 04:58:22.767 7fc37731c198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:22.767 7fc37731c198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 04:58:22.767 7fc37731c198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:24.406 7fc37d657588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 04:58:24.407 7fc37d657588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 04:58:24.408 7fc37d657588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 04:58:24.408 7fc37d657588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:58:24.408 7fc37d657588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 04:58:24.408 7fc37d657588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:58:24.408 7fc37d657588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 04:58:24.408 7fc37d657588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 04:58:24.408 7fc37d657588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:24.408 7fc37d657588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 04:58:24.408 7fc37d657588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:24.409 7fc37d657588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:24.409 7fc37d657588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1759121792351872 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.48:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 04:58:24.409 7fc37d657588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:28.408 7fc37d657588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 04:58:28.409 7fc37d657588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 04:58:28.409 7fc37d657588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 04:58:28.409 7fc37d657588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:58:28.410 7fc37d657588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 04:58:28.410 7fc37d657588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:58:28.410 7fc37d657588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 04:58:28.410 7fc37d657588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 04:58:28.410 7fc37d657588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:28.410 7fc37d657588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 04:58:28.410 7fc37d657588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:28.410 7fc37d657588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:28.410 7fc37d657588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1759121792351872 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.48:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 04:58:28.411 7fc37d657588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:32.410 7fc37d657588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 04:58:32.411 7fc37d657588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 04:58:32.411 7fc37d657588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 04:58:32.411 7fc37d657588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:58:32.411 7fc37d657588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 04:58:32.411 7fc37d657588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:58:32.411 7fc37d657588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 04:58:32.411 7fc37d657588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 04:58:32.412 7fc37d657588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:32.412 7fc37d657588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 04:58:32.412 7fc37d657588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:32.412 7fc37d657588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:32.412 7fc37d657588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1759121792351872 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.48:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 04:58:32.412 7fc37d657588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:36.412 7fc37d657588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 04:58:36.415 7fc37d657588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 04:58:36.415 7fc37d657588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 04:58:36.415 7fc37d657588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:58:36.415 7fc37d657588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 04:58:36.415 7fc37d657588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:58:36.415 7fc37d657588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 04:58:36.416 7fc37d657588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 04:58:36.416 7fc37d657588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:36.417 7fc37d657588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 04:58:36.417 7fc37d657588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:36.417 7fc37d657588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:36.417 7fc37d657588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1759121792351872 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, 10.0.0.48:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 04:58:36.418 7fc37d657588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 04:58:38.175 7fc37731d6d8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4204-2.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.38', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4204-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4204-2.'], 'id_perms': {'created': '2025-09-29T04:58:38.157243', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-29T04:58:38.157243', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13824365191860603067, 'uuid_mslong': 14098411552748553378}}, 'parent_type': 'global-system-config', 'parent_uuid': '9aec2cba-3187-4d11-b071-ec387c400a5a', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'c3a79e9c-52d7-44a2-bfda-02e5ba2048bb'}, 'oper': 'CREATE', 'request-id': 'req-686dc40f-961f-4cfd-a8c7-628a1b10ca98', 'type': 'analytics_node', 'uuid': 'c3a79e9c-52d7-44a2-bfda-02e5ba2048bb'} 09/29/2025 04:58:38.248 7fc37731d6d8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4204-3.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.48', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4204-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4204-3.'], 'id_perms': {'created': '2025-09-29T04:58:38.220407', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-29T04:58:38.220407', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9699208404806942647, 'uuid_mslong': 10451341241109269365}}, 'parent_type': 'global-system-config', 'parent_uuid': '9aec2cba-3187-4d11-b071-ec387c400a5a', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '910a9f7c-36b5-4b75-869a-82a7d3003fb7'}, 'oper': 'CREATE', 'request-id': 'req-a4206025-0116-4b45-8b1a-880f77bdf23e', 'type': 'analytics_node', 'uuid': '910a9f7c-36b5-4b75-869a-82a7d3003fb7'} 09/29/2025 04:58:38.273 7fc37731d6d8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4204-1.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.254', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4204-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4204-1.'], 'id_perms': {'created': '2025-09-29T04:58:38.261081', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-29T04:58:38.261081', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10796085947836182668, 'uuid_mslong': 16830648853535476931}}, 'parent_type': 'global-system-config', 'parent_uuid': '9aec2cba-3187-4d11-b071-ec387c400a5a', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'e99279f5-ae51-4cc3-95d3-66f82d5cb08c'}, 'oper': 'CREATE', 'request-id': 'req-bb6dcea5-0f25-4813-a77c-8fd33e12a0f6', 'type': 'analytics_node', 'uuid': 'e99279f5-ae51-4cc3-95d3-66f82d5cb08c'} 09/29/2025 04:58:40.416 7fc3772b56d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 04:58:40.417 7fc3772b56d8 [contrail-device-manager] [INFO]: Session Event: TCP Connected 09/29/2025 04:58:40.417 7fc3772b56d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 04:58:40.417 7fc3772b56d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:58:40.417 7fc3772b56d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4204-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 04:58:40.418 7fc3772b56d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 04:58:40.418 7fc3772b56d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/29/2025 04:58:40.418 7fc3772b56d8 [contrail-device-manager] [DEBUG]: Send sandesh control message. uve type count # 19 09/29/2025 04:58:40.419 7fc3772b56d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/29/2025 04:58:40.419 7fc3772b56d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/29/2025 04:58:40.419 7fc3772b56d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/29/2025 04:58:40.419 7fc3772b88d0 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/29/2025 04:58:40.421 7fc3772b88d0 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/29/2025 04:58:49.083 7fc3772b8630 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001016 09/29/2025 04:58:49.083 7fc3772b8630 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 09/29/2025 04:58:49.084 7fc3773c0438 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 09/29/2025 04:58:49.084 7fc3773c0438 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 0 09/29/2025 04:58:49.084 7fc3773c0438 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/29/2025 04:58:49.084 7fc3773c0438 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/29/2025 04:58:49.084 7fc3773c0438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 09/29/2025 04:58:49.085 7fc3772b56d8 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/29/2025 04:58:49.086 7fc3772b56d8 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/29/2025 04:58:53.665 7fc37731d6d8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4204-3.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.48', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4204-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4204-3.'], 'id_perms': {'created': '2025-09-29T04:58:53.633432', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-29T04:58:53.633432', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13092907253395228357, 'uuid_mslong': 14856760304057663826}}, 'parent_type': 'global-system-config', 'parent_uuid': '9aec2cba-3187-4d11-b071-ec387c400a5a', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'ce2dd0c8-347b-4152-b5b3-59c71bc206c5'}, 'oper': 'CREATE', 'request-id': 'req-55e206a3-c6b9-42f1-996f-ca6d7bbec21e', 'type': 'analytics_alarm_node', 'uuid': 'ce2dd0c8-347b-4152-b5b3-59c71bc206c5'} 09/29/2025 04:58:53.688 7fc37731d6d8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4204-1.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.254', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4204-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4204-1.'], 'id_perms': {'created': '2025-09-29T04:58:53.656281', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-29T04:58:53.656281', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11838466131538004582, 'uuid_mslong': 10289784940384504335}}, 'parent_type': 'global-system-config', 'parent_uuid': '9aec2cba-3187-4d11-b071-ec387c400a5a', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '8ecca8e2-bac8-4a0f-a44a-ae3947d0ca66'}, 'oper': 'CREATE', 'request-id': 'req-b26886b3-a7b6-47af-9ab2-225d5f032b89', 'type': 'analytics_alarm_node', 'uuid': '8ecca8e2-bac8-4a0f-a44a-ae3947d0ca66'} 09/29/2025 04:58:53.709 7fc37731d6d8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4204-2.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.38', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4204-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4204-2.'], 'id_perms': {'created': '2025-09-29T04:58:53.679561', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-29T04:58:53.679561', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11056601856773158843, 'uuid_mslong': 7890465702391138099}}, 'parent_type': 'global-system-config', 'parent_uuid': '9aec2cba-3187-4d11-b071-ec387c400a5a', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '6d8090c0-376d-4733-9970-f0ceefe377bb'}, 'oper': 'CREATE', 'request-id': 'req-9123a05e-a872-4d36-9f98-5f4b15a232ed', 'type': 'analytics_alarm_node', 'uuid': '6d8090c0-376d-4733-9970-f0ceefe377bb'} 09/29/2025 04:58:59.068 7fc3772b8630 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001296 09/29/2025 04:58:59.069 7fc3772b8630 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 09/29/2025 04:58:59.070 7fc3842e1828 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 09/29/2025 04:58:59.070 7fc3842e1828 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 2 09/29/2025 04:58:59.070 7fc3842e1828 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/29/2025 04:58:59.071 7fc3842e1828 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/29/2025 04:59:02.631 7fc37731d6d8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4204-1.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.254', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4204-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4204-1.'], 'id_perms': {'created': '2025-09-29T04:59:02.616969', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-29T04:59:02.616969', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10586283417989473944, 'uuid_mslong': 10729866215463665958}}, 'parent_type': 'global-system-config', 'parent_uuid': '9aec2cba-3187-4d11-b071-ec387c400a5a', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '94e82479-442d-4126-92ea-08ae1cdee698'}, 'oper': 'CREATE', 'request-id': 'req-a1f23094-b61b-430e-9640-acf55ee8124b', 'type': 'analytics_snmp_node', 'uuid': '94e82479-442d-4126-92ea-08ae1cdee698'} 09/29/2025 04:59:03.015 7fc37731d6d8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4204-2.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.38', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4204-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4204-2.'], 'id_perms': {'created': '2025-09-29T04:59:03.001492', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-29T04:59:03.001492', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11944258675614175289, 'uuid_mslong': 14192546921450980547}}, 'parent_type': 'global-system-config', 'parent_uuid': '9aec2cba-3187-4d11-b071-ec387c400a5a', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'c4f60e3a-cf26-44c3-a5c2-87fb695c5039'}, 'oper': 'CREATE', 'request-id': 'req-edbad1f0-1a3b-4bee-b49e-28a57e34c998', 'type': 'analytics_snmp_node', 'uuid': 'c4f60e3a-cf26-44c3-a5c2-87fb695c5039'} 09/29/2025 04:59:03.071 7fc37731d6d8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4204-3.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.48', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4204-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4204-3.'], 'id_perms': {'created': '2025-09-29T04:59:03.056480', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-29T04:59:03.056480', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13387943245167342631, 'uuid_mslong': 2120593717682981469}}, 'parent_type': 'global-system-config', 'parent_uuid': '9aec2cba-3187-4d11-b071-ec387c400a5a', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '1d6ddcc0-376b-4a5d-b9cb-8772f94a3427'}, 'oper': 'CREATE', 'request-id': 'req-da1913e6-8802-4272-8eec-48b4c47a8385', 'type': 'analytics_snmp_node', 'uuid': '1d6ddcc0-376b-4a5d-b9cb-8772f94a3427'} 09/29/2025 05:00:41.846 7fc37731d6d8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-b2bd7be2-b133-4aa9-91d9-e3e738a43e5a', 'type': 'global_system_config', 'uuid': '9aec2cba-3187-4d11-b071-ec387c400a5a'} 09/29/2025 05:00:41.888 7fc37731d6d8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-89b854a6-13f5-4d08-8380-f72763d2c571', 'type': 'global_system_config', 'uuid': '9aec2cba-3187-4d11-b071-ec387c400a5a'} 09/29/2025 05:00:41.982 7fc37731d6d8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4204-2'], 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-4204-2', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4204-2'], 'id_perms': {'created': '2025-09-29T05:00:41.969536', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-29T05:00:41.969536', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10199467397043869887, 'uuid_mslong': 9884579763059443039}}, 'parent_type': 'global-system-config', 'parent_uuid': '9aec2cba-3187-4d11-b071-ec387c400a5a', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '892d14ec-7b2c-455f-8d8b-c98efafa58bf', 'virtual_router_dpdk_enabled': False, 'virtual_router_ip_address': '10.20.0.193'}, 'oper': 'CREATE', 'request-id': 'req-73030591-7588-4757-ab7a-bbbbcfcacd10', 'type': 'virtual_router', 'uuid': '892d14ec-7b2c-455f-8d8b-c98efafa58bf'} 09/29/2025 05:00:42.027 7fc37731d6d8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4204-1'], 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-4204-1', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4204-1'], 'id_perms': {'created': '2025-09-29T05:00:42.018989', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-29T05:00:42.018989', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11061461850813601081, 'uuid_mslong': 6732567363265905855}}, 'parent_type': 'global-system-config', 'parent_uuid': '9aec2cba-3187-4d11-b071-ec387c400a5a', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '5d6ee258-a1da-44bf-9982-34f27a479539', 'virtual_router_dpdk_enabled': False, 'virtual_router_ip_address': '10.20.0.25'}, 'oper': 'CREATE', 'request-id': 'req-1f6df435-a06f-4694-8b8a-fff3c2c6bf22', 'type': 'virtual_router', 'uuid': '5d6ee258-a1da-44bf-9982-34f27a479539'} 09/29/2025 05:00:42.343 7fc37731d6d8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4204-2', 'vhost0'], 'obj_dict': {'display_name': 'vhost0', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4204-2', 'vhost0'], 'id_perms': {'created': '2025-09-29T05:00:42.328108', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-29T05:00:42.328108', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10671454917610038461, 'uuid_mslong': 894318155548279242}}, 'parent_type': 'virtual-router', 'parent_uuid': '892d14ec-7b2c-455f-8d8b-c98efafa58bf', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '0c69419f-c9a3-41ca-9418-9fb530bde4bd', 'virtual_machine_interface_disable_policy': True, 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:0c:69:41:9f:c9']}, 'virtual_network_refs': [{'to': ['default-domain', 'default-project', 'ip-fabric'], 'uuid': 'ecfae21e-d666-44fa-91f9-942efc1d4284'}]}, 'oper': 'CREATE', 'request-id': 'req-2b585c5c-28e4-48c0-8cb4-e16e245c9928', 'type': 'virtual_machine_interface', 'uuid': '0c69419f-c9a3-41ca-9418-9fb530bde4bd'} 09/29/2025 05:00:42.407 7fc37731d6d8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4204-2', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-2b585c5c-28e4-48c0-8cb4-e16e245c9928', 'type': 'virtual_machine_interface', 'uuid': '0c69419f-c9a3-41ca-9418-9fb530bde4bd'} 09/29/2025 05:00:42.463 7fc37731d6d8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4204-1', 'vhost0'], 'obj_dict': {'display_name': 'vhost0', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4204-1', 'vhost0'], 'id_perms': {'created': '2025-09-29T05:00:42.427590', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-29T05:00:42.427590', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12584377872152932207, 'uuid_mslong': 5648387655549209764}}, 'parent_type': 'virtual-router', 'parent_uuid': '5d6ee258-a1da-44bf-9982-34f27a479539', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '4e631aa5-72b6-4ca4-aea4-b0fff4f62f6f', 'virtual_machine_interface_disable_policy': True, 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:4e:63:1a:a5:72']}, 'virtual_network_refs': [{'to': ['default-domain', 'default-project', 'ip-fabric'], 'uuid': 'ecfae21e-d666-44fa-91f9-942efc1d4284'}]}, 'oper': 'CREATE', 'request-id': 'req-32545b29-7185-4c6e-86ac-f79fe5567cdd', 'type': 'virtual_machine_interface', 'uuid': '4e631aa5-72b6-4ca4-aea4-b0fff4f62f6f'} 09/29/2025 05:00:42.503 7fc37731d6d8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4204-1', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-32545b29-7185-4c6e-86ac-f79fe5567cdd', 'type': 'virtual_machine_interface', 'uuid': '4e631aa5-72b6-4ca4-aea4-b0fff4f62f6f'} 09/29/2025 05:00:42.882 7fc37731d6d8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'oper': 'UPDATE', 'request-id': 'req-b8077051-7c51-4659-b158-a6a550ff0b6b', 'type': 'access_control_list', 'uuid': '064a24c0-9bd5-461a-9eba-03dd0e093b35'} 09/29/2025 05:01:08.273 7fc377592828 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/29/2025 05:01:39.031 7fc3773e4048 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/29/2025 05:03:13.389 7fc37c6f9cc0 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map