09/22/2025 05:05:29.400 7faf1d3943c8 [contrail-device-manager] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/22/2025 05:05:29.400 7faf1d3943c8 [contrail-device-manager] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/22/2025 05:05:29.400 7faf1d3943c8 [contrail-device-manager] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/22/2025 05:05:29.400 7faf14a832e8 [contrail-device-manager] [INFO]: Processing event[EvStart] in state[none] 09/22/2025 05:05:29.400 7faf14a832e8 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectConfigNode:NodeStatusUVE] map 09/22/2025 05:05:29.401 7faf14a832e8 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/22/2025 05:05:29.401 7faf14a832e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/22/2025 05:05:29.401 7faf14a832e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:29.401 7faf14a832e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/22/2025 05:05:29.401 7faf14a832e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:29.401 7faf14a832e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:29.401 7faf14a832e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 start_time = 1758517529401078 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, 10.0.0.254:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/22/2025 05:05:29.401 7faf14a832e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:29.448 7faf1d3943c8 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.254:5673//', 'pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.48:5673//'] 09/22/2025 05:05:29.457 7faf1d3943c8 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 09/22/2025 05:05:29.457 7faf149d5588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:29.458 7faf149d5588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 09/22/2025 05:05:29.458 7faf149d5588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:29.458 7faf149d5588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:29.458 7faf149d5588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 09/22/2025 05:05:29.458 7faf149d5588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:29.462 7faf1d3943c8 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/22/2025 05:05:29.473 7faf1d3943c8 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 09/22/2025 05:05:29.474 7faf149d5240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:29.474 7faf149d5240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/22/2025 05:05:29.474 7faf149d5240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:29.474 7faf149d5240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:29.474 7faf149d5240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 09/22/2025 05:05:29.474 7faf149d5240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:29.478 7faf1d3943c8 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/22/2025 05:05:29.479 7faf149d5588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:29.480 7faf149d5588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/22/2025 05:05:29.480 7faf149d5588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:29.493 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Removed remaining AMQP queue from previous run 09/22/2025 05:05:29.494 7faf1d3943c8 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 09/22/2025 05:05:29.494 7faf1d3943c8 [contrail-device-manager] [WARNING]: initializing cassandra cluster 09/22/2025 05:05:29.494 7faf1d3943c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/22/2025 05:05:29.494 7faf1d37d6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting consumer greenlet 09/22/2025 05:05:29.495 7faf1d37d6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 09/22/2025 05:05:29.495 7faf1a30c6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting publisher greenlet 09/22/2025 05:05:29.495 7faf1a30c6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Publisher connection 09/22/2025 05:05:29.495 7faf23286978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:29.495 7faf23286978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 09/22/2025 05:05:29.495 7faf23286978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:29.501 7faf1d3943c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/22/2025 05:05:29.503 7faf1d37d6d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 09/22/2025 05:05:29.503 7faf1d37d6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Waiting for consumer 09/22/2025 05:05:29.503 7faf1a30c6d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Publisher connection established 09/22/2025 05:05:29.543 7faf1d3943c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/22/2025 05:05:29.545 7faf14826438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:29.546 7faf14826438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/22/2025 05:05:29.546 7faf14826438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:29.553 7faf1d3943c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 09/22/2025 05:05:29.553 7faf148226d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:29.553 7faf148226d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 09/22/2025 05:05:29.554 7faf148226d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:29.561 7faf1d3943c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 09/22/2025 05:05:29.591 7faf1d3943c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 09/22/2025 05:05:30.166 7faf1d3943c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/22/2025 05:05:30.744 7faf1d3943c8 [contrail-device-manager] [WARNING]: table 'obj_uuid_table', created 09/22/2025 05:05:30.749 7faf1d3943c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/22/2025 05:05:32.564 7faf1d3943c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/22/2025 05:05:33.401 7faf14788c18 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/22/2025 05:05:33.403 7faf14788c18 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/22/2025 05:05:33.403 7faf14788c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/22/2025 05:05:33.403 7faf14788c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:05:33.403 7faf14788c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 09/22/2025 05:05:33.403 7faf14788c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:05:33.403 7faf14788c18 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/22/2025 05:05:33.404 7faf14788c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/22/2025 05:05:33.404 7faf14788c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:33.404 7faf14788c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/22/2025 05:05:33.404 7faf14788c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:33.404 7faf14788c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:33.404 7faf14788c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 start_time = 1758517529401078 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.254:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/22/2025 05:05:33.404 7faf14788c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:33.470 7faf1d3943c8 [contrail-device-manager] [WARNING]: table 'obj_uuid_table' fixed with properties {'gc_grace_seconds': 864000} 09/22/2025 05:05:33.476 7faf1d3943c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/22/2025 05:05:35.055 7faf1d3943c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/22/2025 05:05:36.006 7faf1d3943c8 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table', created 09/22/2025 05:05:36.011 7faf1d3943c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/22/2025 05:05:37.404 7faf1770f630 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/22/2025 05:05:37.405 7faf1770f630 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/22/2025 05:05:37.405 7faf1770f630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/22/2025 05:05:37.406 7faf1770f630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:05:37.406 7faf1770f630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 09/22/2025 05:05:37.406 7faf1770f630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:05:37.406 7faf1770f630 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/22/2025 05:05:37.406 7faf1770f630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/22/2025 05:05:37.406 7faf1770f630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:37.406 7faf1770f630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/22/2025 05:05:37.406 7faf1770f630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:37.406 7faf1770f630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:37.406 7faf1770f630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 start_time = 1758517529401078 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.254:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/22/2025 05:05:37.407 7faf1770f630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:37.867 7faf1d3943c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/22/2025 05:05:38.775 7faf1d3943c8 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table' fixed with properties {'gc_grace_seconds': 864000} 09/22/2025 05:05:38.782 7faf1d3943c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/22/2025 05:05:40.633 7faf1d3943c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/22/2025 05:05:40.634 7faf1d3943c8 [contrail-device-manager] [WARNING]: table 'obj_shared_table', created 09/22/2025 05:05:40.640 7faf1d3943c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/22/2025 05:05:41.406 7faf1477cac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/22/2025 05:05:41.407 7faf1477cac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/22/2025 05:05:41.407 7faf1477cac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/22/2025 05:05:41.407 7faf1477cac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:05:41.407 7faf1477cac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 09/22/2025 05:05:41.407 7faf1477cac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:05:41.407 7faf1477cac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/22/2025 05:05:41.408 7faf1477cac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/22/2025 05:05:41.408 7faf1477cac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:41.408 7faf1477cac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/22/2025 05:05:41.408 7faf1477cac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:41.408 7faf1477cac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:41.408 7faf1477cac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 start_time = 1758517529401078 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.254:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/22/2025 05:05:41.408 7faf1477cac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:42.101 7faf1d3943c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/22/2025 05:05:42.778 7faf1d3943c8 [contrail-device-manager] [WARNING]: table 'obj_shared_table' fixed with properties {'gc_grace_seconds': 864000} 09/22/2025 05:05:42.785 7faf1d3943c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/22/2025 05:05:42.785 7faf155bc438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:42.786 7faf155bc438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/22/2025 05:05:42.786 7faf155bc438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:42.815 7faf1d3943c8 [contrail-device-manager] [ERROR]: Error in db walk read 'type' 09/22/2025 05:05:42.816 7faf1477c2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:42.816 7faf1477c2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Zookeeper:Zookeeper[] connection down >>, ] >> 09/22/2025 05:05:42.816 7faf1477c2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:42.825 7faf1476fc18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:42.826 7faf1476fc18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/22/2025 05:05:42.826 7faf1476fc18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:42.826 7faf1d3943c8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_status_consumer.dummy 09/22/2025 05:05:42.826 7faf1d3943c8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_request_consumer 09/22/2025 05:05:42.826 7faf1d37d6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 09/22/2025 05:05:42.826 7faf1d3943c8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-4163-1. 09/22/2025 05:05:42.834 7faf1d37d6d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 09/22/2025 05:05:42.916 7faf1d37d6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-4163-1.'] 09/22/2025 05:05:42.971 7faf1d37d6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 09/22/2025 05:05:42.985 7faf1d37d6d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 09/22/2025 05:05:43.058 7faf1d37d6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-4163-1.'] 09/22/2025 05:05:43.343 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Fetching all containers 09/22/2025 05:05:43.510 7faf1d3943c8 [contrail-device-manager] [INFO]: Restarting dnsmasq docker: config_dnsmasq_1 09/22/2025 05:05:43.765 7faf1d3943c8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4163-1..config_queue 09/22/2025 05:05:43.766 7faf1d3943c8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4163-1..tftp_queue 09/22/2025 05:05:44.074 7faf1d37d6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 09/22/2025 05:05:44.078 7faf1d37d6d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 09/22/2025 05:05:44.212 7faf1d37d6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-4163-1.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4163-1..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4163-1..tftp_queue'] 09/22/2025 05:05:44.266 7faf1d3943c8 [contrail-device-manager] [WARNING]: Process 136 prepared to run in Full mode... 09/22/2025 05:05:44.267 7faf1d3943c8 [contrail-device-manager] [WARNING]: Elected master Device Manager node. Initializing... 09/22/2025 05:05:44.267 7faf1d3943c8 [contrail-device-manager] [INFO]: SANDESH: INTROSPECT IS ON: 0.0.0.0:8096 09/22/2025 05:05:44.270 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add [config_updater.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/config_updater] to home page 09/22/2025 05:05:44.270 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_ConfigApiUpdateReq] 09/22/2025 05:05:44.271 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add [greenlets.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/greenlets] to home page 09/22/2025 05:05:44.271 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_GreenletObjectReq] 09/22/2025 05:05:44.272 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add [cpuinfo.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/nodeinfo/cpuinfo] to home page 09/22/2025 05:05:44.272 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CpuLoadInfoReq] 09/22/2025 05:05:44.278 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add [dm_introspect.xml:/usr/lib/python3.6/site-packages/device_manager/sandesh/dm_introspect] to home page 09/22/2025 05:05:44.278 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_RoutingInstanceList] 09/22/2025 05:05:44.278 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualNetworkList] 09/22/2025 05:05:44.279 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualMachineInterfaceList] 09/22/2025 05:05:44.279 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalInterfaceList] 09/22/2025 05:05:44.279 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalInterfaceList] 09/22/2025 05:05:44.279 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalRouterList] 09/22/2025 05:05:44.279 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_BgpRouterList] 09/22/2025 05:05:44.279 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalRouterList] 09/22/2025 05:05:44.279 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_InstanceIpList] 09/22/2025 05:05:44.279 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FloatingIpList] 09/22/2025 05:05:44.279 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PortTupleList] 09/22/2025 05:05:44.280 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FeatureFlagList] 09/22/2025 05:05:44.282 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add [sandesh_alarm.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_alarm] to home page 09/22/2025 05:05:44.282 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmCacheRequest] 09/22/2025 05:05:44.282 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmTypesRequest] 09/22/2025 05:05:44.282 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmAckRequest] 09/22/2025 05:05:44.284 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add [sandesh_ctrl.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_ctrl] to home page 09/22/2025 05:05:44.284 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlClientToServer] 09/22/2025 05:05:44.284 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlServerToClient] 09/22/2025 05:05:44.285 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add [sandesh_trace.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_trace] to home page 09/22/2025 05:05:44.285 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceRequest] 09/22/2025 05:05:44.285 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferListRequest] 09/22/2025 05:05:44.285 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceEnableDisableReq] 09/22/2025 05:05:44.285 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufStatusReq] 09/22/2025 05:05:44.285 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferEnableDisableReq] 09/22/2025 05:05:44.286 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add [sandesh_uve.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_uve] to home page 09/22/2025 05:05:44.286 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshMessageStatsReq] 09/22/2025 05:05:44.286 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsSet] 09/22/2025 05:05:44.286 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsStatus] 09/22/2025 05:05:44.286 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsSet] 09/22/2025 05:05:44.293 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsStatus] 09/22/2025 05:05:44.293 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsSet] 09/22/2025 05:05:44.293 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsReset] 09/22/2025 05:05:44.294 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsStatus] 09/22/2025 05:05:44.294 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueSet] 09/22/2025 05:05:44.294 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueStatus] 09/22/2025 05:05:44.294 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CollectorInfoRequest] 09/22/2025 05:05:44.295 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVEDSConfReq] 09/22/2025 05:05:44.296 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVECacheReq] 09/22/2025 05:05:44.296 7faf1d3943c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVETypesReq] 09/22/2025 05:05:44.301 7faf1d3943c8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.ztp_queue 09/22/2025 05:05:44.358 7faf14624588 [contrail-device-manager] [ERROR]: Cannot write http_port 8096 to /tmp/contrail-device-manager.1.http_port 09/22/2025 05:05:44.358 7faf14624588 [contrail-device-manager] [ERROR]: Starting Introspect on HTTP Port 8096 09/22/2025 05:05:44.358 7faf0f6034e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:44.359 7faf0f6034e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Initializing description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 09/22/2025 05:05:44.359 7faf0f6034e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:44.370 7faf0f55c6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:44.371 7faf0f55c6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 09/22/2025 05:05:44.371 7faf0f55c6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:45.242 7faf1d37d6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 09/22/2025 05:05:45.247 7faf1d37d6d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 09/22/2025 05:05:45.305 7faf1d37d6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-4163-1.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4163-1..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4163-1..tftp_queue', 'device_manager_ztp.ztp_queue'] 09/22/2025 05:05:45.407 7faf14704438 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/22/2025 05:05:45.408 7faf14704438 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/22/2025 05:05:45.408 7faf14704438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/22/2025 05:05:45.408 7faf14704438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:05:45.409 7faf14704438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 09/22/2025 05:05:45.409 7faf14704438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:05:45.409 7faf14704438 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/22/2025 05:05:45.409 7faf14704438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/22/2025 05:05:45.409 7faf14704438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:45.409 7faf14704438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 09/22/2025 05:05:45.409 7faf14704438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:45.409 7faf14704438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:45.409 7faf14704438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1758517529401078 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.254:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/22/2025 05:05:45.409 7faf14704438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:49.409 7faf0f62dac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/22/2025 05:05:49.410 7faf0f62dac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/22/2025 05:05:49.410 7faf0f62dac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/22/2025 05:05:49.410 7faf0f62dac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:05:49.411 7faf0f62dac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 09/22/2025 05:05:49.411 7faf0f62dac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:05:49.411 7faf0f62dac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/22/2025 05:05:49.411 7faf0f62dac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/22/2025 05:05:49.411 7faf0f62dac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:49.411 7faf0f62dac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 09/22/2025 05:05:49.411 7faf0f62dac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:49.411 7faf0f62dac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:49.411 7faf0f62dac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1758517529401078 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.254:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/22/2025 05:05:49.411 7faf0f62dac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:50.531 7faf0f634d68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:50.532 7faf0f634d68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/22/2025 05:05:50.532 7faf0f634d68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:53.412 7faf0f4aa8d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/22/2025 05:05:53.415 7faf0f4aa8d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/22/2025 05:05:53.415 7faf0f4aa8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/22/2025 05:05:53.415 7faf0f4aa8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:05:53.415 7faf0f4aa8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/22/2025 05:05:53.415 7faf0f4aa8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:05:53.415 7faf0f4aa8d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/22/2025 05:05:53.416 7faf0f4aa8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/22/2025 05:05:53.416 7faf0f4aa8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:53.416 7faf0f4aa8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/22/2025 05:05:53.416 7faf0f4aa8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:53.416 7faf0f4aa8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:53.416 7faf0f4aa8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1758517529401078 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.254:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/22/2025 05:05:53.416 7faf0f4aa8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:57.415 7faf0f3ff630 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/22/2025 05:05:57.416 7faf0f3ff630 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/22/2025 05:05:57.417 7faf0f3ff630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/22/2025 05:05:57.417 7faf0f3ff630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:05:57.417 7faf0f3ff630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/22/2025 05:05:57.417 7faf0f3ff630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:05:57.417 7faf0f3ff630 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/22/2025 05:05:57.417 7faf0f3ff630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/22/2025 05:05:57.417 7faf0f3ff630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:57.417 7faf0f3ff630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/22/2025 05:05:57.417 7faf0f3ff630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:57.417 7faf0f3ff630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:05:57.418 7faf0f3ff630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1758517529401078 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.254:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/22/2025 05:05:57.418 7faf0f3ff630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:01.417 7faf0f541780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/22/2025 05:06:01.418 7faf0f541780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/22/2025 05:06:01.419 7faf0f541780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/22/2025 05:06:01.419 7faf0f541780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:06:01.419 7faf0f541780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/22/2025 05:06:01.419 7faf0f541780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:06:01.419 7faf0f541780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/22/2025 05:06:01.419 7faf0f541780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/22/2025 05:06:01.419 7faf0f541780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:01.419 7faf0f541780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/22/2025 05:06:01.419 7faf0f541780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:01.420 7faf0f541780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:01.420 7faf0f541780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1758517529401078 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.254:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/22/2025 05:06:01.420 7faf0f541780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:05.419 7faf0f484828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/22/2025 05:06:05.420 7faf0f484828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/22/2025 05:06:05.420 7faf0f484828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/22/2025 05:06:05.421 7faf0f484828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:06:05.421 7faf0f484828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/22/2025 05:06:05.421 7faf0f484828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:06:05.421 7faf0f484828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/22/2025 05:06:05.421 7faf0f484828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/22/2025 05:06:05.421 7faf0f484828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:05.421 7faf0f484828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/22/2025 05:06:05.421 7faf0f484828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:05.422 7faf0f484828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:05.422 7faf0f484828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1758517529401078 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.254:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/22/2025 05:06:05.422 7faf0f484828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:09.421 7faf0f484828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/22/2025 05:06:09.423 7faf0f484828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/22/2025 05:06:09.423 7faf0f484828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/22/2025 05:06:09.423 7faf0f484828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:06:09.423 7faf0f484828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/22/2025 05:06:09.423 7faf0f484828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:06:09.423 7faf0f484828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/22/2025 05:06:09.423 7faf0f484828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/22/2025 05:06:09.424 7faf0f484828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:09.424 7faf0f484828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/22/2025 05:06:09.424 7faf0f484828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:09.424 7faf0f484828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:09.424 7faf0f484828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1758517529401078 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.254:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/22/2025 05:06:09.424 7faf0f484828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:13.423 7faf0f484828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/22/2025 05:06:13.424 7faf0f484828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/22/2025 05:06:13.425 7faf0f484828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/22/2025 05:06:13.425 7faf0f484828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:06:13.425 7faf0f484828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/22/2025 05:06:13.425 7faf0f484828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:06:13.425 7faf0f484828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/22/2025 05:06:13.425 7faf0f484828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/22/2025 05:06:13.425 7faf0f484828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:13.425 7faf0f484828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/22/2025 05:06:13.426 7faf0f484828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:13.426 7faf0f484828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:13.426 7faf0f484828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1758517529401078 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.254:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/22/2025 05:06:13.426 7faf0f484828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:17.425 7faf0f484828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/22/2025 05:06:17.427 7faf0f484828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/22/2025 05:06:17.427 7faf0f484828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/22/2025 05:06:17.427 7faf0f484828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:06:17.427 7faf0f484828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/22/2025 05:06:17.427 7faf0f484828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:06:17.427 7faf0f484828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/22/2025 05:06:17.428 7faf0f484828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/22/2025 05:06:17.428 7faf0f484828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:17.428 7faf0f484828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/22/2025 05:06:17.428 7faf0f484828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:17.428 7faf0f484828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:17.428 7faf0f484828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1758517529401078 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.254:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/22/2025 05:06:17.428 7faf0f484828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:21.427 7faf0f484828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/22/2025 05:06:21.429 7faf0f484828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/22/2025 05:06:21.429 7faf0f484828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/22/2025 05:06:21.429 7faf0f484828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:06:21.429 7faf0f484828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/22/2025 05:06:21.429 7faf0f484828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:06:21.429 7faf0f484828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/22/2025 05:06:21.429 7faf0f484828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/22/2025 05:06:21.429 7faf0f484828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:21.429 7faf0f484828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/22/2025 05:06:21.430 7faf0f484828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:21.430 7faf0f484828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:21.430 7faf0f484828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1758517529401078 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.254:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/22/2025 05:06:21.430 7faf0f484828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:25.438 7faf0f484828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/22/2025 05:06:25.439 7faf0f484828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/22/2025 05:06:25.439 7faf0f484828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/22/2025 05:06:25.439 7faf0f484828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:06:25.439 7faf0f484828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/22/2025 05:06:25.439 7faf0f484828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:06:25.440 7faf0f484828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/22/2025 05:06:25.440 7faf0f484828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/22/2025 05:06:25.440 7faf0f484828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:25.440 7faf0f484828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/22/2025 05:06:25.440 7faf0f484828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:25.440 7faf0f484828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:25.440 7faf0f484828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1758517529401078 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.254:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/22/2025 05:06:25.441 7faf0f484828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:29.440 7faf0f484828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/22/2025 05:06:29.441 7faf0f484828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/22/2025 05:06:29.441 7faf0f484828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/22/2025 05:06:29.441 7faf0f484828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:06:29.441 7faf0f484828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/22/2025 05:06:29.441 7faf0f484828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:06:29.441 7faf0f484828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/22/2025 05:06:29.442 7faf0f484828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/22/2025 05:06:29.442 7faf0f484828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:29.442 7faf0f484828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/22/2025 05:06:29.442 7faf0f484828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:29.442 7faf0f484828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:29.442 7faf0f484828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1758517529401078 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.254:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/22/2025 05:06:29.442 7faf0f484828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:33.442 7faf1476ff60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/22/2025 05:06:33.443 7faf1476ff60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/22/2025 05:06:33.443 7faf1476ff60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/22/2025 05:06:33.443 7faf1476ff60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:06:33.443 7faf1476ff60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/22/2025 05:06:33.443 7faf1476ff60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:06:33.443 7faf1476ff60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/22/2025 05:06:33.444 7faf1476ff60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/22/2025 05:06:33.444 7faf1476ff60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:33.444 7faf1476ff60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/22/2025 05:06:33.444 7faf1476ff60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:33.444 7faf1476ff60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:33.444 7faf1476ff60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1758517529401078 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.254:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/22/2025 05:06:33.444 7faf1476ff60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:37.444 7faf0f623c18 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/22/2025 05:06:37.445 7faf0f623c18 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/22/2025 05:06:37.445 7faf0f623c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/22/2025 05:06:37.445 7faf0f623c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:06:37.445 7faf0f623c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/22/2025 05:06:37.445 7faf0f623c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:06:37.445 7faf0f623c18 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/22/2025 05:06:37.446 7faf0f623c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/22/2025 05:06:37.446 7faf0f623c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:37.446 7faf0f623c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/22/2025 05:06:37.446 7faf0f623c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:37.446 7faf0f623c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:37.446 7faf0f623c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1758517529401078 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.254:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/22/2025 05:06:37.446 7faf0f623c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:41.446 7faf15623a20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/22/2025 05:06:41.447 7faf15623a20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/22/2025 05:06:41.447 7faf15623a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/22/2025 05:06:41.447 7faf15623a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:06:41.448 7faf15623a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/22/2025 05:06:41.448 7faf15623a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:06:41.448 7faf15623a20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/22/2025 05:06:41.448 7faf15623a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/22/2025 05:06:41.448 7faf15623a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:41.448 7faf15623a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/22/2025 05:06:41.448 7faf15623a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:41.448 7faf15623a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:41.448 7faf15623a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1758517529401078 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.254:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/22/2025 05:06:41.448 7faf15623a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:45.448 7faf1a4d2630 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/22/2025 05:06:45.450 7faf1a4d2630 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/22/2025 05:06:45.450 7faf1a4d2630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/22/2025 05:06:45.450 7faf1a4d2630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:06:45.450 7faf1a4d2630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/22/2025 05:06:45.450 7faf1a4d2630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:06:45.450 7faf1a4d2630 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/22/2025 05:06:45.450 7faf1a4d2630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/22/2025 05:06:45.451 7faf1a4d2630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:45.451 7faf1a4d2630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/22/2025 05:06:45.451 7faf1a4d2630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:45.451 7faf1a4d2630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:45.451 7faf1a4d2630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1758517529401078 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.254:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/22/2025 05:06:45.451 7faf1a4d2630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:49.453 7faf0f634240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/22/2025 05:06:49.454 7faf0f634240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/22/2025 05:06:49.454 7faf0f634240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/22/2025 05:06:49.455 7faf0f634240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:06:49.455 7faf0f634240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/22/2025 05:06:49.455 7faf0f634240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:06:49.455 7faf0f634240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/22/2025 05:06:49.455 7faf0f634240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/22/2025 05:06:49.455 7faf0f634240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:49.455 7faf0f634240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/22/2025 05:06:49.455 7faf0f634240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:49.455 7faf0f634240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:49.455 7faf0f634240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1758517529401078 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.254:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/22/2025 05:06:49.456 7faf0f634240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:53.456 7faf1c203cc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/22/2025 05:06:53.469 7faf1c203cc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/22/2025 05:06:53.470 7faf1c203cc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/22/2025 05:06:53.470 7faf1c203cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:06:53.470 7faf1c203cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/22/2025 05:06:53.470 7faf1c203cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:06:53.470 7faf1c203cc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/22/2025 05:06:53.470 7faf1c203cc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/22/2025 05:06:53.470 7faf1c203cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:53.470 7faf1c203cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/22/2025 05:06:53.470 7faf1c203cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:53.470 7faf1c203cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:53.471 7faf1c203cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1758517529401078 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.254:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/22/2025 05:06:53.471 7faf1c203cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:57.470 7faf0f563780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/22/2025 05:06:57.471 7faf0f563780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/22/2025 05:06:57.471 7faf0f563780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/22/2025 05:06:57.472 7faf0f563780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:06:57.472 7faf0f563780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/22/2025 05:06:57.472 7faf0f563780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:06:57.472 7faf0f563780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/22/2025 05:06:57.472 7faf0f563780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/22/2025 05:06:57.472 7faf0f563780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:57.472 7faf0f563780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/22/2025 05:06:57.472 7faf0f563780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:57.472 7faf0f563780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:06:57.472 7faf0f563780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1758517529401078 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.254:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/22/2025 05:06:57.473 7faf0f563780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:01.472 7faf1a4f5240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/22/2025 05:07:01.473 7faf1a4f5240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/22/2025 05:07:01.473 7faf1a4f5240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/22/2025 05:07:01.473 7faf1a4f5240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:07:01.473 7faf1a4f5240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/22/2025 05:07:01.473 7faf1a4f5240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:07:01.473 7faf1a4f5240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/22/2025 05:07:01.473 7faf1a4f5240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/22/2025 05:07:01.474 7faf1a4f5240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:01.474 7faf1a4f5240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/22/2025 05:07:01.474 7faf1a4f5240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:01.474 7faf1a4f5240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:01.474 7faf1a4f5240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1758517529401078 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.254:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/22/2025 05:07:01.474 7faf1a4f5240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:05.474 7faf14788b70 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/22/2025 05:07:05.478 7faf14788b70 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/22/2025 05:07:05.478 7faf14788b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/22/2025 05:07:05.478 7faf14788b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:07:05.478 7faf14788b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/22/2025 05:07:05.478 7faf14788b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:07:05.478 7faf14788b70 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/22/2025 05:07:05.479 7faf14788b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/22/2025 05:07:05.479 7faf14788b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:05.479 7faf14788b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/22/2025 05:07:05.479 7faf14788b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:05.479 7faf14788b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:05.479 7faf14788b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1758517529401078 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.254:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/22/2025 05:07:05.479 7faf14788b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:07.442 7faf1d3943c8 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 09/22/2025 05:07:07.442 7faf1d3943c8 [contrail-device-manager] [WARNING]: initializing cassandra cluster 09/22/2025 05:07:07.443 7faf1d3943c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/22/2025 05:07:07.443 7faf1b713240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:07.444 7faf1b713240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/22/2025 05:07:07.444 7faf1b713240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:07.445 7faf1d3943c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/22/2025 05:07:07.482 7faf1d3943c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/22/2025 05:07:07.485 7faf0f4c2eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:07.489 7faf0f4c2eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Initializing description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/22/2025 05:07:07.490 7faf0f4c2eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:07.499 7faf1d3943c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 09/22/2025 05:07:07.500 7faf0f4c2d68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:07.500 7faf0f4c2d68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/22/2025 05:07:07.500 7faf0f4c2d68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:07.520 7faf1d3943c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/22/2025 05:07:08.045 7faf1d3943c8 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace', created 09/22/2025 05:07:08.050 7faf1d3943c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/22/2025 05:07:08.056 7faf1d3943c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/22/2025 05:07:09.134 7faf1d3943c8 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace' fixed with replication {'class': 'SimpleStrategy', 'replication_factor': None} 09/22/2025 05:07:09.146 7faf1d3943c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/22/2025 05:07:09.146 7faf1d3943c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 09/22/2025 05:07:09.210 7faf1d3943c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 09/22/2025 05:07:09.220 7faf1d3943c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/22/2025 05:07:09.478 7faf0f3e7240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/22/2025 05:07:09.482 7faf0f3e7240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/22/2025 05:07:09.482 7faf0f3e7240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/22/2025 05:07:09.482 7faf0f3e7240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:07:09.482 7faf0f3e7240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 09/22/2025 05:07:09.482 7faf0f3e7240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:07:09.482 7faf0f3e7240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/22/2025 05:07:09.483 7faf0f3e7240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/22/2025 05:07:09.483 7faf0f3e7240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:09.483 7faf0f3e7240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/22/2025 05:07:09.483 7faf0f3e7240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:09.483 7faf0f3e7240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:09.483 7faf0f3e7240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1758517529401078 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.254:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/22/2025 05:07:09.483 7faf0f3e7240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:10.174 7faf1d3943c8 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table', created 09/22/2025 05:07:10.183 7faf1d3943c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/22/2025 05:07:10.198 7faf1d3943c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/22/2025 05:07:11.114 7faf1d3943c8 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table' fixed with properties {'gc_grace_seconds': 864000} 09/22/2025 05:07:11.118 7faf1d3943c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/22/2025 05:07:11.124 7faf1d3943c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/22/2025 05:07:12.115 7faf1d3943c8 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table', created 09/22/2025 05:07:12.127 7faf1d3943c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/22/2025 05:07:12.148 7faf1d3943c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/22/2025 05:07:13.057 7faf1d3943c8 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table' fixed with properties {'gc_grace_seconds': 864000} 09/22/2025 05:07:13.061 7faf1d3943c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/22/2025 05:07:13.067 7faf1d3943c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/22/2025 05:07:13.483 7faf0f46d588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/22/2025 05:07:13.484 7faf0f46d588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/22/2025 05:07:13.484 7faf0f46d588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/22/2025 05:07:13.484 7faf0f46d588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:07:13.484 7faf0f46d588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 09/22/2025 05:07:13.484 7faf0f46d588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:07:13.484 7faf0f46d588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/22/2025 05:07:13.484 7faf0f46d588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/22/2025 05:07:13.485 7faf0f46d588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:13.485 7faf0f46d588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/22/2025 05:07:13.485 7faf0f46d588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:13.485 7faf0f46d588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:13.485 7faf0f46d588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1758517529401078 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.254:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/22/2025 05:07:13.485 7faf0f46d588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:13.803 7faf1d3943c8 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table', created 09/22/2025 05:07:13.807 7faf1d3943c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/22/2025 05:07:13.817 7faf1d3943c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/22/2025 05:07:15.557 7faf1d3943c8 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table' fixed with properties {'gc_grace_seconds': 864000} 09/22/2025 05:07:15.561 7faf1d3943c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/22/2025 05:07:15.566 7faf1d3943c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/22/2025 05:07:16.125 7faf1d3943c8 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table', created 09/22/2025 05:07:16.130 7faf1d3943c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/22/2025 05:07:16.135 7faf1d3943c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 09/22/2025 05:07:16.835 7faf1d3943c8 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table' fixed with properties {'gc_grace_seconds': 864000} 09/22/2025 05:07:16.839 7faf1d3943c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 09/22/2025 05:07:16.840 7faf0f54e2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:16.840 7faf0f54e2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/22/2025 05:07:16.840 7faf0f54e2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:16.901 7faf1d3943c8 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.254:5673//', 'pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.48:5673//'] 09/22/2025 05:07:16.902 7faf1d3943c8 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 09/22/2025 05:07:16.902 7faf1a4d2438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:16.903 7faf1a4d2438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Initializing description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 09/22/2025 05:07:16.903 7faf1a4d2438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:16.903 7faf1a4d2438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:16.903 7faf1a4d2438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 09/22/2025 05:07:16.903 7faf1a4d2438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:16.908 7faf1d3943c8 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/22/2025 05:07:16.908 7faf0f38ff60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:16.909 7faf0f38ff60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/22/2025 05:07:16.909 7faf0f38ff60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:16.939 7faf1d3943c8 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 09/22/2025 05:07:16.939 7faf0f3854e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:16.940 7faf0f3854e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 09/22/2025 05:07:16.940 7faf0f3854e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:16.945 7faf1d3943c8 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/22/2025 05:07:16.945 7faf0f38f828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:16.945 7faf0f38f828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/22/2025 05:07:16.945 7faf0f38f828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:16.946 7faf0f38f780 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 09/22/2025 05:07:16.947 7faf0f3e14e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:16.947 7faf0f3e14e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 09/22/2025 05:07:16.947 7faf0f3e14e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:16.955 7faf0f38f780 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/22/2025 05:07:16.956 7faf0f38c588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:16.956 7faf0f38c588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/22/2025 05:07:16.956 7faf0f38c588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:17.484 7faf1569a588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/22/2025 05:07:17.485 7faf1569a588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/22/2025 05:07:17.485 7faf1569a588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/22/2025 05:07:17.485 7faf1569a588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:07:17.486 7faf1569a588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/22/2025 05:07:17.486 7faf1569a588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:07:17.486 7faf1569a588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/22/2025 05:07:17.486 7faf1569a588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/22/2025 05:07:17.486 7faf1569a588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:17.486 7faf1569a588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/22/2025 05:07:17.486 7faf1569a588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:17.486 7faf1569a588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:17.486 7faf1569a588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1758517529401078 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.254:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/22/2025 05:07:17.486 7faf1569a588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:21.486 7faf1569a588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/22/2025 05:07:21.487 7faf1569a588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/22/2025 05:07:21.487 7faf1569a588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/22/2025 05:07:21.487 7faf1569a588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:07:21.487 7faf1569a588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/22/2025 05:07:21.487 7faf1569a588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:07:21.487 7faf1569a588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/22/2025 05:07:21.488 7faf1569a588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/22/2025 05:07:21.488 7faf1569a588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:21.488 7faf1569a588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/22/2025 05:07:21.488 7faf1569a588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:21.488 7faf1569a588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:21.488 7faf1569a588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1758517529401078 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.254:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/22/2025 05:07:21.488 7faf1569a588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:25.488 7faf1569a588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/22/2025 05:07:25.489 7faf1569a588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/22/2025 05:07:25.489 7faf1569a588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/22/2025 05:07:25.489 7faf1569a588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:07:25.489 7faf1569a588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/22/2025 05:07:25.489 7faf1569a588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:07:25.489 7faf1569a588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/22/2025 05:07:25.490 7faf1569a588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/22/2025 05:07:25.490 7faf1569a588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:25.490 7faf1569a588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/22/2025 05:07:25.490 7faf1569a588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:25.490 7faf1569a588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:25.490 7faf1569a588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1758517529401078 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.254:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/22/2025 05:07:25.490 7faf1569a588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:29.489 7faf1569a588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/22/2025 05:07:29.491 7faf1569a588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 09/22/2025 05:07:29.491 7faf1569a588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/22/2025 05:07:29.493 7faf1569a588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:07:29.493 7faf1569a588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/22/2025 05:07:29.493 7faf1569a588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:07:29.493 7faf1569a588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/22/2025 05:07:29.494 7faf1569a588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/22/2025 05:07:29.494 7faf1569a588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:29.494 7faf1569a588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/22/2025 05:07:29.494 7faf1569a588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:29.495 7faf1569a588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:29.495 7faf1569a588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1758517529401078 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.254:8086, 10.0.0.48:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/22/2025 05:07:29.495 7faf1569a588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/22/2025 05:07:33.491 7faf1569a588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/22/2025 05:07:33.493 7faf1569a588 [contrail-device-manager] [INFO]: Session Event: TCP Connected 09/22/2025 05:07:33.493 7faf1569a588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/22/2025 05:07:33.493 7faf1569a588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:07:33.493 7faf1569a588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4163-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/22/2025 05:07:33.493 7faf1569a588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/22/2025 05:07:33.493 7faf1569a588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/22/2025 05:07:33.493 7faf1569a588 [contrail-device-manager] [DEBUG]: Send sandesh control message. uve type count # 19 09/22/2025 05:07:33.493 7faf1569a588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/22/2025 05:07:33.493 7faf1569a588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/22/2025 05:07:33.494 7faf1569a588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/22/2025 05:07:33.494 7faf0f322f60 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/22/2025 05:07:33.495 7faf0f322f60 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/22/2025 05:07:33.792 7faf0f38f828 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4163-2.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.38', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4163-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4163-2.'], 'id_perms': {'created': '2025-09-22T05:07:33.780572', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-22T05:07:33.780572', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9295089912679795216, 'uuid_mslong': 671661107199689183}}, 'parent_type': 'global-system-config', 'parent_uuid': '92396173-5c85-45d4-8bbc-364cd7076f9e', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '09523837-e815-41df-80fe-cb0730988a10'}, 'oper': 'CREATE', 'request-id': 'req-447a4778-4a8b-4c71-b11e-96a97e0a554d', 'type': 'analytics_node', 'uuid': '09523837-e815-41df-80fe-cb0730988a10'} 09/22/2025 05:07:33.842 7faf0f38f828 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4163-3.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.48', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4163-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4163-3.'], 'id_perms': {'created': '2025-09-22T05:07:33.828825', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-22T05:07:33.828825', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12790877269527633552, 'uuid_mslong': 9128350833358357341}}, 'parent_type': 'global-system-config', 'parent_uuid': '92396173-5c85-45d4-8bbc-364cd7076f9e', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '7eae6ab8-0777-435d-b182-531b8fd01e90'}, 'oper': 'CREATE', 'request-id': 'req-b9cb8003-e582-4941-9006-50f7e3ec4e97', 'type': 'analytics_node', 'uuid': '7eae6ab8-0777-435d-b182-531b8fd01e90'} 09/22/2025 05:07:33.939 7faf0f38f828 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4163-1.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.254', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4163-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4163-1.'], 'id_perms': {'created': '2025-09-22T05:07:33.923364', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-22T05:07:33.923364', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13203871848615050806, 'uuid_mslong': 63620828488090200}}, 'parent_type': 'global-system-config', 'parent_uuid': '92396173-5c85-45d4-8bbc-364cd7076f9e', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '00e206ce-7259-4258-b73d-937d3c7afe36'}, 'oper': 'CREATE', 'request-id': 'req-7acaa4e8-aabf-4a15-ae45-b0b37ba19df7', 'type': 'analytics_node', 'uuid': '00e206ce-7259-4258-b73d-937d3c7afe36'} 09/22/2025 05:07:42.541 7faf0f322588 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001016 09/22/2025 05:07:42.542 7faf0f322588 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 09/22/2025 05:07:42.542 7faf0f323a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 09/22/2025 05:07:42.542 7faf0f323a20 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 0 09/22/2025 05:07:42.542 7faf0f323a20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/22/2025 05:07:42.542 7faf0f323a20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/22/2025 05:07:42.543 7faf0f323a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 09/22/2025 05:07:42.543 7faf0f3220f0 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/22/2025 05:07:42.544 7faf0f3220f0 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/22/2025 05:07:47.853 7faf0f38f828 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4163-1.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.254', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4163-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4163-1.'], 'id_perms': {'created': '2025-09-22T05:07:47.828518', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-22T05:07:47.828518', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13806675645593498109, 'uuid_mslong': 1440452931935617124}}, 'parent_type': 'global-system-config', 'parent_uuid': '92396173-5c85-45d4-8bbc-364cd7076f9e', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '13fd844f-4bb1-4064-bf9b-2a5948a331fd'}, 'oper': 'CREATE', 'request-id': 'req-69994e33-f6ba-478e-8662-b83471f6611e', 'type': 'analytics_alarm_node', 'uuid': '13fd844f-4bb1-4064-bf9b-2a5948a331fd'} 09/22/2025 05:07:48.299 7faf0f38f828 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4163-2.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.38', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4163-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4163-2.'], 'id_perms': {'created': '2025-09-22T05:07:48.278282', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-22T05:07:48.278282', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10155011176576721758, 'uuid_mslong': 14946925158796971745}}, 'parent_type': 'global-system-config', 'parent_uuid': '92396173-5c85-45d4-8bbc-364cd7076f9e', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'cf6e253d-5e71-4ee1-8ced-d8dc8dfa235e'}, 'oper': 'CREATE', 'request-id': 'req-2bdd7aa2-152d-459e-ac28-609c70caab22', 'type': 'analytics_alarm_node', 'uuid': 'cf6e253d-5e71-4ee1-8ced-d8dc8dfa235e'} 09/22/2025 05:07:48.346 7faf0f38f828 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4163-3.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.48', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4163-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4163-3.'], 'id_perms': {'created': '2025-09-22T05:07:48.333980', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-22T05:07:48.333980', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11164695223350374678, 'uuid_mslong': 15704269362454413807}}, 'parent_type': 'global-system-config', 'parent_uuid': '92396173-5c85-45d4-8bbc-364cd7076f9e', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'd9f0c5c7-c625-41ef-9af0-f7270d4cb116'}, 'oper': 'CREATE', 'request-id': 'req-3bb9d5b6-4ce3-40d5-bab7-35e28beef716', 'type': 'analytics_alarm_node', 'uuid': 'd9f0c5c7-c625-41ef-9af0-f7270d4cb116'} 09/22/2025 05:07:52.671 7faf0f322588 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001296 09/22/2025 05:07:52.672 7faf0f322588 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 09/22/2025 05:07:52.672 7faf1a567e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 09/22/2025 05:07:52.672 7faf1a567e10 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 2 09/22/2025 05:07:52.672 7faf1a567e10 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/22/2025 05:07:52.672 7faf1a567e10 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/22/2025 05:07:57.510 7faf0f38f828 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4163-3.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.48', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4163-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4163-3.'], 'id_perms': {'created': '2025-09-22T05:07:57.501995', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-22T05:07:57.501995', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11340104008449398032, 'uuid_mslong': 1559179266200584326}}, 'parent_type': 'global-system-config', 'parent_uuid': '92396173-5c85-45d4-8bbc-364cd7076f9e', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '15a35148-1f0a-4086-9d60-2483639c1510'}, 'oper': 'CREATE', 'request-id': 'req-e6633f3a-2320-4a60-8931-6271e66dd1db', 'type': 'analytics_snmp_node', 'uuid': '15a35148-1f0a-4086-9d60-2483639c1510'} 09/22/2025 05:07:57.555 7faf0f38f828 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4163-2.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.38', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4163-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4163-2.'], 'id_perms': {'created': '2025-09-22T05:07:57.549479', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-22T05:07:57.549479', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12607721901159529272, 'uuid_mslong': 14172464313752768299}}, 'parent_type': 'global-system-config', 'parent_uuid': '92396173-5c85-45d4-8bbc-364cd7076f9e', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'c4aeb534-5522-4f2b-aef7-a0454186bb38'}, 'oper': 'CREATE', 'request-id': 'req-881717d2-c09d-411f-b4fb-cb564b35d90b', 'type': 'analytics_snmp_node', 'uuid': 'c4aeb534-5522-4f2b-aef7-a0454186bb38'} 09/22/2025 05:07:57.639 7faf0f38f828 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4163-1.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.254', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4163-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4163-1.'], 'id_perms': {'created': '2025-09-22T05:07:57.633137', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-22T05:07:57.633137', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11084116435932056533, 'uuid_mslong': 13768376024343596197}}, 'parent_type': 'global-system-config', 'parent_uuid': '92396173-5c85-45d4-8bbc-364cd7076f9e', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'bf13190b-d101-4ca5-99d2-b12c1ccc8bd5'}, 'oper': 'CREATE', 'request-id': 'req-3a323118-3315-4a04-808d-4d1417d88ecb', 'type': 'analytics_snmp_node', 'uuid': 'bf13190b-d101-4ca5-99d2-b12c1ccc8bd5'} 09/22/2025 05:09:35.298 7faf0f38f828 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-63dfea73-4735-4c10-ac65-baca56e955c4', 'type': 'global_system_config', 'uuid': '92396173-5c85-45d4-8bbc-364cd7076f9e'} 09/22/2025 05:09:35.349 7faf0f38f828 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-3e3a0f32-ddd1-4fec-abca-5a6d97788ce4', 'type': 'global_system_config', 'uuid': '92396173-5c85-45d4-8bbc-364cd7076f9e'} 09/22/2025 05:09:35.439 7faf0f38f828 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4163-1'], 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-4163-1', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4163-1'], 'id_perms': {'created': '2025-09-22T05:09:35.432414', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-22T05:09:35.432414', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12713609122453497788, 'uuid_mslong': 18430396258315355087}}, 'parent_type': 'global-system-config', 'parent_uuid': '92396173-5c85-45d4-8bbc-364cd7076f9e', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'ffc5ebbf-9e93-43cf-b06f-d0232501a7bc', 'virtual_router_dpdk_enabled': False, 'virtual_router_ip_address': '10.20.0.25'}, 'oper': 'CREATE', 'request-id': 'req-12b181e4-20c1-4d1f-8424-e9d7d0efa182', 'type': 'virtual_router', 'uuid': 'ffc5ebbf-9e93-43cf-b06f-d0232501a7bc'} 09/22/2025 05:09:35.475 7faf0f38f828 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4163-2'], 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-4163-2', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4163-2'], 'id_perms': {'created': '2025-09-22T05:09:35.465382', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-22T05:09:35.465382', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10095230317733317145, 'uuid_mslong': 3123122456147215360}}, 'parent_type': 'global-system-config', 'parent_uuid': '92396173-5c85-45d4-8bbc-364cd7076f9e', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '2b578f53-acb3-4400-8c19-767cb63f8a19', 'virtual_router_dpdk_enabled': False, 'virtual_router_ip_address': '10.20.0.193'}, 'oper': 'CREATE', 'request-id': 'req-44b8ad81-f267-4230-a83c-8951ced76656', 'type': 'virtual_router', 'uuid': '2b578f53-acb3-4400-8c19-767cb63f8a19'} 09/22/2025 05:09:35.782 7faf0f38f828 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4163-1', 'vhost0'], 'obj_dict': {'display_name': 'vhost0', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4163-1', 'vhost0'], 'id_perms': {'created': '2025-09-22T05:09:35.762964', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-22T05:09:35.762964', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10875606784697625994, 'uuid_mslong': 2632598431630641769}}, 'parent_type': 'virtual-router', 'parent_uuid': 'ffc5ebbf-9e93-43cf-b06f-d0232501a7bc', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '2488de53-5198-4a69-96ed-eabfcebaed8a', 'virtual_machine_interface_disable_policy': True, 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:24:88:de:53:51']}, 'virtual_network_refs': [{'to': ['default-domain', 'default-project', 'ip-fabric'], 'uuid': '1e1f479a-998a-4f6a-b582-c1365e6871ff'}]}, 'oper': 'CREATE', 'request-id': 'req-2a052d73-e842-4a28-805d-883cc0b763ad', 'type': 'virtual_machine_interface', 'uuid': '2488de53-5198-4a69-96ed-eabfcebaed8a'} 09/22/2025 05:09:35.828 7faf0f38f828 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4163-1', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-2a052d73-e842-4a28-805d-883cc0b763ad', 'type': 'virtual_machine_interface', 'uuid': '2488de53-5198-4a69-96ed-eabfcebaed8a'} 09/22/2025 05:09:35.853 7faf0f38f828 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4163-2', 'vhost0'], 'obj_dict': {'display_name': 'vhost0', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4163-2', 'vhost0'], 'id_perms': {'created': '2025-09-22T05:09:35.823815', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-22T05:09:35.823815', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13629724266493009525, 'uuid_mslong': 4682909314238859136}}, 'parent_type': 'virtual-router', 'parent_uuid': '2b578f53-acb3-4400-8c19-767cb63f8a19', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '40fd0935-ce93-4780-bd26-8201cad65e75', 'virtual_machine_interface_disable_policy': True, 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:40:fd:09:35:ce']}, 'virtual_network_refs': [{'to': ['default-domain', 'default-project', 'ip-fabric'], 'uuid': '1e1f479a-998a-4f6a-b582-c1365e6871ff'}]}, 'oper': 'CREATE', 'request-id': 'req-fd8bf963-3a72-4472-bd68-48d04ffdd5f8', 'type': 'virtual_machine_interface', 'uuid': '40fd0935-ce93-4780-bd26-8201cad65e75'} 09/22/2025 05:09:35.914 7faf0f38f828 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4163-2', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-fd8bf963-3a72-4472-bd68-48d04ffdd5f8', 'type': 'virtual_machine_interface', 'uuid': '40fd0935-ce93-4780-bd26-8201cad65e75'} 09/22/2025 05:09:36.324 7faf0f38f828 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'oper': 'UPDATE', 'request-id': 'req-ede95668-db03-4a3f-b7a9-ce0e9dd52e08', 'type': 'access_control_list', 'uuid': 'fecfe064-f13e-42d2-a7b4-d04e97975bba'} 09/22/2025 05:09:49.931 7faf0f468eb8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/22/2025 05:10:28.273 7faf0f634240 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/22/2025 05:11:31.807 7faf0f3898d0 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map