08/25/2025 01:36:24.846 7f78d5e43d08 [contrail-device-manager] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 08/25/2025 01:36:24.846 7f78d5e43d08 [contrail-device-manager] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 08/25/2025 01:36:24.846 7f78d5e43d08 [contrail-device-manager] [INFO]: SANDESH: Flow Logging: [None] -> [False] 08/25/2025 01:36:24.846 7f78cd4aa2e8 [contrail-device-manager] [INFO]: Processing event[EvStart] in state[none] 08/25/2025 01:36:24.847 7f78cd4aa2e8 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectConfigNode:NodeStatusUVE] map 08/25/2025 01:36:24.847 7f78cd4aa2e8 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 08/25/2025 01:36:24.847 7f78cd4aa2e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 08/25/2025 01:36:24.847 7f78cd4aa2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:24.848 7f78cd4aa2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 08/25/2025 01:36:24.848 7f78cd4aa2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:24.848 7f78cd4aa2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:24.848 7f78cd4aa2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 start_time = 1756085784847579 collector_name = collector_ip = collector_list = [ 10.0.0.249:8086, 10.0.0.50:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 08/25/2025 01:36:24.848 7f78cd4aa2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:24.901 7f78d5e43d08 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.50:5673//', 'pyamqp://guest:guest@10.0.0.254:5673//', 'pyamqp://guest:guest@10.0.0.249:5673//'] 08/25/2025 01:36:24.911 7f78d5e43d08 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 08/25/2025 01:36:24.911 7f78cd3fd588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:24.911 7f78cd3fd588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 08/25/2025 01:36:24.912 7f78cd3fd588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:24.912 7f78cd3fd588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:24.912 7f78cd3fd588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 08/25/2025 01:36:24.912 7f78cd3fd588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:24.916 7f78d5e43d08 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 08/25/2025 01:36:24.917 7f78cd3fd240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:24.917 7f78cd3fd240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 08/25/2025 01:36:24.917 7f78cd3fd240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:24.944 7f78d5e43d08 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 08/25/2025 01:36:24.944 7f78cd3fd0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:24.945 7f78cd3fd0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 08/25/2025 01:36:24.945 7f78cd3fd0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:24.948 7f78d5e43d08 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 08/25/2025 01:36:24.949 7f78dbcaf978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:24.949 7f78dbcaf978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 08/25/2025 01:36:24.949 7f78dbcaf978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:24.966 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Removed remaining AMQP queue from previous run 08/25/2025 01:36:24.966 7f78d5e43d08 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 08/25/2025 01:36:24.967 7f78d5e43d08 [contrail-device-manager] [WARNING]: initializing cassandra cluster 08/25/2025 01:36:24.967 7f78d5e43d08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 08/25/2025 01:36:24.967 7f78d2d356d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting consumer greenlet 08/25/2025 01:36:24.967 7f78d2d356d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 08/25/2025 01:36:24.968 7f78cfc45978 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting publisher greenlet 08/25/2025 01:36:24.968 7f78cfc45978 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Publisher connection 08/25/2025 01:36:24.968 7f78cd298588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:24.968 7f78cd298588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 08/25/2025 01:36:24.968 7f78cd298588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:24.970 7f78d5e43d08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 08/25/2025 01:36:24.972 7f78d2d356d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 08/25/2025 01:36:24.972 7f78d2d356d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Waiting for consumer 08/25/2025 01:36:24.973 7f78cfc45978 [contrail-device-manager] [INFO]: KombuAmqpClient: Publisher connection established 08/25/2025 01:36:25.012 7f78d5e43d08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 08/25/2025 01:36:25.014 7f78cd1e6f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:25.014 7f78cd1e6f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 08/25/2025 01:36:25.014 7f78cd1e6f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:25.022 7f78d5e43d08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 08/25/2025 01:36:25.022 7f78cd1eed68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:25.022 7f78cd1eed68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 08/25/2025 01:36:25.022 7f78cd1eed68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:25.029 7f78d5e43d08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 08/25/2025 01:36:25.055 7f78d5e43d08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 08/25/2025 01:36:25.380 7f78d5e43d08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 08/25/2025 01:36:28.487 7f78d5e43d08 [contrail-device-manager] [WARNING]: table 'obj_uuid_table', created 08/25/2025 01:36:28.500 7f78d5e43d08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 08/25/2025 01:36:28.848 7f78cd1aeeb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/25/2025 01:36:28.849 7f78cd1aeeb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/25/2025 01:36:28.849 7f78cd1aeeb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/25/2025 01:36:28.849 7f78cd1aeeb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:36:28.850 7f78cd1aeeb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 08/25/2025 01:36:28.850 7f78cd1aeeb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:36:28.850 7f78cd1aeeb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/25/2025 01:36:28.850 7f78cd1aeeb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/25/2025 01:36:28.850 7f78cd1aeeb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:28.850 7f78cd1aeeb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 08/25/2025 01:36:28.850 7f78cd1aeeb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:28.850 7f78cd1aeeb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:28.850 7f78cd1aeeb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 start_time = 1756085784847579 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.50:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/25/2025 01:36:28.850 7f78cd1aeeb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:31.460 7f78d5e43d08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 08/25/2025 01:36:32.774 7f78d5e43d08 [contrail-device-manager] [WARNING]: table 'obj_uuid_table' fixed with properties {'gc_grace_seconds': 864000} 08/25/2025 01:36:32.778 7f78d5e43d08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 08/25/2025 01:36:32.850 7f78cd4aa048 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/25/2025 01:36:32.853 7f78cd4aa048 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/25/2025 01:36:32.853 7f78cd4aa048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/25/2025 01:36:32.853 7f78cd4aa048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:36:32.853 7f78cd4aa048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 08/25/2025 01:36:32.854 7f78cd4aa048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:36:32.854 7f78cd4aa048 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/25/2025 01:36:32.854 7f78cd4aa048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/25/2025 01:36:32.855 7f78cd4aa048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:32.855 7f78cd4aa048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 08/25/2025 01:36:32.855 7f78cd4aa048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:32.855 7f78cd4aa048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:32.855 7f78cd4aa048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 start_time = 1756085784847579 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.50:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/25/2025 01:36:32.855 7f78cd4aa048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:34.376 7f78d5e43d08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 08/25/2025 01:36:35.492 7f78d5e43d08 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table', created 08/25/2025 01:36:35.496 7f78d5e43d08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 08/25/2025 01:36:36.775 7f78d5e43d08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 08/25/2025 01:36:36.854 7f78cd19a240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/25/2025 01:36:36.855 7f78cd19a240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/25/2025 01:36:36.855 7f78cd19a240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/25/2025 01:36:36.855 7f78cd19a240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:36:36.856 7f78cd19a240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 08/25/2025 01:36:36.856 7f78cd19a240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:36:36.856 7f78cd19a240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/25/2025 01:36:36.856 7f78cd19a240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/25/2025 01:36:36.856 7f78cd19a240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:36.856 7f78cd19a240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 08/25/2025 01:36:36.856 7f78cd19a240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:36.856 7f78cd19a240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:36.856 7f78cd19a240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 start_time = 1756085784847579 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.50:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/25/2025 01:36:36.856 7f78cd19a240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:37.463 7f78d5e43d08 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table' fixed with properties {'gc_grace_seconds': 864000} 08/25/2025 01:36:37.467 7f78d5e43d08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 08/25/2025 01:36:39.332 7f78d5e43d08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 08/25/2025 01:36:39.335 7f78d5e43d08 [contrail-device-manager] [WARNING]: table 'obj_shared_table', created 08/25/2025 01:36:39.338 7f78d5e43d08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 08/25/2025 01:36:40.349 7f78d5e43d08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 08/25/2025 01:36:40.856 7f78cc910438 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/25/2025 01:36:40.857 7f78cc910438 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/25/2025 01:36:40.857 7f78cc910438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/25/2025 01:36:40.857 7f78cc910438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:36:40.857 7f78cc910438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 08/25/2025 01:36:40.857 7f78cc910438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:36:40.857 7f78cc910438 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/25/2025 01:36:40.858 7f78cc910438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/25/2025 01:36:40.858 7f78cc910438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:40.858 7f78cc910438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 08/25/2025 01:36:40.858 7f78cc910438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:40.858 7f78cc910438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:40.858 7f78cc910438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 start_time = 1756085784847579 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.50:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/25/2025 01:36:40.858 7f78cc910438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:41.444 7f78d5e43d08 [contrail-device-manager] [WARNING]: table 'obj_shared_table' fixed with properties {'gc_grace_seconds': 864000} 08/25/2025 01:36:41.449 7f78d5e43d08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 08/25/2025 01:36:41.449 7f78cd19a240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:41.449 7f78cd19a240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/25/2025 01:36:41.449 7f78cd19a240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:41.462 7f78d5e43d08 [contrail-device-manager] [ERROR]: Error in db walk read 'type' 08/25/2025 01:36:41.463 7f78cc936240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:41.464 7f78cc936240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Zookeeper:Zookeeper[] connection down >>, ] >> 08/25/2025 01:36:41.464 7f78cc936240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:41.477 7f78cc93f438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:41.477 7f78cc93f438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/25/2025 01:36:41.477 7f78cc93f438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:41.478 7f78d5e43d08 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_status_consumer.dummy 08/25/2025 01:36:41.479 7f78d5e43d08 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_request_consumer 08/25/2025 01:36:41.479 7f78d2d356d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 08/25/2025 01:36:41.479 7f78d5e43d08 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3915-1. 08/25/2025 01:36:41.488 7f78d2d356d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 08/25/2025 01:36:41.570 7f78d2d356d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3915-1.'] 08/25/2025 01:36:41.597 7f78d2d356d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 08/25/2025 01:36:41.602 7f78d2d356d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 08/25/2025 01:36:41.635 7f78d2d356d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3915-1.'] 08/25/2025 01:36:41.992 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Fetching all containers 08/25/2025 01:36:42.126 7f78d5e43d08 [contrail-device-manager] [INFO]: Restarting dnsmasq docker: config_dnsmasq_1 08/25/2025 01:36:42.333 7f78d5e43d08 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3915-1..config_queue 08/25/2025 01:36:42.337 7f78d5e43d08 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3915-1..tftp_queue 08/25/2025 01:36:42.661 7f78d2d356d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 08/25/2025 01:36:42.671 7f78d2d356d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 08/25/2025 01:36:42.814 7f78d2d356d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3915-1.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3915-1..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3915-1..tftp_queue'] 08/25/2025 01:36:42.838 7f78d5e43d08 [contrail-device-manager] [WARNING]: Process 136 prepared to run in Full mode... 08/25/2025 01:36:42.838 7f78d5e43d08 [contrail-device-manager] [WARNING]: Elected master Device Manager node. Initializing... 08/25/2025 01:36:42.838 7f78d5e43d08 [contrail-device-manager] [INFO]: SANDESH: INTROSPECT IS ON: 0.0.0.0:8096 08/25/2025 01:36:42.841 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add [config_updater.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/config_updater] to home page 08/25/2025 01:36:42.841 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_ConfigApiUpdateReq] 08/25/2025 01:36:42.842 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add [greenlets.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/greenlets] to home page 08/25/2025 01:36:42.842 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_GreenletObjectReq] 08/25/2025 01:36:42.842 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add [cpuinfo.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/nodeinfo/cpuinfo] to home page 08/25/2025 01:36:42.842 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CpuLoadInfoReq] 08/25/2025 01:36:42.846 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add [dm_introspect.xml:/usr/lib/python3.6/site-packages/device_manager/sandesh/dm_introspect] to home page 08/25/2025 01:36:42.846 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_RoutingInstanceList] 08/25/2025 01:36:42.846 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualNetworkList] 08/25/2025 01:36:42.846 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualMachineInterfaceList] 08/25/2025 01:36:42.846 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalInterfaceList] 08/25/2025 01:36:42.846 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalInterfaceList] 08/25/2025 01:36:42.846 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalRouterList] 08/25/2025 01:36:42.847 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_BgpRouterList] 08/25/2025 01:36:42.847 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalRouterList] 08/25/2025 01:36:42.847 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_InstanceIpList] 08/25/2025 01:36:42.847 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FloatingIpList] 08/25/2025 01:36:42.847 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PortTupleList] 08/25/2025 01:36:42.847 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FeatureFlagList] 08/25/2025 01:36:42.848 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add [sandesh_alarm.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_alarm] to home page 08/25/2025 01:36:42.848 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmCacheRequest] 08/25/2025 01:36:42.848 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmTypesRequest] 08/25/2025 01:36:42.848 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmAckRequest] 08/25/2025 01:36:42.849 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add [sandesh_ctrl.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_ctrl] to home page 08/25/2025 01:36:42.849 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlClientToServer] 08/25/2025 01:36:42.849 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlServerToClient] 08/25/2025 01:36:42.849 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add [sandesh_trace.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_trace] to home page 08/25/2025 01:36:42.849 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceRequest] 08/25/2025 01:36:42.850 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferListRequest] 08/25/2025 01:36:42.850 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceEnableDisableReq] 08/25/2025 01:36:42.850 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufStatusReq] 08/25/2025 01:36:42.850 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferEnableDisableReq] 08/25/2025 01:36:42.851 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add [sandesh_uve.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_uve] to home page 08/25/2025 01:36:42.851 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshMessageStatsReq] 08/25/2025 01:36:42.851 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsSet] 08/25/2025 01:36:42.851 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsStatus] 08/25/2025 01:36:42.851 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsSet] 08/25/2025 01:36:42.851 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsStatus] 08/25/2025 01:36:42.851 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsSet] 08/25/2025 01:36:42.851 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsReset] 08/25/2025 01:36:42.851 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsStatus] 08/25/2025 01:36:42.851 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueSet] 08/25/2025 01:36:42.851 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueStatus] 08/25/2025 01:36:42.851 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CollectorInfoRequest] 08/25/2025 01:36:42.851 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVEDSConfReq] 08/25/2025 01:36:42.851 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVECacheReq] 08/25/2025 01:36:42.852 7f78d5e43d08 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVETypesReq] 08/25/2025 01:36:42.854 7f78d5e43d08 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.ztp_queue 08/25/2025 01:36:42.909 7f78cc7f2630 [contrail-device-manager] [ERROR]: Cannot write http_port 8096 to /tmp/contrail-device-manager.1.http_port 08/25/2025 01:36:42.909 7f78cc7f2630 [contrail-device-manager] [ERROR]: Starting Introspect on HTTP Port 8096 08/25/2025 01:36:42.909 7f78cc04dcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:42.910 7f78cc04dcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Initializing description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 08/25/2025 01:36:42.910 7f78cc04dcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:42.920 7f78c7f08978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:42.920 7f78c7f08978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 08/25/2025 01:36:42.920 7f78c7f08978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:43.829 7f78d2d356d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 08/25/2025 01:36:43.833 7f78d2d356d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 08/25/2025 01:36:43.891 7f78d2d356d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3915-1.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3915-1..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3915-1..tftp_queue', 'device_manager_ztp.ztp_queue'] 08/25/2025 01:36:44.861 7f78c7f014e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/25/2025 01:36:44.862 7f78c7f014e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/25/2025 01:36:44.862 7f78c7f014e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/25/2025 01:36:44.862 7f78c7f014e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:36:44.863 7f78c7f014e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 08/25/2025 01:36:44.863 7f78c7f014e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:36:44.863 7f78c7f014e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/25/2025 01:36:44.863 7f78c7f014e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/25/2025 01:36:44.863 7f78c7f014e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:44.863 7f78c7f014e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 08/25/2025 01:36:44.863 7f78c7f014e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:44.863 7f78c7f014e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:44.863 7f78c7f014e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1756085784847579 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.50:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/25/2025 01:36:44.863 7f78c7f014e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:48.863 7f78c7f73f60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/25/2025 01:36:48.864 7f78c7f73f60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/25/2025 01:36:48.864 7f78c7f73f60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/25/2025 01:36:48.864 7f78c7f73f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:36:48.864 7f78c7f73f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 08/25/2025 01:36:48.865 7f78c7f73f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:36:48.865 7f78c7f73f60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/25/2025 01:36:48.865 7f78c7f73f60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/25/2025 01:36:48.865 7f78c7f73f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:48.865 7f78c7f73f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 08/25/2025 01:36:48.865 7f78c7f73f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:48.865 7f78c7f73f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:48.865 7f78c7f73f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1756085784847579 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.50:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/25/2025 01:36:48.865 7f78c7f73f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:49.117 7f78c7f4dac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:49.117 7f78c7f4dac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/25/2025 01:36:49.118 7f78c7f4dac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:52.868 7f78c7dc5c18 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/25/2025 01:36:52.869 7f78c7dc5c18 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/25/2025 01:36:52.869 7f78c7dc5c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/25/2025 01:36:52.869 7f78c7dc5c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:36:52.870 7f78c7dc5c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/25/2025 01:36:52.870 7f78c7dc5c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:36:52.870 7f78c7dc5c18 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/25/2025 01:36:52.870 7f78c7dc5c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/25/2025 01:36:52.870 7f78c7dc5c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:52.870 7f78c7dc5c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/25/2025 01:36:52.870 7f78c7dc5c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:52.870 7f78c7dc5c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:52.871 7f78c7dc5c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1756085784847579 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.50:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/25/2025 01:36:52.871 7f78c7dc5c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:56.870 7f78c7f014e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/25/2025 01:36:56.871 7f78c7f014e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/25/2025 01:36:56.872 7f78c7f014e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/25/2025 01:36:56.872 7f78c7f014e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:36:56.872 7f78c7f014e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/25/2025 01:36:56.872 7f78c7f014e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:36:56.872 7f78c7f014e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/25/2025 01:36:56.872 7f78c7f014e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/25/2025 01:36:56.872 7f78c7f014e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:56.873 7f78c7f014e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/25/2025 01:36:56.873 7f78c7f014e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:56.873 7f78c7f014e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:36:56.873 7f78c7f014e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1756085784847579 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.50:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/25/2025 01:36:56.873 7f78c7f014e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:00.873 7f78d01c7390 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/25/2025 01:37:00.874 7f78d01c7390 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/25/2025 01:37:00.875 7f78d01c7390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/25/2025 01:37:00.875 7f78d01c7390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:37:00.875 7f78d01c7390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/25/2025 01:37:00.875 7f78d01c7390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:37:00.875 7f78d01c7390 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/25/2025 01:37:00.875 7f78d01c7390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/25/2025 01:37:00.875 7f78d01c7390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:00.876 7f78d01c7390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/25/2025 01:37:00.876 7f78d01c7390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:00.876 7f78d01c7390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:00.876 7f78d01c7390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1756085784847579 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.50:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/25/2025 01:37:00.876 7f78d01c7390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:04.876 7f78d01c7390 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/25/2025 01:37:04.877 7f78d01c7390 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/25/2025 01:37:04.877 7f78d01c7390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/25/2025 01:37:04.877 7f78d01c7390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:37:04.877 7f78d01c7390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/25/2025 01:37:04.878 7f78d01c7390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:37:04.878 7f78d01c7390 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/25/2025 01:37:04.878 7f78d01c7390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/25/2025 01:37:04.878 7f78d01c7390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:04.878 7f78d01c7390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/25/2025 01:37:04.878 7f78d01c7390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:04.878 7f78d01c7390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:04.878 7f78d01c7390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1756085784847579 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.50:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/25/2025 01:37:04.878 7f78d01c7390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:08.879 7f78d01c7390 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/25/2025 01:37:08.880 7f78d01c7390 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/25/2025 01:37:08.880 7f78d01c7390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/25/2025 01:37:08.880 7f78d01c7390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:37:08.881 7f78d01c7390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/25/2025 01:37:08.881 7f78d01c7390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:37:08.881 7f78d01c7390 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/25/2025 01:37:08.881 7f78d01c7390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/25/2025 01:37:08.881 7f78d01c7390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:08.881 7f78d01c7390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/25/2025 01:37:08.881 7f78d01c7390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:08.882 7f78d01c7390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:08.882 7f78d01c7390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1756085784847579 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.50:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/25/2025 01:37:08.882 7f78d01c7390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:12.881 7f78d01c7390 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/25/2025 01:37:12.882 7f78d01c7390 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/25/2025 01:37:12.882 7f78d01c7390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/25/2025 01:37:12.883 7f78d01c7390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:37:12.883 7f78d01c7390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/25/2025 01:37:12.883 7f78d01c7390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:37:12.883 7f78d01c7390 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/25/2025 01:37:12.883 7f78d01c7390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/25/2025 01:37:12.883 7f78d01c7390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:12.883 7f78d01c7390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/25/2025 01:37:12.883 7f78d01c7390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:12.883 7f78d01c7390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:12.883 7f78d01c7390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1756085784847579 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.50:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/25/2025 01:37:12.884 7f78d01c7390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:16.884 7f78d01c7390 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/25/2025 01:37:16.885 7f78d01c7390 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/25/2025 01:37:16.885 7f78d01c7390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/25/2025 01:37:16.885 7f78d01c7390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:37:16.885 7f78d01c7390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/25/2025 01:37:16.885 7f78d01c7390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:37:16.885 7f78d01c7390 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/25/2025 01:37:16.886 7f78d01c7390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/25/2025 01:37:16.886 7f78d01c7390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:16.886 7f78d01c7390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/25/2025 01:37:16.886 7f78d01c7390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:16.886 7f78d01c7390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:16.886 7f78d01c7390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1756085784847579 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.50:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/25/2025 01:37:16.886 7f78d01c7390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:20.886 7f78d01c7390 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/25/2025 01:37:20.887 7f78d01c7390 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/25/2025 01:37:20.888 7f78d01c7390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/25/2025 01:37:20.888 7f78d01c7390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:37:20.888 7f78d01c7390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/25/2025 01:37:20.888 7f78d01c7390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:37:20.888 7f78d01c7390 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/25/2025 01:37:20.888 7f78d01c7390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/25/2025 01:37:20.888 7f78d01c7390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:20.888 7f78d01c7390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/25/2025 01:37:20.888 7f78d01c7390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:20.889 7f78d01c7390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:20.889 7f78d01c7390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1756085784847579 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.50:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/25/2025 01:37:20.889 7f78d01c7390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:24.889 7f78d01c7390 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/25/2025 01:37:24.890 7f78d01c7390 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/25/2025 01:37:24.890 7f78d01c7390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/25/2025 01:37:24.890 7f78d01c7390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:37:24.890 7f78d01c7390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/25/2025 01:37:24.890 7f78d01c7390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:37:24.890 7f78d01c7390 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/25/2025 01:37:24.890 7f78d01c7390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/25/2025 01:37:24.891 7f78d01c7390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:24.891 7f78d01c7390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/25/2025 01:37:24.891 7f78d01c7390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:24.891 7f78d01c7390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:24.891 7f78d01c7390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1756085784847579 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.50:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/25/2025 01:37:24.891 7f78d01c7390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:28.891 7f78cd1874e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/25/2025 01:37:28.892 7f78cd1874e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/25/2025 01:37:28.892 7f78cd1874e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/25/2025 01:37:28.892 7f78cd1874e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:37:28.892 7f78cd1874e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/25/2025 01:37:28.892 7f78cd1874e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:37:28.892 7f78cd1874e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/25/2025 01:37:28.893 7f78cd1874e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/25/2025 01:37:28.893 7f78cd1874e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:28.893 7f78cd1874e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/25/2025 01:37:28.893 7f78cd1874e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:28.893 7f78cd1874e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:28.893 7f78cd1874e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1756085784847579 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.50:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/25/2025 01:37:28.893 7f78cd1874e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:32.893 7f78c7f58048 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/25/2025 01:37:32.895 7f78c7f58048 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/25/2025 01:37:32.895 7f78c7f58048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/25/2025 01:37:32.895 7f78c7f58048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:37:32.895 7f78c7f58048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/25/2025 01:37:32.895 7f78c7f58048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:37:32.895 7f78c7f58048 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/25/2025 01:37:32.895 7f78c7f58048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/25/2025 01:37:32.896 7f78c7f58048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:32.896 7f78c7f58048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/25/2025 01:37:32.896 7f78c7f58048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:32.896 7f78c7f58048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:32.896 7f78c7f58048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1756085784847579 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.50:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/25/2025 01:37:32.896 7f78c7f58048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:36.897 7f78c7dc12e8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/25/2025 01:37:36.900 7f78c7dc12e8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/25/2025 01:37:36.900 7f78c7dc12e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/25/2025 01:37:36.900 7f78c7dc12e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:37:36.900 7f78c7dc12e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/25/2025 01:37:36.900 7f78c7dc12e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:37:36.900 7f78c7dc12e8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/25/2025 01:37:36.900 7f78c7dc12e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/25/2025 01:37:36.901 7f78c7dc12e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:36.901 7f78c7dc12e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/25/2025 01:37:36.901 7f78c7dc12e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:36.901 7f78c7dc12e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:36.901 7f78c7dc12e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1756085784847579 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.50:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/25/2025 01:37:36.901 7f78c7dc12e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:40.900 7f78d349aa20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/25/2025 01:37:40.901 7f78d349aa20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/25/2025 01:37:40.901 7f78d349aa20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/25/2025 01:37:40.901 7f78d349aa20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:37:40.901 7f78d349aa20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/25/2025 01:37:40.901 7f78d349aa20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:37:40.901 7f78d349aa20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/25/2025 01:37:40.902 7f78d349aa20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/25/2025 01:37:40.902 7f78d349aa20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:40.902 7f78d349aa20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/25/2025 01:37:40.902 7f78d349aa20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:40.902 7f78d349aa20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:40.902 7f78d349aa20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1756085784847579 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.50:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/25/2025 01:37:40.902 7f78d349aa20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:44.902 7f78c7e25eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/25/2025 01:37:44.903 7f78c7e25eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/25/2025 01:37:44.903 7f78c7e25eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/25/2025 01:37:44.903 7f78c7e25eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:37:44.903 7f78c7e25eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/25/2025 01:37:44.903 7f78c7e25eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:37:44.903 7f78c7e25eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/25/2025 01:37:44.903 7f78c7e25eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/25/2025 01:37:44.903 7f78c7e25eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:44.903 7f78c7e25eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/25/2025 01:37:44.904 7f78c7e25eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:44.904 7f78c7e25eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:44.904 7f78c7e25eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1756085784847579 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.50:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/25/2025 01:37:44.904 7f78c7e25eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:48.903 7f78cd18a0f0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/25/2025 01:37:48.904 7f78cd18a0f0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/25/2025 01:37:48.904 7f78cd18a0f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/25/2025 01:37:48.904 7f78cd18a0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:37:48.904 7f78cd18a0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/25/2025 01:37:48.905 7f78cd18a0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:37:48.905 7f78cd18a0f0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/25/2025 01:37:48.905 7f78cd18a0f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/25/2025 01:37:48.905 7f78cd18a0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:48.905 7f78cd18a0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/25/2025 01:37:48.905 7f78cd18a0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:48.905 7f78cd18a0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:48.905 7f78cd18a0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1756085784847579 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.50:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/25/2025 01:37:48.905 7f78cd18a0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:52.904 7f78c7f51630 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/25/2025 01:37:52.906 7f78c7f51630 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/25/2025 01:37:52.906 7f78c7f51630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/25/2025 01:37:52.906 7f78c7f51630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:37:52.906 7f78c7f51630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/25/2025 01:37:52.906 7f78c7f51630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:37:52.906 7f78c7f51630 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/25/2025 01:37:52.906 7f78c7f51630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/25/2025 01:37:52.906 7f78c7f51630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:52.907 7f78c7f51630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/25/2025 01:37:52.907 7f78c7f51630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:52.907 7f78c7f51630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:52.907 7f78c7f51630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1756085784847579 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.50:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/25/2025 01:37:52.907 7f78c7f51630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:56.907 7f78c7e22588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/25/2025 01:37:56.908 7f78c7e22588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/25/2025 01:37:56.908 7f78c7e22588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/25/2025 01:37:56.908 7f78c7e22588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:37:56.908 7f78c7e22588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/25/2025 01:37:56.908 7f78c7e22588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:37:56.908 7f78c7e22588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/25/2025 01:37:56.909 7f78c7e22588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/25/2025 01:37:56.909 7f78c7e22588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:56.909 7f78c7e22588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/25/2025 01:37:56.909 7f78c7e22588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:56.909 7f78c7e22588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:37:56.909 7f78c7e22588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1756085784847579 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.50:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/25/2025 01:37:56.909 7f78c7e22588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:00.909 7f78c7dc5f60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/25/2025 01:38:00.910 7f78c7dc5f60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/25/2025 01:38:00.910 7f78c7dc5f60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/25/2025 01:38:00.910 7f78c7dc5f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:38:00.910 7f78c7dc5f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/25/2025 01:38:00.910 7f78c7dc5f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:38:00.910 7f78c7dc5f60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/25/2025 01:38:00.910 7f78c7dc5f60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/25/2025 01:38:00.910 7f78c7dc5f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:00.911 7f78c7dc5f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/25/2025 01:38:00.911 7f78c7dc5f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:00.911 7f78c7dc5f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:00.911 7f78c7dc5f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1756085784847579 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.50:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/25/2025 01:38:00.911 7f78c7dc5f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:02.538 7f78d5e43d08 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 08/25/2025 01:38:02.538 7f78d5e43d08 [contrail-device-manager] [WARNING]: initializing cassandra cluster 08/25/2025 01:38:02.538 7f78d5e43d08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 08/25/2025 01:38:02.538 7f78c7e22ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:02.539 7f78c7e22ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 08/25/2025 01:38:02.539 7f78c7e22ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:02.545 7f78d5e43d08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 08/25/2025 01:38:02.637 7f78d5e43d08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 08/25/2025 01:38:02.639 7f78c7ebbf60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:02.639 7f78c7ebbf60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Initializing description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 08/25/2025 01:38:02.639 7f78c7ebbf60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:02.657 7f78d5e43d08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 08/25/2025 01:38:02.657 7f78c7ebbe10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:02.658 7f78c7ebbe10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 08/25/2025 01:38:02.658 7f78c7ebbe10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:02.673 7f78d5e43d08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 08/25/2025 01:38:03.376 7f78d5e43d08 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace', created 08/25/2025 01:38:03.380 7f78d5e43d08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 08/25/2025 01:38:03.388 7f78d5e43d08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 08/25/2025 01:38:03.861 7f78d5e43d08 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace' fixed with replication {'class': 'SimpleStrategy', 'replication_factor': None} 08/25/2025 01:38:03.865 7f78d5e43d08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 08/25/2025 01:38:03.866 7f78d5e43d08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 08/25/2025 01:38:03.920 7f78d5e43d08 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 08/25/2025 01:38:03.926 7f78d5e43d08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 08/25/2025 01:38:04.496 7f78d5e43d08 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table', created 08/25/2025 01:38:04.506 7f78d5e43d08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 08/25/2025 01:38:04.513 7f78d5e43d08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 08/25/2025 01:38:04.910 7f78c7ee1048 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/25/2025 01:38:04.911 7f78c7ee1048 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/25/2025 01:38:04.911 7f78c7ee1048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/25/2025 01:38:04.911 7f78c7ee1048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:38:04.911 7f78c7ee1048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 08/25/2025 01:38:04.911 7f78c7ee1048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:38:04.911 7f78c7ee1048 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/25/2025 01:38:04.912 7f78c7ee1048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/25/2025 01:38:04.912 7f78c7ee1048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:04.913 7f78c7ee1048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 08/25/2025 01:38:04.913 7f78c7ee1048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:04.913 7f78c7ee1048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:04.913 7f78c7ee1048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1756085784847579 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.50:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/25/2025 01:38:04.913 7f78c7ee1048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:05.393 7f78d5e43d08 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table' fixed with properties {'gc_grace_seconds': 864000} 08/25/2025 01:38:05.401 7f78d5e43d08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 08/25/2025 01:38:05.409 7f78d5e43d08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 08/25/2025 01:38:05.735 7f78d5e43d08 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table', created 08/25/2025 01:38:05.740 7f78d5e43d08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 08/25/2025 01:38:05.745 7f78d5e43d08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 08/25/2025 01:38:06.832 7f78d5e43d08 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table' fixed with properties {'gc_grace_seconds': 864000} 08/25/2025 01:38:06.847 7f78d5e43d08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 08/25/2025 01:38:06.860 7f78d5e43d08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 08/25/2025 01:38:07.377 7f78d5e43d08 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table', created 08/25/2025 01:38:07.388 7f78d5e43d08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 08/25/2025 01:38:07.396 7f78d5e43d08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 08/25/2025 01:38:08.492 7f78d5e43d08 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table' fixed with properties {'gc_grace_seconds': 864000} 08/25/2025 01:38:08.496 7f78d5e43d08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 08/25/2025 01:38:08.502 7f78d5e43d08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 08/25/2025 01:38:08.911 7f78c7eda048 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/25/2025 01:38:08.912 7f78c7eda048 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/25/2025 01:38:08.912 7f78c7eda048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/25/2025 01:38:08.912 7f78c7eda048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:38:08.912 7f78c7eda048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 08/25/2025 01:38:08.913 7f78c7eda048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:38:08.913 7f78c7eda048 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/25/2025 01:38:08.913 7f78c7eda048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/25/2025 01:38:08.913 7f78c7eda048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:08.913 7f78c7eda048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 08/25/2025 01:38:08.913 7f78c7eda048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:08.913 7f78c7eda048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:08.913 7f78c7eda048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1756085784847579 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.50:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/25/2025 01:38:08.913 7f78c7eda048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:09.467 7f78d5e43d08 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table', created 08/25/2025 01:38:09.472 7f78d5e43d08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 08/25/2025 01:38:09.478 7f78d5e43d08 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 08/25/2025 01:38:10.355 7f78d5e43d08 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table' fixed with properties {'gc_grace_seconds': 864000} 08/25/2025 01:38:10.362 7f78d5e43d08 [contrail-device-manager] [WARNING]: Zookeeper lock released. 08/25/2025 01:38:10.362 7f78c7df1630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:10.362 7f78c7df1630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/25/2025 01:38:10.362 7f78c7df1630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:10.403 7f78d5e43d08 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.50:5673//', 'pyamqp://guest:guest@10.0.0.254:5673//', 'pyamqp://guest:guest@10.0.0.249:5673//'] 08/25/2025 01:38:10.403 7f78d5e43d08 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 08/25/2025 01:38:10.404 7f78c7ececc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:10.404 7f78c7ececc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Initializing description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 08/25/2025 01:38:10.404 7f78c7ececc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:10.404 7f78c7ececc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:10.404 7f78c7ececc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 08/25/2025 01:38:10.404 7f78c7ececc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:10.417 7f78d5e43d08 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 08/25/2025 01:38:10.418 7f78c7ecee10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:10.418 7f78c7ecee10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/25/2025 01:38:10.418 7f78c7ecee10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:10.455 7f78d5e43d08 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 08/25/2025 01:38:10.456 7f78c7ed4828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:10.456 7f78c7ed4828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 08/25/2025 01:38:10.456 7f78c7ed4828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:10.461 7f78d5e43d08 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 08/25/2025 01:38:10.461 7f78c7ed40f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:10.461 7f78c7ed40f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/25/2025 01:38:10.462 7f78c7ed40f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:10.462 7f78c7ebb438 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 08/25/2025 01:38:10.463 7f78c7eb1828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:10.463 7f78c7eb1828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 08/25/2025 01:38:10.463 7f78c7eb1828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:10.467 7f78c7ebb438 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 08/25/2025 01:38:10.467 7f78c7edb438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:10.467 7f78c7edb438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/25/2025 01:38:10.467 7f78c7edb438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:12.913 7f78c7eced68 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/25/2025 01:38:12.914 7f78c7eced68 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/25/2025 01:38:12.914 7f78c7eced68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/25/2025 01:38:12.915 7f78c7eced68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:38:12.915 7f78c7eced68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/25/2025 01:38:12.915 7f78c7eced68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:38:12.915 7f78c7eced68 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/25/2025 01:38:12.915 7f78c7eced68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/25/2025 01:38:12.915 7f78c7eced68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:12.915 7f78c7eced68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/25/2025 01:38:12.915 7f78c7eced68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:12.915 7f78c7eced68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:12.915 7f78c7eced68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1756085784847579 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.50:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/25/2025 01:38:12.915 7f78c7eced68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:16.916 7f78c7eced68 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/25/2025 01:38:16.917 7f78c7eced68 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/25/2025 01:38:16.917 7f78c7eced68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/25/2025 01:38:16.917 7f78c7eced68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:38:16.917 7f78c7eced68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/25/2025 01:38:16.917 7f78c7eced68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:38:16.917 7f78c7eced68 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/25/2025 01:38:16.918 7f78c7eced68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/25/2025 01:38:16.918 7f78c7eced68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:16.918 7f78c7eced68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/25/2025 01:38:16.918 7f78c7eced68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:16.918 7f78c7eced68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:16.918 7f78c7eced68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1756085784847579 collector_name = collector_ip = 10.0.0.249:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.50:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/25/2025 01:38:16.918 7f78c7eced68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:20.918 7f78c7eced68 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/25/2025 01:38:20.919 7f78c7eced68 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/25/2025 01:38:20.919 7f78c7eced68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/25/2025 01:38:20.919 7f78c7eced68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:38:20.919 7f78c7eced68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/25/2025 01:38:20.920 7f78c7eced68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:38:20.920 7f78c7eced68 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/25/2025 01:38:20.920 7f78c7eced68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/25/2025 01:38:20.920 7f78c7eced68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:20.920 7f78c7eced68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/25/2025 01:38:20.920 7f78c7eced68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:20.920 7f78c7eced68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:20.920 7f78c7eced68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1756085784847579 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.50:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/25/2025 01:38:20.920 7f78c7eced68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:24.920 7f78c7eced68 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/25/2025 01:38:24.921 7f78c7eced68 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/25/2025 01:38:24.922 7f78c7eced68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/25/2025 01:38:24.922 7f78c7eced68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:38:24.922 7f78c7eced68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/25/2025 01:38:24.922 7f78c7eced68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:38:24.922 7f78c7eced68 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/25/2025 01:38:24.922 7f78c7eced68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/25/2025 01:38:24.922 7f78c7eced68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:24.922 7f78c7eced68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/25/2025 01:38:24.923 7f78c7eced68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:24.923 7f78c7eced68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:24.923 7f78c7eced68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 136 http_port = 8096 start_time = 1756085784847579 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.249:8086, 10.0.0.50:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/25/2025 01:38:24.923 7f78c7eced68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/25/2025 01:38:26.855 7f78c7ece6d8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3915-2.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.254', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3915-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3915-2.'], 'id_perms': {'created': '2025-08-25T01:38:26.843138', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-08-25T01:38:26.843138', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9395971087771663302, 'uuid_mslong': 8639552283538311975}}, 'parent_type': 'global-system-config', 'parent_uuid': '12147cc1-79df-4a62-b52b-653b9bed2757', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '77e5db07-0eb7-4b27-8265-31ec1e787bc6'}, 'oper': 'CREATE', 'request-id': 'req-d27e6b69-c057-4e84-aa5d-6250f120ec62', 'type': 'analytics_node', 'uuid': '77e5db07-0eb7-4b27-8265-31ec1e787bc6'} 08/25/2025 01:38:26.873 7f78c7ece6d8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3915-3.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.249', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3915-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3915-3.'], 'id_perms': {'created': '2025-08-25T01:38:26.842616', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-08-25T01:38:26.842616', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12390947677931664736, 'uuid_mslong': 1054026850718402500}}, 'parent_type': 'global-system-config', 'parent_uuid': '12147cc1-79df-4a62-b52b-653b9bed2757', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '0ea0a7d6-14cb-47c4-abf5-7d4355917560'}, 'oper': 'CREATE', 'request-id': 'req-35fea01b-7d6f-46c6-b233-d1f9cfb28207', 'type': 'analytics_node', 'uuid': '0ea0a7d6-14cb-47c4-abf5-7d4355917560'} 08/25/2025 01:38:26.911 7f78c7ece6d8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3915-1.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.50', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3915-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3915-1.'], 'id_perms': {'created': '2025-08-25T01:38:26.900291', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-08-25T01:38:26.900291', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11118843779539256940, 'uuid_mslong': 5249494201357126405}}, 'parent_type': 'global-system-config', 'parent_uuid': '12147cc1-79df-4a62-b52b-653b9bed2757', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '48d9f329-c25d-4b05-9a4e-1181e07fc66c'}, 'oper': 'CREATE', 'request-id': 'req-ae94f2ca-cf27-4959-ad33-8f9d41691ac8', 'type': 'analytics_node', 'uuid': '48d9f329-c25d-4b05-9a4e-1181e07fc66c'} 08/25/2025 01:38:28.922 7f78d2ea5240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/25/2025 01:38:28.923 7f78d2ea5240 [contrail-device-manager] [INFO]: Session Event: TCP Connected 08/25/2025 01:38:28.923 7f78d2ea5240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/25/2025 01:38:28.923 7f78d2ea5240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:38:28.923 7f78d2ea5240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3915-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.249:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, 10.0.0.254:5673, 10.0.0.249:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, 10.0.0.254:9041, 10.0.0.249:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, 10.0.0.254:2181, 10.0.0.249:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.50,10.0.0.254,10.0.0.249:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/25/2025 01:38:28.923 7f78d2ea5240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/25/2025 01:38:28.924 7f78d2ea5240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnected] in state[Connect] 08/25/2025 01:38:28.924 7f78d2ea5240 [contrail-device-manager] [DEBUG]: Send sandesh control message. uve type count # 19 08/25/2025 01:38:28.924 7f78d2ea5240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 08/25/2025 01:38:28.925 7f78d2ea5240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 08/25/2025 01:38:28.925 7f78d2ea5240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 08/25/2025 01:38:28.925 7f78c7ed2828 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 08/25/2025 01:38:28.926 7f78c7ed2828 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 08/25/2025 01:38:36.394 7f78c7ed22e8 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001016 08/25/2025 01:38:36.394 7f78c7ed22e8 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 08/25/2025 01:38:36.395 7f78c7f51048 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 08/25/2025 01:38:36.395 7f78c7f51048 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 0 08/25/2025 01:38:36.395 7f78c7f51048 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 08/25/2025 01:38:36.395 7f78c7f51048 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 08/25/2025 01:38:36.395 7f78c7f51048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 08/25/2025 01:38:36.395 7f78c7eb1f60 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 08/25/2025 01:38:36.396 7f78c7eb1f60 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 08/25/2025 01:38:41.232 7f78c7ece6d8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3915-1.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.50', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3915-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3915-1.'], 'id_perms': {'created': '2025-08-25T01:38:41.200587', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-08-25T01:38:41.200587', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11530462426710086698, 'uuid_mslong': 6344567073480854834}}, 'parent_type': 'global-system-config', 'parent_uuid': '12147cc1-79df-4a62-b52b-653b9bed2757', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '580c6e23-6c77-4d32-a004-6e7c700b142a'}, 'oper': 'CREATE', 'request-id': 'req-3a17092e-53f9-44a5-a20b-19598e64c19a', 'type': 'analytics_alarm_node', 'uuid': '580c6e23-6c77-4d32-a004-6e7c700b142a'} 08/25/2025 01:38:41.329 7f78c7ece6d8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3915-2.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.254', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3915-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3915-2.'], 'id_perms': {'created': '2025-08-25T01:38:41.311112', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-08-25T01:38:41.311112', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10675967308102712592, 'uuid_mslong': 16270483402143516960}}, 'parent_type': 'global-system-config', 'parent_uuid': '12147cc1-79df-4a62-b52b-653b9bed2757', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'e1cc5e73-14a7-4920-9428-a7b3f9253d10'}, 'oper': 'CREATE', 'request-id': 'req-d5df7cef-ec0d-4d5c-af23-279068a07aa4', 'type': 'analytics_alarm_node', 'uuid': 'e1cc5e73-14a7-4920-9428-a7b3f9253d10'} 08/25/2025 01:38:41.383 7f78c7ece6d8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3915-3.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.249', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3915-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3915-3.'], 'id_perms': {'created': '2025-08-25T01:38:41.347447', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-08-25T01:38:41.347447', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12535229826656036341, 'uuid_mslong': 2695262520368252717}}, 'parent_type': 'global-system-config', 'parent_uuid': '12147cc1-79df-4a62-b52b-653b9bed2757', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '25677efb-6ef9-472d-adf6-151ce3b6e5f5'}, 'oper': 'CREATE', 'request-id': 'req-3ea25be7-c149-44a8-a33e-9eb5f562c418', 'type': 'analytics_alarm_node', 'uuid': '25677efb-6ef9-472d-adf6-151ce3b6e5f5'} 08/25/2025 01:38:46.380 7f78c7ed22e8 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001296 08/25/2025 01:38:46.381 7f78c7ed22e8 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 08/25/2025 01:38:46.381 7f78d2f17c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 08/25/2025 01:38:46.381 7f78d2f17c18 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 2 08/25/2025 01:38:46.381 7f78d2f17c18 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 08/25/2025 01:38:46.382 7f78d2f17c18 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 08/25/2025 01:38:49.644 7f78c7ece6d8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3915-2.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.254', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3915-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3915-2.'], 'id_perms': {'created': '2025-08-25T01:38:49.635297', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-08-25T01:38:49.635297', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10828204002910978221, 'uuid_mslong': 16055061919354601850}}, 'parent_type': 'global-system-config', 'parent_uuid': '12147cc1-79df-4a62-b52b-653b9bed2757', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'decf09c0-9608-417a-9645-822b9cf8c0ad'}, 'oper': 'CREATE', 'request-id': 'req-9b65e324-4ae7-4ebe-b5f3-4020906f0d2a', 'type': 'analytics_snmp_node', 'uuid': 'decf09c0-9608-417a-9645-822b9cf8c0ad'} 08/25/2025 01:38:49.662 7f78c7ece6d8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3915-1.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.50', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3915-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3915-1.'], 'id_perms': {'created': '2025-08-25T01:38:49.639319', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-08-25T01:38:49.639319', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13716283731245796176, 'uuid_mslong': 4965234318673856556}}, 'parent_type': 'global-system-config', 'parent_uuid': '12147cc1-79df-4a62-b52b-653b9bed2757', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '44e80e4e-4f22-482c-be5a-0761af600b50'}, 'oper': 'CREATE', 'request-id': 'req-ef48116e-9297-4e6f-95bf-dacaade684f8', 'type': 'analytics_snmp_node', 'uuid': '44e80e4e-4f22-482c-be5a-0761af600b50'} 08/25/2025 01:38:49.673 7f78c7ece6d8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3915-3.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.249', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3915-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3915-3.'], 'id_perms': {'created': '2025-08-25T01:38:49.660013', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-08-25T01:38:49.660013', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13737015611474782190, 'uuid_mslong': 12450394291878054111}}, 'parent_type': 'global-system-config', 'parent_uuid': '12147cc1-79df-4a62-b52b-653b9bed2757', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'acc8afa4-b8f1-48df-bea3-aeeab3f327ee'}, 'oper': 'CREATE', 'request-id': 'req-36621839-f734-4451-8ddb-6d456a2791f0', 'type': 'analytics_snmp_node', 'uuid': 'acc8afa4-b8f1-48df-bea3-aeeab3f327ee'} 08/25/2025 01:40:39.022 7f78c7ece6d8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-e65b2aef-e754-47ae-9147-8d0cbba3ce59', 'type': 'global_system_config', 'uuid': '12147cc1-79df-4a62-b52b-653b9bed2757'} 08/25/2025 01:40:39.143 7f78c7ece6d8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3915-1'], 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-3915-1', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3915-1'], 'id_perms': {'created': '2025-08-25T01:40:39.135179', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-08-25T01:40:39.135179', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12694279353608574079, 'uuid_mslong': 12520793770140517749}}, 'parent_type': 'global-system-config', 'parent_uuid': '12147cc1-79df-4a62-b52b-653b9bed2757', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'adc2cb98-8f26-4575-b02b-23d09f6a147f', 'virtual_router_dpdk_enabled': False, 'virtual_router_ip_address': '10.20.0.25'}, 'oper': 'CREATE', 'request-id': 'req-fb082126-b8a7-47a6-b142-cc14a5f32bd1', 'type': 'virtual_router', 'uuid': 'adc2cb98-8f26-4575-b02b-23d09f6a147f'} 08/25/2025 01:40:39.479 7f78c7ece6d8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3915-2'], 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-3915-2', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3915-2'], 'id_perms': {'created': '2025-08-25T01:40:39.468160', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-08-25T01:40:39.468160', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10681402938933818879, 'uuid_mslong': 1514517210951204986}}, 'parent_type': 'global-system-config', 'parent_uuid': '12147cc1-79df-4a62-b52b-653b9bed2757', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '1504a561-0cf6-407a-943b-f76165f6b5ff', 'virtual_router_dpdk_enabled': False, 'virtual_router_ip_address': '10.20.0.193'}, 'oper': 'CREATE', 'request-id': 'req-c6567d2c-19f5-41eb-9e36-6218d7339899', 'type': 'virtual_router', 'uuid': '1504a561-0cf6-407a-943b-f76165f6b5ff'} 08/25/2025 01:40:39.542 7f78c7ece6d8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3915-1', 'vhost0'], 'obj_dict': {'display_name': 'vhost0', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3915-1', 'vhost0'], 'id_perms': {'created': '2025-08-25T01:40:39.521258', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-08-25T01:40:39.521258', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11097736783759197342, 'uuid_mslong': 18413948225832767407}}, 'parent_type': 'virtual-router', 'parent_uuid': 'adc2cb98-8f26-4575-b02b-23d09f6a147f', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'ff8b7c59-a132-4faf-9a03-14ce76c0c49e', 'virtual_machine_interface_disable_policy': True, 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:ff:8b:7c:59:a1']}, 'virtual_network_refs': [{'to': ['default-domain', 'default-project', 'ip-fabric'], 'uuid': '7c5c0f88-0c29-4dce-b6f3-c98ecf3e1ea9'}]}, 'oper': 'CREATE', 'request-id': 'req-6dca7eae-f2b6-4e3a-bcbe-6e16a7a125fc', 'type': 'virtual_machine_interface', 'uuid': 'ff8b7c59-a132-4faf-9a03-14ce76c0c49e'} 08/25/2025 01:40:39.583 7f78c7ece6d8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3915-1', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-6dca7eae-f2b6-4e3a-bcbe-6e16a7a125fc', 'type': 'virtual_machine_interface', 'uuid': 'ff8b7c59-a132-4faf-9a03-14ce76c0c49e'} 08/25/2025 01:40:39.807 7f78c7ece6d8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3915-2', 'vhost0'], 'obj_dict': {'display_name': 'vhost0', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3915-2', 'vhost0'], 'id_perms': {'created': '2025-08-25T01:40:39.792274', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-08-25T01:40:39.792274', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13031038331304298223, 'uuid_mslong': 10065367263946033163}}, 'parent_type': 'virtual-router', 'parent_uuid': '1504a561-0cf6-407a-943b-f76165f6b5ff', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '8baf5e32-ad59-480b-b4d7-8c5227e446ef', 'virtual_machine_interface_disable_policy': True, 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:8b:af:5e:32:ad']}, 'virtual_network_refs': [{'to': ['default-domain', 'default-project', 'ip-fabric'], 'uuid': '7c5c0f88-0c29-4dce-b6f3-c98ecf3e1ea9'}]}, 'oper': 'CREATE', 'request-id': 'req-0e214f11-aec6-475c-a2ff-27a82263571e', 'type': 'virtual_machine_interface', 'uuid': '8baf5e32-ad59-480b-b4d7-8c5227e446ef'} 08/25/2025 01:40:39.854 7f78c7ece6d8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3915-2', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-0e214f11-aec6-475c-a2ff-27a82263571e', 'type': 'virtual_machine_interface', 'uuid': '8baf5e32-ad59-480b-b4d7-8c5227e446ef'} 08/25/2025 01:40:40.083 7f78c7ece6d8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'oper': 'UPDATE', 'request-id': 'req-b281d2fb-0910-4e01-8a32-3e735c715eb3', 'type': 'access_control_list', 'uuid': '3fe53c82-2c21-43f2-993e-21cd9bfe845c'} 08/25/2025 01:41:28.763 7f78d2ea5240 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 08/25/2025 01:41:39.799 7f78c7e56780 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 08/25/2025 01:44:43.879 7f78c7ed4630 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map