03/20/2025 00:45:49.310 7f6adada01c8 [contrail-device-manager] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 03/20/2025 00:45:49.310 7f6adada01c8 [contrail-device-manager] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 03/20/2025 00:45:49.310 7f6adada01c8 [contrail-device-manager] [INFO]: SANDESH: Flow Logging: [None] -> [False] 03/20/2025 00:45:49.311 7f6ad2487390 [contrail-device-manager] [INFO]: Processing event[EvStart] in state[none] 03/20/2025 00:45:49.311 7f6ad2487390 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectConfigNode:NodeStatusUVE] map 03/20/2025 00:45:49.312 7f6ad2487390 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/20/2025 00:45:49.312 7f6ad2487390 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 03/20/2025 00:45:49.312 7f6ad2487390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:45:49.312 7f6ad2487390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/20/2025 00:45:49.312 7f6ad2487390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:45:49.312 7f6ad2487390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:45:49.312 7f6ad2487390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 start_time = 1742431549311970 collector_name = collector_ip = collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 03/20/2025 00:45:49.312 7f6ad2487390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:45:49.369 7f6adada01c8 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.42:5673//', 'pyamqp://guest:guest@10.0.0.12:5673//', 'pyamqp://guest:guest@10.0.0.47:5673//'] 03/20/2025 00:45:49.378 7f6adada01c8 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 03/20/2025 00:45:49.379 7f6ad23da630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:45:49.379 7f6ad23da630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 03/20/2025 00:45:49.379 7f6ad23da630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:45:49.379 7f6ad23da630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:45:49.379 7f6ad23da630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 03/20/2025 00:45:49.379 7f6ad23da630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:45:49.383 7f6adada01c8 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 03/20/2025 00:45:49.384 7f6ad23da2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:45:49.384 7f6ad23da2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/20/2025 00:45:49.384 7f6ad23da2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:45:49.432 7f6adada01c8 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 03/20/2025 00:45:49.433 7f6ad23da8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:45:49.433 7f6ad23da8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 03/20/2025 00:45:49.433 7f6ad23da8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:45:49.464 7f6adada01c8 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 03/20/2025 00:45:49.464 7f6ae0c8c978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:45:49.464 7f6ae0c8c978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/20/2025 00:45:49.464 7f6ae0c8c978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:45:49.500 7f6adada01c8 [contrail-device-manager] [DEBUG]: Removed remaining AMQP queue from previous run 03/20/2025 00:45:49.500 7f6adada01c8 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 03/20/2025 00:45:49.501 7f6adada01c8 [contrail-device-manager] [WARNING]: initializing cassandra cluster 03/20/2025 00:45:49.501 7f6adada01c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 03/20/2025 00:45:49.501 7f6ad7d12780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting consumer greenlet 03/20/2025 00:45:49.501 7f6ad7d12780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 03/20/2025 00:45:49.502 7f6ad4c28a20 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting publisher greenlet 03/20/2025 00:45:49.502 7f6ad4c28a20 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Publisher connection 03/20/2025 00:45:49.502 7f6ad2276240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:45:49.502 7f6ad2276240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 03/20/2025 00:45:49.503 7f6ad2276240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:45:49.504 7f6adada01c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 03/20/2025 00:45:49.508 7f6ad7d12780 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 03/20/2025 00:45:49.508 7f6ad7d12780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Waiting for consumer 03/20/2025 00:45:49.509 7f6ad4c28a20 [contrail-device-manager] [INFO]: KombuAmqpClient: Publisher connection established 03/20/2025 00:45:49.542 7f6adada01c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 03/20/2025 00:45:49.544 7f6ad21cf4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:45:49.544 7f6ad21cf4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 03/20/2025 00:45:49.544 7f6ad21cf4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:45:49.552 7f6adada01c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 03/20/2025 00:45:49.552 7f6ad22140f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:45:49.552 7f6ad22140f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 03/20/2025 00:45:49.552 7f6ad22140f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:45:49.559 7f6adada01c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 03/20/2025 00:45:49.593 7f6adada01c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 03/20/2025 00:45:51.256 7f6adada01c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 03/20/2025 00:45:51.617 7f6adada01c8 [contrail-device-manager] [WARNING]: table 'obj_uuid_table', created 03/20/2025 00:45:51.624 7f6adada01c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 03/20/2025 00:45:53.131 7f6adada01c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 03/20/2025 00:45:53.312 7f6ad21bacc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:45:53.314 7f6ad21bacc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:45:53.314 7f6ad21bacc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:45:53.314 7f6ad21bacc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:45:53.314 7f6ad21bacc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 03/20/2025 00:45:53.314 7f6ad21bacc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:45:53.314 7f6ad21bacc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:45:53.315 7f6ad21bacc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:45:53.315 7f6ad21bacc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:45:53.315 7f6ad21bacc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 03/20/2025 00:45:53.315 7f6ad21bacc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:45:53.315 7f6ad21bacc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:45:53.315 7f6ad21bacc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:45:53.315 7f6ad21bacc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:45:53.627 7f6adada01c8 [contrail-device-manager] [WARNING]: table 'obj_uuid_table' fixed with properties {'gc_grace_seconds': 864000} 03/20/2025 00:45:53.633 7f6adada01c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 03/20/2025 00:45:55.493 7f6adada01c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 03/20/2025 00:45:56.649 7f6adada01c8 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table', created 03/20/2025 00:45:56.658 7f6adada01c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 03/20/2025 00:45:57.315 7f6ad1974438 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:45:57.316 7f6ad1974438 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:45:57.316 7f6ad1974438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:45:57.316 7f6ad1974438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:45:57.316 7f6ad1974438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 03/20/2025 00:45:57.316 7f6ad1974438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:45:57.316 7f6ad1974438 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:45:57.317 7f6ad1974438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:45:57.317 7f6ad1974438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:45:57.317 7f6ad1974438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 03/20/2025 00:45:57.317 7f6ad1974438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:45:57.317 7f6ad1974438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:45:57.317 7f6ad1974438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.12:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:45:57.317 7f6ad1974438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:45:58.653 7f6adada01c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 03/20/2025 00:45:59.554 7f6adada01c8 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table' fixed with properties {'gc_grace_seconds': 864000} 03/20/2025 00:45:59.560 7f6adada01c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 03/20/2025 00:46:01.317 7f6ad194fc18 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:46:01.319 7f6ad194fc18 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:46:01.319 7f6ad194fc18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:46:01.319 7f6ad194fc18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:46:01.319 7f6ad194fc18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 03/20/2025 00:46:01.319 7f6ad194fc18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:46:01.319 7f6ad194fc18 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:46:01.320 7f6ad194fc18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:46:01.320 7f6ad194fc18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:01.320 7f6ad194fc18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 03/20/2025 00:46:01.320 7f6ad194fc18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:01.320 7f6ad194fc18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:01.320 7f6ad194fc18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.47:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:46:01.320 7f6ad194fc18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:01.687 7f6adada01c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 03/20/2025 00:46:01.688 7f6adada01c8 [contrail-device-manager] [WARNING]: table 'obj_shared_table', created 03/20/2025 00:46:01.694 7f6adada01c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 03/20/2025 00:46:02.632 7f6adada01c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 03/20/2025 00:46:04.141 7f6adada01c8 [contrail-device-manager] [WARNING]: table 'obj_shared_table' fixed with properties {'gc_grace_seconds': 864000} 03/20/2025 00:46:04.146 7f6adada01c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 03/20/2025 00:46:04.147 7f6ad69bce10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:04.147 7f6ad69bce10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:46:04.147 7f6ad69bce10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:04.167 7f6adada01c8 [contrail-device-manager] [ERROR]: Error in db walk read 'type' 03/20/2025 00:46:04.168 7f6ad18ed630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:04.168 7f6ad18ed630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Zookeeper:Zookeeper[] connection down >>, ] >> 03/20/2025 00:46:04.168 7f6ad18ed630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:04.178 7f6ad18ed588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:04.178 7f6ad18ed588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:46:04.178 7f6ad18ed588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:04.178 7f6adada01c8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_status_consumer.dummy 03/20/2025 00:46:04.179 7f6adada01c8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_request_consumer 03/20/2025 00:46:04.179 7f6ad7d12780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 03/20/2025 00:46:04.180 7f6adada01c8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-2. 03/20/2025 00:46:04.184 7f6ad7d12780 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 03/20/2025 00:46:04.336 7f6ad7d12780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-2.'] 03/20/2025 00:46:04.356 7f6ad7d12780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 03/20/2025 00:46:04.360 7f6ad7d12780 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 03/20/2025 00:46:04.429 7f6ad7d12780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-2.'] 03/20/2025 00:46:04.690 7f6adada01c8 [contrail-device-manager] [DEBUG]: Fetching all containers 03/20/2025 00:46:04.730 7f6adada01c8 [contrail-device-manager] [INFO]: Restarting dnsmasq docker: config_dnsmasq_1 03/20/2025 00:46:04.917 7f6adada01c8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-2..config_queue 03/20/2025 00:46:04.918 7f6adada01c8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-2..tftp_queue 03/20/2025 00:46:05.320 7f6ad182b780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:46:05.321 7f6ad182b780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:46:05.321 7f6ad182b780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:46:05.321 7f6ad182b780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:46:05.322 7f6ad182b780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:46:05.322 7f6ad182b780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:46:05.322 7f6ad182b780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:46:05.322 7f6ad182b780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:46:05.322 7f6ad182b780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:05.322 7f6ad182b780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:46:05.322 7f6ad182b780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:05.322 7f6ad182b780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:05.322 7f6ad182b780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:46:05.322 7f6ad182b780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:05.419 7f6adada01c8 [contrail-device-manager] [WARNING]: Process 138 prepared to run in Full mode... 03/20/2025 00:46:05.419 7f6adada01c8 [contrail-device-manager] [WARNING]: Elected master Device Manager node. Initializing... 03/20/2025 00:46:05.419 7f6adada01c8 [contrail-device-manager] [INFO]: SANDESH: INTROSPECT IS ON: 0.0.0.0:8096 03/20/2025 00:46:05.422 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add [config_updater.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/config_updater] to home page 03/20/2025 00:46:05.422 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_ConfigApiUpdateReq] 03/20/2025 00:46:05.423 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add [greenlets.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/greenlets] to home page 03/20/2025 00:46:05.423 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_GreenletObjectReq] 03/20/2025 00:46:05.424 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add [cpuinfo.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/nodeinfo/cpuinfo] to home page 03/20/2025 00:46:05.424 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CpuLoadInfoReq] 03/20/2025 00:46:05.427 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add [dm_introspect.xml:/usr/lib/python3.6/site-packages/device_manager/sandesh/dm_introspect] to home page 03/20/2025 00:46:05.427 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_RoutingInstanceList] 03/20/2025 00:46:05.427 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualNetworkList] 03/20/2025 00:46:05.427 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualMachineInterfaceList] 03/20/2025 00:46:05.428 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalInterfaceList] 03/20/2025 00:46:05.428 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalInterfaceList] 03/20/2025 00:46:05.428 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalRouterList] 03/20/2025 00:46:05.428 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_BgpRouterList] 03/20/2025 00:46:05.428 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalRouterList] 03/20/2025 00:46:05.428 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_InstanceIpList] 03/20/2025 00:46:05.428 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FloatingIpList] 03/20/2025 00:46:05.428 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PortTupleList] 03/20/2025 00:46:05.428 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FeatureFlagList] 03/20/2025 00:46:05.429 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add [sandesh_alarm.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_alarm] to home page 03/20/2025 00:46:05.430 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmCacheRequest] 03/20/2025 00:46:05.430 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmTypesRequest] 03/20/2025 00:46:05.430 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmAckRequest] 03/20/2025 00:46:05.430 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add [sandesh_ctrl.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_ctrl] to home page 03/20/2025 00:46:05.430 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlClientToServer] 03/20/2025 00:46:05.430 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlServerToClient] 03/20/2025 00:46:05.431 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add [sandesh_trace.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_trace] to home page 03/20/2025 00:46:05.431 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceRequest] 03/20/2025 00:46:05.431 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferListRequest] 03/20/2025 00:46:05.431 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceEnableDisableReq] 03/20/2025 00:46:05.431 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufStatusReq] 03/20/2025 00:46:05.431 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferEnableDisableReq] 03/20/2025 00:46:05.432 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add [sandesh_uve.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_uve] to home page 03/20/2025 00:46:05.432 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshMessageStatsReq] 03/20/2025 00:46:05.432 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsSet] 03/20/2025 00:46:05.432 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsStatus] 03/20/2025 00:46:05.432 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsSet] 03/20/2025 00:46:05.432 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsStatus] 03/20/2025 00:46:05.433 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsSet] 03/20/2025 00:46:05.433 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsReset] 03/20/2025 00:46:05.433 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsStatus] 03/20/2025 00:46:05.433 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueSet] 03/20/2025 00:46:05.433 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueStatus] 03/20/2025 00:46:05.433 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CollectorInfoRequest] 03/20/2025 00:46:05.433 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVEDSConfReq] 03/20/2025 00:46:05.433 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVECacheReq] 03/20/2025 00:46:05.433 7f6adada01c8 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVETypesReq] 03/20/2025 00:46:05.435 7f6adada01c8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.ztp_queue 03/20/2025 00:46:05.450 7f6ad18a10f0 [contrail-device-manager] [ERROR]: Cannot write http_port 8096 to /tmp/contrail-device-manager.1.http_port 03/20/2025 00:46:05.450 7f6ad18a10f0 [contrail-device-manager] [ERROR]: Starting Introspect on HTTP Port 8096 03/20/2025 00:46:05.495 7f6ad10686d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:05.495 7f6ad10686d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Initializing description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 03/20/2025 00:46:05.495 7f6ad10686d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:05.506 7f6ad0f40a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:05.506 7f6ad0f40a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 03/20/2025 00:46:05.506 7f6ad0f40a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:05.515 7f6ad7d12780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 03/20/2025 00:46:05.520 7f6ad7d12780 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 03/20/2025 00:46:05.694 7f6ad7d12780 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-2.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-2..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-2..tftp_queue', 'device_manager_ztp.ztp_queue'] 03/20/2025 00:46:09.322 7f6ad0e7f198 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:46:09.323 7f6ad0e7f198 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:46:09.323 7f6ad0e7f198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:46:09.323 7f6ad0e7f198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:46:09.324 7f6ad0e7f198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 03/20/2025 00:46:09.324 7f6ad0e7f198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:46:09.324 7f6ad0e7f198 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:46:09.324 7f6ad0e7f198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:46:09.324 7f6ad0e7f198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:09.325 7f6ad0e7f198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 03/20/2025 00:46:09.325 7f6ad0e7f198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:09.325 7f6ad0e7f198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:09.325 7f6ad0e7f198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.12:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:46:09.325 7f6ad0e7f198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:11.692 7f6ad0ee9e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:11.692 7f6ad0ee9e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:46:11.693 7f6ad0ee9e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:13.324 7f6ad0e70438 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:46:13.326 7f6ad0e70438 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:46:13.326 7f6ad0e70438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:46:13.326 7f6ad0e70438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:46:13.326 7f6ad0e70438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:46:13.326 7f6ad0e70438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:46:13.326 7f6ad0e70438 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:46:13.327 7f6ad0e70438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:46:13.327 7f6ad0e70438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:13.327 7f6ad0e70438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:46:13.327 7f6ad0e70438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:13.327 7f6ad0e70438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:13.327 7f6ad0e70438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.47:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:46:13.327 7f6ad0e70438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:17.326 7f6ad2276f60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:46:17.328 7f6ad2276f60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:46:17.328 7f6ad2276f60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:46:17.328 7f6ad2276f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:46:17.328 7f6ad2276f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:46:17.328 7f6ad2276f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:46:17.328 7f6ad2276f60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:46:17.328 7f6ad2276f60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:46:17.329 7f6ad2276f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:17.330 7f6ad2276f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:46:17.330 7f6ad2276f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:17.330 7f6ad2276f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:17.330 7f6ad2276f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:46:17.330 7f6ad2276f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:21.328 7f6ad0e6f2e8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:46:21.329 7f6ad0e6f2e8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:46:21.329 7f6ad0e6f2e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:46:21.329 7f6ad0e6f2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:46:21.329 7f6ad0e6f2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:46:21.329 7f6ad0e6f2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:46:21.330 7f6ad0e6f2e8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:46:21.330 7f6ad0e6f2e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:46:21.330 7f6ad0e6f2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:21.330 7f6ad0e6f2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:46:21.330 7f6ad0e6f2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:21.330 7f6ad0e6f2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:21.330 7f6ad0e6f2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.12:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:46:21.330 7f6ad0e6f2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:25.329 7f6ad4694438 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:46:25.330 7f6ad4694438 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:46:25.331 7f6ad4694438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:46:25.331 7f6ad4694438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:46:25.331 7f6ad4694438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:46:25.331 7f6ad4694438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:46:25.331 7f6ad4694438 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:46:25.331 7f6ad4694438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:46:25.331 7f6ad4694438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:25.331 7f6ad4694438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:46:25.331 7f6ad4694438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:25.331 7f6ad4694438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:25.332 7f6ad4694438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.47:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:46:25.332 7f6ad4694438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:29.331 7f6ad0eed6d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:46:29.333 7f6ad0eed6d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:46:29.333 7f6ad0eed6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:46:29.333 7f6ad0eed6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:46:29.333 7f6ad0eed6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:46:29.333 7f6ad0eed6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:46:29.333 7f6ad0eed6d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:46:29.333 7f6ad0eed6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:46:29.333 7f6ad0eed6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:29.334 7f6ad0eed6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:46:29.334 7f6ad0eed6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:29.334 7f6ad0eed6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:29.334 7f6ad0eed6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:46:29.334 7f6ad0eed6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:33.334 7f6ad196c198 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:46:33.337 7f6ad196c198 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:46:33.337 7f6ad196c198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:46:33.337 7f6ad196c198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:46:33.337 7f6ad196c198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:46:33.337 7f6ad196c198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:46:33.337 7f6ad196c198 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:46:33.338 7f6ad196c198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:46:33.338 7f6ad196c198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:33.338 7f6ad196c198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:46:33.338 7f6ad196c198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:33.338 7f6ad196c198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:33.338 7f6ad196c198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.12:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:46:33.338 7f6ad196c198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:37.338 7f6ad7ed88d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:46:37.349 7f6ad7ed88d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:46:37.349 7f6ad7ed88d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:46:37.349 7f6ad7ed88d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:46:37.349 7f6ad7ed88d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:46:37.349 7f6ad7ed88d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:46:37.349 7f6ad7ed88d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:46:37.350 7f6ad7ed88d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:46:37.350 7f6ad7ed88d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:37.350 7f6ad7ed88d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:46:37.350 7f6ad7ed88d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:37.350 7f6ad7ed88d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:37.350 7f6ad7ed88d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.47:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:46:37.350 7f6ad7ed88d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:41.349 7f6ad1941d68 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:46:41.351 7f6ad1941d68 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:46:41.351 7f6ad1941d68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:46:41.351 7f6ad1941d68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:46:41.352 7f6ad1941d68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:46:41.352 7f6ad1941d68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:46:41.352 7f6ad1941d68 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:46:41.352 7f6ad1941d68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:46:41.352 7f6ad1941d68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:41.353 7f6ad1941d68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:46:41.353 7f6ad1941d68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:41.353 7f6ad1941d68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:41.353 7f6ad1941d68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:46:41.353 7f6ad1941d68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:45.352 7f6ad0e6fd68 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:46:45.353 7f6ad0e6fd68 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:46:45.353 7f6ad0e6fd68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:46:45.353 7f6ad0e6fd68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:46:45.353 7f6ad0e6fd68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:46:45.353 7f6ad0e6fd68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:46:45.353 7f6ad0e6fd68 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:46:45.353 7f6ad0e6fd68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:46:45.354 7f6ad0e6fd68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:45.354 7f6ad0e6fd68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:46:45.354 7f6ad0e6fd68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:45.354 7f6ad0e6fd68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:45.354 7f6ad0e6fd68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.12:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:46:45.354 7f6ad0e6fd68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:49.353 7f6ad0e6f978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:46:49.354 7f6ad0e6f978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:46:49.354 7f6ad0e6f978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:46:49.354 7f6ad0e6f978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:46:49.355 7f6ad0e6f978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:46:49.355 7f6ad0e6f978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:46:49.355 7f6ad0e6f978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:46:49.355 7f6ad0e6f978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:46:49.355 7f6ad0e6f978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:49.355 7f6ad0e6f978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:46:49.355 7f6ad0e6f978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:49.355 7f6ad0e6f978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:49.355 7f6ad0e6f978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.47:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:46:49.355 7f6ad0e6f978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:53.355 7f6ad0e6fb70 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:46:53.357 7f6ad0e6fb70 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:46:53.357 7f6ad0e6fb70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:46:53.357 7f6ad0e6fb70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:46:53.357 7f6ad0e6fb70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:46:53.357 7f6ad0e6fb70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:46:53.357 7f6ad0e6fb70 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:46:53.358 7f6ad0e6fb70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:46:53.358 7f6ad0e6fb70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:53.358 7f6ad0e6fb70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:46:53.358 7f6ad0e6fb70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:53.358 7f6ad0e6fb70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:53.358 7f6ad0e6fb70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:46:53.358 7f6ad0e6fb70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:56.845 7f6adada01c8 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 03/20/2025 00:46:56.845 7f6adada01c8 [contrail-device-manager] [WARNING]: initializing cassandra cluster 03/20/2025 00:46:56.846 7f6adada01c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 03/20/2025 00:46:56.846 7f6ad0f47438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:56.846 7f6ad0f47438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 03/20/2025 00:46:56.846 7f6ad0f47438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:56.847 7f6adada01c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 03/20/2025 00:46:56.878 7f6adada01c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 03/20/2025 00:46:56.879 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:56.879 7f6ad0e30e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Initializing description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 03/20/2025 00:46:56.879 7f6ad0e30e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:56.886 7f6adada01c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 03/20/2025 00:46:56.886 7f6ad0e30c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:56.887 7f6ad0e30c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 03/20/2025 00:46:56.887 7f6ad0e30c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:56.900 7f6adada01c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 03/20/2025 00:46:57.357 7f6ad0de14e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:46:57.359 7f6ad0de14e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:46:57.359 7f6ad0de14e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:46:57.359 7f6ad0de14e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:46:57.359 7f6ad0de14e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 03/20/2025 00:46:57.359 7f6ad0de14e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:46:57.359 7f6ad0de14e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:46:57.360 7f6ad0de14e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:46:57.360 7f6ad0de14e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:57.360 7f6ad0de14e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 03/20/2025 00:46:57.360 7f6ad0de14e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:57.360 7f6ad0de14e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:57.360 7f6ad0de14e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.12:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:46:57.360 7f6ad0de14e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:46:57.570 7f6adada01c8 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace', created 03/20/2025 00:46:57.575 7f6adada01c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 03/20/2025 00:46:57.581 7f6adada01c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 03/20/2025 00:46:58.643 7f6adada01c8 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace' fixed with replication {'class': 'SimpleStrategy', 'replication_factor': None} 03/20/2025 00:46:58.648 7f6adada01c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 03/20/2025 00:46:58.648 7f6adada01c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 03/20/2025 00:46:58.712 7f6adada01c8 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 03/20/2025 00:46:58.719 7f6adada01c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 03/20/2025 00:46:59.643 7f6adada01c8 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table', created 03/20/2025 00:46:59.648 7f6adada01c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 03/20/2025 00:46:59.654 7f6adada01c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 03/20/2025 00:47:00.552 7f6adada01c8 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table' fixed with properties {'gc_grace_seconds': 864000} 03/20/2025 00:47:00.557 7f6adada01c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 03/20/2025 00:47:00.565 7f6adada01c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 03/20/2025 00:47:01.360 7f6ad0dd1a20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:47:01.361 7f6ad0dd1a20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:47:01.361 7f6ad0dd1a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:47:01.361 7f6ad0dd1a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:47:01.361 7f6ad0dd1a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 03/20/2025 00:47:01.362 7f6ad0dd1a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:47:01.362 7f6ad0dd1a20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:47:01.362 7f6ad0dd1a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:47:01.362 7f6ad0dd1a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:01.362 7f6ad0dd1a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 03/20/2025 00:47:01.362 7f6ad0dd1a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:01.362 7f6ad0dd1a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:01.362 7f6ad0dd1a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.47:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:47:01.362 7f6ad0dd1a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:01.710 7f6adada01c8 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table', created 03/20/2025 00:47:01.716 7f6adada01c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 03/20/2025 00:47:01.725 7f6adada01c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 03/20/2025 00:47:02.627 7f6adada01c8 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table' fixed with properties {'gc_grace_seconds': 864000} 03/20/2025 00:47:02.634 7f6adada01c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 03/20/2025 00:47:02.648 7f6adada01c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 03/20/2025 00:47:03.678 7f6adada01c8 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table', created 03/20/2025 00:47:03.683 7f6adada01c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 03/20/2025 00:47:03.702 7f6adada01c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 03/20/2025 00:47:04.592 7f6adada01c8 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table' fixed with properties {'gc_grace_seconds': 864000} 03/20/2025 00:47:04.598 7f6adada01c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 03/20/2025 00:47:04.610 7f6adada01c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 03/20/2025 00:47:05.362 7f6ad0d4a8d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:47:05.364 7f6ad0d4a8d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:47:05.364 7f6ad0d4a8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:47:05.364 7f6ad0d4a8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:47:05.364 7f6ad0d4a8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 03/20/2025 00:47:05.364 7f6ad0d4a8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:47:05.364 7f6ad0d4a8d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:47:05.365 7f6ad0d4a8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:47:05.365 7f6ad0d4a8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:05.365 7f6ad0d4a8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 03/20/2025 00:47:05.365 7f6ad0d4a8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:05.365 7f6ad0d4a8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:05.365 7f6ad0d4a8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:47:05.365 7f6ad0d4a8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:05.747 7f6adada01c8 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table', created 03/20/2025 00:47:05.757 7f6adada01c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 03/20/2025 00:47:05.772 7f6adada01c8 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 03/20/2025 00:47:06.714 7f6adada01c8 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table' fixed with properties {'gc_grace_seconds': 864000} 03/20/2025 00:47:06.722 7f6adada01c8 [contrail-device-manager] [WARNING]: Zookeeper lock released. 03/20/2025 00:47:06.723 7f6ad2fc04e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:06.723 7f6ad2fc04e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:47:06.723 7f6ad2fc04e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:06.788 7f6adada01c8 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.42:5673//', 'pyamqp://guest:guest@10.0.0.12:5673//', 'pyamqp://guest:guest@10.0.0.47:5673//'] 03/20/2025 00:47:06.788 7f6adada01c8 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 03/20/2025 00:47:06.789 7f6ad0fd7438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:06.789 7f6ad0fd7438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Initializing description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 03/20/2025 00:47:06.789 7f6ad0fd7438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:06.790 7f6ad0fd7438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:06.790 7f6ad0fd7438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 03/20/2025 00:47:06.790 7f6ad0fd7438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:06.800 7f6adada01c8 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 03/20/2025 00:47:06.800 7f6ad0daa048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:06.800 7f6ad0daa048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:47:06.800 7f6ad0daa048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:06.848 7f6adada01c8 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 03/20/2025 00:47:06.849 7f6ad0de1a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:06.849 7f6ad0de1a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 03/20/2025 00:47:06.849 7f6ad0de1a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:06.859 7f6adada01c8 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 03/20/2025 00:47:06.859 7f6ad0de1f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:06.860 7f6ad0de1f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:47:06.860 7f6ad0de1f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:06.869 7f6ad0f40780 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 03/20/2025 00:47:06.869 7f6ad0da2eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:06.869 7f6ad0da2eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 03/20/2025 00:47:06.870 7f6ad0da2eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:06.882 7f6ad0f40780 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 03/20/2025 00:47:06.882 7f6ad0e6fc18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:06.882 7f6ad0e6fc18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:47:06.882 7f6ad0e6fc18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:09.365 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:47:09.366 7f6ad0daa6d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:47:09.366 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:47:09.366 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:47:09.367 7f6ad0daa6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:47:09.367 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:47:09.367 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:47:09.367 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:47:09.367 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:09.367 7f6ad0daa6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:47:09.367 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:09.367 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:09.367 7f6ad0daa6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.12:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:47:09.367 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:13.368 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:47:13.369 7f6ad0daa6d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:47:13.369 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:47:13.369 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:47:13.370 7f6ad0daa6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:47:13.370 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:47:13.370 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:47:13.370 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:47:13.370 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:13.370 7f6ad0daa6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:47:13.370 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:13.370 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:13.371 7f6ad0daa6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.47:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:47:13.371 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:17.371 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:47:17.372 7f6ad0daa6d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:47:17.372 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:47:17.372 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:47:17.372 7f6ad0daa6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:47:17.372 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:47:17.372 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:47:17.373 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:47:17.373 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:17.373 7f6ad0daa6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:47:17.373 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:17.373 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:17.373 7f6ad0daa6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:47:17.373 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:21.374 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:47:21.374 7f6ad0daa6d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:47:21.375 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:47:21.375 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:47:21.375 7f6ad0daa6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:47:21.375 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:47:21.375 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:47:21.375 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:47:21.375 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:21.375 7f6ad0daa6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:47:21.375 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:21.375 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:21.376 7f6ad0daa6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.12:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:47:21.376 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:25.375 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:47:25.377 7f6ad0daa6d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:47:25.377 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:47:25.377 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:47:25.377 7f6ad0daa6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:47:25.377 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:47:25.377 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:47:25.378 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:47:25.378 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:25.378 7f6ad0daa6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:47:25.378 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:25.378 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:25.378 7f6ad0daa6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.47:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:47:25.378 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:29.378 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:47:29.380 7f6ad0daa6d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:47:29.380 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:47:29.380 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:47:29.380 7f6ad0daa6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:47:29.380 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:47:29.380 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:47:29.381 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:47:29.381 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:29.381 7f6ad0daa6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:47:29.381 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:29.381 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:29.381 7f6ad0daa6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:47:29.381 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:33.380 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:47:33.381 7f6ad0daa6d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:47:33.381 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:47:33.381 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:47:33.382 7f6ad0daa6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:47:33.382 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:47:33.382 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:47:33.382 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:47:33.382 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:33.382 7f6ad0daa6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:47:33.382 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:33.382 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:33.382 7f6ad0daa6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.12:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:47:33.382 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:37.382 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:47:37.384 7f6ad0daa6d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:47:37.384 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:47:37.384 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:47:37.384 7f6ad0daa6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:47:37.384 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:47:37.384 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:47:37.385 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:47:37.385 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:37.385 7f6ad0daa6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:47:37.385 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:37.385 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:37.385 7f6ad0daa6d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.47:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:47:37.385 7f6ad0daa6d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:38.975 7f6ad0daa048 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-2923-3'], 'obj_dict': {'bgp_router_parameters': {'address': '10.20.0.5', 'address_families': {'family': ['route-target', 'inet-vpn', 'e-vpn', 'erm-vpn', 'inet6-vpn']}, 'admin_down': False, 'auth_data': None, 'autonomous_system': 64512, 'cluster_id': None, 'gateway_address': None, 'hold_time': 0, 'identifier': '10.20.0.5', 'ipv6_gateway_address': None, 'local_autonomous_system': None, 'port': 179, 'router_type': 'control-node', 'source_port': None, 'vendor': 'contrail'}, 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2923-3', 'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-2923-3'], 'id_perms': {'created': '2025-03-20T00:47:38.950620', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-03-20T00:47:38.950620', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13274624960850071091, 'uuid_mslong': 2301072442691766089}}, 'parent_type': 'routing-instance', 'parent_uuid': 'a27ba35d-5cf2-4a3a-ba99-96862116ed7d', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '1fef0d31-ed15-4749-b838-f111e6008e33'}, 'oper': 'CREATE', 'request-id': 'req-5b2a4729-9931-4103-a670-d4f2b0e4be58', 'type': 'bgp_router', 'uuid': '1fef0d31-ed15-4749-b838-f111e6008e33'} 03/20/2025 00:47:39.042 7f6ad0daa048 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-2923-1'], 'obj_dict': {'bgp_router_parameters': {'address': '10.20.0.11', 'address_families': {'family': ['route-target', 'inet-vpn', 'e-vpn', 'erm-vpn', 'inet6-vpn']}, 'admin_down': False, 'auth_data': None, 'autonomous_system': 64512, 'cluster_id': None, 'gateway_address': None, 'hold_time': 0, 'identifier': '10.20.0.11', 'ipv6_gateway_address': None, 'local_autonomous_system': None, 'port': 179, 'router_type': 'control-node', 'source_port': None, 'vendor': 'contrail'}, 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2923-1', 'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-2923-1'], 'id_perms': {'created': '2025-03-20T00:47:39.023674', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-03-20T00:47:39.023674', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12663730660172848833, 'uuid_mslong': 6272712255286559966}}, 'parent_type': 'routing-instance', 'parent_uuid': 'a27ba35d-5cf2-4a3a-ba99-96862116ed7d', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '570d268f-da98-44de-afbe-9bf0aad35ec1'}, 'oper': 'CREATE', 'request-id': 'req-f0d18837-4d42-411f-aeff-34171b07ffdd', 'type': 'bgp_router', 'uuid': '570d268f-da98-44de-afbe-9bf0aad35ec1'} 03/20/2025 00:47:39.130 7f6ad0daa048 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-2923-2'], 'obj_dict': {'bgp_router_parameters': {'address': '10.20.0.23', 'address_families': {'family': ['route-target', 'inet-vpn', 'e-vpn', 'erm-vpn', 'inet6-vpn']}, 'admin_down': False, 'auth_data': None, 'autonomous_system': 64512, 'cluster_id': None, 'gateway_address': None, 'hold_time': 0, 'identifier': '10.20.0.23', 'ipv6_gateway_address': None, 'local_autonomous_system': None, 'port': 179, 'router_type': 'control-node', 'source_port': None, 'vendor': 'contrail'}, 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2923-2', 'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-2923-2'], 'id_perms': {'created': '2025-03-20T00:47:39.095857', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-03-20T00:47:39.095857', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11953260708022157541, 'uuid_mslong': 18309265485222793157}}, 'parent_type': 'routing-instance', 'parent_uuid': 'a27ba35d-5cf2-4a3a-ba99-96862116ed7d', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'fe1793f3-c106-4fc5-a5e2-8348694b80e5'}, 'oper': 'CREATE', 'request-id': 'req-60e48151-5c9d-4b7d-801f-c59df8d55d91', 'type': 'bgp_router', 'uuid': 'fe1793f3-c106-4fc5-a5e2-8348694b80e5'} 03/20/2025 00:47:39.273 7f6ad0daa048 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-8e9f43fa-4138-450d-a9f7-01e0a52da2e2', 'type': 'global_system_config', 'uuid': '92e769a6-5754-45f5-bd68-145125efbf0e'} 03/20/2025 00:47:39.370 7f6ad0daa048 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-7c2f01fa-954b-499d-a1a0-0cb4416898ef', 'type': 'global_system_config', 'uuid': '92e769a6-5754-45f5-bd68-145125efbf0e'} 03/20/2025 00:47:39.402 7f6ad0daa048 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-e1386862-a411-4f91-99ad-2e35acc12cbd', 'type': 'global_system_config', 'uuid': '92e769a6-5754-45f5-bd68-145125efbf0e'} 03/20/2025 00:47:39.583 7f6ad0daa048 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-2923-1'], 'oper': 'UPDATE', 'request-id': 'req-acc6521f-55bb-48e6-99ff-0ec7014c6fbf', 'type': 'bgp_router', 'uuid': '570d268f-da98-44de-afbe-9bf0aad35ec1'} 03/20/2025 00:47:39.594 7f6ad0daa048 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-2923-3'], 'oper': 'UPDATE-IMPLICIT', 'request-id': 'req-acc6521f-55bb-48e6-99ff-0ec7014c6fbf', 'type': 'bgp_router', 'uuid': '1fef0d31-ed15-4749-b838-f111e6008e33'} 03/20/2025 00:47:39.675 7f6ad0daa048 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-2923-2'], 'oper': 'UPDATE', 'request-id': 'req-62ca6847-99ce-4f5a-bd5b-80796fac54c0', 'type': 'bgp_router', 'uuid': 'fe1793f3-c106-4fc5-a5e2-8348694b80e5'} 03/20/2025 00:47:39.692 7f6ad0daa048 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-2923-1'], 'oper': 'UPDATE-IMPLICIT', 'request-id': 'req-62ca6847-99ce-4f5a-bd5b-80796fac54c0', 'type': 'bgp_router', 'uuid': '570d268f-da98-44de-afbe-9bf0aad35ec1'} 03/20/2025 00:47:39.713 7f6ad0daa048 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-2923-2'], 'oper': 'UPDATE', 'request-id': 'req-2a511668-a7be-48f9-b84c-961c794c803a', 'type': 'bgp_router', 'uuid': 'fe1793f3-c106-4fc5-a5e2-8348694b80e5'} 03/20/2025 00:47:39.720 7f6ad0daa048 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-ansible-os-2923-3'], 'oper': 'UPDATE-IMPLICIT', 'request-id': 'req-2a511668-a7be-48f9-b84c-961c794c803a', 'type': 'bgp_router', 'uuid': '1fef0d31-ed15-4749-b838-f111e6008e33'} 03/20/2025 00:47:41.385 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:47:41.386 7f6ad0e6f4e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:47:41.387 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:47:41.387 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:47:41.387 7f6ad0e6f4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:47:41.387 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:47:41.387 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:47:41.388 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:47:41.388 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:41.388 7f6ad0e6f4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:47:41.388 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:41.388 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:41.388 7f6ad0e6f4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:47:41.388 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:45.388 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:47:45.388 7f6ad0e6f4e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:47:45.389 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:47:45.389 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:47:45.389 7f6ad0e6f4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:47:45.389 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:47:45.389 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:47:45.389 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:47:45.389 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:45.389 7f6ad0e6f4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:47:45.389 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:45.389 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:45.389 7f6ad0e6f4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.12:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:47:45.390 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:49.389 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:47:49.391 7f6ad0e6f4e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:47:49.391 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:47:49.391 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:47:49.391 7f6ad0e6f4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:47:49.391 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:47:49.391 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:47:49.392 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:47:49.392 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:49.392 7f6ad0e6f4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:47:49.392 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:49.392 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:49.392 7f6ad0e6f4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.47:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:47:49.392 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:53.392 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:47:53.394 7f6ad0e6f4e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:47:53.394 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:47:53.394 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:47:53.394 7f6ad0e6f4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:47:53.394 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:47:53.394 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:47:53.395 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:47:53.395 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:53.395 7f6ad0e6f4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:47:53.395 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:53.395 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:53.395 7f6ad0e6f4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:47:53.395 7f6ad0e6f4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:57.395 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:47:57.396 7f6ad0e30e10 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:47:57.396 7f6ad0e30e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:47:57.396 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:47:57.397 7f6ad0e30e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:47:57.397 7f6ad0e30e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:47:57.397 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:47:57.397 7f6ad0e30e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:47:57.397 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:57.397 7f6ad0e30e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:47:57.397 7f6ad0e30e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:57.397 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:47:57.398 7f6ad0e30e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.12:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:47:57.398 7f6ad0e30e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:01.397 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:48:01.399 7f6ad0e30e10 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:48:01.399 7f6ad0e30e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:48:01.399 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:01.399 7f6ad0e30e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:48:01.399 7f6ad0e30e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:01.399 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:48:01.400 7f6ad0e30e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:48:01.400 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:01.400 7f6ad0e30e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:48:01.400 7f6ad0e30e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:01.400 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:01.400 7f6ad0e30e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.47:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:48:01.400 7f6ad0e30e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:05.400 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:48:05.401 7f6ad0e30e10 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:48:05.402 7f6ad0e30e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:48:05.402 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:05.402 7f6ad0e30e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:48:05.402 7f6ad0e30e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:05.402 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:48:05.402 7f6ad0e30e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:48:05.402 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:05.402 7f6ad0e30e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:48:05.402 7f6ad0e30e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:05.402 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:05.403 7f6ad0e30e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:48:05.403 7f6ad0e30e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:09.403 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:48:09.404 7f6ad0e30e10 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:48:09.404 7f6ad0e30e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:48:09.404 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:09.404 7f6ad0e30e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:48:09.404 7f6ad0e30e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:09.404 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:48:09.404 7f6ad0e30e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:48:09.404 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:09.404 7f6ad0e30e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:48:09.405 7f6ad0e30e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:09.405 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:09.405 7f6ad0e30e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.12:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:48:09.405 7f6ad0e30e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:13.404 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:48:13.407 7f6ad0e30e10 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:48:13.407 7f6ad0e30e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:48:13.407 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:13.407 7f6ad0e30e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:48:13.407 7f6ad0e30e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:13.407 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:48:13.408 7f6ad0e30e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:48:13.408 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:13.408 7f6ad0e30e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:48:13.408 7f6ad0e30e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:13.408 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:13.408 7f6ad0e30e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.47:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:48:13.408 7f6ad0e30e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:17.408 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:48:17.410 7f6ad0e30e10 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:48:17.410 7f6ad0e30e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:48:17.410 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:17.410 7f6ad0e30e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:48:17.410 7f6ad0e30e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:17.410 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:48:17.411 7f6ad0e30e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:48:17.411 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:17.411 7f6ad0e30e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:48:17.411 7f6ad0e30e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:17.411 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:17.411 7f6ad0e30e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:48:17.411 7f6ad0e30e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:21.411 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:48:21.412 7f6ad0e30e10 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:48:21.412 7f6ad0e30e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:48:21.412 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:21.412 7f6ad0e30e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:48:21.412 7f6ad0e30e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:21.412 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:48:21.413 7f6ad0e30e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:48:21.413 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:21.413 7f6ad0e30e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:48:21.413 7f6ad0e30e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:21.413 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:21.413 7f6ad0e30e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.12:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:48:21.413 7f6ad0e30e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:25.413 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:48:25.415 7f6ad0e30e10 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:48:25.415 7f6ad0e30e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:48:25.415 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:25.415 7f6ad0e30e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:48:25.415 7f6ad0e30e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:25.415 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:48:25.415 7f6ad0e30e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:48:25.416 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:25.416 7f6ad0e30e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:48:25.416 7f6ad0e30e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:25.416 7f6ad0e30e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:25.416 7f6ad0e30e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.47:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:48:25.416 7f6ad0e30e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:29.415 7f6ad0dc3978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:48:29.417 7f6ad0dc3978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:48:29.417 7f6ad0dc3978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:48:29.417 7f6ad0dc3978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:29.417 7f6ad0dc3978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:48:29.417 7f6ad0dc3978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:29.417 7f6ad0dc3978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:48:29.418 7f6ad0dc3978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:48:29.418 7f6ad0dc3978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:29.418 7f6ad0dc3978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:48:29.418 7f6ad0dc3978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:29.418 7f6ad0dc3978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:29.418 7f6ad0dc3978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:48:29.419 7f6ad0dc3978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:33.417 7f6ad0dc3978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:48:33.419 7f6ad0dc3978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:48:33.419 7f6ad0dc3978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:48:33.419 7f6ad0dc3978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:33.419 7f6ad0dc3978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:48:33.419 7f6ad0dc3978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:33.419 7f6ad0dc3978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:48:33.419 7f6ad0dc3978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:48:33.420 7f6ad0dc3978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:33.420 7f6ad0dc3978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:48:33.420 7f6ad0dc3978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:33.420 7f6ad0dc3978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:33.420 7f6ad0dc3978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.12:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:48:33.420 7f6ad0dc3978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:37.201 7f6ad0daa048 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2923-3.'], 'obj_dict': {'database_node_ip_address': '10.0.0.47', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2923-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2923-3.'], 'id_perms': {'created': '2025-03-20T00:48:37.183071', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-03-20T00:48:37.183071', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13640881405911367670, 'uuid_mslong': 3030180836561407535}}, 'parent_type': 'global-system-config', 'parent_uuid': '92e769a6-5754-45f5-bd68-145125efbf0e', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '2a0d5d6a-9070-4e2f-bd4e-255dbe93fff6'}, 'oper': 'CREATE', 'request-id': 'req-cb2fe499-f92e-4f2e-bc7c-9c2dedc5cd19', 'type': 'database_node', 'uuid': '2a0d5d6a-9070-4e2f-bd4e-255dbe93fff6'} 03/20/2025 00:48:37.244 7f6ad0daa048 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2923-2.'], 'obj_dict': {'database_node_ip_address': '10.0.0.12', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2923-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2923-2.'], 'id_perms': {'created': '2025-03-20T00:48:37.229836', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-03-20T00:48:37.229836', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11755233971675740796, 'uuid_mslong': 3245297702601836802}}, 'parent_type': 'global-system-config', 'parent_uuid': '92e769a6-5754-45f5-bd68-145125efbf0e', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '2d099d10-f1f9-4902-a322-fb03ed563a7c'}, 'oper': 'CREATE', 'request-id': 'req-de15b2ad-dc51-4a18-983e-1077f4c853b2', 'type': 'database_node', 'uuid': '2d099d10-f1f9-4902-a322-fb03ed563a7c'} 03/20/2025 00:48:37.339 7f6ad0daa048 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2923-1.'], 'obj_dict': {'database_node_ip_address': '10.0.0.42', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2923-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2923-1.'], 'id_perms': {'created': '2025-03-20T00:48:37.324100', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-03-20T00:48:37.324100', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12834287694014244404, 'uuid_mslong': 17976374462402216555}}, 'parent_type': 'global-system-config', 'parent_uuid': '92e769a6-5754-45f5-bd68-145125efbf0e', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'f978e954-a4e3-426b-b21c-8ca8a2e2e634'}, 'oper': 'CREATE', 'request-id': 'req-a3532bb0-c6aa-4be7-9d94-0fda2a74ad53', 'type': 'database_node', 'uuid': 'f978e954-a4e3-426b-b21c-8ca8a2e2e634'} 03/20/2025 00:48:37.419 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:48:37.420 7f6ad0ff88d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:48:37.420 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:48:37.420 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:37.421 7f6ad0ff88d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:48:37.421 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:37.421 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:48:37.421 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:48:37.421 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:37.421 7f6ad0ff88d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:48:37.421 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:37.421 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:37.421 7f6ad0ff88d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.47:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:48:37.421 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:41.422 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:48:41.423 7f6ad0ff88d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:48:41.423 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:48:41.423 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:41.423 7f6ad0ff88d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:48:41.423 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:41.423 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:48:41.424 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:48:41.424 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:41.424 7f6ad0ff88d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:48:41.424 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:41.424 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:41.424 7f6ad0ff88d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:48:41.424 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:45.424 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:48:45.425 7f6ad0ff88d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:48:45.425 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:48:45.425 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:45.425 7f6ad0ff88d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:48:45.425 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:45.425 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:48:45.426 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:48:45.426 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:45.426 7f6ad0ff88d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:48:45.426 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:45.426 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:45.426 7f6ad0ff88d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.12:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:48:45.426 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:49.426 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:48:49.427 7f6ad0ff88d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:48:49.428 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:48:49.428 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:49.428 7f6ad0ff88d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:48:49.428 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:49.428 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:48:49.428 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:48:49.428 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:49.428 7f6ad0ff88d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:48:49.428 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:49.429 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:49.429 7f6ad0ff88d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.47:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:48:49.429 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:53.429 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:48:53.431 7f6ad0ff88d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:48:53.431 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:48:53.431 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:53.431 7f6ad0ff88d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:48:53.431 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:53.431 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:48:53.432 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:48:53.432 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:53.432 7f6ad0ff88d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:48:53.432 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:53.432 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:53.432 7f6ad0ff88d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:48:53.432 7f6ad0ff88d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:57.432 7f6ad0db9240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:48:57.433 7f6ad0db9240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:48:57.433 7f6ad0db9240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:48:57.433 7f6ad0db9240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:57.434 7f6ad0db9240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:48:57.434 7f6ad0db9240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:57.434 7f6ad0db9240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:48:57.434 7f6ad0db9240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:48:57.434 7f6ad0db9240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:57.434 7f6ad0db9240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:48:57.434 7f6ad0db9240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:57.434 7f6ad0db9240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:57.434 7f6ad0db9240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.12:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:48:57.434 7f6ad0db9240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:01.434 7f6ad0db9240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:49:01.436 7f6ad0db9240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:49:01.436 7f6ad0db9240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:49:01.436 7f6ad0db9240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:49:01.436 7f6ad0db9240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:49:01.436 7f6ad0db9240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:49:01.436 7f6ad0db9240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:49:01.437 7f6ad0db9240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:49:01.437 7f6ad0db9240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:01.437 7f6ad0db9240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:49:01.437 7f6ad0db9240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:01.437 7f6ad0db9240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:01.437 7f6ad0db9240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.47:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:49:01.437 7f6ad0db9240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:05.436 7f6ad0db9240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:49:05.437 7f6ad0db9240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:49:05.438 7f6ad0db9240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:49:05.438 7f6ad0db9240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:49:05.438 7f6ad0db9240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:49:05.438 7f6ad0db9240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:49:05.438 7f6ad0db9240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:49:05.438 7f6ad0db9240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:49:05.438 7f6ad0db9240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:05.438 7f6ad0db9240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:49:05.438 7f6ad0db9240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:05.439 7f6ad0db9240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:05.439 7f6ad0db9240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:49:05.439 7f6ad0db9240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:09.438 7f6ad0db9240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:49:09.439 7f6ad0db9240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:49:09.440 7f6ad0db9240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:49:09.440 7f6ad0db9240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:49:09.440 7f6ad0db9240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:49:09.440 7f6ad0db9240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:49:09.440 7f6ad0db9240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:49:09.440 7f6ad0db9240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:49:09.440 7f6ad0db9240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:09.440 7f6ad0db9240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:49:09.440 7f6ad0db9240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:09.441 7f6ad0db9240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:09.441 7f6ad0db9240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.12:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:49:09.441 7f6ad0db9240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:13.440 7f6ad0db9240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:49:13.442 7f6ad0db9240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:49:13.442 7f6ad0db9240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:49:13.442 7f6ad0db9240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:49:13.442 7f6ad0db9240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:49:13.442 7f6ad0db9240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:49:13.443 7f6ad0db9240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:49:13.443 7f6ad0db9240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:49:13.443 7f6ad0db9240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:13.443 7f6ad0db9240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:49:13.443 7f6ad0db9240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:13.443 7f6ad0db9240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:13.443 7f6ad0db9240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.47:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:49:13.443 7f6ad0db9240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:17.443 7f6ad0d55780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:49:17.445 7f6ad0d55780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:49:17.445 7f6ad0d55780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:49:17.445 7f6ad0d55780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:49:17.445 7f6ad0d55780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:49:17.445 7f6ad0d55780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:49:17.445 7f6ad0d55780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:49:17.446 7f6ad0d55780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:49:17.446 7f6ad0d55780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:17.446 7f6ad0d55780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:49:17.446 7f6ad0d55780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:17.446 7f6ad0d55780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:17.446 7f6ad0d55780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:49:17.446 7f6ad0d55780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:21.445 7f6ad0e6f198 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:49:21.446 7f6ad0e6f198 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:49:21.446 7f6ad0e6f198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:49:21.446 7f6ad0e6f198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:49:21.447 7f6ad0e6f198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:49:21.447 7f6ad0e6f198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:49:21.447 7f6ad0e6f198 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:49:21.447 7f6ad0e6f198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:49:21.447 7f6ad0e6f198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:21.447 7f6ad0e6f198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:49:21.447 7f6ad0e6f198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:21.447 7f6ad0e6f198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:21.447 7f6ad0e6f198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.12:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:49:21.447 7f6ad0e6f198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:25.447 7f6ad0e6f198 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:49:25.499 7f6ad0e6f198 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:49:25.499 7f6ad0e6f198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:49:25.499 7f6ad0e6f198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:49:25.499 7f6ad0e6f198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:49:25.499 7f6ad0e6f198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:49:25.499 7f6ad0e6f198 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:49:25.500 7f6ad0e6f198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:49:25.500 7f6ad0e6f198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:25.500 7f6ad0e6f198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:49:25.500 7f6ad0e6f198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:25.500 7f6ad0e6f198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:25.500 7f6ad0e6f198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 138 http_port = 8096 start_time = 1742431549311970 collector_name = collector_ip = 10.0.0.47:8086 collector_list = [ 10.0.0.42:8086, 10.0.0.12:8086, 10.0.0.47:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:49:25.500 7f6ad0e6f198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:28.251 7f6ad0daa048 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2923-2.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.12', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2923-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2923-2.'], 'id_perms': {'created': '2025-03-20T00:49:28.231460', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-03-20T00:49:28.231460', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11672651625846619010, 'uuid_mslong': 11061318134347023244}}, 'parent_type': 'global-system-config', 'parent_uuid': '92e769a6-5754-45f5-bd68-145125efbf0e', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '9981b23c-e157-438c-a1fd-96cf31793b82'}, 'oper': 'CREATE', 'request-id': 'req-10a978aa-b68b-462e-9664-e8a54ae0d277', 'type': 'analytics_node', 'uuid': '9981b23c-e157-438c-a1fd-96cf31793b82'} 03/20/2025 00:49:28.361 7f6ad0daa048 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2923-1.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.42', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2923-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2923-1.'], 'id_perms': {'created': '2025-03-20T00:49:28.342327', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-03-20T00:49:28.342327', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11271866916915666331, 'uuid_mslong': 2951203326535353567}}, 'parent_type': 'global-system-config', 'parent_uuid': '92e769a6-5754-45f5-bd68-145125efbf0e', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '28f4c7ca-1217-44df-9c6d-b73d723c5d9b'}, 'oper': 'CREATE', 'request-id': 'req-188a9cc7-7cd0-43a4-8daf-57685456e460', 'type': 'analytics_node', 'uuid': '28f4c7ca-1217-44df-9c6d-b73d723c5d9b'} 03/20/2025 00:49:28.448 7f6ad0daa048 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2923-3.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.47', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2923-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2923-3.'], 'id_perms': {'created': '2025-03-20T00:49:28.423018', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-03-20T00:49:28.423018', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11338432313094054592, 'uuid_mslong': 2586010924020680352}}, 'parent_type': 'global-system-config', 'parent_uuid': '92e769a6-5754-45f5-bd68-145125efbf0e', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '23e35b3b-ef88-46a0-9d5a-341d7bc822c0'}, 'oper': 'CREATE', 'request-id': 'req-40f4421b-09e6-4058-aac6-9fee94b0c38f', 'type': 'analytics_node', 'uuid': '23e35b3b-ef88-46a0-9d5a-341d7bc822c0'} 03/20/2025 00:49:29.500 7f6ad30a5780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:49:29.501 7f6ad30a5780 [contrail-device-manager] [INFO]: Session Event: TCP Connected 03/20/2025 00:49:29.501 7f6ad30a5780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:49:29.501 7f6ad30a5780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:49:29.501 7f6ad30a5780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-2. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.42:5673, 10.0.0.12:5673, 10.0.0.47:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.42:9041, 10.0.0.12:9041, 10.0.0.47:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.42:2181, 10.0.0.12:2181, 10.0.0.47:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.42,10.0.0.12,10.0.0.47:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:49:29.502 7f6ad30a5780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:49:29.502 7f6ad30a5780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/20/2025 00:49:29.502 7f6ad30a5780 [contrail-device-manager] [DEBUG]: Send sandesh control message. uve type count # 19 03/20/2025 00:49:29.502 7f6ad30a5780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/20/2025 00:49:29.502 7f6ad30a5780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/20/2025 00:49:29.502 7f6ad30a5780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/20/2025 00:49:29.502 7f6ad0d3fac8 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 03/20/2025 00:49:29.504 7f6ad0d3fac8 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 03/20/2025 00:49:38.100 7f6ad0d55438 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001016 03/20/2025 00:49:38.100 7f6ad0d55438 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 03/20/2025 00:49:38.101 7f6ad0deef60 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 03/20/2025 00:49:38.101 7f6ad0deef60 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 0 03/20/2025 00:49:38.101 7f6ad0deef60 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/20/2025 00:49:38.101 7f6ad0deef60 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/20/2025 00:49:38.102 7f6ad0deef60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 03/20/2025 00:49:38.103 7f6ad0dce0f0 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 03/20/2025 00:49:38.104 7f6ad0dce0f0 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 03/20/2025 00:49:48.616 7f6ad0daa048 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2923-1.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.42', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2923-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2923-1.'], 'id_perms': {'created': '2025-03-20T00:49:48.605380', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-03-20T00:49:48.605380', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13658454429170963811, 'uuid_mslong': 4565978857557151589}}, 'parent_type': 'global-system-config', 'parent_uuid': '92e769a6-5754-45f5-bd68-145125efbf0e', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '3f5d9d94-50a3-4f65-bd8c-93f00f989563'}, 'oper': 'CREATE', 'request-id': 'req-530ea9f4-208f-4297-93ee-eb15dece8567', 'type': 'analytics_alarm_node', 'uuid': '3f5d9d94-50a3-4f65-bd8c-93f00f989563'} 03/20/2025 00:49:48.711 7f6ad0daa048 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2923-2.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.12', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2923-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2923-2.'], 'id_perms': {'created': '2025-03-20T00:49:48.681609', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-03-20T00:49:48.681609', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12401597909814865341, 'uuid_mslong': 4006576900372647049}}, 'parent_type': 'global-system-config', 'parent_uuid': '92e769a6-5754-45f5-bd68-145125efbf0e', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '379a3876-8fff-4c89-ac1b-5397adbdedbd'}, 'oper': 'CREATE', 'request-id': 'req-63fd8c15-0cf7-479d-90ef-1d5aa999f8f1', 'type': 'analytics_alarm_node', 'uuid': '379a3876-8fff-4c89-ac1b-5397adbdedbd'} 03/20/2025 00:49:48.744 7f6ad0daa048 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2923-3.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.47', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2923-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2923-3.'], 'id_perms': {'created': '2025-03-20T00:49:48.730673', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-03-20T00:49:48.730673', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10925185346693394117, 'uuid_mslong': 16581986432498288668}}, 'parent_type': 'global-system-config', 'parent_uuid': '92e769a6-5754-45f5-bd68-145125efbf0e', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'e61f0cce-2368-481c-979e-0e304ef66ec5'}, 'oper': 'CREATE', 'request-id': 'req-5e922a80-8bbd-470a-82dd-99ac6d8bcf22', 'type': 'analytics_alarm_node', 'uuid': 'e61f0cce-2368-481c-979e-0e304ef66ec5'} 03/20/2025 00:49:50.647 7f6ad0d55438 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001297 03/20/2025 00:49:50.647 7f6ad0d55438 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 03/20/2025 00:49:50.648 7f6ad7db9eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 03/20/2025 00:49:50.648 7f6ad7db9eb8 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 2 03/20/2025 00:49:50.648 7f6ad7db9eb8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/20/2025 00:49:50.648 7f6ad7db9eb8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/20/2025 00:50:02.790 7f6ad0daa048 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2923-2.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.12', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2923-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2923-2.'], 'id_perms': {'created': '2025-03-20T00:50:02.777415', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-03-20T00:50:02.777415', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10617362835679415336, 'uuid_mslong': 11882705669797989828}}, 'parent_type': 'global-system-config', 'parent_uuid': '92e769a6-5754-45f5-bd68-145125efbf0e', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'a4e7d9d9-72d4-4dc4-9358-733ef01b0c28'}, 'oper': 'CREATE', 'request-id': 'req-802d163d-5e8a-4aa1-961f-ea58a5e17404', 'type': 'analytics_snmp_node', 'uuid': 'a4e7d9d9-72d4-4dc4-9358-733ef01b0c28'} 03/20/2025 00:50:02.905 7f6ad0daa048 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2923-3.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.47', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2923-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2923-3.'], 'id_perms': {'created': '2025-03-20T00:50:02.886669', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-03-20T00:50:02.886669', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13232706780590423501, 'uuid_mslong': 10112174715567752156}}, 'parent_type': 'global-system-config', 'parent_uuid': '92e769a6-5754-45f5-bd68-145125efbf0e', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '8c55a953-c41a-43dc-b7a4-04b501febdcd'}, 'oper': 'CREATE', 'request-id': 'req-2a267fa5-c990-4ec2-9228-42e704a4e9c2', 'type': 'analytics_snmp_node', 'uuid': '8c55a953-c41a-43dc-b7a4-04b501febdcd'} 03/20/2025 00:50:02.954 7f6ad0daa048 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2923-1.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.42', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-2923-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-2923-1.'], 'id_perms': {'created': '2025-03-20T00:50:02.941273', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-03-20T00:50:02.941273', 'permissions': {'group': 'member', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9728461416749403925, 'uuid_mslong': 5487816183473718790}}, 'parent_type': 'global-system-config', 'parent_uuid': '92e769a6-5754-45f5-bd68-145125efbf0e', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '4c28a3be-4097-4606-8702-701d6c710f15'}, 'oper': 'CREATE', 'request-id': 'req-b92d2174-569f-4a69-b699-eba9d330b2ee', 'type': 'analytics_snmp_node', 'uuid': '4c28a3be-4097-4606-8702-701d6c710f15'} 03/20/2025 00:51:59.966 7f6ad0daa048 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-984a1923-f84b-4004-b379-9cd96cece1f2', 'type': 'global_system_config', 'uuid': '92e769a6-5754-45f5-bd68-145125efbf0e'} 03/20/2025 00:52:00.052 7f6ad0daa048 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-4f2b1bb3-d714-48ba-bbc7-456f6e58f1eb', 'type': 'global_system_config', 'uuid': '92e769a6-5754-45f5-bd68-145125efbf0e'} 03/20/2025 00:52:00.108 7f6ad0daa048 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-2923-2'], 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-2923-2', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-2923-2'], 'id_perms': {'created': '2025-03-20T00:52:00.100651', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-03-20T00:52:00.100651', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13175873738772514205, 'uuid_mslong': 13685377141349435613}}, 'parent_type': 'global-system-config', 'parent_uuid': '92e769a6-5754-45f5-bd68-145125efbf0e', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'bdec3a00-7762-48dd-b6da-1b5b4cef059d', 'virtual_router_dpdk_enabled': False, 'virtual_router_ip_address': '10.20.0.14'}, 'oper': 'CREATE', 'request-id': 'req-d6d43db3-6d26-4f59-8c69-ed7fca1c0fcb', 'type': 'virtual_router', 'uuid': 'bdec3a00-7762-48dd-b6da-1b5b4cef059d'} 03/20/2025 00:52:00.145 7f6ad0daa048 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-2923-1'], 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-2923-1', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-2923-1'], 'id_perms': {'created': '2025-03-20T00:52:00.137485', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-03-20T00:52:00.137485', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12676109258359580077, 'uuid_mslong': 17961036342243706285}}, 'parent_type': 'global-system-config', 'parent_uuid': '92e769a6-5754-45f5-bd68-145125efbf0e', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'f9426b64-4151-45ad-afea-9635a69025ad', 'virtual_router_dpdk_enabled': False, 'virtual_router_ip_address': '10.20.0.12'}, 'oper': 'CREATE', 'request-id': 'req-0b7cf66f-65ff-4828-8b26-affdf35cd08b', 'type': 'virtual_router', 'uuid': 'f9426b64-4151-45ad-afea-9635a69025ad'} 03/20/2025 00:52:00.475 7f6ad0daa048 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-2923-2', 'vhost0'], 'obj_dict': {'display_name': 'vhost0', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-2923-2', 'vhost0'], 'id_perms': {'created': '2025-03-20T00:52:00.449759', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-03-20T00:52:00.449759', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9297926490923739149, 'uuid_mslong': 13754707928961011415}}, 'parent_type': 'virtual-router', 'parent_uuid': 'bdec3a00-7762-48dd-b6da-1b5b4cef059d', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'bee289fc-5ef8-4ed7-8108-dee18734280d', 'virtual_machine_interface_disable_policy': True, 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:be:e2:89:fc:5e']}, 'virtual_network_refs': [{'to': ['default-domain', 'default-project', 'ip-fabric'], 'uuid': '25e81aa2-d36c-4b63-8506-67606ce5b850'}]}, 'oper': 'CREATE', 'request-id': 'req-1dc098ff-d058-4683-bf61-974472c54893', 'type': 'virtual_machine_interface', 'uuid': 'bee289fc-5ef8-4ed7-8108-dee18734280d'} 03/20/2025 00:52:00.517 7f6ad0daa048 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-2923-2', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-1dc098ff-d058-4683-bf61-974472c54893', 'type': 'virtual_machine_interface', 'uuid': 'bee289fc-5ef8-4ed7-8108-dee18734280d'} 03/20/2025 00:52:00.542 7f6ad0daa048 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-2923-1', 'vhost0'], 'obj_dict': {'display_name': 'vhost0', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-2923-1', 'vhost0'], 'id_perms': {'created': '2025-03-20T00:52:00.499107', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-03-20T00:52:00.499107', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13779479366413766183, 'uuid_mslong': 11085803862912486831}}, 'parent_type': 'virtual-router', 'parent_uuid': 'f9426b64-4151-45ad-afea-9635a69025ad', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '99d8afe0-d28d-4daf-bf3a-8b7a18eace27', 'virtual_machine_interface_disable_policy': True, 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:99:d8:af:e0:d2']}, 'virtual_network_refs': [{'to': ['default-domain', 'default-project', 'ip-fabric'], 'uuid': '25e81aa2-d36c-4b63-8506-67606ce5b850'}]}, 'oper': 'CREATE', 'request-id': 'req-33c88b99-65db-4d70-9153-30837a111f59', 'type': 'virtual_machine_interface', 'uuid': '99d8afe0-d28d-4daf-bf3a-8b7a18eace27'} 03/20/2025 00:52:00.605 7f6ad0daa048 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-2923-1', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-33c88b99-65db-4d70-9153-30837a111f59', 'type': 'virtual_machine_interface', 'uuid': '99d8afe0-d28d-4daf-bf3a-8b7a18eace27'} 03/20/2025 00:52:01.046 7f6ad0daa048 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'oper': 'UPDATE', 'request-id': 'req-3df90d24-2981-4c26-8724-ab618b878130', 'type': 'access_control_list', 'uuid': 'df83a9c3-866e-4ef4-8457-9d8f1ab72fe8'} 03/20/2025 00:52:16.204 7f6ad0ddd2e8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/20/2025 00:52:59.192 7f6ad3039630 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map