06/12/2025 00:42:45.516 7f9516d0a348 [contrail-device-manager] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 06/12/2025 00:42:45.516 7f9516d0a348 [contrail-device-manager] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 06/12/2025 00:42:45.516 7f9516d0a348 [contrail-device-manager] [INFO]: SANDESH: Flow Logging: [None] -> [False] 06/12/2025 00:42:45.517 7f950e3f02e8 [contrail-device-manager] [INFO]: Processing event[EvStart] in state[none] 06/12/2025 00:42:45.517 7f950e3f02e8 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectConfigNode:NodeStatusUVE] map 06/12/2025 00:42:45.517 7f950e3f02e8 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 06/12/2025 00:42:45.517 7f950e3f02e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 06/12/2025 00:42:45.517 7f950e3f02e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:45.518 7f950e3f02e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 06/12/2025 00:42:45.518 7f950e3f02e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:45.518 7f950e3f02e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:45.518 7f950e3f02e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 135 start_time = 1749688965517660 collector_name = collector_ip = collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.19:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 06/12/2025 00:42:45.518 7f950e3f02e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:45.560 7f9516d0a348 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.19:5673//', 'pyamqp://guest:guest@10.0.0.41:5673//', 'pyamqp://guest:guest@10.0.0.61:5673//'] 06/12/2025 00:42:45.569 7f9516d0a348 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 06/12/2025 00:42:45.570 7f950e242588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:45.570 7f950e242588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 06/12/2025 00:42:45.570 7f950e242588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:45.570 7f950e242588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:45.570 7f950e242588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 06/12/2025 00:42:45.570 7f950e242588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:45.580 7f9516d0a348 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 06/12/2025 00:42:45.589 7f950e242588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:45.590 7f950e242588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 06/12/2025 00:42:45.590 7f950e242588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:45.627 7f9516d0a348 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 06/12/2025 00:42:45.627 7f950e242c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:45.627 7f950e242c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 06/12/2025 00:42:45.628 7f950e242c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:45.631 7f9516d0a348 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 06/12/2025 00:42:45.631 7f950e242588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:45.631 7f950e242588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 06/12/2025 00:42:45.631 7f950e242588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:45.654 7f9516d0a348 [contrail-device-manager] [DEBUG]: Removed remaining AMQP queue from previous run 06/12/2025 00:42:45.654 7f9516d0a348 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 06/12/2025 00:42:45.655 7f9516d0a348 [contrail-device-manager] [WARNING]: initializing cassandra cluster 06/12/2025 00:42:45.655 7f9516d0a348 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 06/12/2025 00:42:45.655 7f950e2690f0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting consumer greenlet 06/12/2025 00:42:45.655 7f950e2690f0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 06/12/2025 00:42:45.656 7f9510b91978 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting publisher greenlet 06/12/2025 00:42:45.656 7f9510b91978 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Publisher connection 06/12/2025 00:42:45.656 7f950e1d9048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:45.656 7f950e1d9048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 06/12/2025 00:42:45.656 7f950e1d9048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:45.657 7f9516d0a348 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 06/12/2025 00:42:45.663 7f9510b91978 [contrail-device-manager] [INFO]: KombuAmqpClient: Publisher connection established 06/12/2025 00:42:45.663 7f950e2690f0 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 06/12/2025 00:42:45.663 7f950e2690f0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Waiting for consumer 06/12/2025 00:42:45.729 7f9516d0a348 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 06/12/2025 00:42:45.730 7f950e136438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:45.730 7f950e136438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 06/12/2025 00:42:45.730 7f950e136438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:45.736 7f9516d0a348 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 06/12/2025 00:42:45.736 7f950e132198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:45.736 7f950e132198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 06/12/2025 00:42:45.737 7f950e132198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:45.744 7f9516d0a348 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 06/12/2025 00:42:45.776 7f9516d0a348 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 06/12/2025 00:42:46.366 7f9516d0a348 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/12/2025 00:42:47.092 7f9516d0a348 [contrail-device-manager] [WARNING]: table 'obj_uuid_table', created 06/12/2025 00:42:47.097 7f9516d0a348 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/12/2025 00:42:48.935 7f9516d0a348 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/12/2025 00:42:49.518 7f950e0d7438 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/12/2025 00:42:49.519 7f950e0d7438 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/12/2025 00:42:49.519 7f950e0d7438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/12/2025 00:42:49.519 7f950e0d7438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:42:49.519 7f950e0d7438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 06/12/2025 00:42:49.519 7f950e0d7438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:42:49.520 7f950e0d7438 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/12/2025 00:42:49.520 7f950e0d7438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/12/2025 00:42:49.520 7f950e0d7438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:49.520 7f950e0d7438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 06/12/2025 00:42:49.520 7f950e0d7438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:49.520 7f950e0d7438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:49.520 7f950e0d7438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 135 start_time = 1749688965517660 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.19:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/12/2025 00:42:49.520 7f950e0d7438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:49.832 7f9516d0a348 [contrail-device-manager] [WARNING]: table 'obj_uuid_table' fixed with properties {'gc_grace_seconds': 864000} 06/12/2025 00:42:49.837 7f9516d0a348 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/12/2025 00:42:51.411 7f9516d0a348 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/12/2025 00:42:52.149 7f9516d0a348 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table', created 06/12/2025 00:42:52.155 7f9516d0a348 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/12/2025 00:42:53.140 7f9516d0a348 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/12/2025 00:42:53.520 7f950efa2a20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/12/2025 00:42:53.521 7f950efa2a20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/12/2025 00:42:53.522 7f950efa2a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/12/2025 00:42:53.522 7f950efa2a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:42:53.522 7f950efa2a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 06/12/2025 00:42:53.522 7f950efa2a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:42:53.522 7f950efa2a20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/12/2025 00:42:53.522 7f950efa2a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/12/2025 00:42:53.522 7f950efa2a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:53.522 7f950efa2a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 06/12/2025 00:42:53.522 7f950efa2a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:53.522 7f950efa2a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:53.523 7f950efa2a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 135 start_time = 1749688965517660 collector_name = collector_ip = 10.0.0.61:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.19:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/12/2025 00:42:53.523 7f950efa2a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:54.081 7f9516d0a348 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table' fixed with properties {'gc_grace_seconds': 864000} 06/12/2025 00:42:54.087 7f9516d0a348 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/12/2025 00:42:55.292 7f9516d0a348 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/12/2025 00:42:56.441 7f9516d0a348 [contrail-device-manager] [WARNING]: table 'obj_shared_table', created 06/12/2025 00:42:56.446 7f9516d0a348 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/12/2025 00:42:57.179 7f9516d0a348 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/12/2025 00:42:57.522 7f950ef8e4e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/12/2025 00:42:57.523 7f950ef8e4e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/12/2025 00:42:57.523 7f950ef8e4e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/12/2025 00:42:57.523 7f950ef8e4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:42:57.523 7f950ef8e4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 06/12/2025 00:42:57.523 7f950ef8e4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:42:57.523 7f950ef8e4e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/12/2025 00:42:57.524 7f950ef8e4e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/12/2025 00:42:57.524 7f950ef8e4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:57.524 7f950ef8e4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 06/12/2025 00:42:57.524 7f950ef8e4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:57.524 7f950ef8e4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:57.524 7f950ef8e4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 135 start_time = 1749688965517660 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.19:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/12/2025 00:42:57.524 7f950ef8e4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:58.076 7f9516d0a348 [contrail-device-manager] [WARNING]: table 'obj_shared_table' fixed with properties {'gc_grace_seconds': 864000} 06/12/2025 00:42:58.081 7f9516d0a348 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/12/2025 00:42:58.081 7f950ef8e4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:58.082 7f950ef8e4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/12/2025 00:42:58.082 7f950ef8e4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:58.108 7f9516d0a348 [contrail-device-manager] [ERROR]: Error in db walk read 'type' 06/12/2025 00:42:58.108 7f950e0d1438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:58.109 7f950e0d1438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Zookeeper:Zookeeper[] connection down >>, ] >> 06/12/2025 00:42:58.109 7f950e0d1438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:58.120 7f950e0c8eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:58.121 7f950e0c8eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/12/2025 00:42:58.121 7f950e0c8eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:58.121 7f9516d0a348 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_status_consumer.dummy 06/12/2025 00:42:58.122 7f9516d0a348 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_request_consumer 06/12/2025 00:42:58.122 7f950e2690f0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 06/12/2025 00:42:58.127 7f9516d0a348 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3446-1. 06/12/2025 00:42:58.132 7f950e2690f0 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 06/12/2025 00:42:58.260 7f950e2690f0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3446-1.'] 06/12/2025 00:42:58.305 7f950e2690f0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 06/12/2025 00:42:58.311 7f950e2690f0 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 06/12/2025 00:42:58.353 7f950e2690f0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3446-1.'] 06/12/2025 00:42:58.639 7f9516d0a348 [contrail-device-manager] [DEBUG]: Fetching all containers 06/12/2025 00:42:58.755 7f9516d0a348 [contrail-device-manager] [INFO]: Restarting dnsmasq docker: config_dnsmasq_1 06/12/2025 00:42:59.141 7f9516d0a348 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3446-1..config_queue 06/12/2025 00:42:59.141 7f9516d0a348 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3446-1..tftp_queue 06/12/2025 00:42:59.402 7f950e2690f0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 06/12/2025 00:42:59.415 7f950e2690f0 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 06/12/2025 00:42:59.606 7f950e2690f0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3446-1.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3446-1..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3446-1..tftp_queue'] 06/12/2025 00:42:59.642 7f9516d0a348 [contrail-device-manager] [WARNING]: Process 135 prepared to run in Full mode... 06/12/2025 00:42:59.642 7f9516d0a348 [contrail-device-manager] [WARNING]: Elected master Device Manager node. Initializing... 06/12/2025 00:42:59.642 7f9516d0a348 [contrail-device-manager] [INFO]: SANDESH: INTROSPECT IS ON: 0.0.0.0:8096 06/12/2025 00:42:59.645 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add [config_updater.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/config_updater] to home page 06/12/2025 00:42:59.645 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_ConfigApiUpdateReq] 06/12/2025 00:42:59.646 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add [greenlets.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/greenlets] to home page 06/12/2025 00:42:59.646 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_GreenletObjectReq] 06/12/2025 00:42:59.647 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add [cpuinfo.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/nodeinfo/cpuinfo] to home page 06/12/2025 00:42:59.647 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CpuLoadInfoReq] 06/12/2025 00:42:59.651 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add [dm_introspect.xml:/usr/lib/python3.6/site-packages/device_manager/sandesh/dm_introspect] to home page 06/12/2025 00:42:59.651 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_RoutingInstanceList] 06/12/2025 00:42:59.651 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualNetworkList] 06/12/2025 00:42:59.651 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualMachineInterfaceList] 06/12/2025 00:42:59.651 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalInterfaceList] 06/12/2025 00:42:59.651 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalInterfaceList] 06/12/2025 00:42:59.651 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalRouterList] 06/12/2025 00:42:59.652 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_BgpRouterList] 06/12/2025 00:42:59.652 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalRouterList] 06/12/2025 00:42:59.652 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_InstanceIpList] 06/12/2025 00:42:59.652 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FloatingIpList] 06/12/2025 00:42:59.652 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PortTupleList] 06/12/2025 00:42:59.652 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FeatureFlagList] 06/12/2025 00:42:59.653 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add [sandesh_alarm.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_alarm] to home page 06/12/2025 00:42:59.653 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmCacheRequest] 06/12/2025 00:42:59.653 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmTypesRequest] 06/12/2025 00:42:59.653 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmAckRequest] 06/12/2025 00:42:59.655 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add [sandesh_ctrl.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_ctrl] to home page 06/12/2025 00:42:59.655 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlClientToServer] 06/12/2025 00:42:59.655 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlServerToClient] 06/12/2025 00:42:59.656 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add [sandesh_trace.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_trace] to home page 06/12/2025 00:42:59.656 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceRequest] 06/12/2025 00:42:59.656 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferListRequest] 06/12/2025 00:42:59.656 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceEnableDisableReq] 06/12/2025 00:42:59.656 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufStatusReq] 06/12/2025 00:42:59.656 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferEnableDisableReq] 06/12/2025 00:42:59.661 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add [sandesh_uve.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_uve] to home page 06/12/2025 00:42:59.661 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshMessageStatsReq] 06/12/2025 00:42:59.661 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsSet] 06/12/2025 00:42:59.661 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsStatus] 06/12/2025 00:42:59.661 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsSet] 06/12/2025 00:42:59.661 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsStatus] 06/12/2025 00:42:59.662 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsSet] 06/12/2025 00:42:59.662 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsReset] 06/12/2025 00:42:59.662 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsStatus] 06/12/2025 00:42:59.662 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueSet] 06/12/2025 00:42:59.662 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueStatus] 06/12/2025 00:42:59.662 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CollectorInfoRequest] 06/12/2025 00:42:59.662 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVEDSConfReq] 06/12/2025 00:42:59.662 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVECacheReq] 06/12/2025 00:42:59.662 7f9516d0a348 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVETypesReq] 06/12/2025 00:42:59.665 7f9516d0a348 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.ztp_queue 06/12/2025 00:42:59.669 7f950d718ac8 [contrail-device-manager] [ERROR]: Cannot write http_port 8096 to /tmp/contrail-device-manager.1.http_port 06/12/2025 00:42:59.669 7f950d718ac8 [contrail-device-manager] [ERROR]: Starting Introspect on HTTP Port 8096 06/12/2025 00:42:59.720 7f950d6fee10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:59.720 7f950d6fee10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Initializing description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 06/12/2025 00:42:59.720 7f950d6fee10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:59.729 7f950ced5d68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:42:59.730 7f950ced5d68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 06/12/2025 00:42:59.730 7f950ced5d68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:00.635 7f950e2690f0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 06/12/2025 00:43:00.640 7f950e2690f0 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 06/12/2025 00:43:00.755 7f950e2690f0 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3446-1.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3446-1..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3446-1..tftp_queue', 'device_manager_ztp.ztp_queue'] 06/12/2025 00:43:01.524 7f950ced0cc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/12/2025 00:43:01.525 7f950ced0cc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/12/2025 00:43:01.525 7f950ced0cc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/12/2025 00:43:01.525 7f950ced0cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:43:01.525 7f950ced0cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 06/12/2025 00:43:01.526 7f950ced0cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:43:01.526 7f950ced0cc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/12/2025 00:43:01.526 7f950ced0cc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/12/2025 00:43:01.526 7f950ced0cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:01.526 7f950ced0cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 06/12/2025 00:43:01.526 7f950ced0cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:01.526 7f950ced0cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:01.526 7f950ced0cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 135 http_port = 8096 start_time = 1749688965517660 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.19:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/12/2025 00:43:01.526 7f950ced0cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:05.526 7f950cee7198 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/12/2025 00:43:05.528 7f950cee7198 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/12/2025 00:43:05.528 7f950cee7198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/12/2025 00:43:05.528 7f950cee7198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:43:05.528 7f950cee7198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 06/12/2025 00:43:05.528 7f950cee7198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:43:05.528 7f950cee7198 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/12/2025 00:43:05.529 7f950cee7198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/12/2025 00:43:05.529 7f950cee7198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:05.529 7f950cee7198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 06/12/2025 00:43:05.529 7f950cee7198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:05.529 7f950cee7198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:05.529 7f950cee7198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 135 http_port = 8096 start_time = 1749688965517660 collector_name = collector_ip = 10.0.0.61:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.19:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/12/2025 00:43:05.529 7f950cee7198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:05.853 7f950cee7438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:05.853 7f950cee7438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/12/2025 00:43:05.853 7f950cee7438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:09.528 7f950e0f46d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/12/2025 00:43:09.529 7f950e0f46d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/12/2025 00:43:09.529 7f950e0f46d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/12/2025 00:43:09.529 7f950e0f46d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:43:09.530 7f950e0f46d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/12/2025 00:43:09.530 7f950e0f46d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:43:09.530 7f950e0f46d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/12/2025 00:43:09.530 7f950e0f46d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/12/2025 00:43:09.530 7f950e0f46d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:09.530 7f950e0f46d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/12/2025 00:43:09.531 7f950e0f46d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:09.531 7f950e0f46d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:09.531 7f950e0f46d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 135 http_port = 8096 start_time = 1749688965517660 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.19:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/12/2025 00:43:09.531 7f950e0f46d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:13.531 7f950cd1d978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/12/2025 00:43:13.532 7f950cd1d978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/12/2025 00:43:13.532 7f950cd1d978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/12/2025 00:43:13.532 7f950cd1d978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:43:13.533 7f950cd1d978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/12/2025 00:43:13.533 7f950cd1d978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:43:13.533 7f950cd1d978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/12/2025 00:43:13.533 7f950cd1d978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/12/2025 00:43:13.533 7f950cd1d978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:13.533 7f950cd1d978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/12/2025 00:43:13.533 7f950cd1d978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:13.533 7f950cd1d978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:13.533 7f950cd1d978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 135 http_port = 8096 start_time = 1749688965517660 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.19:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/12/2025 00:43:13.534 7f950cd1d978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:17.533 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/12/2025 00:43:17.535 7f950ce06c18 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/12/2025 00:43:17.535 7f950ce06c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/12/2025 00:43:17.535 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:43:17.535 7f950ce06c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/12/2025 00:43:17.535 7f950ce06c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:43:17.535 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/12/2025 00:43:17.535 7f950ce06c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/12/2025 00:43:17.535 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:17.536 7f950ce06c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/12/2025 00:43:17.536 7f950ce06c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:17.536 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:17.536 7f950ce06c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 135 http_port = 8096 start_time = 1749688965517660 collector_name = collector_ip = 10.0.0.61:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.19:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/12/2025 00:43:17.536 7f950ce06c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:21.536 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/12/2025 00:43:21.536 7f950ce06c18 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/12/2025 00:43:21.536 7f950ce06c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/12/2025 00:43:21.537 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:43:21.537 7f950ce06c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/12/2025 00:43:21.537 7f950ce06c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:43:21.537 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/12/2025 00:43:21.537 7f950ce06c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/12/2025 00:43:21.537 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:21.537 7f950ce06c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/12/2025 00:43:21.537 7f950ce06c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:21.537 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:21.538 7f950ce06c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 135 http_port = 8096 start_time = 1749688965517660 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.19:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/12/2025 00:43:21.538 7f950ce06c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:25.538 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/12/2025 00:43:25.539 7f950ce06c18 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/12/2025 00:43:25.539 7f950ce06c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/12/2025 00:43:25.539 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:43:25.539 7f950ce06c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/12/2025 00:43:25.539 7f950ce06c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:43:25.539 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/12/2025 00:43:25.540 7f950ce06c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/12/2025 00:43:25.540 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:25.540 7f950ce06c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/12/2025 00:43:25.540 7f950ce06c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:25.540 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:25.540 7f950ce06c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 135 http_port = 8096 start_time = 1749688965517660 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.19:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/12/2025 00:43:25.540 7f950ce06c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:29.540 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/12/2025 00:43:29.541 7f950ce06c18 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/12/2025 00:43:29.541 7f950ce06c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/12/2025 00:43:29.541 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:43:29.541 7f950ce06c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/12/2025 00:43:29.541 7f950ce06c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:43:29.541 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/12/2025 00:43:29.542 7f950ce06c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/12/2025 00:43:29.542 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:29.542 7f950ce06c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/12/2025 00:43:29.542 7f950ce06c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:29.542 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:29.542 7f950ce06c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 135 http_port = 8096 start_time = 1749688965517660 collector_name = collector_ip = 10.0.0.61:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.19:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/12/2025 00:43:29.542 7f950ce06c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:33.542 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/12/2025 00:43:33.543 7f950ce06c18 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/12/2025 00:43:33.543 7f950ce06c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/12/2025 00:43:33.543 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:43:33.543 7f950ce06c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/12/2025 00:43:33.543 7f950ce06c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:43:33.543 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/12/2025 00:43:33.543 7f950ce06c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/12/2025 00:43:33.544 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:33.544 7f950ce06c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/12/2025 00:43:33.544 7f950ce06c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:33.544 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:33.544 7f950ce06c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 135 http_port = 8096 start_time = 1749688965517660 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.19:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/12/2025 00:43:33.544 7f950ce06c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:37.544 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/12/2025 00:43:37.545 7f950ce06c18 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/12/2025 00:43:37.545 7f950ce06c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/12/2025 00:43:37.545 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:43:37.546 7f950ce06c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/12/2025 00:43:37.546 7f950ce06c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:43:37.546 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/12/2025 00:43:37.546 7f950ce06c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/12/2025 00:43:37.546 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:37.546 7f950ce06c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/12/2025 00:43:37.546 7f950ce06c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:37.546 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:37.546 7f950ce06c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 135 http_port = 8096 start_time = 1749688965517660 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.19:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/12/2025 00:43:37.546 7f950ce06c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:41.546 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/12/2025 00:43:41.558 7f950ce06c18 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/12/2025 00:43:41.558 7f950ce06c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/12/2025 00:43:41.558 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:43:41.558 7f950ce06c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/12/2025 00:43:41.559 7f950ce06c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:43:41.559 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/12/2025 00:43:41.559 7f950ce06c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/12/2025 00:43:41.559 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:41.559 7f950ce06c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/12/2025 00:43:41.559 7f950ce06c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:41.559 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:41.559 7f950ce06c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 135 http_port = 8096 start_time = 1749688965517660 collector_name = collector_ip = 10.0.0.61:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.19:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/12/2025 00:43:41.559 7f950ce06c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:45.559 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/12/2025 00:43:45.560 7f950ce06c18 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/12/2025 00:43:45.560 7f950ce06c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/12/2025 00:43:45.560 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:43:45.560 7f950ce06c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/12/2025 00:43:45.560 7f950ce06c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:43:45.560 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/12/2025 00:43:45.561 7f950ce06c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/12/2025 00:43:45.561 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:45.561 7f950ce06c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/12/2025 00:43:45.561 7f950ce06c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:45.561 7f950ce06c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:45.561 7f950ce06c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 135 http_port = 8096 start_time = 1749688965517660 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.19:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/12/2025 00:43:45.561 7f950ce06c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:49.561 7f950e0c94e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/12/2025 00:43:49.563 7f950e0c94e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/12/2025 00:43:49.563 7f950e0c94e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/12/2025 00:43:49.563 7f950e0c94e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:43:49.563 7f950e0c94e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/12/2025 00:43:49.563 7f950e0c94e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:43:49.564 7f950e0c94e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/12/2025 00:43:49.564 7f950e0c94e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/12/2025 00:43:49.564 7f950e0c94e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:49.564 7f950e0c94e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/12/2025 00:43:49.564 7f950e0c94e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:49.564 7f950e0c94e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:49.564 7f950e0c94e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 135 http_port = 8096 start_time = 1749688965517660 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.19:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/12/2025 00:43:49.564 7f950e0c94e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:53.567 7f950cd1f8d0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/12/2025 00:43:53.586 7f950cd1f8d0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/12/2025 00:43:53.586 7f950cd1f8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/12/2025 00:43:53.586 7f950cd1f8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:43:53.586 7f950cd1f8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/12/2025 00:43:53.586 7f950cd1f8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:43:53.586 7f950cd1f8d0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/12/2025 00:43:53.588 7f950cd1f8d0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/12/2025 00:43:53.588 7f950cd1f8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:53.588 7f950cd1f8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/12/2025 00:43:53.588 7f950cd1f8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:53.588 7f950cd1f8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:53.588 7f950cd1f8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 135 http_port = 8096 start_time = 1749688965517660 collector_name = collector_ip = 10.0.0.61:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.19:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/12/2025 00:43:53.588 7f950cd1f8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:57.587 7f950cd9c588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/12/2025 00:43:57.587 7f950cd9c588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/12/2025 00:43:57.588 7f950cd9c588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/12/2025 00:43:57.588 7f950cd9c588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:43:57.588 7f950cd9c588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/12/2025 00:43:57.588 7f950cd9c588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:43:57.588 7f950cd9c588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/12/2025 00:43:57.588 7f950cd9c588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/12/2025 00:43:57.588 7f950cd9c588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:57.588 7f950cd9c588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/12/2025 00:43:57.588 7f950cd9c588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:57.589 7f950cd9c588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:43:57.589 7f950cd9c588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 135 http_port = 8096 start_time = 1749688965517660 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.19:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/12/2025 00:43:57.589 7f950cd9c588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:01.588 7f9513ea2438 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/12/2025 00:44:01.589 7f9513ea2438 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/12/2025 00:44:01.589 7f9513ea2438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/12/2025 00:44:01.589 7f9513ea2438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:44:01.589 7f9513ea2438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/12/2025 00:44:01.589 7f9513ea2438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:44:01.589 7f9513ea2438 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/12/2025 00:44:01.590 7f9513ea2438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/12/2025 00:44:01.590 7f9513ea2438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:01.590 7f9513ea2438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/12/2025 00:44:01.590 7f9513ea2438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:01.590 7f9513ea2438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:01.590 7f9513ea2438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 135 http_port = 8096 start_time = 1749688965517660 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.19:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/12/2025 00:44:01.590 7f9513ea2438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:05.589 7f950cd24f60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/12/2025 00:44:05.590 7f950cd24f60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/12/2025 00:44:05.590 7f950cd24f60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/12/2025 00:44:05.591 7f950cd24f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:44:05.591 7f950cd24f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/12/2025 00:44:05.591 7f950cd24f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:44:05.591 7f950cd24f60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/12/2025 00:44:05.591 7f950cd24f60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/12/2025 00:44:05.591 7f950cd24f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:05.591 7f950cd24f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/12/2025 00:44:05.591 7f950cd24f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:05.591 7f950cd24f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:05.592 7f950cd24f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 135 http_port = 8096 start_time = 1749688965517660 collector_name = collector_ip = 10.0.0.61:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.19:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/12/2025 00:44:05.592 7f950cd24f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:09.593 7f950ef8e438 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/12/2025 00:44:09.593 7f950ef8e438 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/12/2025 00:44:09.597 7f950ef8e438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/12/2025 00:44:09.597 7f950ef8e438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:44:09.597 7f950ef8e438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/12/2025 00:44:09.597 7f950ef8e438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:44:09.597 7f950ef8e438 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/12/2025 00:44:09.598 7f950ef8e438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/12/2025 00:44:09.598 7f950ef8e438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:09.598 7f950ef8e438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/12/2025 00:44:09.598 7f950ef8e438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:09.598 7f950ef8e438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:09.598 7f950ef8e438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 135 http_port = 8096 start_time = 1749688965517660 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.19:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/12/2025 00:44:09.598 7f950ef8e438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:13.594 7f9513ed7e10 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/12/2025 00:44:13.595 7f9513ed7e10 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/12/2025 00:44:13.595 7f9513ed7e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/12/2025 00:44:13.595 7f9513ed7e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:44:13.595 7f9513ed7e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/12/2025 00:44:13.595 7f9513ed7e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:44:13.595 7f9513ed7e10 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/12/2025 00:44:13.596 7f9513ed7e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/12/2025 00:44:13.596 7f9513ed7e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:13.596 7f9513ed7e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/12/2025 00:44:13.596 7f9513ed7e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:13.596 7f9513ed7e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:13.596 7f9513ed7e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 135 http_port = 8096 start_time = 1749688965517660 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.19:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/12/2025 00:44:13.596 7f9513ed7e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:17.596 7f950e0c8630 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/12/2025 00:44:17.597 7f950e0c8630 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/12/2025 00:44:17.598 7f950e0c8630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/12/2025 00:44:17.598 7f950e0c8630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:44:17.598 7f950e0c8630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/12/2025 00:44:17.598 7f950e0c8630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:44:17.598 7f950e0c8630 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/12/2025 00:44:17.598 7f950e0c8630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/12/2025 00:44:17.598 7f950e0c8630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:17.598 7f950e0c8630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/12/2025 00:44:17.598 7f950e0c8630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:17.599 7f950e0c8630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:17.599 7f950e0c8630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 135 http_port = 8096 start_time = 1749688965517660 collector_name = collector_ip = 10.0.0.61:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.19:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/12/2025 00:44:17.599 7f950e0c8630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:21.395 7f9516d0a348 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 06/12/2025 00:44:21.395 7f9516d0a348 [contrail-device-manager] [WARNING]: initializing cassandra cluster 06/12/2025 00:44:21.395 7f9516d0a348 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 06/12/2025 00:44:21.395 7f9513ea2438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:21.396 7f9513ea2438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 06/12/2025 00:44:21.396 7f9513ea2438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:21.396 7f9516d0a348 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 06/12/2025 00:44:21.466 7f9516d0a348 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 06/12/2025 00:44:21.467 7f950cd04c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:21.467 7f950cd04c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Initializing description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 06/12/2025 00:44:21.467 7f950cd04c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:21.484 7f9516d0a348 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 06/12/2025 00:44:21.485 7f950cd04a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:21.485 7f950cd04a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 06/12/2025 00:44:21.485 7f950cd04a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:21.516 7f9516d0a348 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/12/2025 00:44:21.600 7f950cd55d68 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/12/2025 00:44:21.601 7f950cd55d68 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/12/2025 00:44:21.601 7f950cd55d68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/12/2025 00:44:21.601 7f950cd55d68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:44:21.609 7f950cd55d68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 06/12/2025 00:44:21.609 7f950cd55d68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:44:21.609 7f950cd55d68 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/12/2025 00:44:21.609 7f950cd55d68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/12/2025 00:44:21.610 7f950cd55d68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:21.610 7f950cd55d68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 06/12/2025 00:44:21.610 7f950cd55d68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:21.610 7f950cd55d68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:21.610 7f950cd55d68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 135 http_port = 8096 start_time = 1749688965517660 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.19:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/12/2025 00:44:21.610 7f950cd55d68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:22.445 7f9516d0a348 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace', created 06/12/2025 00:44:22.451 7f9516d0a348 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/12/2025 00:44:22.462 7f9516d0a348 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/12/2025 00:44:22.933 7f9516d0a348 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace' fixed with replication {'class': 'SimpleStrategy', 'replication_factor': None} 06/12/2025 00:44:22.942 7f9516d0a348 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/12/2025 00:44:22.942 7f9516d0a348 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 06/12/2025 00:44:22.988 7f9516d0a348 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 06/12/2025 00:44:23.017 7f9516d0a348 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/12/2025 00:44:23.982 7f9516d0a348 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table', created 06/12/2025 00:44:23.988 7f9516d0a348 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/12/2025 00:44:23.994 7f9516d0a348 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/12/2025 00:44:24.923 7f9516d0a348 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table' fixed with properties {'gc_grace_seconds': 864000} 06/12/2025 00:44:24.927 7f9516d0a348 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/12/2025 00:44:24.933 7f9516d0a348 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/12/2025 00:44:25.602 7f950ccebd68 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/12/2025 00:44:25.603 7f950ccebd68 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/12/2025 00:44:25.604 7f950ccebd68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/12/2025 00:44:25.604 7f950ccebd68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:44:25.604 7f950ccebd68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 06/12/2025 00:44:25.604 7f950ccebd68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:44:25.604 7f950ccebd68 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/12/2025 00:44:25.604 7f950ccebd68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/12/2025 00:44:25.604 7f950ccebd68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:25.604 7f950ccebd68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 06/12/2025 00:44:25.604 7f950ccebd68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:25.604 7f950ccebd68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:25.605 7f950ccebd68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 135 http_port = 8096 start_time = 1749688965517660 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.19:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/12/2025 00:44:25.605 7f950ccebd68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:26.093 7f9516d0a348 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table', created 06/12/2025 00:44:26.101 7f9516d0a348 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/12/2025 00:44:26.118 7f9516d0a348 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/12/2025 00:44:26.308 7f9516d0a348 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table' fixed with properties {'gc_grace_seconds': 864000} 06/12/2025 00:44:26.318 7f9516d0a348 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/12/2025 00:44:26.342 7f9516d0a348 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/12/2025 00:44:27.316 7f9516d0a348 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table', created 06/12/2025 00:44:27.320 7f9516d0a348 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/12/2025 00:44:27.329 7f9516d0a348 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/12/2025 00:44:29.031 7f9516d0a348 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table' fixed with properties {'gc_grace_seconds': 864000} 06/12/2025 00:44:29.038 7f9516d0a348 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/12/2025 00:44:29.055 7f9516d0a348 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/12/2025 00:44:29.604 7f950cd55eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/12/2025 00:44:29.605 7f950cd55eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/12/2025 00:44:29.605 7f950cd55eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/12/2025 00:44:29.605 7f950cd55eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:44:29.605 7f950cd55eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 06/12/2025 00:44:29.606 7f950cd55eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:44:29.606 7f950cd55eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/12/2025 00:44:29.606 7f950cd55eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/12/2025 00:44:29.606 7f950cd55eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:29.606 7f950cd55eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 06/12/2025 00:44:29.606 7f950cd55eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:29.606 7f950cd55eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:29.606 7f950cd55eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 135 http_port = 8096 start_time = 1749688965517660 collector_name = collector_ip = 10.0.0.61:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.19:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/12/2025 00:44:29.606 7f950cd55eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:30.418 7f9516d0a348 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table', created 06/12/2025 00:44:30.423 7f9516d0a348 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/12/2025 00:44:30.427 7f9516d0a348 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 06/12/2025 00:44:30.923 7f9516d0a348 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table' fixed with properties {'gc_grace_seconds': 864000} 06/12/2025 00:44:30.928 7f9516d0a348 [contrail-device-manager] [WARNING]: Zookeeper lock released. 06/12/2025 00:44:30.928 7f950ccdee10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:30.929 7f950ccdee10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/12/2025 00:44:30.929 7f950ccdee10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:30.979 7f9516d0a348 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.19:5673//', 'pyamqp://guest:guest@10.0.0.41:5673//', 'pyamqp://guest:guest@10.0.0.61:5673//'] 06/12/2025 00:44:30.980 7f9516d0a348 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 06/12/2025 00:44:30.980 7f950cce7780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:30.980 7f950cce7780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Initializing description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 06/12/2025 00:44:30.980 7f950cce7780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:30.980 7f950cce7780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:30.981 7f950cce7780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 06/12/2025 00:44:30.981 7f950cce7780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:30.984 7f9516d0a348 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 06/12/2025 00:44:30.985 7f950cce7eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:30.985 7f950cce7eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/12/2025 00:44:30.985 7f950cce7eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:31.040 7f9516d0a348 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 06/12/2025 00:44:31.041 7f950cce2780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:31.041 7f950cce2780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 06/12/2025 00:44:31.041 7f950cce2780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:31.047 7f9516d0a348 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 06/12/2025 00:44:31.047 7f950cce7780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:31.047 7f950cce7780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/12/2025 00:44:31.047 7f950cce7780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:31.049 7f950ccdeeb8 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 06/12/2025 00:44:31.050 7f950ccfbf60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:31.050 7f950ccfbf60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 06/12/2025 00:44:31.050 7f950ccfbf60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:31.056 7f950ccdeeb8 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 06/12/2025 00:44:31.056 7f950cceb048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:31.056 7f950cceb048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/12/2025 00:44:31.056 7f950cceb048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:33.606 7f950f000c18 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/12/2025 00:44:33.607 7f950f000c18 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/12/2025 00:44:33.607 7f950f000c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/12/2025 00:44:33.608 7f950f000c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:44:33.608 7f950f000c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/12/2025 00:44:33.608 7f950f000c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:44:33.608 7f950f000c18 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/12/2025 00:44:33.608 7f950f000c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/12/2025 00:44:33.608 7f950f000c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:33.608 7f950f000c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/12/2025 00:44:33.608 7f950f000c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:33.608 7f950f000c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:33.609 7f950f000c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 135 http_port = 8096 start_time = 1749688965517660 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.19:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/12/2025 00:44:33.609 7f950f000c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:37.608 7f950f000c18 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/12/2025 00:44:37.610 7f950f000c18 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/12/2025 00:44:37.610 7f950f000c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/12/2025 00:44:37.610 7f950f000c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:44:37.610 7f950f000c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/12/2025 00:44:37.610 7f950f000c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:44:37.610 7f950f000c18 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/12/2025 00:44:37.610 7f950f000c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/12/2025 00:44:37.610 7f950f000c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:37.611 7f950f000c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/12/2025 00:44:37.611 7f950f000c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:37.611 7f950f000c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:37.611 7f950f000c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 135 http_port = 8096 start_time = 1749688965517660 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.19:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/12/2025 00:44:37.611 7f950f000c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:41.610 7f950f000c18 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/12/2025 00:44:41.612 7f950f000c18 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/12/2025 00:44:41.612 7f950f000c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/12/2025 00:44:41.612 7f950f000c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:44:41.612 7f950f000c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/12/2025 00:44:41.612 7f950f000c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:44:41.612 7f950f000c18 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/12/2025 00:44:41.613 7f950f000c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/12/2025 00:44:41.613 7f950f000c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:41.613 7f950f000c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.61:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/12/2025 00:44:41.613 7f950f000c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:41.613 7f950f000c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:41.613 7f950f000c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 135 http_port = 8096 start_time = 1749688965517660 collector_name = collector_ip = 10.0.0.61:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.19:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/12/2025 00:44:41.613 7f950f000c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:45.613 7f950f000c18 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/12/2025 00:44:45.614 7f950f000c18 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 06/12/2025 00:44:45.614 7f950f000c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/12/2025 00:44:45.614 7f950f000c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:44:45.614 7f950f000c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/12/2025 00:44:45.614 7f950f000c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:44:45.614 7f950f000c18 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/12/2025 00:44:45.615 7f950f000c18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/12/2025 00:44:45.615 7f950f000c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:45.615 7f950f000c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/12/2025 00:44:45.615 7f950f000c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:45.615 7f950f000c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:45.615 7f950f000c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 135 http_port = 8096 start_time = 1749688965517660 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.61:8086, 10.0.0.19:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/12/2025 00:44:45.615 7f950f000c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/12/2025 00:44:48.956 7f950cce7b70 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3446-3.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.61', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3446-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3446-3.'], 'id_perms': {'created': '2025-06-12T00:44:48.941875', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-06-12T00:44:48.941875', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10240540907898776879, 'uuid_mslong': 17366190958630094585}}, 'parent_type': 'global-system-config', 'parent_uuid': '7c0a6328-77ce-4796-90dd-48f4b5e097c4', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'f1011aaa-b6bf-46f9-8e1d-b5b2f329ad2f'}, 'oper': 'CREATE', 'request-id': 'req-8973a8f0-9763-414c-b507-6d92bf3cc97a', 'type': 'analytics_node', 'uuid': 'f1011aaa-b6bf-46f9-8e1d-b5b2f329ad2f'} 06/12/2025 00:44:49.002 7f950cce7b70 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3446-1.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.19', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3446-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3446-1.'], 'id_perms': {'created': '2025-06-12T00:44:48.988044', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-06-12T00:44:48.988044', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12586442096916095326, 'uuid_mslong': 15657191261787803602}}, 'parent_type': 'global-system-config', 'parent_uuid': '7c0a6328-77ce-4796-90dd-48f4b5e097c4', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'd949847f-4b36-4fd2-aeac-0666bcb4e15e'}, 'oper': 'CREATE', 'request-id': 'req-b2bec854-6994-4507-8633-23f113902daf', 'type': 'analytics_node', 'uuid': 'd949847f-4b36-4fd2-aeac-0666bcb4e15e'} 06/12/2025 00:44:49.023 7f950cce7b70 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3446-2.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.41', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3446-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3446-2.'], 'id_perms': {'created': '2025-06-12T00:44:48.993921', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-06-12T00:44:48.993921', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12101588990268358425, 'uuid_mslong': 3832278514183327371}}, 'parent_type': 'global-system-config', 'parent_uuid': '7c0a6328-77ce-4796-90dd-48f4b5e097c4', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '352efd01-1e42-4e8b-a7f1-7b11d6b93f19'}, 'oper': 'CREATE', 'request-id': 'req-6db17fb1-339e-4f17-9535-03711af65616', 'type': 'analytics_node', 'uuid': '352efd01-1e42-4e8b-a7f1-7b11d6b93f19'} 06/12/2025 00:44:49.615 7f950cce74e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/12/2025 00:44:49.618 7f950cce74e0 [contrail-device-manager] [INFO]: Session Event: TCP Connected 06/12/2025 00:44:49.618 7f950cce74e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/12/2025 00:44:49.618 7f950cce74e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:44:49.619 7f950cce74e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3446-1. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.19:5673, 10.0.0.41:5673, 10.0.0.61:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.19:9041, 10.0.0.41:9041, 10.0.0.61:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.19:2181, 10.0.0.41:2181, 10.0.0.61:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.19,10.0.0.41,10.0.0.61:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/12/2025 00:44:49.619 7f950cce74e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/12/2025 00:44:49.619 7f950cce74e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnected] in state[Connect] 06/12/2025 00:44:49.619 7f950cce74e0 [contrail-device-manager] [DEBUG]: Send sandesh control message. uve type count # 19 06/12/2025 00:44:49.619 7f950cce74e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 06/12/2025 00:44:49.619 7f950cce74e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 06/12/2025 00:44:49.619 7f950cce74e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 06/12/2025 00:44:49.619 7f950cc82240 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 06/12/2025 00:44:49.620 7f950cc82240 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 06/12/2025 00:44:58.036 7f950cc82d68 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001016 06/12/2025 00:44:58.036 7f950cc82d68 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 06/12/2025 00:44:58.037 7f950cee7630 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 06/12/2025 00:44:58.037 7f950cee7630 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 0 06/12/2025 00:44:58.037 7f950cee7630 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/12/2025 00:44:58.037 7f950cee7630 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 06/12/2025 00:44:58.037 7f950cee7630 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 06/12/2025 00:44:58.038 7f950cce74e0 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 06/12/2025 00:44:58.040 7f950cce74e0 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 06/12/2025 00:45:04.042 7f950cce7b70 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3446-3.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.61', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3446-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3446-3.'], 'id_perms': {'created': '2025-06-12T00:45:03.990448', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-06-12T00:45:03.990448', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10197105479066279121, 'uuid_mslong': 5360133277482765151}}, 'parent_type': 'global-system-config', 'parent_uuid': '7c0a6328-77ce-4796-90dd-48f4b5e097c4', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '4a6304d1-111c-4b5f-8d83-656818ef5cd1'}, 'oper': 'CREATE', 'request-id': 'req-d1c587e8-055e-4e0e-a2ad-0e2b80d5d501', 'type': 'analytics_alarm_node', 'uuid': '4a6304d1-111c-4b5f-8d83-656818ef5cd1'} 06/12/2025 00:45:04.074 7f950cce7b70 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3446-1.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.19', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3446-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3446-1.'], 'id_perms': {'created': '2025-06-12T00:45:04.003715', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-06-12T00:45:04.003715', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11842807861842407815, 'uuid_mslong': 8485399594716187029}}, 'parent_type': 'global-system-config', 'parent_uuid': '7c0a6328-77ce-4796-90dd-48f4b5e097c4', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '75c231f9-4c4d-4995-a45a-1b012451f187'}, 'oper': 'CREATE', 'request-id': 'req-91842d32-5277-4644-9544-0dece9303685', 'type': 'analytics_alarm_node', 'uuid': '75c231f9-4c4d-4995-a45a-1b012451f187'} 06/12/2025 00:45:04.106 7f950cce7b70 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3446-2.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.41', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3446-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3446-2.'], 'id_perms': {'created': '2025-06-12T00:45:04.089116', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-06-12T00:45:04.089116', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10983638542749733735, 'uuid_mslong': 17793885632076204052}}, 'parent_type': 'global-system-config', 'parent_uuid': '7c0a6328-77ce-4796-90dd-48f4b5e097c4', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'f6f094b4-d1d4-4814-986d-b90f91e31367'}, 'oper': 'CREATE', 'request-id': 'req-2e9d7ba3-e406-480a-92f5-e6c03694c00c', 'type': 'analytics_alarm_node', 'uuid': 'f6f094b4-d1d4-4814-986d-b90f91e31367'} 06/12/2025 00:45:08.181 7f950cc82d68 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001296 06/12/2025 00:45:08.181 7f950cc82d68 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 06/12/2025 00:45:08.182 7f950cce7cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 06/12/2025 00:45:08.182 7f950cce7cc0 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 2 06/12/2025 00:45:08.182 7f950cce7cc0 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/12/2025 00:45:08.182 7f950cce7cc0 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 06/12/2025 00:45:12.150 7f950cce7b70 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3446-1.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.19', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3446-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3446-1.'], 'id_perms': {'created': '2025-06-12T00:45:12.138842', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-06-12T00:45:12.138842', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13756512362946539515, 'uuid_mslong': 1328588081253401268}}, 'parent_type': 'global-system-config', 'parent_uuid': '7c0a6328-77ce-4796-90dd-48f4b5e097c4', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '127017d2-1bed-42b4-bee8-f31be5b2affb'}, 'oper': 'CREATE', 'request-id': 'req-655bae44-0bbf-4c2f-9356-b27cd0a73fc6', 'type': 'analytics_snmp_node', 'uuid': '127017d2-1bed-42b4-bee8-f31be5b2affb'} 06/12/2025 00:45:12.716 7f950cce7b70 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3446-3.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.61', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3446-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3446-3.'], 'id_perms': {'created': '2025-06-12T00:45:12.708507', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-06-12T00:45:12.708507', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10483400794948848021, 'uuid_mslong': 13655904981810761015}}, 'parent_type': 'global-system-config', 'parent_uuid': '7c0a6328-77ce-4796-90dd-48f4b5e097c4', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'bd83853a-9751-4d37-917c-857ac4f94995'}, 'oper': 'CREATE', 'request-id': 'req-6161333e-ea43-44ca-8f4a-47ddac4e76ac', 'type': 'analytics_snmp_node', 'uuid': 'bd83853a-9751-4d37-917c-857ac4f94995'} 06/12/2025 00:45:13.164 7f950cce7b70 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3446-2.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.41', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3446-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3446-2.'], 'id_perms': {'created': '2025-06-12T00:45:13.154835', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-06-12T00:45:13.154835', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10608386865069963832, 'uuid_mslong': 10377872734122296128}}, 'parent_type': 'global-system-config', 'parent_uuid': '7c0a6328-77ce-4796-90dd-48f4b5e097c4', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '90059c44-7192-4340-9338-8fa5ec6a3a38'}, 'oper': 'CREATE', 'request-id': 'req-3f98050d-cc24-4590-8690-b4185004b118', 'type': 'analytics_snmp_node', 'uuid': '90059c44-7192-4340-9338-8fa5ec6a3a38'} 06/12/2025 00:46:53.782 7f950cce7b70 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-42244d85-84fb-4164-8621-914106b5426b', 'type': 'global_system_config', 'uuid': '7c0a6328-77ce-4796-90dd-48f4b5e097c4'} 06/12/2025 00:46:53.836 7f950cce7b70 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-b01e8abd-9a27-40d0-82c8-34726df9bc2d', 'type': 'global_system_config', 'uuid': '7c0a6328-77ce-4796-90dd-48f4b5e097c4'} 06/12/2025 00:46:53.920 7f950cce7b70 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3446-2'], 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-3446-2', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3446-2'], 'id_perms': {'created': '2025-06-12T00:46:53.899143', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-06-12T00:46:53.899143', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11811900349914616401, 'uuid_mslong': 14641612240177088190}}, 'parent_type': 'global-system-config', 'parent_uuid': '7c0a6328-77ce-4796-90dd-48f4b5e097c4', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'cb3174c2-02c4-46be-a3ec-4cc93ef91251', 'virtual_router_dpdk_enabled': False, 'virtual_router_ip_address': '10.20.0.129'}, 'oper': 'CREATE', 'request-id': 'req-8b188425-d42c-4e12-ba63-8c45777d1ac2', 'type': 'virtual_router', 'uuid': 'cb3174c2-02c4-46be-a3ec-4cc93ef91251'} 06/12/2025 00:46:53.969 7f950cce7b70 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3446-1'], 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-3446-1', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3446-1'], 'id_perms': {'created': '2025-06-12T00:46:53.958193', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-06-12T00:46:53.958193', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9400935888582983888, 'uuid_mslong': 7323532122820854827}}, 'parent_type': 'global-system-config', 'parent_uuid': '7c0a6328-77ce-4796-90dd-48f4b5e097c4', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '65a269a9-f9e6-442b-8276-d561e335c0d0', 'virtual_router_dpdk_enabled': False, 'virtual_router_ip_address': '10.20.0.193'}, 'oper': 'CREATE', 'request-id': 'req-b7448d2e-3799-41d0-ad2a-a151a81e0d58', 'type': 'virtual_router', 'uuid': '65a269a9-f9e6-442b-8276-d561e335c0d0'} 06/12/2025 00:46:54.253 7f950cce7b70 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3446-2', 'vhost0'], 'obj_dict': {'display_name': 'vhost0', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3446-2', 'vhost0'], 'id_perms': {'created': '2025-06-12T00:46:54.236264', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-06-12T00:46:54.236264', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9824764134322357566, 'uuid_mslong': 4654986189375883916}}, 'parent_type': 'virtual-router', 'parent_uuid': 'cb3174c2-02c4-46be-a3ec-4cc93ef91251', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '4099d546-ac19-4a8c-8858-92ed2448453e', 'virtual_machine_interface_disable_policy': True, 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:40:99:d5:46:ac']}, 'virtual_network_refs': [{'to': ['default-domain', 'default-project', 'ip-fabric'], 'uuid': 'bc3fc7ef-8328-43b2-ad58-4283ef2a301b'}]}, 'oper': 'CREATE', 'request-id': 'req-7469cdc4-e4dd-45fb-b7e8-7fe56c1c6aa9', 'type': 'virtual_machine_interface', 'uuid': '4099d546-ac19-4a8c-8858-92ed2448453e'} 06/12/2025 00:46:54.299 7f950cce7b70 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3446-2', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-7469cdc4-e4dd-45fb-b7e8-7fe56c1c6aa9', 'type': 'virtual_machine_interface', 'uuid': '4099d546-ac19-4a8c-8858-92ed2448453e'} 06/12/2025 00:46:54.335 7f950cce7b70 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3446-1', 'vhost0'], 'obj_dict': {'display_name': 'vhost0', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3446-1', 'vhost0'], 'id_perms': {'created': '2025-06-12T00:46:54.315850', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-06-12T00:46:54.315850', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10775715232968750969, 'uuid_mslong': 3826580026299076240}}, 'parent_type': 'virtual-router', 'parent_uuid': '65a269a9-f9e6-442b-8276-d561e335c0d0', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '351abe42-8429-4e90-958b-07e985de3779', 'virtual_machine_interface_disable_policy': True, 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:35:1a:be:42:84']}, 'virtual_network_refs': [{'to': ['default-domain', 'default-project', 'ip-fabric'], 'uuid': 'bc3fc7ef-8328-43b2-ad58-4283ef2a301b'}]}, 'oper': 'CREATE', 'request-id': 'req-4feadb9d-729a-45bd-8610-ffc7181323a6', 'type': 'virtual_machine_interface', 'uuid': '351abe42-8429-4e90-958b-07e985de3779'} 06/12/2025 00:46:54.374 7f950cce7b70 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3446-1', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-4feadb9d-729a-45bd-8610-ffc7181323a6', 'type': 'virtual_machine_interface', 'uuid': '351abe42-8429-4e90-958b-07e985de3779'} 06/12/2025 00:46:54.790 7f950cce7b70 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'oper': 'UPDATE', 'request-id': 'req-2b94d7de-c01a-440f-89f1-3dd4d33adccb', 'type': 'access_control_list', 'uuid': '6568efea-bdb0-4a45-91fc-59835cc1b799'} 06/12/2025 00:47:08.754 7f950cebde10 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/12/2025 00:47:48.282 7f950cd05c18 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/12/2025 01:43:24.506 7f950cceba20 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map