08/26/2025 02:59:11.737 7fd74dc4ef88 [contrail-device-manager] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 08/26/2025 02:59:11.738 7fd74dc4ef88 [contrail-device-manager] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 08/26/2025 02:59:11.738 7fd74dc4ef88 [contrail-device-manager] [INFO]: SANDESH: Flow Logging: [None] -> [False] 08/26/2025 02:59:11.738 7fd74535b2e8 [contrail-device-manager] [INFO]: Processing event[EvStart] in state[none] 08/26/2025 02:59:11.738 7fd74535b2e8 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectConfigNode:NodeStatusUVE] map 08/26/2025 02:59:11.739 7fd74535b2e8 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 08/26/2025 02:59:11.739 7fd74535b2e8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 08/26/2025 02:59:11.739 7fd74535b2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:11.739 7fd74535b2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 08/26/2025 02:59:11.739 7fd74535b2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:11.739 7fd74535b2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:11.739 7fd74535b2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 start_time = 1756177151739083 collector_name = collector_ip = collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 08/26/2025 02:59:11.739 7fd74535b2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:11.789 7fd74dc4ef88 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.254:5673//', 'pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.48:5673//'] 08/26/2025 02:59:11.798 7fd74dc4ef88 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 08/26/2025 02:59:11.798 7fd74526a588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:11.799 7fd74526a588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 08/26/2025 02:59:11.799 7fd74526a588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:11.799 7fd74526a588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:11.799 7fd74526a588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 08/26/2025 02:59:11.799 7fd74526a588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:13.814 7fd74dc4ef88 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 08/26/2025 02:59:13.816 7fd7452a8588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:13.816 7fd7452a8588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 08/26/2025 02:59:13.816 7fd7452a8588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:13.844 7fd74dc4ef88 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 08/26/2025 02:59:13.844 7fd745278198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:13.845 7fd745278198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 08/26/2025 02:59:13.845 7fd745278198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:13.852 7fd74dc4ef88 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 08/26/2025 02:59:13.854 7fd753b5e978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:13.854 7fd753b5e978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 08/26/2025 02:59:13.854 7fd753b5e978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:13.876 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Removed remaining AMQP queue from previous run 08/26/2025 02:59:13.877 7fd74dc4ef88 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 08/26/2025 02:59:13.878 7fd74dc4ef88 [contrail-device-manager] [WARNING]: initializing cassandra cluster 08/26/2025 02:59:13.878 7fd74dc4ef88 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 08/26/2025 02:59:13.879 7fd74dc51a20 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting consumer greenlet 08/26/2025 02:59:13.879 7fd74dc51a20 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 08/26/2025 02:59:13.880 7fd74526a240 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting publisher greenlet 08/26/2025 02:59:13.880 7fd74526a240 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Publisher connection 08/26/2025 02:59:13.881 7fd7452a8eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:13.881 7fd7452a8eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 08/26/2025 02:59:13.881 7fd7452a8eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:13.883 7fd74dc4ef88 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 08/26/2025 02:59:13.896 7fd74526a240 [contrail-device-manager] [INFO]: KombuAmqpClient: Publisher connection established 08/26/2025 02:59:13.926 7fd74dc51a20 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 08/26/2025 02:59:13.927 7fd74dc51a20 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Waiting for consumer 08/26/2025 02:59:13.946 7fd74dc4ef88 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 08/26/2025 02:59:13.948 7fd7450c66d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:13.949 7fd7450c66d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 08/26/2025 02:59:13.949 7fd7450c66d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:13.956 7fd74dc4ef88 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 08/26/2025 02:59:13.957 7fd745031a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:13.957 7fd745031a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 08/26/2025 02:59:13.957 7fd745031a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:14.971 7fd74dc4ef88 [contrail-device-manager] [WARNING]: Waiting for keyspace config_db_uuid to be created 08/26/2025 02:59:15.146 7fd74dc4ef88 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 08/26/2025 02:59:15.186 7fd74dc4ef88 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 08/26/2025 02:59:15.739 7fd74503d048 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/26/2025 02:59:15.742 7fd74503d048 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/26/2025 02:59:15.742 7fd74503d048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/26/2025 02:59:15.742 7fd74503d048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/26/2025 02:59:15.743 7fd74503d048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 08/26/2025 02:59:15.743 7fd74503d048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/26/2025 02:59:15.743 7fd74503d048 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/26/2025 02:59:15.743 7fd74503d048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/26/2025 02:59:15.743 7fd74503d048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:15.744 7fd74503d048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 08/26/2025 02:59:15.744 7fd74503d048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:15.744 7fd74503d048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:15.744 7fd74503d048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 start_time = 1756177151739083 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/26/2025 02:59:15.744 7fd74503d048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:16.835 7fd74dc4ef88 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 08/26/2025 02:59:17.855 7fd74dc4ef88 [contrail-device-manager] [WARNING]: table 'obj_uuid_table', created 08/26/2025 02:59:17.861 7fd74dc4ef88 [contrail-device-manager] [WARNING]: Zookeeper lock released. 08/26/2025 02:59:19.743 7fd745047b70 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/26/2025 02:59:19.745 7fd745047b70 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/26/2025 02:59:19.745 7fd745047b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/26/2025 02:59:19.745 7fd745047b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/26/2025 02:59:19.745 7fd745047b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 08/26/2025 02:59:19.745 7fd745047b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/26/2025 02:59:19.745 7fd745047b70 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/26/2025 02:59:19.746 7fd745047b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/26/2025 02:59:19.746 7fd745047b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:19.746 7fd745047b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 08/26/2025 02:59:19.746 7fd745047b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:19.746 7fd745047b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:19.746 7fd745047b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 start_time = 1756177151739083 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/26/2025 02:59:19.746 7fd745047b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:19.821 7fd74dc4ef88 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 08/26/2025 02:59:20.931 7fd74dc4ef88 [contrail-device-manager] [WARNING]: table 'obj_uuid_table' fixed with properties {'gc_grace_seconds': 864000} 08/26/2025 02:59:20.942 7fd74dc4ef88 [contrail-device-manager] [WARNING]: Zookeeper lock released. 08/26/2025 02:59:22.643 7fd74dc4ef88 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 08/26/2025 02:59:23.745 7fd74503d438 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/26/2025 02:59:23.747 7fd74503d438 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/26/2025 02:59:23.747 7fd74503d438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/26/2025 02:59:23.747 7fd74503d438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/26/2025 02:59:23.747 7fd74503d438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 08/26/2025 02:59:23.747 7fd74503d438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/26/2025 02:59:23.747 7fd74503d438 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/26/2025 02:59:23.748 7fd74503d438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/26/2025 02:59:23.748 7fd74503d438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:23.748 7fd74503d438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 08/26/2025 02:59:23.748 7fd74503d438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:23.748 7fd74503d438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:23.748 7fd74503d438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 start_time = 1756177151739083 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/26/2025 02:59:23.748 7fd74503d438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:23.782 7fd74dc4ef88 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table', created 08/26/2025 02:59:23.787 7fd74dc4ef88 [contrail-device-manager] [WARNING]: Zookeeper lock released. 08/26/2025 02:59:25.707 7fd74dc4ef88 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 08/26/2025 02:59:26.819 7fd74dc4ef88 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table' fixed with properties {'gc_grace_seconds': 864000} 08/26/2025 02:59:26.824 7fd74dc4ef88 [contrail-device-manager] [WARNING]: Zookeeper lock released. 08/26/2025 02:59:27.747 7fd74503dc18 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/26/2025 02:59:27.748 7fd74503dc18 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/26/2025 02:59:27.748 7fd74503dc18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/26/2025 02:59:27.749 7fd74503dc18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/26/2025 02:59:27.749 7fd74503dc18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 08/26/2025 02:59:27.749 7fd74503dc18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/26/2025 02:59:27.749 7fd74503dc18 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/26/2025 02:59:27.749 7fd74503dc18 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/26/2025 02:59:27.749 7fd74503dc18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:27.749 7fd74503dc18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 08/26/2025 02:59:27.749 7fd74503dc18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:27.749 7fd74503dc18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:27.749 7fd74503dc18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 start_time = 1756177151739083 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/26/2025 02:59:27.749 7fd74503dc18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:28.793 7fd74dc4ef88 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 08/26/2025 02:59:28.794 7fd74dc4ef88 [contrail-device-manager] [WARNING]: table 'obj_shared_table', created 08/26/2025 02:59:28.805 7fd74dc4ef88 [contrail-device-manager] [WARNING]: Zookeeper lock released. 08/26/2025 02:59:30.865 7fd74dc4ef88 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 08/26/2025 02:59:31.749 7fd7447ebe10 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/26/2025 02:59:31.750 7fd7447ebe10 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/26/2025 02:59:31.751 7fd7447ebe10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/26/2025 02:59:31.751 7fd7447ebe10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/26/2025 02:59:31.751 7fd7447ebe10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 08/26/2025 02:59:31.751 7fd7447ebe10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/26/2025 02:59:31.751 7fd7447ebe10 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/26/2025 02:59:31.751 7fd7447ebe10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/26/2025 02:59:31.751 7fd7447ebe10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:31.751 7fd7447ebe10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 08/26/2025 02:59:31.752 7fd7447ebe10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:31.752 7fd7447ebe10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:31.752 7fd7447ebe10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 start_time = 1756177151739083 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/26/2025 02:59:31.752 7fd7447ebe10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:31.982 7fd74dc4ef88 [contrail-device-manager] [WARNING]: table 'obj_shared_table' fixed with properties {'gc_grace_seconds': 864000} 08/26/2025 02:59:31.987 7fd74dc4ef88 [contrail-device-manager] [WARNING]: Zookeeper lock released. 08/26/2025 02:59:31.987 7fd74a93a048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:31.987 7fd74a93a048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/26/2025 02:59:31.987 7fd74a93a048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:32.031 7fd74dc4ef88 [contrail-device-manager] [ERROR]: Error in db walk read 'type' 08/26/2025 02:59:32.031 7fd7447e7e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:32.032 7fd7447e7e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Zookeeper:Zookeeper[] connection down >>, ] >> 08/26/2025 02:59:32.032 7fd7447e7e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:32.042 7fd7447e6c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:32.042 7fd7447e6c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/26/2025 02:59:32.042 7fd7447e6c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:32.042 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_status_consumer.dummy 08/26/2025 02:59:32.042 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_request_consumer 08/26/2025 02:59:32.043 7fd74dc51a20 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 08/26/2025 02:59:32.043 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3935-3. 08/26/2025 02:59:32.047 7fd74dc51a20 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 08/26/2025 02:59:32.058 7fd74dc51a20 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3935-3.'] 08/26/2025 02:59:32.066 7fd74dc51a20 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 08/26/2025 02:59:32.070 7fd74dc51a20 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 08/26/2025 02:59:32.079 7fd74dc51a20 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3935-3.'] 08/26/2025 02:59:32.553 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Fetching all containers 08/26/2025 02:59:32.596 7fd74dc4ef88 [contrail-device-manager] [INFO]: Restarting dnsmasq docker: config_dnsmasq_1 08/26/2025 02:59:32.750 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3935-3..config_queue 08/26/2025 02:59:32.751 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3935-3..tftp_queue 08/26/2025 02:59:33.087 7fd74dc51a20 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 08/26/2025 02:59:33.091 7fd74dc51a20 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 08/26/2025 02:59:33.104 7fd74dc51a20 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3935-3.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3935-3..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3935-3..tftp_queue'] 08/26/2025 02:59:33.252 7fd74dc4ef88 [contrail-device-manager] [WARNING]: Process 140 prepared to run in Full mode... 08/26/2025 02:59:33.252 7fd74dc4ef88 [contrail-device-manager] [WARNING]: Elected master Device Manager node. Initializing... 08/26/2025 02:59:33.252 7fd74dc4ef88 [contrail-device-manager] [INFO]: SANDESH: INTROSPECT IS ON: 0.0.0.0:8096 08/26/2025 02:59:33.255 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add [config_updater.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/config_updater] to home page 08/26/2025 02:59:33.255 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_ConfigApiUpdateReq] 08/26/2025 02:59:33.256 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add [greenlets.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/greenlets] to home page 08/26/2025 02:59:33.256 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_GreenletObjectReq] 08/26/2025 02:59:33.257 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add [cpuinfo.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/nodeinfo/cpuinfo] to home page 08/26/2025 02:59:33.257 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CpuLoadInfoReq] 08/26/2025 02:59:33.260 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add [dm_introspect.xml:/usr/lib/python3.6/site-packages/device_manager/sandesh/dm_introspect] to home page 08/26/2025 02:59:33.261 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_RoutingInstanceList] 08/26/2025 02:59:33.261 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualNetworkList] 08/26/2025 02:59:33.261 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualMachineInterfaceList] 08/26/2025 02:59:33.261 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalInterfaceList] 08/26/2025 02:59:33.261 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalInterfaceList] 08/26/2025 02:59:33.261 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalRouterList] 08/26/2025 02:59:33.261 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_BgpRouterList] 08/26/2025 02:59:33.261 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalRouterList] 08/26/2025 02:59:33.261 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_InstanceIpList] 08/26/2025 02:59:33.261 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FloatingIpList] 08/26/2025 02:59:33.261 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PortTupleList] 08/26/2025 02:59:33.261 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FeatureFlagList] 08/26/2025 02:59:33.263 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add [sandesh_alarm.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_alarm] to home page 08/26/2025 02:59:33.263 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmCacheRequest] 08/26/2025 02:59:33.263 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmTypesRequest] 08/26/2025 02:59:33.263 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmAckRequest] 08/26/2025 02:59:33.263 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add [sandesh_ctrl.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_ctrl] to home page 08/26/2025 02:59:33.263 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlClientToServer] 08/26/2025 02:59:33.264 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlServerToClient] 08/26/2025 02:59:33.264 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add [sandesh_trace.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_trace] to home page 08/26/2025 02:59:33.264 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceRequest] 08/26/2025 02:59:33.264 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferListRequest] 08/26/2025 02:59:33.264 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceEnableDisableReq] 08/26/2025 02:59:33.264 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufStatusReq] 08/26/2025 02:59:33.265 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferEnableDisableReq] 08/26/2025 02:59:33.265 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add [sandesh_uve.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_uve] to home page 08/26/2025 02:59:33.265 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshMessageStatsReq] 08/26/2025 02:59:33.266 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsSet] 08/26/2025 02:59:33.266 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsStatus] 08/26/2025 02:59:33.266 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsSet] 08/26/2025 02:59:33.266 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsStatus] 08/26/2025 02:59:33.266 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsSet] 08/26/2025 02:59:33.266 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsReset] 08/26/2025 02:59:33.266 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsStatus] 08/26/2025 02:59:33.266 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueSet] 08/26/2025 02:59:33.266 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueStatus] 08/26/2025 02:59:33.266 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CollectorInfoRequest] 08/26/2025 02:59:33.266 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVEDSConfReq] 08/26/2025 02:59:33.266 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVECacheReq] 08/26/2025 02:59:33.267 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVETypesReq] 08/26/2025 02:59:33.269 7fd74dc4ef88 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.ztp_queue 08/26/2025 02:59:33.280 7fd7447eef60 [contrail-device-manager] [ERROR]: Cannot write http_port 8096 to /tmp/contrail-device-manager.1.http_port 08/26/2025 02:59:33.280 7fd7447eef60 [contrail-device-manager] [ERROR]: Starting Introspect on HTTP Port 8096 08/26/2025 02:59:33.332 7fd73fed12e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:33.332 7fd73fed12e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Initializing description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 08/26/2025 02:59:33.332 7fd73fed12e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:33.344 7fd73fe11c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:33.344 7fd73fe11c18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 08/26/2025 02:59:33.344 7fd73fe11c18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:34.115 7fd74dc51a20 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 08/26/2025 02:59:34.121 7fd74dc51a20 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 08/26/2025 02:59:34.140 7fd74dc51a20 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-3935-3.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3935-3..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-3935-3..tftp_queue', 'device_manager_ztp.ztp_queue'] 08/26/2025 02:59:35.751 7fd73fe18ac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/26/2025 02:59:35.752 7fd73fe18ac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/26/2025 02:59:35.752 7fd73fe18ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/26/2025 02:59:35.752 7fd73fe18ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/26/2025 02:59:35.752 7fd73fe18ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 08/26/2025 02:59:35.752 7fd73fe18ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/26/2025 02:59:35.752 7fd73fe18ac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/26/2025 02:59:35.753 7fd73fe18ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/26/2025 02:59:35.753 7fd73fe18ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:35.753 7fd73fe18ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 08/26/2025 02:59:35.753 7fd73fe18ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:35.753 7fd73fe18ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:35.753 7fd73fe18ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1756177151739083 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/26/2025 02:59:35.753 7fd73fe18ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:36.716 7fd73fdbd8d0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:36.716 7fd73fdbd8d0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/26/2025 02:59:36.716 7fd73fdbd8d0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:39.753 7fd73fd39588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/26/2025 02:59:39.754 7fd73fd39588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/26/2025 02:59:39.754 7fd73fd39588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/26/2025 02:59:39.754 7fd73fd39588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/26/2025 02:59:39.754 7fd73fd39588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/26/2025 02:59:39.754 7fd73fd39588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/26/2025 02:59:39.754 7fd73fd39588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/26/2025 02:59:39.755 7fd73fd39588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/26/2025 02:59:39.755 7fd73fd39588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:39.755 7fd73fd39588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/26/2025 02:59:39.755 7fd73fd39588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:39.755 7fd73fd39588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:39.755 7fd73fd39588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1756177151739083 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/26/2025 02:59:39.755 7fd73fd39588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:43.754 7fd73fd3dcc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/26/2025 02:59:43.755 7fd73fd3dcc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/26/2025 02:59:43.755 7fd73fd3dcc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/26/2025 02:59:43.756 7fd73fd3dcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/26/2025 02:59:43.756 7fd73fd3dcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/26/2025 02:59:43.756 7fd73fd3dcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/26/2025 02:59:43.756 7fd73fd3dcc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/26/2025 02:59:43.756 7fd73fd3dcc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/26/2025 02:59:43.756 7fd73fd3dcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:43.756 7fd73fd3dcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/26/2025 02:59:43.756 7fd73fd3dcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:43.756 7fd73fd3dcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:43.756 7fd73fd3dcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1756177151739083 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/26/2025 02:59:43.757 7fd73fd3dcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:47.756 7fd7450316d8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/26/2025 02:59:47.757 7fd7450316d8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/26/2025 02:59:47.757 7fd7450316d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/26/2025 02:59:47.757 7fd7450316d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/26/2025 02:59:47.757 7fd7450316d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/26/2025 02:59:47.757 7fd7450316d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/26/2025 02:59:47.757 7fd7450316d8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/26/2025 02:59:47.758 7fd7450316d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/26/2025 02:59:47.758 7fd7450316d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:47.758 7fd7450316d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/26/2025 02:59:47.758 7fd7450316d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:47.758 7fd7450316d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:47.758 7fd7450316d8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1756177151739083 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/26/2025 02:59:47.758 7fd7450316d8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:51.758 7fd74bf12588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/26/2025 02:59:51.759 7fd74bf12588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/26/2025 02:59:51.759 7fd74bf12588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/26/2025 02:59:51.760 7fd74bf12588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/26/2025 02:59:51.760 7fd74bf12588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/26/2025 02:59:51.760 7fd74bf12588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/26/2025 02:59:51.760 7fd74bf12588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/26/2025 02:59:51.760 7fd74bf12588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/26/2025 02:59:51.760 7fd74bf12588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:51.760 7fd74bf12588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/26/2025 02:59:51.760 7fd74bf12588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:51.760 7fd74bf12588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:51.761 7fd74bf12588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1756177151739083 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/26/2025 02:59:51.761 7fd74bf12588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:55.761 7fd74bf12588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/26/2025 02:59:55.763 7fd74bf12588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/26/2025 02:59:55.763 7fd74bf12588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/26/2025 02:59:55.763 7fd74bf12588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/26/2025 02:59:55.763 7fd74bf12588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/26/2025 02:59:55.763 7fd74bf12588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/26/2025 02:59:55.763 7fd74bf12588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/26/2025 02:59:55.763 7fd74bf12588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/26/2025 02:59:55.764 7fd74bf12588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:55.764 7fd74bf12588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/26/2025 02:59:55.764 7fd74bf12588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:55.764 7fd74bf12588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:55.764 7fd74bf12588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1756177151739083 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/26/2025 02:59:55.764 7fd74bf12588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:59.764 7fd74bf12588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/26/2025 02:59:59.765 7fd74bf12588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/26/2025 02:59:59.765 7fd74bf12588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/26/2025 02:59:59.765 7fd74bf12588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/26/2025 02:59:59.765 7fd74bf12588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/26/2025 02:59:59.765 7fd74bf12588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/26/2025 02:59:59.765 7fd74bf12588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/26/2025 02:59:59.766 7fd74bf12588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/26/2025 02:59:59.766 7fd74bf12588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:59.766 7fd74bf12588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/26/2025 02:59:59.766 7fd74bf12588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:59.766 7fd74bf12588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 02:59:59.766 7fd74bf12588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1756177151739083 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/26/2025 02:59:59.766 7fd74bf12588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:03.766 7fd73fcd0240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/26/2025 03:00:03.767 7fd73fcd0240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/26/2025 03:00:03.767 7fd73fcd0240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/26/2025 03:00:03.767 7fd73fcd0240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:00:03.767 7fd73fcd0240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/26/2025 03:00:03.767 7fd73fcd0240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:00:03.767 7fd73fcd0240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/26/2025 03:00:03.768 7fd73fcd0240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/26/2025 03:00:03.768 7fd73fcd0240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:03.768 7fd73fcd0240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/26/2025 03:00:03.768 7fd73fcd0240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:03.768 7fd73fcd0240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:03.768 7fd73fcd0240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1756177151739083 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/26/2025 03:00:03.768 7fd73fcd0240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:07.768 7fd73fcd0240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/26/2025 03:00:07.769 7fd73fcd0240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/26/2025 03:00:07.769 7fd73fcd0240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/26/2025 03:00:07.769 7fd73fcd0240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:00:07.770 7fd73fcd0240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/26/2025 03:00:07.770 7fd73fcd0240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:00:07.770 7fd73fcd0240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/26/2025 03:00:07.770 7fd73fcd0240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/26/2025 03:00:07.770 7fd73fcd0240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:07.770 7fd73fcd0240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/26/2025 03:00:07.770 7fd73fcd0240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:07.770 7fd73fcd0240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:07.770 7fd73fcd0240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1756177151739083 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/26/2025 03:00:07.770 7fd73fcd0240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:11.770 7fd73fcd0240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/26/2025 03:00:11.771 7fd73fcd0240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/26/2025 03:00:11.771 7fd73fcd0240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/26/2025 03:00:11.771 7fd73fcd0240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:00:11.772 7fd73fcd0240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/26/2025 03:00:11.772 7fd73fcd0240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:00:11.772 7fd73fcd0240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/26/2025 03:00:11.772 7fd73fcd0240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/26/2025 03:00:11.772 7fd73fcd0240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:11.772 7fd73fcd0240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/26/2025 03:00:11.772 7fd73fcd0240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:11.772 7fd73fcd0240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:11.772 7fd73fcd0240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1756177151739083 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/26/2025 03:00:11.772 7fd73fcd0240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:15.772 7fd7447d9240 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/26/2025 03:00:15.773 7fd7447d9240 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/26/2025 03:00:15.773 7fd7447d9240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/26/2025 03:00:15.774 7fd7447d9240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:00:15.774 7fd7447d9240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/26/2025 03:00:15.774 7fd7447d9240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:00:15.774 7fd7447d9240 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/26/2025 03:00:15.774 7fd7447d9240 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/26/2025 03:00:15.774 7fd7447d9240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:15.774 7fd7447d9240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/26/2025 03:00:15.774 7fd7447d9240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:15.774 7fd7447d9240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:15.774 7fd7447d9240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1756177151739083 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/26/2025 03:00:15.775 7fd7447d9240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:19.774 7fd745056b70 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/26/2025 03:00:19.775 7fd745056b70 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/26/2025 03:00:19.775 7fd745056b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/26/2025 03:00:19.775 7fd745056b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:00:19.775 7fd745056b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/26/2025 03:00:19.776 7fd745056b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:00:19.776 7fd745056b70 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/26/2025 03:00:19.776 7fd745056b70 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/26/2025 03:00:19.776 7fd745056b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:19.776 7fd745056b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/26/2025 03:00:19.776 7fd745056b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:19.776 7fd745056b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:19.777 7fd745056b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1756177151739083 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/26/2025 03:00:19.777 7fd745056b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:23.775 7fd73fcd4588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/26/2025 03:00:23.778 7fd73fcd4588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/26/2025 03:00:23.779 7fd73fcd4588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/26/2025 03:00:23.779 7fd73fcd4588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:00:23.779 7fd73fcd4588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/26/2025 03:00:23.779 7fd73fcd4588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:00:23.779 7fd73fcd4588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/26/2025 03:00:23.779 7fd73fcd4588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/26/2025 03:00:23.779 7fd73fcd4588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:23.780 7fd73fcd4588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/26/2025 03:00:23.780 7fd73fcd4588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:23.780 7fd73fcd4588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:23.780 7fd73fcd4588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1756177151739083 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/26/2025 03:00:23.780 7fd73fcd4588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:27.779 7fd73fd39e10 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/26/2025 03:00:27.780 7fd73fd39e10 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/26/2025 03:00:27.780 7fd73fd39e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/26/2025 03:00:27.780 7fd73fd39e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:00:27.780 7fd73fd39e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/26/2025 03:00:27.781 7fd73fd39e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:00:27.781 7fd73fd39e10 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/26/2025 03:00:27.781 7fd73fd39e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/26/2025 03:00:27.781 7fd73fd39e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:27.781 7fd73fd39e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/26/2025 03:00:27.781 7fd73fd39e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:27.781 7fd73fd39e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:27.781 7fd73fd39e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1756177151739083 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/26/2025 03:00:27.781 7fd73fd39e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:31.781 7fd74bf12588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/26/2025 03:00:31.783 7fd74bf12588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/26/2025 03:00:31.783 7fd74bf12588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/26/2025 03:00:31.783 7fd74bf12588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:00:31.783 7fd74bf12588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/26/2025 03:00:31.783 7fd74bf12588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:00:31.783 7fd74bf12588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/26/2025 03:00:31.784 7fd74bf12588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/26/2025 03:00:31.784 7fd74bf12588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:31.784 7fd74bf12588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/26/2025 03:00:31.784 7fd74bf12588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:31.784 7fd74bf12588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:31.784 7fd74bf12588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1756177151739083 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/26/2025 03:00:31.784 7fd74bf12588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:35.783 7fd73fd39f60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/26/2025 03:00:35.784 7fd73fd39f60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/26/2025 03:00:35.784 7fd73fd39f60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/26/2025 03:00:35.785 7fd73fd39f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:00:35.785 7fd73fd39f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/26/2025 03:00:35.785 7fd73fd39f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:00:35.785 7fd73fd39f60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/26/2025 03:00:35.785 7fd73fd39f60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/26/2025 03:00:35.785 7fd73fd39f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:35.785 7fd73fd39f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/26/2025 03:00:35.785 7fd73fd39f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:35.785 7fd73fd39f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:35.785 7fd73fd39f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1756177151739083 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/26/2025 03:00:35.786 7fd73fd39f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:39.785 7fd73fd39eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/26/2025 03:00:39.786 7fd73fd39eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/26/2025 03:00:39.786 7fd73fd39eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/26/2025 03:00:39.786 7fd73fd39eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:00:39.786 7fd73fd39eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/26/2025 03:00:39.786 7fd73fd39eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:00:39.787 7fd73fd39eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/26/2025 03:00:39.787 7fd73fd39eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/26/2025 03:00:39.787 7fd73fd39eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:39.787 7fd73fd39eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/26/2025 03:00:39.787 7fd73fd39eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:39.787 7fd73fd39eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:39.787 7fd73fd39eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1756177151739083 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/26/2025 03:00:39.787 7fd73fd39eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:43.787 7fd74adc6e10 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/26/2025 03:00:43.789 7fd74adc6e10 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/26/2025 03:00:43.789 7fd74adc6e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/26/2025 03:00:43.789 7fd74adc6e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:00:43.789 7fd74adc6e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/26/2025 03:00:43.789 7fd74adc6e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:00:43.789 7fd74adc6e10 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/26/2025 03:00:43.789 7fd74adc6e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/26/2025 03:00:43.790 7fd74adc6e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:43.790 7fd74adc6e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/26/2025 03:00:43.790 7fd74adc6e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:43.790 7fd74adc6e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:43.790 7fd74adc6e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1756177151739083 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/26/2025 03:00:43.790 7fd74adc6e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:47.790 7fd73fcd0588 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/26/2025 03:00:47.793 7fd73fcd0588 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/26/2025 03:00:47.793 7fd73fcd0588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/26/2025 03:00:47.793 7fd73fcd0588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:00:47.793 7fd73fcd0588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/26/2025 03:00:47.793 7fd73fcd0588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:00:47.793 7fd73fcd0588 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/26/2025 03:00:47.794 7fd73fcd0588 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/26/2025 03:00:47.794 7fd73fcd0588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:47.794 7fd73fcd0588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/26/2025 03:00:47.794 7fd73fcd0588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:47.794 7fd73fcd0588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:47.794 7fd73fcd0588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1756177151739083 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/26/2025 03:00:47.794 7fd73fcd0588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:51.794 7fd74adaa828 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/26/2025 03:00:51.795 7fd74adaa828 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/26/2025 03:00:51.795 7fd74adaa828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/26/2025 03:00:51.795 7fd74adaa828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:00:51.796 7fd74adaa828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/26/2025 03:00:51.796 7fd74adaa828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:00:51.796 7fd74adaa828 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/26/2025 03:00:51.796 7fd74adaa828 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/26/2025 03:00:51.796 7fd74adaa828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:51.796 7fd74adaa828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/26/2025 03:00:51.797 7fd74adaa828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:51.797 7fd74adaa828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:51.797 7fd74adaa828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1756177151739083 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/26/2025 03:00:51.797 7fd74adaa828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:54.363 7fd74dc4ef88 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 08/26/2025 03:00:54.363 7fd74dc4ef88 [contrail-device-manager] [WARNING]: initializing cassandra cluster 08/26/2025 03:00:54.363 7fd74dc4ef88 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 08/26/2025 03:00:54.364 7fd74ae3fe10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:54.364 7fd74ae3fe10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 08/26/2025 03:00:54.365 7fd74ae3fe10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:54.366 7fd74dc4ef88 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 08/26/2025 03:00:54.464 7fd74dc4ef88 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 08/26/2025 03:00:54.466 7fd73fcb8eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:54.466 7fd73fcb8eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Initializing description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 08/26/2025 03:00:54.466 7fd73fcb8eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:54.473 7fd74dc4ef88 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 08/26/2025 03:00:54.473 7fd73fcb8cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:54.474 7fd73fcb8cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 08/26/2025 03:00:54.474 7fd73fcb8cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:54.486 7fd74dc4ef88 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 08/26/2025 03:00:54.741 7fd74dc4ef88 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace', created 08/26/2025 03:00:54.746 7fd74dc4ef88 [contrail-device-manager] [WARNING]: Zookeeper lock released. 08/26/2025 03:00:54.752 7fd74dc4ef88 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 08/26/2025 03:00:55.796 7fd74adfd048 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/26/2025 03:00:55.798 7fd74adfd048 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/26/2025 03:00:55.798 7fd74adfd048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/26/2025 03:00:55.798 7fd74adfd048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:00:55.798 7fd74adfd048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 08/26/2025 03:00:55.798 7fd74adfd048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:00:55.798 7fd74adfd048 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/26/2025 03:00:55.798 7fd74adfd048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/26/2025 03:00:55.798 7fd74adfd048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:55.799 7fd74adfd048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 08/26/2025 03:00:55.799 7fd74adfd048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:55.799 7fd74adfd048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:55.799 7fd74adfd048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1756177151739083 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/26/2025 03:00:55.799 7fd74adfd048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:55.835 7fd74dc4ef88 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace' fixed with replication {'class': 'SimpleStrategy', 'replication_factor': None} 08/26/2025 03:00:55.839 7fd74dc4ef88 [contrail-device-manager] [WARNING]: Zookeeper lock released. 08/26/2025 03:00:55.839 7fd74dc4ef88 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 08/26/2025 03:00:55.932 7fd74dc4ef88 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 08/26/2025 03:00:55.941 7fd74dc4ef88 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 08/26/2025 03:00:57.090 7fd74dc4ef88 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table', created 08/26/2025 03:00:57.095 7fd74dc4ef88 [contrail-device-manager] [WARNING]: Zookeeper lock released. 08/26/2025 03:00:57.103 7fd74dc4ef88 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 08/26/2025 03:00:57.857 7fd74dc4ef88 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table' fixed with properties {'gc_grace_seconds': 864000} 08/26/2025 03:00:57.861 7fd74dc4ef88 [contrail-device-manager] [WARNING]: Zookeeper lock released. 08/26/2025 03:00:57.869 7fd74dc4ef88 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 08/26/2025 03:00:59.002 7fd74dc4ef88 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table', created 08/26/2025 03:00:59.006 7fd74dc4ef88 [contrail-device-manager] [WARNING]: Zookeeper lock released. 08/26/2025 03:00:59.012 7fd74dc4ef88 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 08/26/2025 03:00:59.799 7fd73fe7e978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/26/2025 03:00:59.800 7fd73fe7e978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/26/2025 03:00:59.800 7fd73fe7e978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/26/2025 03:00:59.800 7fd73fe7e978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:00:59.800 7fd73fe7e978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 08/26/2025 03:00:59.800 7fd73fe7e978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:00:59.800 7fd73fe7e978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/26/2025 03:00:59.801 7fd73fe7e978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/26/2025 03:00:59.801 7fd73fe7e978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:59.801 7fd73fe7e978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 08/26/2025 03:00:59.801 7fd73fe7e978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:59.801 7fd73fe7e978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:59.801 7fd73fe7e978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1756177151739083 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/26/2025 03:00:59.801 7fd73fe7e978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:00:59.903 7fd74dc4ef88 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table' fixed with properties {'gc_grace_seconds': 864000} 08/26/2025 03:00:59.908 7fd74dc4ef88 [contrail-device-manager] [WARNING]: Zookeeper lock released. 08/26/2025 03:00:59.914 7fd74dc4ef88 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 08/26/2025 03:01:01.713 7fd74dc4ef88 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table', created 08/26/2025 03:01:01.726 7fd74dc4ef88 [contrail-device-manager] [WARNING]: Zookeeper lock released. 08/26/2025 03:01:01.734 7fd74dc4ef88 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 08/26/2025 03:01:02.830 7fd74dc4ef88 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table' fixed with properties {'gc_grace_seconds': 864000} 08/26/2025 03:01:02.835 7fd74dc4ef88 [contrail-device-manager] [WARNING]: Zookeeper lock released. 08/26/2025 03:01:02.843 7fd74dc4ef88 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 08/26/2025 03:01:03.790 7fd74dc4ef88 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table', created 08/26/2025 03:01:03.797 7fd74dc4ef88 [contrail-device-manager] [WARNING]: Zookeeper lock released. 08/26/2025 03:01:03.800 7fd73fe98e10 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/26/2025 03:01:03.801 7fd73fe98e10 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/26/2025 03:01:03.801 7fd73fe98e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/26/2025 03:01:03.801 7fd73fe98e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:01:03.801 7fd73fe98e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 08/26/2025 03:01:03.802 7fd73fe98e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:01:03.802 7fd73fe98e10 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/26/2025 03:01:03.802 7fd73fe98e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/26/2025 03:01:03.802 7fd73fe98e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:01:03.802 7fd73fe98e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 08/26/2025 03:01:03.802 7fd73fe98e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:01:03.802 7fd73fe98e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:01:03.803 7fd73fe98e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1756177151739083 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/26/2025 03:01:03.803 7fd73fe98e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:01:03.804 7fd74dc4ef88 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 08/26/2025 03:01:04.732 7fd74dc4ef88 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table' fixed with properties {'gc_grace_seconds': 864000} 08/26/2025 03:01:04.737 7fd74dc4ef88 [contrail-device-manager] [WARNING]: Zookeeper lock released. 08/26/2025 03:01:04.737 7fd74adaa438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:01:04.738 7fd74adaa438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/26/2025 03:01:04.738 7fd74adaa438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:01:04.770 7fd74dc4ef88 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.254:5673//', 'pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.48:5673//'] 08/26/2025 03:01:04.771 7fd74dc4ef88 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 08/26/2025 03:01:04.771 7fd73fea22e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:01:04.772 7fd73fea22e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Initializing description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 08/26/2025 03:01:04.772 7fd73fea22e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:01:04.772 7fd73fea22e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:01:04.772 7fd73fea22e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 08/26/2025 03:01:04.772 7fd73fea22e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:01:04.776 7fd74dc4ef88 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 08/26/2025 03:01:04.776 7fd73feaf2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:01:04.776 7fd73feaf2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/26/2025 03:01:04.776 7fd73feaf2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:01:04.800 7fd74dc4ef88 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 08/26/2025 03:01:04.800 7fd73febc438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:01:04.801 7fd73febc438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 08/26/2025 03:01:04.801 7fd73febc438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:01:04.804 7fd74dc4ef88 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 08/26/2025 03:01:04.804 7fd73fea2438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:01:04.804 7fd73fea2438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/26/2025 03:01:04.804 7fd73fea2438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:01:04.806 7fd73fe50390 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 08/26/2025 03:01:04.806 7fd73fe98390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:01:04.807 7fd73fe98390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 08/26/2025 03:01:04.807 7fd73fe98390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:01:04.814 7fd73fe50390 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 08/26/2025 03:01:04.814 7fd73fea8cc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:01:04.815 7fd73fea8cc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/26/2025 03:01:04.815 7fd73fea8cc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:01:07.802 7fd745fcbd68 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/26/2025 03:01:07.803 7fd745fcbd68 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/26/2025 03:01:07.803 7fd745fcbd68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/26/2025 03:01:07.803 7fd745fcbd68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:01:07.803 7fd745fcbd68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/26/2025 03:01:07.803 7fd745fcbd68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:01:07.803 7fd745fcbd68 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/26/2025 03:01:07.804 7fd745fcbd68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/26/2025 03:01:07.804 7fd745fcbd68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:01:07.804 7fd745fcbd68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/26/2025 03:01:07.804 7fd745fcbd68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:01:07.804 7fd745fcbd68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:01:07.804 7fd745fcbd68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1756177151739083 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/26/2025 03:01:07.804 7fd745fcbd68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:01:11.804 7fd745fcbd68 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/26/2025 03:01:11.806 7fd745fcbd68 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 08/26/2025 03:01:11.806 7fd745fcbd68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/26/2025 03:01:11.806 7fd745fcbd68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:01:11.806 7fd745fcbd68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/26/2025 03:01:11.806 7fd745fcbd68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:01:11.806 7fd745fcbd68 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 08/26/2025 03:01:11.806 7fd745fcbd68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 08/26/2025 03:01:11.807 7fd745fcbd68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:01:11.807 7fd745fcbd68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 08/26/2025 03:01:11.807 7fd745fcbd68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:01:11.807 7fd745fcbd68 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:01:11.807 7fd745fcbd68 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1756177151739083 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 08/26/2025 03:01:11.807 7fd745fcbd68 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/26/2025 03:01:15.643 7fd73feafac8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3935-1.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.254', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3935-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3935-1.'], 'id_perms': {'created': '2025-08-26T03:01:15.628103', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-08-26T03:01:15.628103', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12178445837787870300, 'uuid_mslong': 3533454921223194398}}, 'parent_type': 'global-system-config', 'parent_uuid': '951a748f-d93e-4fbd-81f4-5adea4ed5912', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '31095a87-a15d-471e-a902-87f71dae345c'}, 'oper': 'CREATE', 'request-id': 'req-5eebae05-8d83-484c-9470-00129505b500', 'type': 'analytics_node', 'uuid': '31095a87-a15d-471e-a902-87f71dae345c'} 08/26/2025 03:01:15.652 7fd73feafac8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3935-2.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.38', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3935-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3935-2.'], 'id_perms': {'created': '2025-08-26T03:01:15.624252', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-08-26T03:01:15.624252', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13566589813890706025, 'uuid_mslong': 10996376726786100750}}, 'parent_type': 'global-system-config', 'parent_uuid': '951a748f-d93e-4fbd-81f4-5adea4ed5912', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '989afa5f-2875-4e0e-bc46-358e8e636a69'}, 'oper': 'CREATE', 'request-id': 'req-dd5dbdb2-c651-4c52-aeac-ad6328cfeef5', 'type': 'analytics_node', 'uuid': '989afa5f-2875-4e0e-bc46-358e8e636a69'} 08/26/2025 03:01:15.678 7fd73feafac8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3935-3.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.48', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3935-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3935-3.'], 'id_perms': {'created': '2025-08-26T03:01:15.668644', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-08-26T03:01:15.668644', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11997497433063912927, 'uuid_mslong': 7425151483330643056}}, 'parent_type': 'global-system-config', 'parent_uuid': '951a748f-d93e-4fbd-81f4-5adea4ed5912', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '670b6fef-1763-4470-a67f-ac59936f75df'}, 'oper': 'CREATE', 'request-id': 'req-08e1980e-ec5d-41d6-a80c-16e420bf75e7', 'type': 'analytics_node', 'uuid': '670b6fef-1763-4470-a67f-ac59936f75df'} 08/26/2025 03:01:15.807 7fd73fe98198 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/26/2025 03:01:15.808 7fd73fe98198 [contrail-device-manager] [INFO]: Session Event: TCP Connected 08/26/2025 03:01:15.808 7fd73fe98198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/26/2025 03:01:15.808 7fd73fe98198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:01:15.808 7fd73fe98198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3935-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/26/2025 03:01:15.808 7fd73fe98198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/26/2025 03:01:15.808 7fd73fe98198 [contrail-device-manager] [INFO]: Processing event[EvTcpConnected] in state[Connect] 08/26/2025 03:01:15.808 7fd73fe98198 [contrail-device-manager] [DEBUG]: Send sandesh control message. uve type count # 19 08/26/2025 03:01:15.809 7fd73fe98198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 08/26/2025 03:01:15.809 7fd73fe98198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 08/26/2025 03:01:15.809 7fd73fe98198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 08/26/2025 03:01:15.809 7fd73fbc1198 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 08/26/2025 03:01:15.810 7fd73fbc1198 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 08/26/2025 03:01:29.274 7fd73fbc1eb8 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001016 08/26/2025 03:01:29.274 7fd73fbc1eb8 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 08/26/2025 03:01:29.275 7fd73fd6ad68 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 08/26/2025 03:01:29.275 7fd73fd6ad68 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 0 08/26/2025 03:01:29.275 7fd73fd6ad68 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 08/26/2025 03:01:29.275 7fd73fd6ad68 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 08/26/2025 03:01:29.277 7fd73fd6ad68 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 08/26/2025 03:01:29.278 7fd73fe98198 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 08/26/2025 03:01:29.279 7fd73fe98198 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 08/26/2025 03:01:29.952 7fd73feafac8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3935-1.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.254', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3935-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3935-1.'], 'id_perms': {'created': '2025-08-26T03:01:29.942771', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-08-26T03:01:29.942771', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10227270016004811401, 'uuid_mslong': 1250278128509469989}}, 'parent_type': 'global-system-config', 'parent_uuid': '951a748f-d93e-4fbd-81f4-5adea4ed5912', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '1159e155-683f-4525-8dee-8fe4a5f86a89'}, 'oper': 'CREATE', 'request-id': 'req-1ba04439-83c4-4aed-a3de-e613f3914636', 'type': 'analytics_alarm_node', 'uuid': '1159e155-683f-4525-8dee-8fe4a5f86a89'} 08/26/2025 03:01:30.318 7fd73feafac8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3935-2.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.38', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3935-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3935-2.'], 'id_perms': {'created': '2025-08-26T03:01:30.307334', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-08-26T03:01:30.307334', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9968675704548168019, 'uuid_mslong': 2710809455430156494}}, 'parent_type': 'global-system-config', 'parent_uuid': '951a748f-d93e-4fbd-81f4-5adea4ed5912', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '259ebad6-54af-40ce-8a57-d9bcaaa91153'}, 'oper': 'CREATE', 'request-id': 'req-59fd6660-c0c7-42a8-b15b-2a248cda204f', 'type': 'analytics_alarm_node', 'uuid': '259ebad6-54af-40ce-8a57-d9bcaaa91153'} 08/26/2025 03:01:31.178 7fd73feafac8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3935-3.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.48', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3935-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3935-3.'], 'id_perms': {'created': '2025-08-26T03:01:31.168111', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-08-26T03:01:31.168111', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13116514455941892211, 'uuid_mslong': 18196981413305338554}}, 'parent_type': 'global-system-config', 'parent_uuid': '951a748f-d93e-4fbd-81f4-5adea4ed5912', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'fc88aa2f-0473-4aba-b607-386745b14873'}, 'oper': 'CREATE', 'request-id': 'req-258ed61b-63cd-4b89-9073-33407a13e179', 'type': 'analytics_alarm_node', 'uuid': 'fc88aa2f-0473-4aba-b607-386745b14873'} 08/26/2025 03:01:39.249 7fd73fbc1eb8 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001296 08/26/2025 03:01:39.250 7fd73fbc1eb8 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 08/26/2025 03:01:39.250 7fd73fe98c18 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 08/26/2025 03:01:39.250 7fd73fe98c18 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 2 08/26/2025 03:01:39.250 7fd73fe98c18 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 08/26/2025 03:01:39.250 7fd73fe98c18 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 08/26/2025 03:01:40.468 7fd73feafac8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3935-3.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.48', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3935-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3935-3.'], 'id_perms': {'created': '2025-08-26T03:01:40.451802', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-08-26T03:01:40.451802', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9524016932053388155, 'uuid_mslong': 426434654505422506}}, 'parent_type': 'global-system-config', 'parent_uuid': '951a748f-d93e-4fbd-81f4-5adea4ed5912', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '05eb000f-15b6-4eaa-842c-1af075d90b7b'}, 'oper': 'CREATE', 'request-id': 'req-d5126797-6c41-4433-9f7d-f88ae09b7824', 'type': 'analytics_snmp_node', 'uuid': '05eb000f-15b6-4eaa-842c-1af075d90b7b'} 08/26/2025 03:01:40.526 7fd73feafac8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3935-2.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.38', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3935-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3935-2.'], 'id_perms': {'created': '2025-08-26T03:01:40.516776', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-08-26T03:01:40.516776', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9369979047306839658, 'uuid_mslong': 13523966199557472383}}, 'parent_type': 'global-system-config', 'parent_uuid': '951a748f-d93e-4fbd-81f4-5adea4ed5912', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'bbaec79a-ff03-407f-8208-da4c9b93c66a'}, 'oper': 'CREATE', 'request-id': 'req-fd9a8bb7-645c-4c02-8671-1508d22a028f', 'type': 'analytics_snmp_node', 'uuid': 'bbaec79a-ff03-407f-8208-da4c9b93c66a'} 08/26/2025 03:01:40.632 7fd73feafac8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3935-1.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.254', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-3935-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-3935-1.'], 'id_perms': {'created': '2025-08-26T03:01:40.616545', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-08-26T03:01:40.616545', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10969043171520249509, 'uuid_mslong': 172930259979747331}}, 'parent_type': 'global-system-config', 'parent_uuid': '951a748f-d93e-4fbd-81f4-5adea4ed5912', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '02665f27-bd02-4003-9839-dea5e4637ea5'}, 'oper': 'CREATE', 'request-id': 'req-31ea3252-1530-42d6-891b-c0e653081cce', 'type': 'analytics_snmp_node', 'uuid': '02665f27-bd02-4003-9839-dea5e4637ea5'} 08/26/2025 03:03:23.790 7fd73feafac8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-5d7de10b-4a95-475f-87c3-9f4ae27467b5', 'type': 'global_system_config', 'uuid': '951a748f-d93e-4fbd-81f4-5adea4ed5912'} 08/26/2025 03:03:23.850 7fd73feafac8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-a058c0c4-eca1-41de-9bca-0a65fe685de8', 'type': 'global_system_config', 'uuid': '951a748f-d93e-4fbd-81f4-5adea4ed5912'} 08/26/2025 03:03:23.928 7fd73feafac8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3935-1'], 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-3935-1', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3935-1'], 'id_perms': {'created': '2025-08-26T03:03:23.920620', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-08-26T03:03:23.920620', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12901422293053527624, 'uuid_mslong': 1070111475532974354}}, 'parent_type': 'global-system-config', 'parent_uuid': '951a748f-d93e-4fbd-81f4-5adea4ed5912', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '0ed9ccb7-a136-4912-b30b-0f388ac2d648', 'virtual_router_dpdk_enabled': False, 'virtual_router_ip_address': '10.20.0.25'}, 'oper': 'CREATE', 'request-id': 'req-8ef909cb-1df0-48f5-a3be-6e6f5ee39b1e', 'type': 'virtual_router', 'uuid': '0ed9ccb7-a136-4912-b30b-0f388ac2d648'} 08/26/2025 03:03:23.945 7fd73feafac8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3935-2'], 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-3935-2', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3935-2'], 'id_perms': {'created': '2025-08-26T03:03:23.925218', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-08-26T03:03:23.925218', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12110133203293651995, 'uuid_mslong': 6182494975770708079}}, 'parent_type': 'global-system-config', 'parent_uuid': '951a748f-d93e-4fbd-81f4-5adea4ed5912', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '55cca26c-9873-486f-a80f-d5fc74fb441b', 'virtual_router_dpdk_enabled': False, 'virtual_router_ip_address': '10.20.0.193'}, 'oper': 'CREATE', 'request-id': 'req-36a0e8ab-c6e1-4a2e-9db4-698e722bf4de', 'type': 'virtual_router', 'uuid': '55cca26c-9873-486f-a80f-d5fc74fb441b'} 08/26/2025 03:03:24.308 7fd73feafac8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3935-1', 'vhost0'], 'obj_dict': {'display_name': 'vhost0', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3935-1', 'vhost0'], 'id_perms': {'created': '2025-08-26T03:03:24.282413', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-08-26T03:03:24.282413', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11262494764253201902, 'uuid_mslong': 6323859377745905077}}, 'parent_type': 'virtual-router', 'parent_uuid': '0ed9ccb7-a136-4912-b30b-0f388ac2d648', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '57c2dc99-4c11-45b5-9c4c-6b511c0245ee', 'virtual_machine_interface_disable_policy': True, 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:57:c2:dc:99:4c']}, 'virtual_network_refs': [{'to': ['default-domain', 'default-project', 'ip-fabric'], 'uuid': '3f99dc03-204c-48da-b9dd-6093c5aa8264'}]}, 'oper': 'CREATE', 'request-id': 'req-f5459cac-2645-4d6d-b01e-1b3d07811f89', 'type': 'virtual_machine_interface', 'uuid': '57c2dc99-4c11-45b5-9c4c-6b511c0245ee'} 08/26/2025 03:03:24.345 7fd73feafac8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3935-2', 'vhost0'], 'obj_dict': {'display_name': 'vhost0', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3935-2', 'vhost0'], 'id_perms': {'created': '2025-08-26T03:03:24.294064', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-08-26T03:03:24.294064', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9841646601541565072, 'uuid_mslong': 12276190320718594620}}, 'parent_type': 'virtual-router', 'parent_uuid': '55cca26c-9873-486f-a80f-d5fc74fb441b', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'aa5dca0d-fd85-423c-8894-8d70d9d0ba90', 'virtual_machine_interface_disable_policy': True, 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:aa:5d:ca:0d:fd']}, 'virtual_network_refs': [{'to': ['default-domain', 'default-project', 'ip-fabric'], 'uuid': '3f99dc03-204c-48da-b9dd-6093c5aa8264'}]}, 'oper': 'CREATE', 'request-id': 'req-b94ed69d-4d6a-4e90-8bcf-400782fdd3d5', 'type': 'virtual_machine_interface', 'uuid': 'aa5dca0d-fd85-423c-8894-8d70d9d0ba90'} 08/26/2025 03:03:24.376 7fd73feafac8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3935-1', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-f5459cac-2645-4d6d-b01e-1b3d07811f89', 'type': 'virtual_machine_interface', 'uuid': '57c2dc99-4c11-45b5-9c4c-6b511c0245ee'} 08/26/2025 03:03:24.407 7fd73feafac8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-3935-2', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-b94ed69d-4d6a-4e90-8bcf-400782fdd3d5', 'type': 'virtual_machine_interface', 'uuid': 'aa5dca0d-fd85-423c-8894-8d70d9d0ba90'} 08/26/2025 03:03:24.847 7fd73feafac8 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'oper': 'UPDATE', 'request-id': 'req-07b06493-c089-4b15-be26-492ff17a3251', 'type': 'access_control_list', 'uuid': '42df6ff7-56c0-4d9a-b2e3-3882fd34d088'} 08/26/2025 03:03:57.809 7fd73fe6cf60 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 08/26/2025 03:04:21.075 7fd73fe6cf60 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 08/26/2025 03:06:14.148 7fd73fe7ec18 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map