02/13/2025 02:16:36.181 7f0bc4ce9c48 [contrail-api] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 02/13/2025 02:16:36.181 7f0bc4ce9c48 [contrail-api] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 02/13/2025 02:16:36.181 7f0bc4ce9c48 [contrail-api] [INFO]: SANDESH: Flow Logging: [None] -> [False] 02/13/2025 02:16:36.182 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Connecting to zookeeper on 10.0.0.17:2181 02/13/2025 02:16:36.182 7f0bc4ce9c48 [contrail-api] [DEBUG]: Add uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:16:36.183 7f0bbe59de10 [contrail-api] [ERROR]: Cannot write http_port 8084 to /tmp/contrail-api.0.http_port 02/13/2025 02:16:36.183 7f0bbe59de10 [contrail-api] [ERROR]: Starting Introspect on HTTP Port 8084 02/13/2025 02:16:36.183 7f0bbe5a0780 [contrail-api] [INFO]: Processing event[EvStart] in state[none] 02/13/2025 02:16:36.184 7f0bbe5a0780 [contrail-api] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/13/2025 02:16:36.184 7f0bbe5a0780 [contrail-api] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 02/13/2025 02:16:36.184 7f0bbe5a0780 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:16:36.185 7f0bbe5a0780 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Initializing description = >>, ] description = Zookeeper:Zookeeper[] connection down >>, ] >> 02/13/2025 02:16:36.185 7f0bbe5a0780 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:16:36.185 7f0bbe5a0780 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:16:36.185 7f0bbe5a0780 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Initializing description = >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Zookeeper:Zookeeper[], Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/13/2025 02:16:36.185 7f0bbe5a0780 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:16:36.185 7f0bbe5a0780 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:16:36.185 7f0bbe5a0780 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 2 >> 02/13/2025 02:16:36.185 7f0bbe5a0780 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:16:36.192 7f0bbe508390 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:16:36.193 7f0bbe508390 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/13/2025 02:16:36.193 7f0bbe508390 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:16:36.267 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Connecting to database on ['10.0.0.17:9041'] 02/13/2025 02:16:36.268 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: VNCCassandra started with driver cql 02/13/2025 02:16:36.268 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: initializing cassandra cluster 02/13/2025 02:16:36.268 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: CassandraDriverCQL.__init__Cluster before create 02/13/2025 02:16:36.271 7f0bbe0336d8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:16:36.272 7f0bbe0336d8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 02/13/2025 02:16:36.272 7f0bbe0336d8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:16:36.276 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: CassandraDriverCQL.__init__Cluster created 02/13/2025 02:16:36.745 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: CassandraDriverCQL.__init__Cluster DB connected 02/13/2025 02:16:36.747 7f0bbd469978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:16:36.747 7f0bbd469978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Initializing description = >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Initializing description = >>, ] description = Zookeeper:Zookeeper[], Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 02/13/2025 02:16:36.747 7f0bbd469978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:16:36.753 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: CassandraDriverCQL.__init__Cluster init KS 02/13/2025 02:16:36.754 7f0bbe04b828 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:16:36.754 7f0bbe04b828 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 02/13/2025 02:16:36.754 7f0bbe04b828 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:16:36.816 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock acquired. Performing schema change... 02/13/2025 02:16:36.975 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: keyspace 'config_db_uuid', created 02/13/2025 02:16:36.981 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock released. 02/13/2025 02:16:36.989 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock acquired. Performing schema change... 02/13/2025 02:16:37.151 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: keyspace 'config_db_uuid' fixed with replication {'class': 'SimpleStrategy', 'replication_factor': None} 02/13/2025 02:16:37.155 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock released. 02/13/2025 02:16:37.163 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock acquired. Performing schema change... 02/13/2025 02:16:37.297 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: keyspace 'useragent', created 02/13/2025 02:16:37.301 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock released. 02/13/2025 02:16:37.308 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock acquired. Performing schema change... 02/13/2025 02:16:37.400 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: keyspace 'useragent' fixed with replication {'class': 'SimpleStrategy', 'replication_factor': None} 02/13/2025 02:16:37.404 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock released. 02/13/2025 02:16:37.405 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: CassandraDriverCQL.__init__Cluster KS are ready 02/13/2025 02:16:37.566 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: CassandraDriverCQL.__init__Cluster CF are ready 02/13/2025 02:16:37.577 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock acquired. Performing schema change... 02/13/2025 02:16:37.777 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: table 'obj_uuid_table', created 02/13/2025 02:16:37.782 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock released. 02/13/2025 02:16:37.797 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock acquired. Performing schema change... 02/13/2025 02:16:37.935 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: table 'obj_uuid_table' fixed with properties {'gc_grace_seconds': 864000} 02/13/2025 02:16:37.951 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock released. 02/13/2025 02:16:37.957 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock acquired. Performing schema change... 02/13/2025 02:16:38.158 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: table 'obj_fq_name_table', created 02/13/2025 02:16:38.162 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock released. 02/13/2025 02:16:38.172 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock acquired. Performing schema change... 02/13/2025 02:16:38.298 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: table 'obj_fq_name_table' fixed with properties {'gc_grace_seconds': 864000} 02/13/2025 02:16:38.303 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock released. 02/13/2025 02:16:38.312 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock acquired. Performing schema change... 02/13/2025 02:16:38.473 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: table 'obj_shared_table', created 02/13/2025 02:16:38.478 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock released. 02/13/2025 02:16:38.486 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock acquired. Performing schema change... 02/13/2025 02:16:38.610 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: table 'obj_shared_table' fixed with properties {'gc_grace_seconds': 864000} 02/13/2025 02:16:38.615 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock released. 02/13/2025 02:16:38.622 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock acquired. Performing schema change... 02/13/2025 02:16:38.809 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: table 'useragent_keyval_table', created 02/13/2025 02:16:38.816 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock released. 02/13/2025 02:16:38.826 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock acquired. Performing schema change... 02/13/2025 02:16:38.936 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: table 'useragent_keyval_table' fixed with properties {'gc_grace_seconds': 864000} 02/13/2025 02:16:38.940 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock released. 02/13/2025 02:16:38.941 7f0bbe54dc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:16:38.941 7f0bbe54dc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/13/2025 02:16:38.941 7f0bbe54dc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:16:39.064 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.17:5673//'] 02/13/2025 02:16:39.073 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: RabbitMQ drainer connection down 02/13/2025 02:16:39.074 7f0bbe54f198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:16:39.075 7f0bbe54f198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 02/13/2025 02:16:39.075 7f0bbe54f198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:16:39.075 7f0bbe54f198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:16:39.075 7f0bbe54f198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 02/13/2025 02:16:39.075 7f0bbe54f198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:16:40.184 7f0bbe5b7588 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:16:40.192 7f0bbe5b7588 [contrail-api] [INFO]: Session Event: TCP Connected 02/13/2025 02:16:40.192 7f0bbe5b7588 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:16:40.192 7f0bbe5b7588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:16:40.192 7f0bbe5b7588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Down description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:RabbitMQ[] connection down >>, ] >> 02/13/2025 02:16:40.192 7f0bbe5b7588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:16:40.192 7f0bbe5b7588 [contrail-api] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/13/2025 02:16:40.192 7f0bbe5b7588 [contrail-api] [DEBUG]: Send sandesh control message. uve type count # 19 02/13/2025 02:16:40.193 7f0bbe5b7588 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/13/2025 02:16:40.193 7f0bbe5b7588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/13/2025 02:16:40.193 7f0bbe5b7588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/13/2025 02:16:40.193 7f0bbe5ad978 [contrail-api] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/13/2025 02:16:40.195 7f0bbe5ad978 [contrail-api] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/13/2025 02:16:40.197 7f0bbe536b70 [contrail-api] [DEBUG]: Extracted sandesh length: 0000001011 02/13/2025 02:16:40.197 7f0bbe536b70 [contrail-api] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 02/13/2025 02:16:40.198 7f0bbe4efcc0 [contrail-api] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 02/13/2025 02:16:40.198 7f0bbe4efcc0 [contrail-api] [DEBUG]: Number of uve types in sandesh control message is 0 02/13/2025 02:16:40.198 7f0bbe4efcc0 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:16:40.198 7f0bbe4efcc0 [contrail-api] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/13/2025 02:16:40.198 7f0bbe4efcc0 [contrail-api] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 02/13/2025 02:16:40.199 7f0bbe5ad4e0 [contrail-api] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/13/2025 02:16:40.200 7f0bbe5ad4e0 [contrail-api] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/13/2025 02:16:59.122 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: RabbitMQ drainer connection ESTABLISHED 02/13/2025 02:16:59.145 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: RabbitMQ publish connection down 02/13/2025 02:16:59.167 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: RabbitMQ publish connection ESTABLISHED 02/13/2025 02:16:59.171 7f0bbe62c828 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: RabbitMQ publish connection down 02/13/2025 02:16:59.177 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-global-system-config']: Unknown id: global_system_config default-global-system-config 02/13/2025 02:16:59.199 7f0bbe62c828 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: RabbitMQ publish connection ESTABLISHED 02/13/2025 02:16:59.242 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-global-system-config', 'default-global-vrouter-config']: Unknown id: global_vrouter_config default-global-system-config:default-global-vrouter-config 02/13/2025 02:16:59.320 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-domain']: Unknown id: domain default-domain 02/13/2025 02:16:59.569 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-global-system-config', 'default-fabric']: Unknown id: fabric default-global-system-config:default-fabric 02/13/2025 02:16:59.668 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-policy-management']: Unknown id: policy_management default-policy-management 02/13/2025 02:16:59.718 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-policy-management', 'default-application-policy-set']: Unknown id: application_policy_set default-policy-management:default-application-policy-set 02/13/2025 02:16:59.874 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-domain', 'default-project', 'ip-fabric']: Unknown id: virtual_network default-domain:default-project:ip-fabric 02/13/2025 02:17:00.030 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric']: Unknown id: routing_instance default-domain:default-project:ip-fabric:ip-fabric 02/13/2025 02:17:00.092 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-domain', 'default-project', 'ip-fabric', '__default__']: Unknown id: routing_instance default-domain:default-project:ip-fabric:__default__ 02/13/2025 02:17:00.123 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-domain', 'default-project', '__link_local__']: Unknown id: virtual_network default-domain:default-project:__link_local__ 02/13/2025 02:17:00.240 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-domain', 'default-project', '__link_local__', '__link_local__']: Unknown id: routing_instance default-domain:default-project:__link_local__:__link_local__ 02/13/2025 02:17:00.276 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-domain', 'default-project', 'dci-network']: Unknown id: virtual_network default-domain:default-project:dci-network 02/13/2025 02:17:00.394 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-domain', 'default-project', 'dci-network', 'dci-network']: Unknown id: routing_instance default-domain:default-project:dci-network:dci-network 02/13/2025 02:17:00.454 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-domain', 'default-project', 'dci-network', '__default__']: Unknown id: routing_instance default-domain:default-project:dci-network:__default__ 02/13/2025 02:17:00.535 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-global-system-config', 'system-defined-bottle-request-size-limit']: Unknown id: alarm default-global-system-config:system-defined-bottle-request-size-limit 02/13/2025 02:17:00.578 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-domain', 'default-project', 'default-virtual-network', 'default-virtual-network']: Unknown id: routing_instance default-domain:default-project:default-virtual-network:default-virtual-network 02/13/2025 02:17:00.645 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-domain', 'default-project', '__no_rule__']: Unknown id: security_group default-domain:default-project:__no_rule__ 02/13/2025 02:17:00.705 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-discovery-service-assignment']: Unknown id: discovery_service_assignment default-discovery-service-assignment 02/13/2025 02:17:00.733 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-global-system-config', 'default-global-qos-config']: Unknown id: global_qos_config default-global-system-config:default-global-qos-config 02/13/2025 02:17:00.768 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-domain', 'default-project', 'service-chain-flat-ipam']: Unknown id: network_ipam default-domain:default-project:service-chain-flat-ipam 02/13/2025 02:17:00.801 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['label']: Unknown id: tag_type label 02/13/2025 02:17:00.836 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['application']: Unknown id: tag_type application 02/13/2025 02:17:00.873 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['tier']: Unknown id: tag_type tier 02/13/2025 02:17:00.903 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['deployment']: Unknown id: tag_type deployment 02/13/2025 02:17:00.951 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['site']: Unknown id: tag_type site 02/13/2025 02:17:00.981 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['neutron_fwaas']: Unknown id: tag_type neutron_fwaas 02/13/2025 02:17:01.012 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['neutron_tag']: Unknown id: tag_type neutron_tag 02/13/2025 02:17:01.048 7f0bbe5ebac8 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: Executing db_resync on api-server instance 10.0.0.17_0 02/13/2025 02:17:01.048 7f0bbe5ebac8 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: Contrail Version is(nightly) 02/13/2025 02:17:01.085 7f0bbe5ebac8 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: zk_sync_node data (None) 02/13/2025 02:17:01.085 7f0bbe5ebac8 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: DBERESYNC: running dbe-resync 02/13/2025 02:17:01.094 7f0bbe5ebac8 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: DB walk: obj_type namespace len 1 02/13/2025 02:17:01.094 7f0bbe5ebac8 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Start DB Resync for namespace 02/13/2025 02:17:01.107 7f0bbe5ebac8 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Finished DB Resync for namespace 02/13/2025 02:17:01.107 7f0bbe5ebac8 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: DB walk: obj_type tag_type len 7 02/13/2025 02:17:01.107 7f0bbe5ebac8 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Start DB Resync for tag_type 02/13/2025 02:17:01.127 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Error in _create_default_rbac_rule: NoIdError: Unknown id: api_access_list default-global-system-config:default-api-access-list 02/13/2025 02:17:01.130 7f0bbe5ebac8 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Finished DB Resync for tag_type 02/13/2025 02:17:01.130 7f0bbe5ebac8 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: DB walk: obj_type policy_management len 1 02/13/2025 02:17:01.130 7f0bbe5ebac8 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Start DB Resync for policy_management 02/13/2025 02:17:01.131 7f0bc4ce9c48 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-global-system-config', 'default-api-access-list']: Unknown id: api_access_list default-global-system-config:default-api-access-list 02/13/2025 02:17:01.141 7f0bbe5ebac8 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Finished DB Resync for policy_management 02/13/2025 02:17:01.141 7f0bbe5ebac8 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: DB walk: obj_type global_vrouter_config len 1 02/13/2025 02:17:01.142 7f0bbe5ebac8 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Start DB Resync for global_vrouter_config 02/13/2025 02:17:01.150 7f0bbe5ebac8 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Finished DB Resync for global_vrouter_config 02/13/2025 02:17:01.150 7f0bbe5ebac8 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: DB walk: obj_type virtual_network len 4 02/13/2025 02:17:01.150 7f0bbe5ebac8 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Start DB Resync for virtual_network 02/13/2025 02:17:01.258 7f0bbe59bac8 [contrail-api] [DEBUG]: Add uve in the [ObjectVNTable:ContrailConfigTrace] map 02/13/2025 02:17:01.259 7f0bbe5366d8 [contrail-api] [DEBUG]: Add uve in the [ObjectVNTable:ContrailConfigTrace] map 02/13/2025 02:17:01.259 7f0bbe5bc978 [contrail-api] [DEBUG]: Add uve in the [ObjectVNTable:ContrailConfigTrace] map 02/13/2025 02:17:01.260 7f0bbe62c438 [contrail-api] [DEBUG]: Add uve in the [ObjectVNTable:ContrailConfigTrace] map 02/13/2025 02:17:01.260 7f0bbd469978 [contrail-api] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/13/2025 02:17:01.262 7f0bbd469978 [contrail-api] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/13/2025 02:17:01.263 7f0bbe5ebac8 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Finished DB Resync for virtual_network 02/13/2025 02:17:01.263 7f0bbe5ebac8 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: DB walk: obj_type project len 1 02/13/2025 02:17:01.263 7f0bbe5ebac8 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Start DB Resync for project 02/13/2025 02:17:01.272 7f0bbe54f198 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-domain', 'default-project', 'default-application-policy-set']: Unknown id: application_policy_set default-domain:default-project:default-application-policy-set 02/13/2025 02:17:01.615 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: Starting consumer greenlet 02/13/2025 02:17:01.616 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: Ensuring Consumer connection 02/13/2025 02:17:01.616 7f0bbdb29a20 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: Starting publisher greenlet 02/13/2025 02:17:01.617 7f0bbdb29a20 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: Ensuring Publisher connection 02/13/2025 02:17:01.619 7f0bc4ce9c48 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: Added consumer: amqp_request.cn-jenkins-deploy-platform-juju-os-473-1.dummy 02/13/2025 02:17:01.620 7f0bc4ce9c48 [contrail-api] [DEBUG]: Add uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:17:01.633 7f0bbde2da20 [contrail-api] [DEBUG]: Add uve in the [ObjectProjectTable:ContrailConfigTrace] map 02/13/2025 02:17:01.634 7f0bbe5ebac8 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Finished DB Resync for project 02/13/2025 02:17:01.635 7f0bbe5ebac8 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: DB walk: obj_type global_system_config len 1 02/13/2025 02:17:01.635 7f0bbe5ebac8 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Start DB Resync for global_system_config 02/13/2025 02:17:01.660 7f0bbe5ebac8 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Finished DB Resync for global_system_config 02/13/2025 02:17:01.664 7f0bbe5ebac8 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: DB walk: obj_type application_policy_set len 1 02/13/2025 02:17:01.664 7f0bbe5ebac8 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Start DB Resync for application_policy_set 02/13/2025 02:17:01.676 7f0bbe5f6780 [contrail-api] [DEBUG]: Add uve in the [ObjectApplicationPolicySetTable:ContrailConfigTrace] map 02/13/2025 02:17:01.677 7f0bbe5ebac8 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Finished DB Resync for application_policy_set 02/13/2025 02:17:01.677 7f0bbe5ebac8 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: DB walk: obj_type security_group len 1 02/13/2025 02:17:01.677 7f0bbe5ebac8 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Start DB Resync for security_group 02/13/2025 02:17:01.690 7f0bbe5ebac8 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Finished DB Resync for security_group 02/13/2025 02:17:01.691 7f0bbe5ebac8 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: DB walk: obj_type discovery_service_assignment len 1 02/13/2025 02:17:01.691 7f0bbe5ebac8 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Start DB Resync for discovery_service_assignment 02/13/2025 02:17:01.695 7f0bbdb298d0 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: KombuAmqpClient: Consumer connection established 02/13/2025 02:17:01.697 7f0bbe5ebac8 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Finished DB Resync for discovery_service_assignment 02/13/2025 02:17:01.697 7f0bbe5ebac8 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: DB walk: obj_type domain len 1 02/13/2025 02:17:01.697 7f0bbe5ebac8 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Start DB Resync for domain 02/13/2025 02:17:01.704 7f0bbdb29a20 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: KombuAmqpClient: Publisher connection established 02/13/2025 02:17:01.718 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: Created consumers ['amqp_request.cn-jenkins-deploy-platform-juju-os-473-1.dummy'] 02/13/2025 02:17:01.726 7f0bbe5ebac8 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Finished DB Resync for domain 02/13/2025 02:17:01.726 7f0bbe5ebac8 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: DB walk: obj_type network_ipam len 2 02/13/2025 02:17:01.726 7f0bbe5ebac8 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Start DB Resync for network_ipam 02/13/2025 02:17:01.732 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: Ensuring Consumer connection 02/13/2025 02:17:01.735 7f0bbe5ebac8 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Finished DB Resync for network_ipam 02/13/2025 02:17:01.736 7f0bbe5ebac8 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: DB walk: obj_type global_qos_config len 1 02/13/2025 02:17:01.736 7f0bbe5ebac8 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Start DB Resync for global_qos_config 02/13/2025 02:17:01.741 7f0bbe5ebac8 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Finished DB Resync for global_qos_config 02/13/2025 02:17:01.741 7f0bbe5ebac8 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: DB walk: obj_type alarm len 1 02/13/2025 02:17:01.741 7f0bbe5ebac8 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Start DB Resync for alarm 02/13/2025 02:17:01.745 7f0bbe5ebac8 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Finished DB Resync for alarm 02/13/2025 02:17:01.745 7f0bbe5ebac8 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: DB walk: obj_type routing_instance len 6 02/13/2025 02:17:01.745 7f0bbe5ebac8 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Start DB Resync for routing_instance 02/13/2025 02:17:01.756 7f0bbdb298d0 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: KombuAmqpClient: Consumer connection established 02/13/2025 02:17:01.763 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: Created consumers ['amqp_request.cn-jenkins-deploy-platform-juju-os-473-1.dummy'] 02/13/2025 02:17:01.769 7f0bbe5ebac8 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Finished DB Resync for routing_instance 02/13/2025 02:17:01.769 7f0bbe5ebac8 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: DB walk: obj_type fabric len 1 02/13/2025 02:17:01.769 7f0bbe5ebac8 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Start DB Resync for fabric 02/13/2025 02:17:01.773 7f0bbe5ebac8 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Finished DB Resync for fabric 02/13/2025 02:17:01.777 7f0bbe5ebac8 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: DBERESYNC: Completed successfully 02/13/2025 02:17:01.781 7f0bbe5ebac8 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: Cassandra DB walk completed. 02/13/2025 02:17:01.799 7f0bbe5ebac8 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Time elapsed in resyncing db: 0:00:00.750895 02/13/2025 02:17:01.801 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config'], 'obj_dict': {'autonomous_system': 64512, 'config_version': '1.0', 'enable_4byte_as': False, 'enable_security_policy_draft': False, 'fq_name': ['default-global-system-config'], 'ibgp_auto_mesh': True, 'id_perms': {'created': '2025-02-13T02:16:59.213351', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:16:59.213351', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9893212966702303077, 'uuid_mslong': 4206571152036286153}}, 'igmp_enable': False, 'mac_aging_time': 300, 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '3a60be2c-e787-4ec9-894b-c0c6d7944765'}, 'oper': 'CREATE', 'request-id': 'req-5ea00caa-b00f-4dd7-abda-3f951289565b', 'type': 'global_system_config', 'uuid': '3a60be2c-e787-4ec9-894b-c0c6d7944765'} 02/13/2025 02:17:01.808 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'default-global-vrouter-config'], 'obj_dict': {'enable_security_logging': True, 'encryption_mode': 'none', 'fq_name': ['default-global-system-config', 'default-global-vrouter-config'], 'id_perms': {'created': '2025-02-13T02:16:59.287792', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:16:59.287792', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11270242003857873719, 'uuid_mslong': 15315686202061767009}}, 'parent_type': 'global-system-config', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'd48c3f74-6d3e-4161-9c67-f163e4a29b37'}, 'oper': 'CREATE', 'request-id': 'req-34304362-0d89-40e9-939e-fabe9a7a9feb', 'type': 'global_vrouter_config', 'uuid': 'd48c3f74-6d3e-4161-9c67-f163e4a29b37'} 02/13/2025 02:17:01.812 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain'], 'obj_dict': {'fq_name': ['default-domain'], 'id_perms': {'created': '2025-02-13T02:16:59.336974', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:16:59.336974', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11257628960460771073, 'uuid_mslong': 5121569286561218665}}, 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '47137821-8208-4069-9c3b-21e4e16a0f01'}, 'oper': 'CREATE', 'request-id': 'req-dc039515-1818-48e4-88fb-05fcede82025', 'type': 'domain', 'uuid': '47137821-8208-4069-9c3b-21e4e16a0f01'} 02/13/2025 02:17:01.814 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-namespace'], 'obj_dict': {'_type': 'namespace', '_uuid': None, 'fq_name': ['default-domain', 'default-namespace'], 'id_perms': {'created': '2025-02-13T02:16:59.399727', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:16:59.399727', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9800113867070885766, 'uuid_mslong': 8483151534024311537}}, 'name': 'default-namespace', 'parent_type': 'domain', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '75ba355f-e154-42f1-8800-ffa38d23cf86'}, 'oper': 'CREATE', 'request-id': 'req-27668f03-57b9-4123-a647-873f568beba6', 'type': 'namespace', 'uuid': '75ba355f-e154-42f1-8800-ffa38d23cf86'} 02/13/2025 02:17:01.817 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project'], 'obj_dict': {'_enable_security_policy_draft': False, '_type': 'project', '_uuid': None, 'fq_name': ['default-domain', 'default-project'], 'id_perms': {'created': '2025-02-13T02:16:59.451021', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:16:59.451021', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13386698883600576163, 'uuid_mslong': 17239180990840914962}}, 'name': 'default-project', 'parent_type': 'domain', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'ef3ddfc6-29bd-4412-b9c7-1bb578216ea3'}, 'oper': 'CREATE', 'request-id': 'req-c8378c2a-9b80-472c-9397-1979062accfa', 'type': 'project', 'uuid': 'ef3ddfc6-29bd-4412-b9c7-1bb578216ea3'} 02/13/2025 02:17:01.822 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'default-virtual-network'], 'obj_dict': {'_fabric_snat': False, '_flood_unknown_unicast': False, '_igmp_enable': False, '_is_provider_network': False, '_layer2_control_word': False, '_mac_aging_time': 300, '_mac_ip_learning_enable': False, '_mac_learning_enabled': False, '_pbb_etree_enable': False, '_pbb_evpn_enable': False, '_port_security_enabled': True, '_type': 'virtual-network', '_uuid': None, '_virtual_network_category': 'tenant', 'fq_name': ['default-domain', 'default-project', 'default-virtual-network'], 'id_perms': {'created': '2025-02-13T02:16:59.502876', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:16:59.502876', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11507453288406421738, 'uuid_mslong': 525346901882127951}}, 'name': 'default-virtual-network', 'parent_type': 'project', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '074a6839-4e89-464f-9fb2-afcbf4dd90ea', 'virtual_network_network_id': 1}, 'oper': 'CREATE', 'request-id': 'req-7388ddab-04ce-45f6-8dec-ce9b5c9d3bcb', 'type': 'virtual_network', 'uuid': '074a6839-4e89-464f-9fb2-afcbf4dd90ea'} 02/13/2025 02:17:01.824 7f0bc26c3048 [contrail-api] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/13/2025 02:17:01.830 7f0bc26c3048 [contrail-api] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/13/2025 02:17:01.836 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'default-network-ipam'], 'obj_dict': {'_type': 'network-ipam', '_uuid': None, 'fq_name': ['default-domain', 'default-project', 'default-network-ipam'], 'id_perms': {'created': '2025-02-13T02:16:59.534321', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:16:59.534321', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9546217687268875350, 'uuid_mslong': 10483617627098401469}}, 'name': 'default-network-ipam', 'parent_type': 'project', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '917d4aaf-f039-4abd-847a-fa6896365c56'}, 'oper': 'CREATE', 'request-id': 'req-99faaca7-39ed-4d90-afd3-3865fe950b0c', 'type': 'network_ipam', 'uuid': '917d4aaf-f039-4abd-847a-fa6896365c56'} 02/13/2025 02:17:01.838 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'default-fabric'], 'obj_dict': {'disable_vlan_vn_uniqueness_check': False, 'fabric_enterprise_style': True, 'fq_name': ['default-global-system-config', 'default-fabric'], 'id_perms': {'created': '2025-02-13T02:16:59.588988', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:16:59.588988', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9534951498705997355, 'uuid_mslong': 13165060110593510353}}, 'parent_type': 'global-system-config', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'b6b3b06b-4a97-4bd1-8452-f3dea6f30e2b'}, 'oper': 'CREATE', 'request-id': 'req-2b225399-e688-4b4a-b0bf-d35347b03613', 'type': 'fabric', 'uuid': 'b6b3b06b-4a97-4bd1-8452-f3dea6f30e2b'} 02/13/2025 02:17:01.840 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management'], 'obj_dict': {'fq_name': ['default-policy-management'], 'id_perms': {'created': '2025-02-13T02:16:59.691770', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:16:59.691770', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11158475401282616834, 'uuid_mslong': 8276385234333353001}}, 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '72dba085-3cba-4429-9ada-de41e0627e02'}, 'oper': 'CREATE', 'request-id': 'req-c6e8da7d-00b1-42a1-a5c8-c06b3000f46e', 'type': 'policy_management', 'uuid': '72dba085-3cba-4429-9ada-de41e0627e02'} 02/13/2025 02:17:01.842 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'default-application-policy-set'], 'obj_dict': {'all_applications': True, 'fq_name': ['default-policy-management', 'default-application-policy-set'], 'id_perms': {'created': '2025-02-13T02:16:59.745329', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:16:59.745329', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10862885150791774229, 'uuid_mslong': 14175996238254655550}}, 'parent_type': 'policy-management', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'c4bb4178-9671-483e-96c0-b87d98cf3015'}, 'oper': 'CREATE', 'request-id': 'req-cbf20923-a25e-437d-b930-b5e1a4ba2f26', 'type': 'application_policy_set', 'uuid': 'c4bb4178-9671-483e-96c0-b87d98cf3015'} 02/13/2025 02:17:01.843 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'default-application-policy-set'], 'oper': 'UPDATE', 'request-id': 'req-439f63c5-ef35-4a7a-96c5-b2510ad8b3c4', 'type': 'application_policy_set', 'uuid': 'c4bb4178-9671-483e-96c0-b87d98cf3015'} 02/13/2025 02:17:01.851 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric'], 'obj_dict': {'fabric_snat': False, 'flood_unknown_unicast': False, 'fq_name': ['default-domain', 'default-project', 'ip-fabric'], 'id_perms': {'created': '2025-02-13T02:16:59.924499', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:16:59.924499', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11165455373526359449, 'uuid_mslong': 3272829523783995366}}, 'igmp_enable': False, 'is_provider_network': True, 'layer2_control_word': False, 'mac_aging_time': 300, 'mac_ip_learning_enable': False, 'mac_learning_enabled': False, 'parent_type': 'project', 'pbb_etree_enable': False, 'pbb_evpn_enable': False, 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'port_security_enabled': True, 'uuid': '2d6b6d1c-978e-4fe6-9af3-aa814e83dd99', 'virtual_network_category': 'tenant', 'virtual_network_network_id': 2}, 'oper': 'CREATE', 'request-id': 'req-2b9665c3-5517-42ed-80ee-1305c600d361', 'type': 'virtual_network', 'uuid': '2d6b6d1c-978e-4fe6-9af3-aa814e83dd99'} 02/13/2025 02:17:01.852 7f0bbb5b1f60 [contrail-api] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/13/2025 02:17:01.853 7f0bbb5b1f60 [contrail-api] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/13/2025 02:17:01.856 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'obj_dict': {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'id_perms': {'created': '2025-02-13T02:17:00.060194', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:00.060194', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12255776485531849281, 'uuid_mslong': 5878541369752506314}}, 'parent_type': 'virtual-network', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'routing_instance_fabric_snat': False, 'routing_instance_has_pnf': False, 'routing_instance_is_default': True, 'uuid': '5194c63a-dd89-4fca-aa15-43c79ab61241'}, 'oper': 'CREATE', 'request-id': 'req-d525b0d9-e00d-48c9-8f6a-9e66986f02ce', 'type': 'routing_instance', 'uuid': '5194c63a-dd89-4fca-aa15-43c79ab61241'} 02/13/2025 02:17:01.858 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__'], 'obj_dict': {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__'], 'id_perms': {'created': '2025-02-13T02:17:00.110494', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:00.110494', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13011856619525677486, 'uuid_mslong': 17184692117069316113}}, 'parent_type': 'virtual-network', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'routing_instance_fabric_snat': False, 'routing_instance_has_pnf': False, 'routing_instance_is_default': False, 'uuid': 'ee7c4a6e-9c5f-4011-b493-66a7db24a1ae'}, 'oper': 'CREATE', 'request-id': 'req-4bb0e0e8-cc39-4f46-afc7-c70518427aca', 'type': 'routing_instance', 'uuid': 'ee7c4a6e-9c5f-4011-b493-66a7db24a1ae'} 02/13/2025 02:17:01.860 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', '__link_local__'], 'obj_dict': {'fabric_snat': False, 'flood_unknown_unicast': False, 'fq_name': ['default-domain', 'default-project', '__link_local__'], 'id_perms': {'created': '2025-02-13T02:17:00.178690', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:00.178690', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10362948394797075631, 'uuid_mslong': 16618390374396676048}}, 'igmp_enable': False, 'is_provider_network': False, 'layer2_control_word': False, 'mac_aging_time': 300, 'mac_ip_learning_enable': False, 'mac_learning_enabled': False, 'parent_type': 'project', 'pbb_etree_enable': False, 'pbb_evpn_enable': False, 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'port_security_enabled': True, 'uuid': 'e6a061ff-5caf-47d0-8fd0-96a8e9958caf', 'virtual_network_category': 'tenant', 'virtual_network_network_id': 3}, 'oper': 'CREATE', 'request-id': 'req-5e244afc-6325-46a7-b1c3-52e74ce2f2a2', 'type': 'virtual_network', 'uuid': 'e6a061ff-5caf-47d0-8fd0-96a8e9958caf'} 02/13/2025 02:17:01.867 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', '__link_local__', '__link_local__'], 'obj_dict': {'fq_name': ['default-domain', 'default-project', '__link_local__', '__link_local__'], 'id_perms': {'created': '2025-02-13T02:17:00.261660', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:00.261660', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10261148180442331270, 'uuid_mslong': 717702438821511569}}, 'parent_type': 'virtual-network', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'routing_instance_fabric_snat': False, 'routing_instance_has_pnf': False, 'routing_instance_is_default': True, 'uuid': '09f5ca90-c863-4191-8e66-ebe77ae93886'}, 'oper': 'CREATE', 'request-id': 'req-eb5f5bd2-49d5-4070-8e8b-c35e4b938738', 'type': 'routing_instance', 'uuid': '09f5ca90-c863-4191-8e66-ebe77ae93886'} 02/13/2025 02:17:01.869 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'dci-network'], 'obj_dict': {'fabric_snat': False, 'flood_unknown_unicast': False, 'fq_name': ['default-domain', 'default-project', 'dci-network'], 'id_perms': {'created': '2025-02-13T02:17:00.317963', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:00.317963', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12471431978892876205, 'uuid_mslong': 16826328056217292449}}, 'igmp_enable': False, 'is_provider_network': False, 'layer2_control_word': False, 'mac_aging_time': 300, 'mac_ip_learning_enable': False, 'mac_learning_enabled': False, 'parent_type': 'project', 'pbb_etree_enable': False, 'pbb_evpn_enable': False, 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'port_security_enabled': True, 'uuid': 'e9832037-a905-4ea1-ad13-6d4eee5b59ad', 'virtual_network_category': 'tenant', 'virtual_network_network_id': 4}, 'oper': 'CREATE', 'request-id': 'req-88563299-acd9-4c75-b4da-ff4819581740', 'type': 'virtual_network', 'uuid': 'e9832037-a905-4ea1-ad13-6d4eee5b59ad'} 02/13/2025 02:17:01.873 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'dci-network', 'dci-network'], 'obj_dict': {'fq_name': ['default-domain', 'default-project', 'dci-network', 'dci-network'], 'id_perms': {'created': '2025-02-13T02:17:00.432421', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:00.432421', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10870412341313302897, 'uuid_mslong': 4878378034692112689}}, 'parent_type': 'virtual-network', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'routing_instance_fabric_snat': False, 'routing_instance_has_pnf': False, 'routing_instance_is_default': True, 'uuid': '43b37af9-cde4-4131-96db-766e36007171'}, 'oper': 'CREATE', 'request-id': 'req-a0335dc0-bc16-4cc9-930f-8258c4cad3a2', 'type': 'routing_instance', 'uuid': '43b37af9-cde4-4131-96db-766e36007171'} 02/13/2025 02:17:01.874 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'dci-network', '__default__'], 'obj_dict': {'fq_name': ['default-domain', 'default-project', 'dci-network', '__default__'], 'id_perms': {'created': '2025-02-13T02:17:00.502864', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:00.502864', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12580909801303719650, 'uuid_mslong': 18392820622744240358}}, 'parent_type': 'virtual-network', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'routing_instance_fabric_snat': False, 'routing_instance_has_pnf': False, 'routing_instance_is_default': False, 'uuid': 'ff406ce8-3426-40e6-ae98-5ecec9f03ee2'}, 'oper': 'CREATE', 'request-id': 'req-699b19e6-f64c-432a-a70b-66356a57efa0', 'type': 'routing_instance', 'uuid': 'ff406ce8-3426-40e6-ae98-5ecec9f03ee2'} 02/13/2025 02:17:01.876 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-bottle-request-size-limit'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'UveConfigReq.err_info.*.', 'operand2': {'json_value': 'True'}, 'operation': '=='}]}]}, 'alarm_severity': 1, 'fq_name': ['default-global-system-config', 'system-defined-bottle-request-size-limit'], 'id_perms': {'created': '2025-02-13T02:17:00.558923', 'description': 'Bottle request size limit exceeded.', 'last_modified': '2025-02-13T02:17:00.558923', 'uuid': {'uuid_lslong': 9673849343481797259, 'uuid_mslong': 12666295861181237950}}, 'parent_type': 'global-system-config', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'afc7b8fa-11b5-4abe-8640-6ab940e0da8b', 'uve_keys': {'uve_key': ['config-node']}}, 'oper': 'CREATE', 'request-id': 'req-4163ce3c-bd9f-4f11-bbbb-01fc0f533134', 'type': 'alarm', 'uuid': 'afc7b8fa-11b5-4abe-8640-6ab940e0da8b'} 02/13/2025 02:17:01.888 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'default-virtual-network', 'default-virtual-network'], 'obj_dict': {'fq_name': ['default-domain', 'default-project', 'default-virtual-network', 'default-virtual-network'], 'id_perms': {'created': '2025-02-13T02:17:00.615709', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:00.615709', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9901146490289166680, 'uuid_mslong': 1428566404547169648}}, 'parent_type': 'virtual-network', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'routing_instance_fabric_snat': False, 'routing_instance_has_pnf': False, 'routing_instance_is_default': True, 'uuid': '13d34993-7917-4d70-8967-f0463eca9958'}, 'oper': 'CREATE', 'request-id': 'req-941d357e-52d9-4f55-bb02-0b058c4f4fdc', 'type': 'routing_instance', 'uuid': '13d34993-7917-4d70-8967-f0463eca9958'} 02/13/2025 02:17:01.890 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', '__no_rule__'], 'obj_dict': {'configured_security_group_id': 0, 'fq_name': ['default-domain', 'default-project', '__no_rule__'], 'id_perms': {'created': '2025-02-13T02:17:00.688637', 'creator': None, 'description': 'Security group with no rules', 'enable': True, 'last_modified': '2025-02-13T02:17:00.688637', 'permissions': None, 'user_visible': True, 'uuid': {'uuid_lslong': 13130116402852972668, 'uuid_mslong': 16043000299676913208}}, 'parent_type': 'project', 'perms2': {'global_access': 5, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'security_group_entries': {'policy_rule': []}, 'security_group_id': 8000001, 'uuid': 'dea42fc5-e9ba-4e38-b637-8b4d5394c87c'}, 'oper': 'CREATE', 'request-id': 'req-00ab7bf4-38f8-4dae-8861-3a4b1f383906', 'type': 'security_group', 'uuid': 'dea42fc5-e9ba-4e38-b637-8b4d5394c87c'} 02/13/2025 02:17:01.895 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-discovery-service-assignment'], 'obj_dict': {'fq_name': ['default-discovery-service-assignment'], 'id_perms': {'created': '2025-02-13T02:17:00.717035', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:00.717035', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11873252090251726008, 'uuid_mslong': 2775280736595627643}}, 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '2683c720-38c4-4e7b-a4c6-43de701e4cb8'}, 'oper': 'CREATE', 'request-id': 'req-8e3c9252-a058-4508-a9df-17a341c80b24', 'type': 'discovery_service_assignment', 'uuid': '2683c720-38c4-4e7b-a4c6-43de701e4cb8'} 02/13/2025 02:17:01.896 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'default-global-qos-config'], 'obj_dict': {'fq_name': ['default-global-system-config', 'default-global-qos-config'], 'id_perms': {'created': '2025-02-13T02:17:00.756127', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:00.756127', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10513404855683423960, 'uuid_mslong': 17896880204364140032}}, 'parent_type': 'global-system-config', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'f85e7db9-60d8-4e00-91e7-1e038ae09ed8'}, 'oper': 'CREATE', 'request-id': 'req-1341e5bb-8845-4689-af2c-3ba1a5c1f0ca', 'type': 'global_qos_config', 'uuid': 'f85e7db9-60d8-4e00-91e7-1e038ae09ed8'} 02/13/2025 02:17:01.899 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'service-chain-flat-ipam'], 'obj_dict': {'fq_name': ['default-domain', 'default-project', 'service-chain-flat-ipam'], 'id_perms': {'created': '2025-02-13T02:17:00.791900', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:00.791900', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13472980830641205384, 'uuid_mslong': 2892936446380887322}}, 'ipam_subnet_method': 'flat-subnet', 'ipam_subnets': {'subnets': [{'addr_from_start': None, 'alloc_unit': 1, 'allocation_pools': [], 'created': None, 'default_gateway': None, 'dhcp_option_list': None, 'dhcp_relay_server': [], 'dns_nameservers': [], 'dns_server_address': None, 'enable_dhcp': True, 'host_routes': None, 'last_modified': None, 'subnet': {'ip_prefix': '0.0.0.0', 'ip_prefix_len': 8}, 'subnet_name': None, 'subnet_uuid': None, 'subscriber_tag': None, 'vlan_tag': None}, {'addr_from_start': None, 'alloc_unit': 1, 'allocation_pools': [], 'created': None, 'default_gateway': None, 'dhcp_option_list': None, 'dhcp_relay_server': [], 'dns_nameservers': [], 'dns_server_address': None, 'enable_dhcp': True, 'host_routes': None, 'last_modified': None, 'subnet': {'ip_prefix': '::ffff', 'ip_prefix_len': 104}, 'subnet_name': None, 'subnet_uuid': None, 'subscriber_tag': None, 'vlan_tag': None}]}, 'parent_type': 'project', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '2825c65e-dc50-4d1a-baf9-a4aebc043088'}, 'oper': 'CREATE', 'request-id': 'req-ff0f1e39-f5dd-404d-8cc3-2c43b151cb3a', 'type': 'network_ipam', 'uuid': '2825c65e-dc50-4d1a-baf9-a4aebc043088'} 02/13/2025 02:17:01.905 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['label'], 'obj_dict': {'display_name': 'label', 'fq_name': ['label'], 'id_perms': {'created': '2025-02-13T02:17:00.820882', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:00.820882', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': False, 'uuid': {'uuid_lslong': 11314862015438450008, 'uuid_mslong': 7329441829533600969}}, 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'tag_type_id': '0x0000', 'uuid': '65b76882-cb3d-4cc9-9d06-770de8c4ed58'}, 'oper': 'CREATE', 'request-id': 'req-30252232-1cf0-49a0-be84-f7223ee48e07', 'type': 'tag_type', 'uuid': '65b76882-cb3d-4cc9-9d06-770de8c4ed58'} 02/13/2025 02:17:01.908 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['application'], 'obj_dict': {'display_name': 'application', 'fq_name': ['application'], 'id_perms': {'created': '2025-02-13T02:17:00.858337', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:00.858337', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': False, 'uuid': {'uuid_lslong': 11471014860306371633, 'uuid_mslong': 15690566120018821446}}, 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'tag_type_id': '0x0001', 'uuid': 'd9c016c1-0480-4146-9f31-3b3d49c9d431'}, 'oper': 'CREATE', 'request-id': 'req-3929f99a-f13d-4c79-a98e-637eeca886f8', 'type': 'tag_type', 'uuid': 'd9c016c1-0480-4146-9f31-3b3d49c9d431'} 02/13/2025 02:17:01.910 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['tier'], 'obj_dict': {'display_name': 'tier', 'fq_name': ['tier'], 'id_perms': {'created': '2025-02-13T02:17:00.884348', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:00.884348', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': False, 'uuid': {'uuid_lslong': 10982655463039759316, 'uuid_mslong': 8197998431213732993}}, 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'tag_type_id': '0x0002', 'uuid': '71c52423-69b4-4481-986a-3af47cfcf3d4'}, 'oper': 'CREATE', 'request-id': 'req-9af79abf-f1c6-40fc-94ad-f774828da2ba', 'type': 'tag_type', 'uuid': '71c52423-69b4-4481-986a-3af47cfcf3d4'} 02/13/2025 02:17:01.914 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['deployment'], 'obj_dict': {'display_name': 'deployment', 'fq_name': ['deployment'], 'id_perms': {'created': '2025-02-13T02:17:00.925335', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:00.925335', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': False, 'uuid': {'uuid_lslong': 11608105999922164052, 'uuid_mslong': 12669811088694526716}}, 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'tag_type_id': '0x0003', 'uuid': 'afd4360e-c0f1-42fc-a118-46e79070f154'}, 'oper': 'CREATE', 'request-id': 'req-0085a499-7836-4f50-92c8-4ec5584dce93', 'type': 'tag_type', 'uuid': 'afd4360e-c0f1-42fc-a118-46e79070f154'} 02/13/2025 02:17:01.916 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['site'], 'obj_dict': {'display_name': 'site', 'fq_name': ['site'], 'id_perms': {'created': '2025-02-13T02:17:00.968531', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:00.968531', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': False, 'uuid': {'uuid_lslong': 12503701078331410566, 'uuid_mslong': 9148612426004384717}}, 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'tag_type_id': '0x0004', 'uuid': '7ef66687-afb7-4bcd-ad86-11e21f9cc886'}, 'oper': 'CREATE', 'request-id': 'req-9f1c0505-a857-4f8f-ac5b-653e0ada6602', 'type': 'tag_type', 'uuid': '7ef66687-afb7-4bcd-ad86-11e21f9cc886'} 02/13/2025 02:17:01.919 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['neutron_fwaas'], 'obj_dict': {'display_name': 'neutron_fwaas', 'fq_name': ['neutron_fwaas'], 'id_perms': {'created': '2025-02-13T02:17:00.996581', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:00.996581', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': False, 'uuid': {'uuid_lslong': 11340026082467490479, 'uuid_mslong': 14520805449210284523}}, 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'tag_type_id': '0x0005', 'uuid': 'c984439f-1cbc-49eb-9d5f-dda3d523aaaf'}, 'oper': 'CREATE', 'request-id': 'req-49882c32-320b-47b2-b59a-bb253e85776e', 'type': 'tag_type', 'uuid': 'c984439f-1cbc-49eb-9d5f-dda3d523aaaf'} 02/13/2025 02:17:01.922 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['neutron_tag'], 'obj_dict': {'display_name': 'neutron_tag', 'fq_name': ['neutron_tag'], 'id_perms': {'created': '2025-02-13T02:17:01.031789', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:01.031789', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': False, 'uuid': {'uuid_lslong': 13456054159377417055, 'uuid_mslong': 548564745640625338}}, 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'tag_type_id': '0x0006', 'uuid': '079ce4ba-d036-44ba-babd-81f6f81bc35f'}, 'oper': 'CREATE', 'request-id': 'req-e8da5353-dfe9-4081-bdf0-c2c23caf0519', 'type': 'tag_type', 'uuid': '079ce4ba-d036-44ba-babd-81f6f81bc35f'} 02/13/2025 02:17:01.925 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'default-network-ipam'], 'oper': 'UPDATE', 'request-id': 'req-54408455-1b69-4f85-8d21-c28fb4a6c7fb', 'type': 'network_ipam', 'uuid': '917d4aaf-f039-4abd-847a-fa6896365c56'} 02/13/2025 02:17:01.996 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'default-api-access-list'], 'obj_dict': {'api_access_list_entries': {'rbac_rule': [{'rule_field': '', 'rule_object': 'fqname-to-id', 'rule_perms': [{'role_crud': 'CRUD', 'role_name': '*'}]}, {'rule_field': '', 'rule_object': 'id-to-fqname', 'rule_perms': [{'role_crud': 'CRUD', 'role_name': '*'}]}, {'rule_field': '', 'rule_object': 'useragent-kv', 'rule_perms': [{'role_crud': 'CRUD', 'role_name': '*'}]}, {'rule_field': '', 'rule_object': 'documentation', 'rule_perms': [{'role_crud': 'R', 'role_name': '*'}]}, {'rule_field': '', 'rule_object': '/', 'rule_perms': [{'role_crud': 'R', 'role_name': '*'}]}, {'rule_field': '', 'rule_object': 'watch', 'rule_perms': [{'role_crud': 'R', 'role_name': '*'}]}, {'rule_field': 'logical_router_gateway_external', 'rule_object': 'logical-router', 'rule_perms': [{'role_crud': 'R', 'role_name': '*'}]}]}, 'fq_name': ['default-global-system-config', 'default-api-access-list'], 'id_perms': {'created': '2025-02-13T02:17:01.352002', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:01.352002', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12545112365249150924, 'uuid_mslong': 7917816975025063024}}, 'parent_type': 'global-system-config', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '6de1bc96-a0ed-4470-ae19-313ab6093fcc'}, 'oper': 'CREATE', 'request-id': 'req-26685fb2-fb5b-4ab2-bd6a-77051ff2d768', 'type': 'api_access_list', 'uuid': '6de1bc96-a0ed-4470-ae19-313ab6093fcc'} 02/13/2025 02:17:01.998 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'default-application-policy-set'], 'obj_dict': {'all_applications': True, 'fq_name': ['default-domain', 'default-project', 'default-application-policy-set'], 'id_perms': {'created': '2025-02-13T02:17:01.360561', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:01.360561', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10091411539965468164, 'uuid_mslong': 14778099202453816952}}, 'parent_type': 'project', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'cd165aed-49be-4678-8c0b-e55439eb5e04'}, 'oper': 'CREATE', 'request-id': 'req-24e641b0-bd83-4c9c-93b2-767411f9ee70', 'type': 'application_policy_set', 'uuid': 'cd165aed-49be-4678-8c0b-e55439eb5e04'} 02/13/2025 02:17:01.999 7f0bbe5bcc18 [contrail-api] [DEBUG]: Add uve in the [ObjectApplicationPolicySetTable:ContrailConfigTrace] map 02/13/2025 02:17:02.000 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project'], 'oper': 'UPDATE', 'request-id': 'req-37bf910a-bbc6-43d1-b66a-440fe7ccfec8', 'type': 'project', 'uuid': 'ef3ddfc6-29bd-4412-b9c7-1bb578216ea3'} 02/13/2025 02:17:02.766 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:03.030 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource global-system-config: 'parent_type' 02/13/2025 02:17:03.031 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flags in resource global-system-config: 'feature_flags' 02/13/2025 02:17:03.031 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zones in resource global-system-config: 'control_node_zones' 02/13/2025 02:17:03.031 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_configs in resource global-system-config: 'global_vrouter_configs' 02/13/2025 02:17:03.031 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_configs in resource global-system-config: 'global_qos_configs' 02/13/2025 02:17:03.031 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_routers in resource global-system-config: 'virtual_routers' 02/13/2025 02:17:03.031 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_nodes in resource global-system-config: 'config_nodes' 02/13/2025 02:17:03.032 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_nodes in resource global-system-config: 'analytics_nodes' 02/13/2025 02:17:03.032 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_nodes in resource global-system-config: 'flow_nodes' 02/13/2025 02:17:03.032 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_nodes in resource global-system-config: 'devicemgr_nodes' 02/13/2025 02:17:03.032 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_nodes in resource global-system-config: 'database_nodes' 02/13/2025 02:17:03.032 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_nodes in resource global-system-config: 'webui_nodes' 02/13/2025 02:17:03.032 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_nodes in resource global-system-config: 'config_database_nodes' 02/13/2025 02:17:03.032 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_nodes in resource global-system-config: 'analytics_alarm_nodes' 02/13/2025 02:17:03.032 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_nodes in resource global-system-config: 'analytics_snmp_nodes' 02/13/2025 02:17:03.032 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_sets in resource global-system-config: 'service_appliance_sets' 02/13/2025 02:17:03.032 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource global-system-config: 'api_access_lists' 02/13/2025 02:17:03.032 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource global-system-config: 'alarms' 02/13/2025 02:17:03.033 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_propertiess in resource global-system-config: 'config_propertiess' 02/13/2025 02:17:03.033 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_templates in resource global-system-config: 'job_templates' 02/13/2025 02:17:03.033 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnects in resource global-system-config: 'data_center_interconnects' 02/13/2025 02:17:03.033 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_maps in resource global-system-config: 'intent_maps' 02/13/2025 02:17:03.033 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabrics in resource global-system-config: 'fabrics' 02/13/2025 02:17:03.033 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profiles in resource global-system-config: 'node_profiles' 02/13/2025 02:17:03.033 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_routers in resource global-system-config: 'physical_routers' 02/13/2025 02:17:03.033 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_images in resource global-system-config: 'device_images' 02/13/2025 02:17:03.033 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field nodes in resource global-system-config: 'nodes' 02/13/2025 02:17:03.033 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field features in resource global-system-config: 'features' 02/13/2025 02:17:03.034 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_roles in resource global-system-config: 'physical_roles' 02/13/2025 02:17:03.034 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_roles in resource global-system-config: 'overlay_roles' 02/13/2025 02:17:03.034 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definitions in resource global-system-config: 'role_definitions' 02/13/2025 02:17:03.034 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_configs in resource global-system-config: 'global_analytics_configs' 02/13/2025 02:17:03.035 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_refs in resource global-system-config: 'bgp_router_refs' 02/13/2025 02:17:03.035 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource global-system-config: 'tag_refs' 02/13/2025 02:17:03.035 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource global-system-config: 'qos_config_back_refs' 02/13/2025 02:17:03.141 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-os-473-1'] not found in DB: Unknown id: database_node default-global-system-config:cn-jenkins-deploy-platform-juju-os-473-1 02/13/2025 02:17:03.399 7f0bbdd032e8 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = database_node identifier_name = default-global-system-config:cn-jenkins-deploy-platform-juju-os-473-1 url = https://10.0.0.17:8082/database-nodes operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_database_node.py remote_ip = 10.0.0.17 body = {"database-node": {"uuid": null, "fq_name": ["default-global-system-config", "cn-jenkins-deploy-platform-juju-os-473-1"], "parent_type": "global-system-config", "database_node_ip_address": "10.0.0.17", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "cn-jenkins-deploy-platform-juju-os-473-1"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:03.443 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-os-473-1'], 'obj_dict': {'database_node_ip_address': '10.0.0.17', 'display_name': 'cn-jenkins-deploy-platform-juju-os-473-1', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-os-473-1'], 'id_perms': {'created': '2025-02-13T02:17:03.433735', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:03.433735', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11627579018681471091, 'uuid_mslong': 11923421886576479440}}, 'parent_type': 'global-system-config', 'parent_uuid': '3a60be2c-e787-4ec9-894b-c0c6d7944765', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'a5788108-6882-4cd0-a15d-75830e4b5073'}, 'oper': 'CREATE', 'request-id': 'req-2c5c6ba8-1a74-44c2-adae-40460a6cd54d', 'type': 'database_node', 'uuid': 'a5788108-6882-4cd0-a15d-75830e4b5073'} 02/13/2025 02:17:03.444 7f0bbe5bcc18 [contrail-api] [DEBUG]: Add uve in the [ObjectDatabaseInfo:ContrailConfigTrace] map 02/13/2025 02:17:03.768 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:04.094 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource global-system-config: 'parent_type' 02/13/2025 02:17:04.094 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flags in resource global-system-config: 'feature_flags' 02/13/2025 02:17:04.094 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zones in resource global-system-config: 'control_node_zones' 02/13/2025 02:17:04.094 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_configs in resource global-system-config: 'global_vrouter_configs' 02/13/2025 02:17:04.095 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_configs in resource global-system-config: 'global_qos_configs' 02/13/2025 02:17:04.095 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_routers in resource global-system-config: 'virtual_routers' 02/13/2025 02:17:04.095 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_nodes in resource global-system-config: 'config_nodes' 02/13/2025 02:17:04.095 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_nodes in resource global-system-config: 'analytics_nodes' 02/13/2025 02:17:04.095 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_nodes in resource global-system-config: 'flow_nodes' 02/13/2025 02:17:04.095 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_nodes in resource global-system-config: 'devicemgr_nodes' 02/13/2025 02:17:04.095 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_nodes in resource global-system-config: 'database_nodes' 02/13/2025 02:17:04.095 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_nodes in resource global-system-config: 'webui_nodes' 02/13/2025 02:17:04.095 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_nodes in resource global-system-config: 'config_database_nodes' 02/13/2025 02:17:04.095 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_nodes in resource global-system-config: 'analytics_alarm_nodes' 02/13/2025 02:17:04.096 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_nodes in resource global-system-config: 'analytics_snmp_nodes' 02/13/2025 02:17:04.096 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_sets in resource global-system-config: 'service_appliance_sets' 02/13/2025 02:17:04.096 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource global-system-config: 'api_access_lists' 02/13/2025 02:17:04.096 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource global-system-config: 'alarms' 02/13/2025 02:17:04.096 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_propertiess in resource global-system-config: 'config_propertiess' 02/13/2025 02:17:04.096 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_templates in resource global-system-config: 'job_templates' 02/13/2025 02:17:04.096 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnects in resource global-system-config: 'data_center_interconnects' 02/13/2025 02:17:04.096 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_maps in resource global-system-config: 'intent_maps' 02/13/2025 02:17:04.096 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabrics in resource global-system-config: 'fabrics' 02/13/2025 02:17:04.096 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profiles in resource global-system-config: 'node_profiles' 02/13/2025 02:17:04.096 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_routers in resource global-system-config: 'physical_routers' 02/13/2025 02:17:04.097 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_images in resource global-system-config: 'device_images' 02/13/2025 02:17:04.097 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field nodes in resource global-system-config: 'nodes' 02/13/2025 02:17:04.097 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field features in resource global-system-config: 'features' 02/13/2025 02:17:04.097 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_roles in resource global-system-config: 'physical_roles' 02/13/2025 02:17:04.097 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_roles in resource global-system-config: 'overlay_roles' 02/13/2025 02:17:04.097 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definitions in resource global-system-config: 'role_definitions' 02/13/2025 02:17:04.097 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_configs in resource global-system-config: 'global_analytics_configs' 02/13/2025 02:17:04.097 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_refs in resource global-system-config: 'bgp_router_refs' 02/13/2025 02:17:04.097 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource global-system-config: 'tag_refs' 02/13/2025 02:17:04.097 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource global-system-config: 'qos_config_back_refs' 02/13/2025 02:17:04.197 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-os-473-1'] not found in DB: Unknown id: analytics_snmp_node default-global-system-config:cn-jenkins-deploy-platform-juju-os-473-1 02/13/2025 02:17:04.211 7f0bbde69780 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = analytics_snmp_node identifier_name = default-global-system-config:cn-jenkins-deploy-platform-juju-os-473-1 url = https://10.0.0.17:8082/analytics-snmp-nodes operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_analytics_snmp_node.py remote_ip = 10.0.0.17 body = {"analytics-snmp-node": {"uuid": null, "fq_name": ["default-global-system-config", "cn-jenkins-deploy-platform-juju-os-473-1"], "parent_type": "global-system-config", "analytics_snmp_node_ip_address": "10.0.0.17", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "cn-jenkins-deploy-platform-juju-os-473-1"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:04.277 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-os-473-1'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.17', 'display_name': 'cn-jenkins-deploy-platform-juju-os-473-1', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-os-473-1'], 'id_perms': {'created': '2025-02-13T02:17:04.265165', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:04.265165', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12099273495173911835, 'uuid_mslong': 9198079041706410262}}, 'parent_type': 'global-system-config', 'parent_uuid': '3a60be2c-e787-4ec9-894b-c0c6d7944765', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '7fa62427-a8e0-4116-a7e9-4123a023d51b'}, 'oper': 'CREATE', 'request-id': 'req-1ee0d9ee-3402-4a1a-8db9-fbc0fb5767e1', 'type': 'analytics_snmp_node', 'uuid': '7fa62427-a8e0-4116-a7e9-4123a023d51b'} 02/13/2025 02:17:04.277 7f0bbe5bcc18 [contrail-api] [DEBUG]: Add uve in the [ObjectAnalyticsSNMPInfo:ContrailConfigTrace] map 02/13/2025 02:17:04.843 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:05.139 7f0bc1ea36d8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource global-system-config: 'parent_type' 02/13/2025 02:17:05.140 7f0bc1ea36d8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flags in resource global-system-config: 'feature_flags' 02/13/2025 02:17:05.140 7f0bc1ea36d8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zones in resource global-system-config: 'control_node_zones' 02/13/2025 02:17:05.140 7f0bc1ea36d8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_configs in resource global-system-config: 'global_vrouter_configs' 02/13/2025 02:17:05.140 7f0bc1ea36d8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_configs in resource global-system-config: 'global_qos_configs' 02/13/2025 02:17:05.140 7f0bc1ea36d8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_routers in resource global-system-config: 'virtual_routers' 02/13/2025 02:17:05.140 7f0bc1ea36d8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_nodes in resource global-system-config: 'config_nodes' 02/13/2025 02:17:05.141 7f0bc1ea36d8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_nodes in resource global-system-config: 'analytics_nodes' 02/13/2025 02:17:05.141 7f0bc1ea36d8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_nodes in resource global-system-config: 'flow_nodes' 02/13/2025 02:17:05.141 7f0bc1ea36d8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_nodes in resource global-system-config: 'devicemgr_nodes' 02/13/2025 02:17:05.141 7f0bc1ea36d8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_nodes in resource global-system-config: 'database_nodes' 02/13/2025 02:17:05.141 7f0bc1ea36d8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_nodes in resource global-system-config: 'webui_nodes' 02/13/2025 02:17:05.142 7f0bc1ea36d8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_nodes in resource global-system-config: 'config_database_nodes' 02/13/2025 02:17:05.142 7f0bc1ea36d8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_nodes in resource global-system-config: 'analytics_alarm_nodes' 02/13/2025 02:17:05.142 7f0bc1ea36d8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_nodes in resource global-system-config: 'analytics_snmp_nodes' 02/13/2025 02:17:05.142 7f0bc1ea36d8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_sets in resource global-system-config: 'service_appliance_sets' 02/13/2025 02:17:05.142 7f0bc1ea36d8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource global-system-config: 'api_access_lists' 02/13/2025 02:17:05.143 7f0bc1ea36d8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource global-system-config: 'alarms' 02/13/2025 02:17:05.143 7f0bc1ea36d8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_propertiess in resource global-system-config: 'config_propertiess' 02/13/2025 02:17:05.143 7f0bc1ea36d8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_templates in resource global-system-config: 'job_templates' 02/13/2025 02:17:05.143 7f0bc1ea36d8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnects in resource global-system-config: 'data_center_interconnects' 02/13/2025 02:17:05.143 7f0bc1ea36d8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_maps in resource global-system-config: 'intent_maps' 02/13/2025 02:17:05.143 7f0bc1ea36d8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabrics in resource global-system-config: 'fabrics' 02/13/2025 02:17:05.144 7f0bc1ea36d8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profiles in resource global-system-config: 'node_profiles' 02/13/2025 02:17:05.144 7f0bc1ea36d8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_routers in resource global-system-config: 'physical_routers' 02/13/2025 02:17:05.144 7f0bc1ea36d8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_images in resource global-system-config: 'device_images' 02/13/2025 02:17:05.144 7f0bc1ea36d8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field nodes in resource global-system-config: 'nodes' 02/13/2025 02:17:05.144 7f0bc1ea36d8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field features in resource global-system-config: 'features' 02/13/2025 02:17:05.144 7f0bc1ea36d8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_roles in resource global-system-config: 'physical_roles' 02/13/2025 02:17:05.144 7f0bc1ea36d8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_roles in resource global-system-config: 'overlay_roles' 02/13/2025 02:17:05.144 7f0bc1ea36d8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definitions in resource global-system-config: 'role_definitions' 02/13/2025 02:17:05.144 7f0bc1ea36d8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_configs in resource global-system-config: 'global_analytics_configs' 02/13/2025 02:17:05.145 7f0bc1ea36d8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_refs in resource global-system-config: 'bgp_router_refs' 02/13/2025 02:17:05.145 7f0bc1ea36d8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource global-system-config: 'tag_refs' 02/13/2025 02:17:05.145 7f0bc1ea36d8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource global-system-config: 'qos_config_back_refs' 02/13/2025 02:17:05.272 7f0bc1ea36d8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-os-473-1'] not found in DB: Unknown id: config_node default-global-system-config:cn-jenkins-deploy-platform-juju-os-473-1 02/13/2025 02:17:05.326 7f0bbdc9d048 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource global-system-config: 'parent_type' 02/13/2025 02:17:05.327 7f0bbdc9d048 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flags in resource global-system-config: 'feature_flags' 02/13/2025 02:17:05.327 7f0bbdc9d048 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zones in resource global-system-config: 'control_node_zones' 02/13/2025 02:17:05.327 7f0bbdc9d048 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_configs in resource global-system-config: 'global_vrouter_configs' 02/13/2025 02:17:05.327 7f0bbdc9d048 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_configs in resource global-system-config: 'global_qos_configs' 02/13/2025 02:17:05.327 7f0bbdc9d048 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_routers in resource global-system-config: 'virtual_routers' 02/13/2025 02:17:05.327 7f0bbdc9d048 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_nodes in resource global-system-config: 'config_nodes' 02/13/2025 02:17:05.327 7f0bbdc9d048 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_nodes in resource global-system-config: 'analytics_nodes' 02/13/2025 02:17:05.327 7f0bbdc9d048 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_nodes in resource global-system-config: 'flow_nodes' 02/13/2025 02:17:05.327 7f0bbdc9d048 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_nodes in resource global-system-config: 'devicemgr_nodes' 02/13/2025 02:17:05.327 7f0bbdc9d048 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_nodes in resource global-system-config: 'database_nodes' 02/13/2025 02:17:05.328 7f0bbdc9d048 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_nodes in resource global-system-config: 'webui_nodes' 02/13/2025 02:17:05.328 7f0bbdc9d048 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_nodes in resource global-system-config: 'config_database_nodes' 02/13/2025 02:17:05.328 7f0bbdc9d048 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_nodes in resource global-system-config: 'analytics_alarm_nodes' 02/13/2025 02:17:05.328 7f0bbdc9d048 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_nodes in resource global-system-config: 'analytics_snmp_nodes' 02/13/2025 02:17:05.328 7f0bbdc9d048 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_sets in resource global-system-config: 'service_appliance_sets' 02/13/2025 02:17:05.328 7f0bbdc9d048 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource global-system-config: 'api_access_lists' 02/13/2025 02:17:05.328 7f0bbdc9d048 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource global-system-config: 'alarms' 02/13/2025 02:17:05.328 7f0bbdc9d048 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_propertiess in resource global-system-config: 'config_propertiess' 02/13/2025 02:17:05.328 7f0bbdc9d048 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_templates in resource global-system-config: 'job_templates' 02/13/2025 02:17:05.328 7f0bbdc9d048 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnects in resource global-system-config: 'data_center_interconnects' 02/13/2025 02:17:05.328 7f0bbdc9d048 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_maps in resource global-system-config: 'intent_maps' 02/13/2025 02:17:05.329 7f0bbdc9d048 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabrics in resource global-system-config: 'fabrics' 02/13/2025 02:17:05.329 7f0bbdc9d048 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profiles in resource global-system-config: 'node_profiles' 02/13/2025 02:17:05.329 7f0bbdc9d048 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_routers in resource global-system-config: 'physical_routers' 02/13/2025 02:17:05.330 7f0bbdc9d048 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_images in resource global-system-config: 'device_images' 02/13/2025 02:17:05.330 7f0bbdc9d048 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field nodes in resource global-system-config: 'nodes' 02/13/2025 02:17:05.330 7f0bbdc9d048 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field features in resource global-system-config: 'features' 02/13/2025 02:17:05.330 7f0bbdc9d048 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_roles in resource global-system-config: 'physical_roles' 02/13/2025 02:17:05.330 7f0bbdc9d048 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_roles in resource global-system-config: 'overlay_roles' 02/13/2025 02:17:05.331 7f0bbdc9d048 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definitions in resource global-system-config: 'role_definitions' 02/13/2025 02:17:05.331 7f0bbdc9d048 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_configs in resource global-system-config: 'global_analytics_configs' 02/13/2025 02:17:05.331 7f0bbdc9d048 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_refs in resource global-system-config: 'bgp_router_refs' 02/13/2025 02:17:05.331 7f0bbdc9d048 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource global-system-config: 'tag_refs' 02/13/2025 02:17:05.331 7f0bbdc9d048 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource global-system-config: 'qos_config_back_refs' 02/13/2025 02:17:05.335 7f0bbd9ac240 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = config_node identifier_name = default-global-system-config:cn-jenkins-deploy-platform-juju-os-473-1 url = https://10.0.0.17:8082/config-nodes operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_config_node.py remote_ip = 10.0.0.17 body = {"config-node": {"uuid": null, "fq_name": ["default-global-system-config", "cn-jenkins-deploy-platform-juju-os-473-1"], "parent_type": "global-system-config", "config_node_ip_address": "10.0.0.17", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "cn-jenkins-deploy-platform-juju-os-473-1"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:05.380 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-os-473-1'], 'obj_dict': {'config_node_ip_address': '10.0.0.17', 'display_name': 'cn-jenkins-deploy-platform-juju-os-473-1', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-os-473-1'], 'id_perms': {'created': '2025-02-13T02:17:05.356330', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:05.356330', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10805991921260987215, 'uuid_mslong': 5611449628528887512}}, 'parent_type': 'global-system-config', 'parent_uuid': '3a60be2c-e787-4ec9-894b-c0c6d7944765', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '4ddfdfb4-d776-46d8-95f6-98665880834f'}, 'oper': 'CREATE', 'request-id': 'req-d8c65a7f-e204-4f60-8766-9ab0754aed55', 'type': 'config_node', 'uuid': '4ddfdfb4-d776-46d8-95f6-98665880834f'} 02/13/2025 02:17:05.388 7f0bbe5bcc18 [contrail-api] [DEBUG]: Add uve in the [ObjectConfigNode:ContrailConfigTrace] map 02/13/2025 02:17:05.454 7f0bbdc9d048 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-os-473-1'] not found in DB: Unknown id: analytics_node default-global-system-config:cn-jenkins-deploy-platform-juju-os-473-1 02/13/2025 02:17:05.474 7f0bbda04f60 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = analytics_node identifier_name = default-global-system-config:cn-jenkins-deploy-platform-juju-os-473-1 url = https://10.0.0.17:8082/analytics-nodes operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_analytics_node.py remote_ip = 10.0.0.17 body = {"analytics-node": {"uuid": null, "fq_name": ["default-global-system-config", "cn-jenkins-deploy-platform-juju-os-473-1"], "parent_type": "global-system-config", "analytics_node_ip_address": "10.0.0.17", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "cn-jenkins-deploy-platform-juju-os-473-1"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:05.554 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-os-473-1'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.17', 'display_name': 'cn-jenkins-deploy-platform-juju-os-473-1', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-os-473-1'], 'id_perms': {'created': '2025-02-13T02:17:05.524634', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:05.524634', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13101537829918691961, 'uuid_mslong': 13989129838696023217}}, 'parent_type': 'global-system-config', 'parent_uuid': '3a60be2c-e787-4ec9-894b-c0c6d7944765', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'c2235f78-804b-44b1-b5d2-033dcba55279'}, 'oper': 'CREATE', 'request-id': 'req-98eba210-bcd1-4ab4-802f-75db0525bd13', 'type': 'analytics_node', 'uuid': 'c2235f78-804b-44b1-b5d2-033dcba55279'} 02/13/2025 02:17:05.554 7f0bbe5bcc18 [contrail-api] [DEBUG]: Add uve in the [ObjectCollectorInfo:ContrailConfigTrace] map 02/13/2025 02:17:05.690 7f0bbda05ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource global-system-config: 'parent_type' 02/13/2025 02:17:05.691 7f0bbda05ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flags in resource global-system-config: 'feature_flags' 02/13/2025 02:17:05.691 7f0bbda05ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zones in resource global-system-config: 'control_node_zones' 02/13/2025 02:17:05.691 7f0bbda05ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_configs in resource global-system-config: 'global_vrouter_configs' 02/13/2025 02:17:05.691 7f0bbda05ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_configs in resource global-system-config: 'global_qos_configs' 02/13/2025 02:17:05.691 7f0bbda05ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_routers in resource global-system-config: 'virtual_routers' 02/13/2025 02:17:05.691 7f0bbda05ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_nodes in resource global-system-config: 'config_nodes' 02/13/2025 02:17:05.691 7f0bbda05ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_nodes in resource global-system-config: 'analytics_nodes' 02/13/2025 02:17:05.691 7f0bbda05ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_nodes in resource global-system-config: 'flow_nodes' 02/13/2025 02:17:05.691 7f0bbda05ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_nodes in resource global-system-config: 'devicemgr_nodes' 02/13/2025 02:17:05.691 7f0bbda05ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_nodes in resource global-system-config: 'database_nodes' 02/13/2025 02:17:05.692 7f0bbda05ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_nodes in resource global-system-config: 'webui_nodes' 02/13/2025 02:17:05.692 7f0bbda05ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_nodes in resource global-system-config: 'config_database_nodes' 02/13/2025 02:17:05.692 7f0bbda05ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_nodes in resource global-system-config: 'analytics_alarm_nodes' 02/13/2025 02:17:05.692 7f0bbda05ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_nodes in resource global-system-config: 'analytics_snmp_nodes' 02/13/2025 02:17:05.692 7f0bbda05ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_sets in resource global-system-config: 'service_appliance_sets' 02/13/2025 02:17:05.692 7f0bbda05ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource global-system-config: 'api_access_lists' 02/13/2025 02:17:05.692 7f0bbda05ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource global-system-config: 'alarms' 02/13/2025 02:17:05.692 7f0bbda05ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_propertiess in resource global-system-config: 'config_propertiess' 02/13/2025 02:17:05.692 7f0bbda05ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_templates in resource global-system-config: 'job_templates' 02/13/2025 02:17:05.692 7f0bbda05ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnects in resource global-system-config: 'data_center_interconnects' 02/13/2025 02:17:05.692 7f0bbda05ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_maps in resource global-system-config: 'intent_maps' 02/13/2025 02:17:05.693 7f0bbda05ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabrics in resource global-system-config: 'fabrics' 02/13/2025 02:17:05.693 7f0bbda05ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profiles in resource global-system-config: 'node_profiles' 02/13/2025 02:17:05.693 7f0bbda05ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_routers in resource global-system-config: 'physical_routers' 02/13/2025 02:17:05.693 7f0bbda05ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_images in resource global-system-config: 'device_images' 02/13/2025 02:17:05.693 7f0bbda05ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field nodes in resource global-system-config: 'nodes' 02/13/2025 02:17:05.693 7f0bbda05ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field features in resource global-system-config: 'features' 02/13/2025 02:17:05.693 7f0bbda05ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_roles in resource global-system-config: 'physical_roles' 02/13/2025 02:17:05.693 7f0bbda05ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_roles in resource global-system-config: 'overlay_roles' 02/13/2025 02:17:05.693 7f0bbda05ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definitions in resource global-system-config: 'role_definitions' 02/13/2025 02:17:05.693 7f0bbda05ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_configs in resource global-system-config: 'global_analytics_configs' 02/13/2025 02:17:05.693 7f0bbda05ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_refs in resource global-system-config: 'bgp_router_refs' 02/13/2025 02:17:05.694 7f0bbda05ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource global-system-config: 'tag_refs' 02/13/2025 02:17:05.695 7f0bbda05ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource global-system-config: 'qos_config_back_refs' 02/13/2025 02:17:05.819 7f0bbda05ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-os-473-1'] not found in DB: Unknown id: analytics_alarm_node default-global-system-config:cn-jenkins-deploy-platform-juju-os-473-1 02/13/2025 02:17:05.839 7f0bbddb5588 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = analytics_alarm_node identifier_name = default-global-system-config:cn-jenkins-deploy-platform-juju-os-473-1 url = https://10.0.0.17:8082/analytics-alarm-nodes operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_analytics_alarm_node.py remote_ip = 10.0.0.17 body = {"analytics-alarm-node": {"uuid": null, "fq_name": ["default-global-system-config", "cn-jenkins-deploy-platform-juju-os-473-1"], "parent_type": "global-system-config", "analytics_alarm_node_ip_address": "10.0.0.17", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "cn-jenkins-deploy-platform-juju-os-473-1"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:05.844 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:05.885 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-os-473-1'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.17', 'display_name': 'cn-jenkins-deploy-platform-juju-os-473-1', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-os-473-1'], 'id_perms': {'created': '2025-02-13T02:17:05.863974', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:05.863974', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13382302193502317432, 'uuid_mslong': 14242828075096427915}}, 'parent_type': 'global-system-config', 'parent_uuid': '3a60be2c-e787-4ec9-894b-c0c6d7944765', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'c5a8b0ac-2d15-4d8b-b9b7-7cf148a2eb78'}, 'oper': 'CREATE', 'request-id': 'req-d94eec8b-8a18-45e3-a806-a61b961c71a8', 'type': 'analytics_alarm_node', 'uuid': 'c5a8b0ac-2d15-4d8b-b9b7-7cf148a2eb78'} 02/13/2025 02:17:05.886 7f0bbe5bcc18 [contrail-api] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:ContrailConfigTrace] map 02/13/2025 02:17:06.844 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:07.011 7f0bbd9c00f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource global-system-config: 'parent_type' 02/13/2025 02:17:07.011 7f0bbd9c00f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flags in resource global-system-config: 'feature_flags' 02/13/2025 02:17:07.011 7f0bbd9c00f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zones in resource global-system-config: 'control_node_zones' 02/13/2025 02:17:07.011 7f0bbd9c00f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_configs in resource global-system-config: 'global_vrouter_configs' 02/13/2025 02:17:07.011 7f0bbd9c00f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_configs in resource global-system-config: 'global_qos_configs' 02/13/2025 02:17:07.011 7f0bbd9c00f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_routers in resource global-system-config: 'virtual_routers' 02/13/2025 02:17:07.011 7f0bbd9c00f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_nodes in resource global-system-config: 'config_nodes' 02/13/2025 02:17:07.012 7f0bbd9c00f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_nodes in resource global-system-config: 'analytics_nodes' 02/13/2025 02:17:07.012 7f0bbd9c00f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_nodes in resource global-system-config: 'flow_nodes' 02/13/2025 02:17:07.012 7f0bbd9c00f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_nodes in resource global-system-config: 'devicemgr_nodes' 02/13/2025 02:17:07.012 7f0bbd9c00f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_nodes in resource global-system-config: 'database_nodes' 02/13/2025 02:17:07.012 7f0bbd9c00f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_nodes in resource global-system-config: 'webui_nodes' 02/13/2025 02:17:07.012 7f0bbd9c00f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_nodes in resource global-system-config: 'config_database_nodes' 02/13/2025 02:17:07.012 7f0bbd9c00f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_nodes in resource global-system-config: 'analytics_alarm_nodes' 02/13/2025 02:17:07.012 7f0bbd9c00f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_nodes in resource global-system-config: 'analytics_snmp_nodes' 02/13/2025 02:17:07.012 7f0bbd9c00f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_sets in resource global-system-config: 'service_appliance_sets' 02/13/2025 02:17:07.012 7f0bbd9c00f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource global-system-config: 'api_access_lists' 02/13/2025 02:17:07.012 7f0bbd9c00f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource global-system-config: 'alarms' 02/13/2025 02:17:07.013 7f0bbd9c00f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_propertiess in resource global-system-config: 'config_propertiess' 02/13/2025 02:17:07.013 7f0bbd9c00f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_templates in resource global-system-config: 'job_templates' 02/13/2025 02:17:07.014 7f0bbd9c00f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnects in resource global-system-config: 'data_center_interconnects' 02/13/2025 02:17:07.014 7f0bbd9c00f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_maps in resource global-system-config: 'intent_maps' 02/13/2025 02:17:07.014 7f0bbd9c00f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabrics in resource global-system-config: 'fabrics' 02/13/2025 02:17:07.014 7f0bbd9c00f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profiles in resource global-system-config: 'node_profiles' 02/13/2025 02:17:07.014 7f0bbd9c00f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_routers in resource global-system-config: 'physical_routers' 02/13/2025 02:17:07.014 7f0bbd9c00f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_images in resource global-system-config: 'device_images' 02/13/2025 02:17:07.014 7f0bbd9c00f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field nodes in resource global-system-config: 'nodes' 02/13/2025 02:17:07.014 7f0bbd9c00f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field features in resource global-system-config: 'features' 02/13/2025 02:17:07.015 7f0bbd9c00f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_roles in resource global-system-config: 'physical_roles' 02/13/2025 02:17:07.015 7f0bbd9c00f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_roles in resource global-system-config: 'overlay_roles' 02/13/2025 02:17:07.015 7f0bbd9c00f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definitions in resource global-system-config: 'role_definitions' 02/13/2025 02:17:07.015 7f0bbd9c00f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_configs in resource global-system-config: 'global_analytics_configs' 02/13/2025 02:17:07.015 7f0bbd9c00f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_refs in resource global-system-config: 'bgp_router_refs' 02/13/2025 02:17:07.015 7f0bbd9c00f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource global-system-config: 'tag_refs' 02/13/2025 02:17:07.015 7f0bbd9c00f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource global-system-config: 'qos_config_back_refs' 02/13/2025 02:17:07.117 7f0bbd9c00f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-os-473-1'] not found in DB: Unknown id: config_database_node default-global-system-config:cn-jenkins-deploy-platform-juju-os-473-1 02/13/2025 02:17:07.125 7f0bbdff9438 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = config_database_node identifier_name = default-global-system-config:cn-jenkins-deploy-platform-juju-os-473-1 url = https://10.0.0.17:8082/config-database-nodes operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_config_database_node.py remote_ip = 10.0.0.17 body = {"config-database-node": {"uuid": null, "fq_name": ["default-global-system-config", "cn-jenkins-deploy-platform-juju-os-473-1"], "parent_type": "global-system-config", "config_database_node_ip_address": "10.0.0.17", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "cn-jenkins-deploy-platform-juju-os-473-1"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:07.156 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-os-473-1'], 'obj_dict': {'config_database_node_ip_address': '10.0.0.17', 'display_name': 'cn-jenkins-deploy-platform-juju-os-473-1', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-os-473-1'], 'id_perms': {'created': '2025-02-13T02:17:07.144498', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:07.144498', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9476992846031022248, 'uuid_mslong': 13158323782092474584}}, 'parent_type': 'global-system-config', 'parent_uuid': '3a60be2c-e787-4ec9-894b-c0c6d7944765', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'b69bc1c3-9732-48d8-8385-0ac83e98e8a8'}, 'oper': 'CREATE', 'request-id': 'req-299e64c2-ca0c-4839-9db2-6619f14e922d', 'type': 'config_database_node', 'uuid': 'b69bc1c3-9732-48d8-8385-0ac83e98e8a8'} 02/13/2025 02:17:07.156 7f0bbe5bcc18 [contrail-api] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:ContrailConfigTrace] map 02/13/2025 02:17:07.845 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:08.846 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:09.234 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_routers in resource routing-instance: 'bgp_routers' 02/13/2025 02:17:09.234 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_refs in resource routing-instance: 'routing_instance_refs' 02/13/2025 02:17:09.234 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_target_refs in resource routing-instance: 'route_target_refs' 02/13/2025 02:17:09.235 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource routing-instance: 'tag_refs' 02/13/2025 02:17:09.235 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource routing-instance: 'virtual_machine_interface_back_refs' 02/13/2025 02:17:09.235 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregate_back_refs in resource routing-instance: 'route_aggregate_back_refs' 02/13/2025 02:17:09.235 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_back_refs in resource routing-instance: 'routing_policy_back_refs' 02/13/2025 02:17:09.235 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_back_refs in resource routing-instance: 'routing_instance_back_refs' 02/13/2025 02:17:09.290 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-juju-os-473-1'] not found in DB: Unknown id: bgp_router default-domain:default-project:ip-fabric:__default__:cn-jenkins-deploy-platform-juju-os-473-1 02/13/2025 02:17:09.296 7f0bbdd00588 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = bgp_router identifier_name = default-domain:default-project:ip-fabric:__default__:cn-jenkins-deploy-platform-juju-os-473-1 url = https://10.0.0.17:8082/bgp-routers operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_control.py remote_ip = 10.0.0.17 body = {"bgp-router": {"uuid": null, "fq_name": ["default-domain", "default-project", "ip-fabric", "__default__", "cn-jenkins-deploy-platform-juju-os-473-1"], "parent_type": "routing-instance", "bgp_router_parameters": {"admin_down": false, "vendor": "contrail", "cluster_id": null, "autonomous_system": 64512, "identifier": "10.0.0.17", "address": "10.0.0.17", "port": 179, "source_port": null, "hold_time": 0, "address_families": {"family": ["route-target", "inet-vpn", "e-vpn", "erm-vpn", "inet6-vpn"]}, "auth_data": null, "local_autonomous_system": null, "router_type": "control-node", "gateway_address": null, "ipv6_gateway_address": null}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "cn-jenkins-deploy-platform-juju-os-473-1"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:09.325 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-juju-os-473-1'], 'obj_dict': {'bgp_router_parameters': {'address': '10.0.0.17', 'address_families': {'family': ['route-target', 'inet-vpn', 'e-vpn', 'erm-vpn', 'inet6-vpn']}, 'admin_down': False, 'auth_data': None, 'autonomous_system': 64512, 'cluster_id': None, 'gateway_address': None, 'hold_time': 0, 'identifier': '10.0.0.17', 'ipv6_gateway_address': None, 'local_autonomous_system': None, 'port': 179, 'router_type': 'control-node', 'source_port': None, 'vendor': 'contrail'}, 'display_name': 'cn-jenkins-deploy-platform-juju-os-473-1', 'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-juju-os-473-1'], 'id_perms': {'created': '2025-02-13T02:17:09.316738', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:09.316738', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9820898330021686608, 'uuid_mslong': 11686432484423910657}}, 'parent_type': 'routing-instance', 'parent_uuid': 'ee7c4a6e-9c5f-4011-b493-66a7db24a1ae', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'a22e8c6d-5dac-4501-884a-d6ff702b6d50'}, 'oper': 'CREATE', 'request-id': 'req-4f494846-b568-4d74-a6b4-cb059585cf6a', 'type': 'bgp_router', 'uuid': 'a22e8c6d-5dac-4501-884a-d6ff702b6d50'} 02/13/2025 02:17:09.325 7f0bbe5bcc18 [contrail-api] [DEBUG]: Add uve in the [ObjectBgpRouter:ContrailConfigTrace] map 02/13/2025 02:17:09.378 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zone_refs in resource bgp-router: 'control_node_zone_refs' 02/13/2025 02:17:09.379 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource bgp-router: 'sub_cluster_refs' 02/13/2025 02:17:09.379 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_refs in resource bgp-router: 'bgp_router_refs' 02/13/2025 02:17:09.379 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource bgp-router: 'tag_refs' 02/13/2025 02:17:09.379 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_system_config_back_refs in resource bgp-router: 'global_system_config_back_refs' 02/13/2025 02:17:09.379 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource bgp-router: 'physical_router_back_refs' 02/13/2025 02:17:09.379 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource bgp-router: 'virtual_machine_interface_back_refs' 02/13/2025 02:17:09.380 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource bgp-router: 'bgp_as_a_service_back_refs' 02/13/2025 02:17:09.380 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_back_refs in resource bgp-router: 'bgp_router_back_refs' 02/13/2025 02:17:09.478 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource global-system-config: 'parent_type' 02/13/2025 02:17:09.479 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flags in resource global-system-config: 'feature_flags' 02/13/2025 02:17:09.479 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zones in resource global-system-config: 'control_node_zones' 02/13/2025 02:17:09.480 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_configs in resource global-system-config: 'global_vrouter_configs' 02/13/2025 02:17:09.480 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_configs in resource global-system-config: 'global_qos_configs' 02/13/2025 02:17:09.480 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_routers in resource global-system-config: 'virtual_routers' 02/13/2025 02:17:09.480 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_nodes in resource global-system-config: 'config_nodes' 02/13/2025 02:17:09.480 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_nodes in resource global-system-config: 'analytics_nodes' 02/13/2025 02:17:09.480 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_nodes in resource global-system-config: 'flow_nodes' 02/13/2025 02:17:09.481 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_nodes in resource global-system-config: 'devicemgr_nodes' 02/13/2025 02:17:09.481 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_nodes in resource global-system-config: 'database_nodes' 02/13/2025 02:17:09.481 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_nodes in resource global-system-config: 'webui_nodes' 02/13/2025 02:17:09.481 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_nodes in resource global-system-config: 'config_database_nodes' 02/13/2025 02:17:09.482 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_nodes in resource global-system-config: 'analytics_alarm_nodes' 02/13/2025 02:17:09.482 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_nodes in resource global-system-config: 'analytics_snmp_nodes' 02/13/2025 02:17:09.482 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_sets in resource global-system-config: 'service_appliance_sets' 02/13/2025 02:17:09.482 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource global-system-config: 'api_access_lists' 02/13/2025 02:17:09.482 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource global-system-config: 'alarms' 02/13/2025 02:17:09.483 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_propertiess in resource global-system-config: 'config_propertiess' 02/13/2025 02:17:09.483 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_templates in resource global-system-config: 'job_templates' 02/13/2025 02:17:09.483 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnects in resource global-system-config: 'data_center_interconnects' 02/13/2025 02:17:09.483 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_maps in resource global-system-config: 'intent_maps' 02/13/2025 02:17:09.483 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabrics in resource global-system-config: 'fabrics' 02/13/2025 02:17:09.483 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profiles in resource global-system-config: 'node_profiles' 02/13/2025 02:17:09.483 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_routers in resource global-system-config: 'physical_routers' 02/13/2025 02:17:09.483 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_images in resource global-system-config: 'device_images' 02/13/2025 02:17:09.483 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field nodes in resource global-system-config: 'nodes' 02/13/2025 02:17:09.483 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field features in resource global-system-config: 'features' 02/13/2025 02:17:09.484 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_roles in resource global-system-config: 'physical_roles' 02/13/2025 02:17:09.484 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_roles in resource global-system-config: 'overlay_roles' 02/13/2025 02:17:09.487 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definitions in resource global-system-config: 'role_definitions' 02/13/2025 02:17:09.487 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_configs in resource global-system-config: 'global_analytics_configs' 02/13/2025 02:17:09.487 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_refs in resource global-system-config: 'bgp_router_refs' 02/13/2025 02:17:09.487 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource global-system-config: 'tag_refs' 02/13/2025 02:17:09.487 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource global-system-config: 'qos_config_back_refs' 02/13/2025 02:17:09.537 7f0bbac35ac8 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 3a60be2c-e787-4ec9-894b-c0c6d7944765 object_type = global_system_config identifier_name = default-global-system-config url = https://10.0.0.17:8082/ref-relax-for-delete operation = ref-relax-for-delete useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_control.py remote_ip = 10.0.0.17 body = {"uuid": "3a60be2c-e787-4ec9-894b-c0c6d7944765", "ref-uuid": "a22e8c6d-5dac-4501-884a-d6ff702b6d50"} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:09.588 7f0bbddbdd68 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 3a60be2c-e787-4ec9-894b-c0c6d7944765 object_type = global_system_config identifier_name = default-global-system-config url = https://10.0.0.17:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_control.py remote_ip = 10.0.0.17 body = {"type": "global-system-config", "uuid": "3a60be2c-e787-4ec9-894b-c0c6d7944765", "ref-type": "bgp-router", "ref-uuid": "a22e8c6d-5dac-4501-884a-d6ff702b6d50", "ref-fq-name": ["default-domain", "default-project", "ip-fabric", "__default__", "cn-jenkins-deploy-platform-juju-os-473-1"], "operation": "ADD", "attr": null} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:09.602 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-ba6366b8-ff09-493c-876e-843a9469bb24', 'type': 'global_system_config', 'uuid': '3a60be2c-e787-4ec9-894b-c0c6d7944765'} 02/13/2025 02:17:09.848 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:09.929 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_refs in resource network-ipam: 'virtual_DNS_refs' 02/13/2025 02:17:09.929 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource network-ipam: 'tag_refs' 02/13/2025 02:17:09.930 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource network-ipam: 'virtual_network_back_refs' 02/13/2025 02:17:09.930 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_back_refs in resource network-ipam: 'virtual_router_back_refs' 02/13/2025 02:17:09.930 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource network-ipam: 'instance_ip_back_refs' 02/13/2025 02:17:10.548 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'default-project', '__no_rule__', 'ingress-access-control-list'] not found in DB: Unknown id: access_control_list default-domain:default-project:__no_rule__:ingress-access-control-list 02/13/2025 02:17:10.552 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource global-system-config: 'parent_type' 02/13/2025 02:17:10.553 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flags in resource global-system-config: 'feature_flags' 02/13/2025 02:17:10.553 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zones in resource global-system-config: 'control_node_zones' 02/13/2025 02:17:10.553 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_configs in resource global-system-config: 'global_vrouter_configs' 02/13/2025 02:17:10.553 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_configs in resource global-system-config: 'global_qos_configs' 02/13/2025 02:17:10.553 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_routers in resource global-system-config: 'virtual_routers' 02/13/2025 02:17:10.553 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_nodes in resource global-system-config: 'config_nodes' 02/13/2025 02:17:10.554 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_nodes in resource global-system-config: 'analytics_nodes' 02/13/2025 02:17:10.554 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_nodes in resource global-system-config: 'flow_nodes' 02/13/2025 02:17:10.554 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_nodes in resource global-system-config: 'devicemgr_nodes' 02/13/2025 02:17:10.554 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_nodes in resource global-system-config: 'database_nodes' 02/13/2025 02:17:10.554 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_nodes in resource global-system-config: 'webui_nodes' 02/13/2025 02:17:10.554 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_nodes in resource global-system-config: 'config_database_nodes' 02/13/2025 02:17:10.554 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_nodes in resource global-system-config: 'analytics_alarm_nodes' 02/13/2025 02:17:10.554 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_nodes in resource global-system-config: 'analytics_snmp_nodes' 02/13/2025 02:17:10.555 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_sets in resource global-system-config: 'service_appliance_sets' 02/13/2025 02:17:10.555 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource global-system-config: 'api_access_lists' 02/13/2025 02:17:10.555 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource global-system-config: 'alarms' 02/13/2025 02:17:10.555 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_propertiess in resource global-system-config: 'config_propertiess' 02/13/2025 02:17:10.555 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_templates in resource global-system-config: 'job_templates' 02/13/2025 02:17:10.555 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnects in resource global-system-config: 'data_center_interconnects' 02/13/2025 02:17:10.555 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_maps in resource global-system-config: 'intent_maps' 02/13/2025 02:17:10.555 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabrics in resource global-system-config: 'fabrics' 02/13/2025 02:17:10.555 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profiles in resource global-system-config: 'node_profiles' 02/13/2025 02:17:10.556 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_routers in resource global-system-config: 'physical_routers' 02/13/2025 02:17:10.556 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_images in resource global-system-config: 'device_images' 02/13/2025 02:17:10.556 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field nodes in resource global-system-config: 'nodes' 02/13/2025 02:17:10.556 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field features in resource global-system-config: 'features' 02/13/2025 02:17:10.556 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_roles in resource global-system-config: 'physical_roles' 02/13/2025 02:17:10.556 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_roles in resource global-system-config: 'overlay_roles' 02/13/2025 02:17:10.556 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definitions in resource global-system-config: 'role_definitions' 02/13/2025 02:17:10.556 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_configs in resource global-system-config: 'global_analytics_configs' 02/13/2025 02:17:10.557 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource global-system-config: 'tag_refs' 02/13/2025 02:17:10.557 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource global-system-config: 'qos_config_back_refs' 02/13/2025 02:17:10.559 7f0bbddc9048 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = access_control_list identifier_name = default-domain:default-project:__no_rule__:ingress-access-control-list url = https://10.0.0.17:8082/access-control-lists operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.17 body = {"access-control-list": {"uuid": null, "fq_name": ["default-domain", "default-project", "__no_rule__", "ingress-access-control-list"], "parent_type": "security-group", "access_control_list_entries": {"dynamic": null, "acl_rule": []}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "ingress-access-control-list"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:10.580 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', '__no_rule__', 'ingress-access-control-list'], 'obj_dict': {'access_control_list_entries': {'acl_rule': [], 'dynamic': None}, 'display_name': 'ingress-access-control-list', 'fq_name': ['default-domain', 'default-project', '__no_rule__', 'ingress-access-control-list'], 'id_perms': {'created': '2025-02-13T02:17:10.571281', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:10.571281', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13318336300922553187, 'uuid_mslong': 418697305953028445}}, 'parent_type': 'security-group', 'parent_uuid': 'dea42fc5-e9ba-4e38-b637-8b4d5394c87c', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '05cf82fb-3dbe-495d-b8d4-3c4d57adc363'}, 'oper': 'CREATE', 'request-id': 'req-0ddfbaeb-5b83-4974-9135-90ffb5ec8447', 'type': 'access_control_list', 'uuid': '05cf82fb-3dbe-495d-b8d4-3c4d57adc363'} 02/13/2025 02:17:10.611 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'opencontrail'] not found in DB: Unknown id: service_appliance_set default-global-system-config:opencontrail 02/13/2025 02:17:10.613 7f0bbe54dcc0 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = service_appliance_set identifier_name = default-global-system-config:opencontrail url = https://10.0.0.17:8082/service-appliance-sets operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/usr/local/bin/contrail-svc-monitor remote_ip = 10.0.0.17 body = {"service-appliance-set": {"uuid": null, "fq_name": ["default-global-system-config", "opencontrail"], "parent_type": "global-system-config", "service_appliance_driver": "svc_monitor.services.loadbalancer.drivers.ha_proxy.driver.OpencontrailLoadbalancerDriver", "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 5, "share": []}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "display_name": "opencontrail"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:10.638 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'opencontrail'], 'obj_dict': {'display_name': 'opencontrail', 'fq_name': ['default-global-system-config', 'opencontrail'], 'id_perms': {'created': '2025-02-13T02:17:10.631505', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:10.631505', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9658517111908841868, 'uuid_mslong': 527910570601235231}}, 'parent_type': 'global-system-config', 'parent_uuid': '3a60be2c-e787-4ec9-894b-c0c6d7944765', 'perms2': {'global_access': 5, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'service_appliance_driver': 'svc_monitor.services.loadbalancer.drivers.ha_proxy.driver.OpencontrailLoadbalancerDriver', 'uuid': '075383dd-f1dc-431f-8609-f223e85bb98c'}, 'oper': 'CREATE', 'request-id': 'req-e4eff886-02f1-4da5-9370-12061c8c8e05', 'type': 'service_appliance_set', 'uuid': '075383dd-f1dc-431f-8609-f223e85bb98c'} 02/13/2025 02:17:10.678 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'default-project', '__no_rule__', 'egress-access-control-list'] not found in DB: Unknown id: access_control_list default-domain:default-project:__no_rule__:egress-access-control-list 02/13/2025 02:17:10.684 7f0bbdd00e10 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = access_control_list identifier_name = default-domain:default-project:__no_rule__:egress-access-control-list url = https://10.0.0.17:8082/access-control-lists operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.17 body = {"access-control-list": {"uuid": null, "fq_name": ["default-domain", "default-project", "__no_rule__", "egress-access-control-list"], "parent_type": "security-group", "access_control_list_entries": {"dynamic": null, "acl_rule": []}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "egress-access-control-list"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:10.709 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', '__no_rule__', 'egress-access-control-list'], 'obj_dict': {'access_control_list_entries': {'acl_rule': [], 'dynamic': None}, 'display_name': 'egress-access-control-list', 'fq_name': ['default-domain', 'default-project', '__no_rule__', 'egress-access-control-list'], 'id_perms': {'created': '2025-02-13T02:17:10.700147', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:10.700147', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10084730549226137756, 'uuid_mslong': 4944233602971159138}}, 'parent_type': 'security-group', 'parent_uuid': 'dea42fc5-e9ba-4e38-b637-8b4d5394c87c', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '449d7244-27a5-4a62-8bf4-2900da27689c'}, 'oper': 'CREATE', 'request-id': 'req-91c61c10-07fd-443b-93ad-772967402a3c', 'type': 'access_control_list', 'uuid': '449d7244-27a5-4a62-8bf4-2900da27689c'} 02/13/2025 02:17:10.799 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['target:64512:8000000'] not found in DB: Unknown id: route_target target:64512:8000000 02/13/2025 02:17:10.800 7f0bbda0bb70 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = route_target identifier_name = target:64512:8000000 url = https://10.0.0.17:8082/route-targets operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.17 body = {"route-target": {"uuid": null, "fq_name": ["target:64512:8000000"], "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "e13ef443d9ab406792437e7ea9d9e4c9", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "target:64512:8000000"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:10.802 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource global-system-config: 'parent_type' 02/13/2025 02:17:10.802 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flags in resource global-system-config: 'feature_flags' 02/13/2025 02:17:10.802 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zones in resource global-system-config: 'control_node_zones' 02/13/2025 02:17:10.802 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_configs in resource global-system-config: 'global_vrouter_configs' 02/13/2025 02:17:10.802 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_configs in resource global-system-config: 'global_qos_configs' 02/13/2025 02:17:10.803 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_routers in resource global-system-config: 'virtual_routers' 02/13/2025 02:17:10.803 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_nodes in resource global-system-config: 'config_nodes' 02/13/2025 02:17:10.803 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_nodes in resource global-system-config: 'analytics_nodes' 02/13/2025 02:17:10.803 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_nodes in resource global-system-config: 'flow_nodes' 02/13/2025 02:17:10.803 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_nodes in resource global-system-config: 'devicemgr_nodes' 02/13/2025 02:17:10.803 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_nodes in resource global-system-config: 'database_nodes' 02/13/2025 02:17:10.803 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_nodes in resource global-system-config: 'webui_nodes' 02/13/2025 02:17:10.803 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_nodes in resource global-system-config: 'config_database_nodes' 02/13/2025 02:17:10.803 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_nodes in resource global-system-config: 'analytics_alarm_nodes' 02/13/2025 02:17:10.803 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_nodes in resource global-system-config: 'analytics_snmp_nodes' 02/13/2025 02:17:10.803 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_sets in resource global-system-config: 'service_appliance_sets' 02/13/2025 02:17:10.804 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource global-system-config: 'api_access_lists' 02/13/2025 02:17:10.804 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource global-system-config: 'alarms' 02/13/2025 02:17:10.804 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_propertiess in resource global-system-config: 'config_propertiess' 02/13/2025 02:17:10.804 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_templates in resource global-system-config: 'job_templates' 02/13/2025 02:17:10.804 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnects in resource global-system-config: 'data_center_interconnects' 02/13/2025 02:17:10.804 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_maps in resource global-system-config: 'intent_maps' 02/13/2025 02:17:10.804 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabrics in resource global-system-config: 'fabrics' 02/13/2025 02:17:10.804 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profiles in resource global-system-config: 'node_profiles' 02/13/2025 02:17:10.804 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_routers in resource global-system-config: 'physical_routers' 02/13/2025 02:17:10.804 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_images in resource global-system-config: 'device_images' 02/13/2025 02:17:10.804 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field nodes in resource global-system-config: 'nodes' 02/13/2025 02:17:10.805 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field features in resource global-system-config: 'features' 02/13/2025 02:17:10.805 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_roles in resource global-system-config: 'physical_roles' 02/13/2025 02:17:10.805 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_roles in resource global-system-config: 'overlay_roles' 02/13/2025 02:17:10.805 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definitions in resource global-system-config: 'role_definitions' 02/13/2025 02:17:10.805 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_configs in resource global-system-config: 'global_analytics_configs' 02/13/2025 02:17:10.805 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource global-system-config: 'tag_refs' 02/13/2025 02:17:10.805 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource global-system-config: 'qos_config_back_refs' 02/13/2025 02:17:10.824 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['target:64512:8000000'], 'obj_dict': {'display_name': 'target:64512:8000000', 'fq_name': ['target:64512:8000000'], 'id_perms': {'created': '2025-02-13T02:17:10.811187', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:10.811187', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12772439888617361680, 'uuid_mslong': 16486111357440183425}}, 'perms2': {'global_access': 0, 'owner': 'e13ef443d9ab406792437e7ea9d9e4c9', 'owner_access': 7, 'share': []}, 'uuid': 'e4ca6eee-b3c1-4c81-b140-d2684c115110'}, 'oper': 'CREATE', 'request-id': 'req-5f70f208-7bfe-4cde-9f89-285e93556c9c', 'type': 'route_target', 'uuid': 'e4ca6eee-b3c1-4c81-b140-d2684c115110'} 02/13/2025 02:17:10.833 7f0bbd9c2438 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = ff406ce8-3426-40e6-ae98-5ecec9f03ee2 object_type = routing_instance identifier_name = default-domain:default-project:dci-network:__default__ url = https://10.0.0.17:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-os-473-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.17 body = {"type": "routing-instance", "uuid": "ff406ce8-3426-40e6-ae98-5ecec9f03ee2", "ref-type": "route-target", "ref-uuid": "e4ca6eee-b3c1-4c81-b140-d2684c115110", "ref-fq-name": ["target:64512:8000000"], "operation": "ADD", "attr": {"import_export": null}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:10.849 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:10.851 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'dci-network', '__default__'], 'oper': 'UPDATE', 'request-id': 'req-adacf85d-a50a-4030-9e50-13384a0f8657', 'type': 'routing_instance', 'uuid': 'ff406ce8-3426-40e6-ae98-5ecec9f03ee2'} 02/13/2025 02:17:10.862 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'native'] not found in DB: Unknown id: service_appliance_set default-global-system-config:native 02/13/2025 02:17:10.863 7f0bbde64c18 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = service_appliance_set identifier_name = default-global-system-config:native url = https://10.0.0.17:8082/service-appliance-sets operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/usr/local/bin/contrail-svc-monitor remote_ip = 10.0.0.17 body = {"service-appliance-set": {"uuid": null, "fq_name": ["default-global-system-config", "native"], "parent_type": "global-system-config", "service_appliance_driver": "svc_monitor.services.loadbalancer.drivers.native.driver.OpencontrailLoadbalancerDriver", "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 5, "share": []}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "display_name": "native"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:10.889 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'native'], 'obj_dict': {'display_name': 'native', 'fq_name': ['default-global-system-config', 'native'], 'id_perms': {'created': '2025-02-13T02:17:10.881238', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:10.881238', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10202533193716968233, 'uuid_mslong': 5725000121503403422}}, 'parent_type': 'global-system-config', 'parent_uuid': '3a60be2c-e787-4ec9-894b-c0c6d7944765', 'perms2': {'global_access': 5, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'service_appliance_driver': 'svc_monitor.services.loadbalancer.drivers.native.driver.OpencontrailLoadbalancerDriver', 'uuid': '4f734947-3547-4d9e-8d96-ade264a1ab29'}, 'oper': 'CREATE', 'request-id': 'req-c6292874-0a97-4a4d-ad23-5c47b20dad11', 'type': 'service_appliance_set', 'uuid': '4f734947-3547-4d9e-8d96-ade264a1ab29'} 02/13/2025 02:17:10.913 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['target:64512:8000001'] not found in DB: Unknown id: route_target target:64512:8000001 02/13/2025 02:17:10.914 7f0bbddc9198 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = route_target identifier_name = target:64512:8000001 url = https://10.0.0.17:8082/route-targets operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.17 body = {"route-target": {"uuid": null, "fq_name": ["target:64512:8000001"], "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "e13ef443d9ab406792437e7ea9d9e4c9", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "target:64512:8000001"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:10.935 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['target:64512:8000001'], 'obj_dict': {'display_name': 'target:64512:8000001', 'fq_name': ['target:64512:8000001'], 'id_perms': {'created': '2025-02-13T02:17:10.925981', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:10.925981', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11231058139336376450, 'uuid_mslong': 1474997349150379837}}, 'perms2': {'global_access': 0, 'owner': 'e13ef443d9ab406792437e7ea9d9e4c9', 'owner_access': 7, 'share': []}, 'uuid': '14783e46-30c0-4f3d-9bdc-bbdf7ee78082'}, 'oper': 'CREATE', 'request-id': 'req-5abda940-9c7a-4378-83e9-4e1886aeaa1e', 'type': 'route_target', 'uuid': '14783e46-30c0-4f3d-9bdc-bbdf7ee78082'} 02/13/2025 02:17:10.985 7f0bbd933a20 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 43b37af9-cde4-4131-96db-766e36007171 object_type = routing_instance identifier_name = default-domain:default-project:dci-network:dci-network url = https://10.0.0.17:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-os-473-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.17 body = {"type": "routing-instance", "uuid": "43b37af9-cde4-4131-96db-766e36007171", "ref-type": "route-target", "ref-uuid": "14783e46-30c0-4f3d-9bdc-bbdf7ee78082", "ref-fq-name": ["target:64512:8000001"], "operation": "ADD", "attr": {"import_export": null}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:11.002 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'dci-network', 'dci-network'], 'oper': 'UPDATE', 'request-id': 'req-cc4828f7-38c0-40c2-9310-04f42cddf586', 'type': 'routing_instance', 'uuid': '43b37af9-cde4-4131-96db-766e36007171'} 02/13/2025 02:17:11.010 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_refs in resource network-ipam: 'virtual_DNS_refs' 02/13/2025 02:17:11.010 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource network-ipam: 'tag_refs' 02/13/2025 02:17:11.010 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource network-ipam: 'virtual_network_back_refs' 02/13/2025 02:17:11.010 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_back_refs in resource network-ipam: 'virtual_router_back_refs' 02/13/2025 02:17:11.010 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource network-ipam: 'instance_ip_back_refs' 02/13/2025 02:17:11.060 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['target:64512:8000002'] not found in DB: Unknown id: route_target target:64512:8000002 02/13/2025 02:17:11.061 7f0bbda0b630 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = route_target identifier_name = target:64512:8000002 url = https://10.0.0.17:8082/route-targets operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.17 body = {"route-target": {"uuid": null, "fq_name": ["target:64512:8000002"], "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "e13ef443d9ab406792437e7ea9d9e4c9", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "target:64512:8000002"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:11.075 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['target:64512:8000002'], 'obj_dict': {'display_name': 'target:64512:8000002', 'fq_name': ['target:64512:8000002'], 'id_perms': {'created': '2025-02-13T02:17:11.067453', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:11.067453', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12866571567265791731, 'uuid_mslong': 18442817170272636097}}, 'perms2': {'global_access': 0, 'owner': 'e13ef443d9ab406792437e7ea9d9e4c9', 'owner_access': 7, 'share': []}, 'uuid': 'fff20c80-8b93-4cc1-b28f-3eaba004bef3'}, 'oper': 'CREATE', 'request-id': 'req-5edf8ac4-1501-4797-863b-b96771c5f553', 'type': 'route_target', 'uuid': 'fff20c80-8b93-4cc1-b28f-3eaba004bef3'} 02/13/2025 02:17:11.130 7f0bbd937c18 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 13d34993-7917-4d70-8967-f0463eca9958 object_type = routing_instance identifier_name = default-domain:default-project:default-virtual-network:default-virtual-network url = https://10.0.0.17:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-os-473-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.17 body = {"type": "routing-instance", "uuid": "13d34993-7917-4d70-8967-f0463eca9958", "ref-type": "route-target", "ref-uuid": "fff20c80-8b93-4cc1-b28f-3eaba004bef3", "ref-fq-name": ["target:64512:8000002"], "operation": "ADD", "attr": {"import_export": null}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:11.148 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'default-virtual-network', 'default-virtual-network'], 'oper': 'UPDATE', 'request-id': 'req-3b551a41-d5af-419c-af0c-f1831b46378f', 'type': 'routing_instance', 'uuid': '13d34993-7917-4d70-8967-f0463eca9958'} 02/13/2025 02:17:11.172 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'analyzer-template'] not found in DB: Unknown id: service_template default-domain:analyzer-template 02/13/2025 02:17:11.174 7f0bbd941eb8 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = service_template identifier_name = default-domain:analyzer-template url = https://10.0.0.17:8082/service-templates operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/usr/local/bin/contrail-svc-monitor remote_ip = 10.0.0.17 body = {"service-template": {"uuid": null, "fq_name": ["default-domain", "analyzer-template"], "parent_type": "domain", "service_template_properties": {"version": 1, "service_mode": null, "service_type": "analyzer", "image_name": "analyzer", "service_scaling": false, "interface_type": [{"service_interface_type": "left", "shared_ip": false, "static_route_enable": false}], "flavor": "m1.medium", "ordered_interfaces": true, "service_virtualization_type": "virtual-machine", "availability_zone_enable": false, "vrouter_instance_type": null, "instance_data": null}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "e13ef443d9ab406792437e7ea9d9e4c9", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "analyzer-template"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:11.198 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'analyzer-template'], 'obj_dict': {'display_name': 'analyzer-template', 'fq_name': ['default-domain', 'analyzer-template'], 'id_perms': {'created': '2025-02-13T02:17:11.191131', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:11.191131', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9468529288444057509, 'uuid_mslong': 693988657968270999}}, 'parent_type': 'domain', 'parent_uuid': '47137821-8208-4069-9c3b-21e4e16a0f01', 'perms2': {'global_access': 0, 'owner': 'e13ef443d9ab406792437e7ea9d9e4c9', 'owner_access': 7, 'share': [{'tenant': 'domain:47137821-8208-4069-9c3b-21e4e16a0f01', 'tenant_access': 5}]}, 'service_template_properties': {'availability_zone_enable': False, 'flavor': 'm1.medium', 'image_name': 'analyzer', 'instance_data': None, 'interface_type': [{'service_interface_type': 'left', 'shared_ip': False, 'static_route_enable': False}], 'ordered_interfaces': True, 'service_mode': None, 'service_scaling': False, 'service_type': 'analyzer', 'service_virtualization_type': 'virtual-machine', 'version': 1, 'vrouter_instance_type': None}, 'uuid': '09a18b01-ec59-4e97-8366-f938af1717a5'}, 'oper': 'CREATE', 'request-id': 'req-d40b18ea-ab10-4f2b-83e7-bf7b5c2b2450', 'type': 'service_template', 'uuid': '09a18b01-ec59-4e97-8366-f938af1717a5'} 02/13/2025 02:17:11.213 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['target:64512:8000003'] not found in DB: Unknown id: route_target target:64512:8000003 02/13/2025 02:17:11.214 7f0bbd9ff2e8 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = route_target identifier_name = target:64512:8000003 url = https://10.0.0.17:8082/route-targets operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.17 body = {"route-target": {"uuid": null, "fq_name": ["target:64512:8000003"], "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "e13ef443d9ab406792437e7ea9d9e4c9", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "target:64512:8000003"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:11.215 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'nat-template'] not found in DB: Unknown id: service_template default-domain:nat-template 02/13/2025 02:17:11.216 7f0bbd9c7630 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = service_template identifier_name = default-domain:nat-template url = https://10.0.0.17:8082/service-templates operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/usr/local/bin/contrail-svc-monitor remote_ip = 10.0.0.17 body = {"service-template": {"uuid": null, "fq_name": ["default-domain", "nat-template"], "parent_type": "domain", "service_template_properties": {"version": 1, "service_mode": "in-network-nat", "service_type": "firewall", "image_name": "analyzer", "service_scaling": false, "interface_type": [{"service_interface_type": "management", "shared_ip": false, "static_route_enable": false}, {"service_interface_type": "left", "shared_ip": false, "static_route_enable": false}, {"service_interface_type": "right", "shared_ip": false, "static_route_enable": false}], "flavor": "m1.medium", "ordered_interfaces": true, "service_virtualization_type": "virtual-machine", "availability_zone_enable": false, "vrouter_instance_type": null, "instance_data": null}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "e13ef443d9ab406792437e7ea9d9e4c9", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "nat-template"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:11.237 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['target:64512:8000003'], 'obj_dict': {'display_name': 'target:64512:8000003', 'fq_name': ['target:64512:8000003'], 'id_perms': {'created': '2025-02-13T02:17:11.226592', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:11.226592', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12093415891999580938, 'uuid_mslong': 16214369266211766523}}, 'perms2': {'global_access': 0, 'owner': 'e13ef443d9ab406792437e7ea9d9e4c9', 'owner_access': 7, 'share': []}, 'uuid': 'e10502f0-b190-40fb-a7d4-71ae1bb2e30a'}, 'oper': 'CREATE', 'request-id': 'req-5a6a7d3c-61f9-4260-bbef-89a874f96f87', 'type': 'route_target', 'uuid': 'e10502f0-b190-40fb-a7d4-71ae1bb2e30a'} 02/13/2025 02:17:11.246 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'nat-template'], 'obj_dict': {'display_name': 'nat-template', 'fq_name': ['default-domain', 'nat-template'], 'id_perms': {'created': '2025-02-13T02:17:11.233950', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:11.233950', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9696234691826422685, 'uuid_mslong': 13873115063541646902}}, 'parent_type': 'domain', 'parent_uuid': '47137821-8208-4069-9c3b-21e4e16a0f01', 'perms2': {'global_access': 0, 'owner': 'e13ef443d9ab406792437e7ea9d9e4c9', 'owner_access': 7, 'share': [{'tenant': 'domain:47137821-8208-4069-9c3b-21e4e16a0f01', 'tenant_access': 5}]}, 'service_template_properties': {'availability_zone_enable': False, 'flavor': 'm1.medium', 'image_name': 'analyzer', 'instance_data': None, 'interface_type': [{'service_interface_type': 'management', 'shared_ip': False, 'static_route_enable': False}, {'service_interface_type': 'left', 'shared_ip': False, 'static_route_enable': False}, {'service_interface_type': 'right', 'shared_ip': False, 'static_route_enable': False}], 'ordered_interfaces': True, 'service_mode': 'in-network-nat', 'service_scaling': False, 'service_type': 'firewall', 'service_virtualization_type': 'virtual-machine', 'version': 1, 'vrouter_instance_type': None}, 'uuid': 'c08734a5-ba85-4636-868f-f2145122139d'}, 'oper': 'CREATE', 'request-id': 'req-0a1d28c8-534f-4e6a-a2cc-22ffe4edce56', 'type': 'service_template', 'uuid': 'c08734a5-ba85-4636-868f-f2145122139d'} 02/13/2025 02:17:11.291 7f0bbd950e10 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 5194c63a-dd89-4fca-aa15-43c79ab61241 object_type = routing_instance identifier_name = default-domain:default-project:ip-fabric:ip-fabric url = https://10.0.0.17:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-os-473-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.17 body = {"type": "routing-instance", "uuid": "5194c63a-dd89-4fca-aa15-43c79ab61241", "ref-type": "route-target", "ref-uuid": "e10502f0-b190-40fb-a7d4-71ae1bb2e30a", "ref-fq-name": ["target:64512:8000003"], "operation": "ADD", "attr": {"import_export": null}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:11.307 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'netns-snat-template'] not found in DB: Unknown id: service_template default-domain:netns-snat-template 02/13/2025 02:17:11.308 7f0bbac326d8 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = service_template identifier_name = default-domain:netns-snat-template url = https://10.0.0.17:8082/service-templates operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/usr/local/bin/contrail-svc-monitor remote_ip = 10.0.0.17 body = {"service-template": {"uuid": null, "fq_name": ["default-domain", "netns-snat-template"], "parent_type": "domain", "service_template_properties": {"version": 1, "service_mode": "in-network-nat", "service_type": "source-nat", "image_name": null, "service_scaling": true, "interface_type": [{"service_interface_type": "right", "shared_ip": true, "static_route_enable": false}, {"service_interface_type": "left", "shared_ip": true, "static_route_enable": false}], "flavor": null, "ordered_interfaces": true, "service_virtualization_type": "network-namespace", "availability_zone_enable": false, "vrouter_instance_type": null, "instance_data": null}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "e13ef443d9ab406792437e7ea9d9e4c9", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "netns-snat-template"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:11.318 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'oper': 'UPDATE', 'request-id': 'req-c8972369-d3ba-448a-b1cf-406a74994a5c', 'type': 'routing_instance', 'uuid': '5194c63a-dd89-4fca-aa15-43c79ab61241'} 02/13/2025 02:17:11.336 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'netns-snat-template'], 'obj_dict': {'display_name': 'netns-snat-template', 'fq_name': ['default-domain', 'netns-snat-template'], 'id_perms': {'created': '2025-02-13T02:17:11.328349', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:11.328349', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11619248890244165126, 'uuid_mslong': 5817549109588870858}}, 'parent_type': 'domain', 'parent_uuid': '47137821-8208-4069-9c3b-21e4e16a0f01', 'perms2': {'global_access': 0, 'owner': 'e13ef443d9ab406792437e7ea9d9e4c9', 'owner_access': 7, 'share': [{'tenant': 'domain:47137821-8208-4069-9c3b-21e4e16a0f01', 'tenant_access': 5}]}, 'service_template_properties': {'availability_zone_enable': False, 'flavor': None, 'image_name': None, 'instance_data': None, 'interface_type': [{'service_interface_type': 'right', 'shared_ip': True, 'static_route_enable': False}, {'service_interface_type': 'left', 'shared_ip': True, 'static_route_enable': False}], 'ordered_interfaces': True, 'service_mode': 'in-network-nat', 'service_scaling': True, 'service_type': 'source-nat', 'service_virtualization_type': 'network-namespace', 'version': 1, 'vrouter_instance_type': None}, 'uuid': '50bc1617-ac6f-46ca-a13f-dd4de3daee06'}, 'oper': 'CREATE', 'request-id': 'req-b3b39497-8bdc-4111-b80f-2ab08a597f11', 'type': 'service_template', 'uuid': '50bc1617-ac6f-46ca-a13f-dd4de3daee06'} 02/13/2025 02:17:11.396 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'haproxy-loadbalancer-template'] not found in DB: Unknown id: service_template default-domain:haproxy-loadbalancer-template 02/13/2025 02:17:11.397 7f0bbddb04e0 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = service_template identifier_name = default-domain:haproxy-loadbalancer-template url = https://10.0.0.17:8082/service-templates operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/usr/local/bin/contrail-svc-monitor remote_ip = 10.0.0.17 body = {"service-template": {"uuid": null, "fq_name": ["default-domain", "haproxy-loadbalancer-template"], "parent_type": "domain", "service_template_properties": {"version": 1, "service_mode": "in-network-nat", "service_type": "loadbalancer", "image_name": null, "service_scaling": true, "interface_type": [{"service_interface_type": "right", "shared_ip": true, "static_route_enable": false}, {"service_interface_type": "left", "shared_ip": true, "static_route_enable": false}], "flavor": null, "ordered_interfaces": true, "service_virtualization_type": "network-namespace", "availability_zone_enable": false, "vrouter_instance_type": null, "instance_data": null}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "e13ef443d9ab406792437e7ea9d9e4c9", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "haproxy-loadbalancer-template"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:11.409 7f0bbddc2240 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_objects in resource global-vrouter-config: 'security_logging_objects' 02/13/2025 02:17:11.409 7f0bbddc2240 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource global-vrouter-config: 'tag_refs' 02/13/2025 02:17:11.409 7f0bbddc2240 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource global-vrouter-config: 'application_policy_set_back_refs' 02/13/2025 02:17:11.431 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'haproxy-loadbalancer-template'], 'obj_dict': {'display_name': 'haproxy-loadbalancer-template', 'fq_name': ['default-domain', 'haproxy-loadbalancer-template'], 'id_perms': {'created': '2025-02-13T02:17:11.420992', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:11.420992', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12002035704871159465, 'uuid_mslong': 3216286324416791258}}, 'parent_type': 'domain', 'parent_uuid': '47137821-8208-4069-9c3b-21e4e16a0f01', 'perms2': {'global_access': 0, 'owner': 'e13ef443d9ab406792437e7ea9d9e4c9', 'owner_access': 7, 'share': [{'tenant': 'domain:47137821-8208-4069-9c3b-21e4e16a0f01', 'tenant_access': 5}]}, 'service_template_properties': {'availability_zone_enable': False, 'flavor': None, 'image_name': None, 'instance_data': None, 'interface_type': [{'service_interface_type': 'right', 'shared_ip': True, 'static_route_enable': False}, {'service_interface_type': 'left', 'shared_ip': True, 'static_route_enable': False}], 'ordered_interfaces': True, 'service_mode': 'in-network-nat', 'service_scaling': True, 'service_type': 'loadbalancer', 'service_virtualization_type': 'network-namespace', 'version': 1, 'vrouter_instance_type': None}, 'uuid': '2ca28b5f-16cf-42da-a68f-cbe2526686a9'}, 'oper': 'CREATE', 'request-id': 'req-dbc82179-f1e8-4a6f-af4b-8bbd1c9a06d4', 'type': 'service_template', 'uuid': '2ca28b5f-16cf-42da-a68f-cbe2526686a9'} 02/13/2025 02:17:11.464 7f0bbe547c18 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'] not found in DB: Unknown id: access_control_list default-domain:default-project:ip-fabric:ip-fabric 02/13/2025 02:17:11.469 7f0bbe54da20 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = access_control_list identifier_name = default-domain:default-project:ip-fabric:ip-fabric url = https://10.0.0.17:8082/access-control-lists operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.17 body = {"access-control-list": {"uuid": null, "fq_name": ["default-domain", "default-project", "ip-fabric", "ip-fabric"], "parent_type": "virtual-network", "access_control_list_entries": {"dynamic": false, "acl_rule": [{"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "default-domain:default-project:ip-fabric", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": "default-domain:default-project:ip-fabric", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": null}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "00000000-0000-0000-0000-100000000001", "direction": null}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "default-domain:default-project:ip-fabric", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": "any", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": null}, "action_list": {"simple_action": "deny", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "00000000-0000-0000-0000-100000000002", "direction": "<>"}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "any", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": "any", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": null}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "00000000-0000-0000-0000-100000000001", "direction": null}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "ip-fabric"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:11.491 7f0bbd9c38d0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'docker-template'] not found in DB: Unknown id: service_template default-domain:docker-template 02/13/2025 02:17:11.493 7f0bbd90d828 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = service_template identifier_name = default-domain:docker-template url = https://10.0.0.17:8082/service-templates operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/usr/local/bin/contrail-svc-monitor remote_ip = 10.0.0.17 body = {"service-template": {"uuid": null, "fq_name": ["default-domain", "docker-template"], "parent_type": "domain", "service_template_properties": {"version": 1, "service_mode": "transparent", "service_type": "firewall", "image_name": "ubuntu", "service_scaling": false, "interface_type": [{"service_interface_type": "management", "shared_ip": false, "static_route_enable": false}, {"service_interface_type": "left", "shared_ip": false, "static_route_enable": false}, {"service_interface_type": "right", "shared_ip": false, "static_route_enable": false}], "flavor": null, "ordered_interfaces": true, "service_virtualization_type": "vrouter-instance", "availability_zone_enable": false, "vrouter_instance_type": "docker", "instance_data": "{\"command\":\"/bin/bash\"}"}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "e13ef443d9ab406792437e7ea9d9e4c9", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "docker-template"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:11.540 7f0bbd919828 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = d48c3f74-6d3e-4161-9c67-f163e4a29b37 object_type = global_vrouter_config identifier_name = default-global-system-config:default-global-vrouter-config url = https://10.0.0.17:8082/global-vrouter-config/d48c3f74-6d3e-4161-9c67-f163e4a29b37 operation = http_put useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_linklocal.py remote_ip = 10.0.0.17 body = {"global-vrouter-config": {"uuid": "d48c3f74-6d3e-4161-9c67-f163e4a29b37", "fq_name": ["default-global-system-config", "default-global-vrouter-config"], "parent_type": "global-system-config", "linklocal_services": {"linklocal_service_entry": [{"linklocal_service_name": "metadata", "linklocal_service_ip": "169.254.169.254", "linklocal_service_port": 80, "ip_fabric_DNS_service_name": "", "ip_fabric_service_port": 8775, "ip_fabric_service_ip": ["127.0.0.1"]}]}}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:11.552 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'obj_dict': {'access_control_list_entries': {'acl_rule': [{'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'default-domain:default-project:ip-fabric'}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': None, 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'default-domain:default-project:ip-fabric'}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '00000000-0000-0000-0000-100000000001'}, {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'deny'}, 'direction': '<>', 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'any'}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': None, 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'default-domain:default-project:ip-fabric'}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '00000000-0000-0000-0000-100000000002'}, {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'any'}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': None, 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'any'}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '00000000-0000-0000-0000-100000000001'}], 'dynamic': False}, 'display_name': 'ip-fabric', 'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'id_perms': {'created': '2025-02-13T02:17:11.490175', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:11.490175', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13350108471765147960, 'uuid_mslong': 12808626817308773375}}, 'parent_type': 'virtual-network', 'parent_uuid': '2d6b6d1c-978e-4fe6-9af3-aa814e83dd99', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'b1c1623a-323b-4bff-b945-1cec547ea138'}, 'oper': 'CREATE', 'request-id': 'req-71f17a82-b929-4bf7-9d22-1174e68cbfb6', 'type': 'access_control_list', 'uuid': 'b1c1623a-323b-4bff-b945-1cec547ea138'} 02/13/2025 02:17:11.565 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'docker-template'], 'obj_dict': {'display_name': 'docker-template', 'fq_name': ['default-domain', 'docker-template'], 'id_perms': {'created': '2025-02-13T02:17:11.545131', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:17:11.545131', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10117267765424162005, 'uuid_mslong': 11436160503068246944}}, 'parent_type': 'domain', 'parent_uuid': '47137821-8208-4069-9c3b-21e4e16a0f01', 'perms2': {'global_access': 0, 'owner': 'e13ef443d9ab406792437e7ea9d9e4c9', 'owner_access': 7, 'share': [{'tenant': 'domain:47137821-8208-4069-9c3b-21e4e16a0f01', 'tenant_access': 5}]}, 'service_template_properties': {'availability_zone_enable': False, 'flavor': None, 'image_name': 'ubuntu', 'instance_data': '{"command":"/bin/bash"}', 'interface_type': [{'service_interface_type': 'management', 'shared_ip': False, 'static_route_enable': False}, {'service_interface_type': 'left', 'shared_ip': False, 'static_route_enable': False}, {'service_interface_type': 'right', 'shared_ip': False, 'static_route_enable': False}], 'ordered_interfaces': True, 'service_mode': 'transparent', 'service_scaling': False, 'service_type': 'firewall', 'service_virtualization_type': 'vrouter-instance', 'version': 1, 'vrouter_instance_type': 'docker'}, 'uuid': '9eb56762-dbbc-43a0-8c67-c16dd79e04d5'}, 'oper': 'CREATE', 'request-id': 'req-b4335d0f-409b-4d73-8c88-17dcfc56e3cb', 'type': 'service_template', 'uuid': '9eb56762-dbbc-43a0-8c67-c16dd79e04d5'} 02/13/2025 02:17:11.566 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'default-global-vrouter-config'], 'oper': 'UPDATE', 'request-id': 'req-2adfa053-ebe4-414e-a0f2-b34939af0d25', 'type': 'global_vrouter_config', 'uuid': 'd48c3f74-6d3e-4161-9c67-f163e4a29b37'} 02/13/2025 02:17:11.624 7f0bbd957828 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 05cf82fb-3dbe-495d-b8d4-3c4d57adc363 object_type = access_control_list identifier_name = default-domain:default-project:__no_rule__:ingress-access-control-list url = https://10.0.0.17:8082/access-control-list/05cf82fb-3dbe-495d-b8d4-3c4d57adc363 operation = http_put useragent = cn-jenkins-deploy-platform-juju-os-473-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.17 body = {"access-control-list": {"uuid": "05cf82fb-3dbe-495d-b8d4-3c4d57adc363", "access_control_list_entries": {"dynamic": null, "acl_rule": []}, "access_control_list_hash": -782339169199063927}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:11.657 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', '__no_rule__', 'ingress-access-control-list'], 'oper': 'UPDATE', 'request-id': 'req-59edb2a1-8d61-4e75-8b57-e77ea0837778', 'type': 'access_control_list', 'uuid': '05cf82fb-3dbe-495d-b8d4-3c4d57adc363'} 02/13/2025 02:17:11.712 7f0bbd8f08d0 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 449d7244-27a5-4a62-8bf4-2900da27689c object_type = access_control_list identifier_name = default-domain:default-project:__no_rule__:egress-access-control-list url = https://10.0.0.17:8082/access-control-list/449d7244-27a5-4a62-8bf4-2900da27689c operation = http_put useragent = cn-jenkins-deploy-platform-juju-os-473-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.17 body = {"access-control-list": {"uuid": "449d7244-27a5-4a62-8bf4-2900da27689c", "access_control_list_entries": {"dynamic": null, "acl_rule": []}, "access_control_list_hash": -782339169199063927}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:11.731 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', '__no_rule__', 'egress-access-control-list'], 'oper': 'UPDATE', 'request-id': 'req-e759bd2f-9632-47c8-a251-7b668e3db008', 'type': 'access_control_list', 'uuid': '449d7244-27a5-4a62-8bf4-2900da27689c'} 02/13/2025 02:17:11.850 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:12.622 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-conf-elements-inconsistency'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-conf-elements-inconsistency 02/13/2025 02:17:12.628 7f0bbd9c2a20 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-conf-elements-inconsistency url = https://10.0.0.17:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.17 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-conf-elements-inconsistency"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["analytics-node", "analytics-alarm-node", "analytics-snmp-node", "config-node", "control-node", "database-node", "config-database-node", "vrouter"]}, "alarm_severity": 1, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "ContrailConfig.elements", "operation": "size==", "operand2": {"json_value": "1"}}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "ContrailConfig has a list of elements - it should be a dict. Controllers have inconsistency in their data.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-conf-elements-inconsistency"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:12.651 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-conf-elements-inconsistency'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'ContrailConfig.elements', 'operand2': {'json_value': '1'}, 'operation': 'size=='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-conf-elements-inconsistency', 'fq_name': ['default-global-system-config', 'system-defined-conf-elements-inconsistency'], 'id_perms': {'created': '2025-02-13T02:17:12.641057', 'creator': None, 'description': 'ContrailConfig has a list of ' 'elements - it should be a dict. ' 'Controllers have inconsistency in ' 'their data.', 'enable': True, 'last_modified': '2025-02-13T02:17:12.641057', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13432294173492414738, 'uuid_mslong': 3974898253768704336}}, 'parent_type': 'global-system-config', 'parent_uuid': '3a60be2c-e787-4ec9-894b-c0c6d7944765', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '3729ace6-e234-4150-ba69-186229908912', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-473ff2c3-53b4-4f3e-993e-b65281730ce1', 'type': 'alarm', 'uuid': '3729ace6-e234-4150-ba69-186229908912'} 02/13/2025 02:17:12.657 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-address-mismatch-compute'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-address-mismatch-compute 02/13/2025 02:17:12.664 7f0bbd9e5048 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-address-mismatch-compute url = https://10.0.0.17:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.17 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-address-mismatch-compute"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["vrouter"]}, "alarm_severity": 1, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "ContrailConfig.elements.virtual_router_ip_address", "operation": "!=", "operand2": {"uve_attribute": "VrouterAgent.control_ip"}}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Compute Node IP Address mismatch.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-address-mismatch-compute"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:12.700 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-address-mismatch-compute'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'ContrailConfig.elements.virtual_router_ip_address', 'operand2': {'uve_attribute': 'VrouterAgent.control_ip'}, 'operation': '!='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-address-mismatch-compute', 'fq_name': ['default-global-system-config', 'system-defined-address-mismatch-compute'], 'id_perms': {'created': '2025-02-13T02:17:12.688887', 'creator': None, 'description': 'Compute Node IP Address mismatch.', 'enable': True, 'last_modified': '2025-02-13T02:17:12.688887', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12258673767952049066, 'uuid_mslong': 18277364393149548240}}, 'parent_type': 'global-system-config', 'parent_uuid': '3a60be2c-e787-4ec9-894b-c0c6d7944765', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'fda63e13-f40d-46d0-aa1f-8ed7bc2e63aa', 'uve_keys': {'uve_key': ['vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-cebfe62c-e650-43db-9de6-8a6c444a516b', 'type': 'alarm', 'uuid': 'fda63e13-f40d-46d0-aa1f-8ed7bc2e63aa'} 02/13/2025 02:17:12.748 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-address-mismatch-control'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-address-mismatch-control 02/13/2025 02:17:12.758 7f0bc30b72e8 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-address-mismatch-control url = https://10.0.0.17:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.17 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-address-mismatch-control"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["control-node"]}, "alarm_severity": 1, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "ContrailConfig.elements.bgp_router_parameters.address", "operation": "not in", "operand2": {"uve_attribute": "BgpRouterState.bgp_router_ip_list"}}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Control Node IP Address mismatch.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-address-mismatch-control"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:12.800 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-address-mismatch-control'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'ContrailConfig.elements.bgp_router_parameters.address', 'operand2': {'uve_attribute': 'BgpRouterState.bgp_router_ip_list'}, 'operation': 'not ' 'in'}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-address-mismatch-control', 'fq_name': ['default-global-system-config', 'system-defined-address-mismatch-control'], 'id_perms': {'created': '2025-02-13T02:17:12.776417', 'creator': None, 'description': 'Control Node IP Address mismatch.', 'enable': True, 'last_modified': '2025-02-13T02:17:12.776417', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13512348603409696639, 'uuid_mslong': 8836410424997659095}}, 'parent_type': 'global-system-config', 'parent_uuid': '3a60be2c-e787-4ec9-894b-c0c6d7944765', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '7aa13c76-d78b-41d7-bb85-81769b7b077f', 'uve_keys': {'uve_key': ['control-node']}}, 'oper': 'CREATE', 'request-id': 'req-6f49feec-af15-4f17-9c11-28349c156747', 'type': 'alarm', 'uuid': '7aa13c76-d78b-41d7-bb85-81769b7b077f'} 02/13/2025 02:17:12.834 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-bgp-connectivity'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-bgp-connectivity 02/13/2025 02:17:12.841 7f0bbd9dc048 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-bgp-connectivity url = https://10.0.0.17:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.17 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-bgp-connectivity"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["control-node"]}, "alarm_severity": 1, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "BgpRouterState.num_up_bgp_peer", "operation": "==", "operand2": {"json_value": "null"}}]}, {"and_list": [{"operand1": "BgpRouterState.num_up_bgp_peer", "operation": "!=", "operand2": {"uve_attribute": "BgpRouterState.num_bgp_peer"}}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "BGP peer mismatch. Not enough BGP peers are up.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-bgp-connectivity"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:12.851 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:12.872 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-bgp-connectivity'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'BgpRouterState.num_up_bgp_peer', 'operand2': {'json_value': 'null'}, 'operation': '=='}]}, {'and_list': [{'operand1': 'BgpRouterState.num_up_bgp_peer', 'operand2': {'uve_attribute': 'BgpRouterState.num_bgp_peer'}, 'operation': '!='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-bgp-connectivity', 'fq_name': ['default-global-system-config', 'system-defined-bgp-connectivity'], 'id_perms': {'created': '2025-02-13T02:17:12.855359', 'creator': None, 'description': 'BGP peer mismatch. Not enough BGP ' 'peers are up.', 'enable': True, 'last_modified': '2025-02-13T02:17:12.855359', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13698925487134144996, 'uuid_mslong': 15590156810511534065}}, 'parent_type': 'global-system-config', 'parent_uuid': '3a60be2c-e787-4ec9-894b-c0c6d7944765', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'd85b5d04-db2d-47f1-be1c-5c26892cf9e4', 'uve_keys': {'uve_key': ['control-node']}}, 'oper': 'CREATE', 'request-id': 'req-2756c088-8197-478a-8478-943bcc2d188e', 'type': 'alarm', 'uuid': 'd85b5d04-db2d-47f1-be1c-5c26892cf9e4'} 02/13/2025 02:17:12.918 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-conf-incorrect'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-conf-incorrect 02/13/2025 02:17:12.925 7f0bbd9be588 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-conf-incorrect url = https://10.0.0.17:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.17 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-conf-incorrect"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["analytics-node", "analytics-alarm-node", "analytics-snmp-node", "config-node", "control-node", "database-node", "config-database-node", "vrouter"]}, "alarm_severity": 1, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "ContrailConfig", "operation": "==", "operand2": {"json_value": "null"}}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "ContrailConfig missing or incorrect. Configuration pushed to Ifmap as ContrailConfig is missing/incorrect.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-conf-incorrect"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:12.955 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-conf-incorrect'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'ContrailConfig', 'operand2': {'json_value': 'null'}, 'operation': '=='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-conf-incorrect', 'fq_name': ['default-global-system-config', 'system-defined-conf-incorrect'], 'id_perms': {'created': '2025-02-13T02:17:12.941370', 'creator': None, 'description': 'ContrailConfig missing or ' 'incorrect. Configuration pushed to ' 'Ifmap as ContrailConfig is ' 'missing/incorrect.', 'enable': True, 'last_modified': '2025-02-13T02:17:12.941370', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11646259208410657462, 'uuid_mslong': 53062919311544114}}, 'parent_type': 'global-system-config', 'parent_uuid': '3a60be2c-e787-4ec9-894b-c0c6d7944765', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '00bc8471-a84f-4f32-a19f-d30ba77072b6', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-bf471259-3099-4008-a4f8-6fc1c75a3c82', 'type': 'alarm', 'uuid': '00bc8471-a84f-4f32-a19f-d30ba77072b6'} 02/13/2025 02:17:13.001 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-disk-usage-high'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-disk-usage-high 02/13/2025 02:17:13.008 7f0bbde3b4e0 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-disk-usage-high url = https://10.0.0.17:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.17 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-disk-usage-high"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["analytics-node", "analytics-alarm-node", "analytics-snmp-node", "config-node", "control-node", "database-node", "config-database-node", "vrouter"]}, "alarm_severity": 1, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "NodeStatus.disk_usage_info.*.percentage_partition_space_used", "operation": "range", "operand2": {"json_value": "[70, 90]"}, "variables": ["NodeStatus.disk_usage_info.__key"]}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Disk usage crosses high threshold limit.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-disk-usage-high"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:13.065 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-disk-usage-high'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'NodeStatus.disk_usage_info.*.percentage_partition_space_used', 'operand2': {'json_value': '[70, ' '90]'}, 'operation': 'range', 'variables': ['NodeStatus.disk_usage_info.__key']}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-disk-usage-high', 'fq_name': ['default-global-system-config', 'system-defined-disk-usage-high'], 'id_perms': {'created': '2025-02-13T02:17:13.028207', 'creator': None, 'description': 'Disk usage crosses high threshold ' 'limit.', 'enable': True, 'last_modified': '2025-02-13T02:17:13.028207', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12426579982713904521, 'uuid_mslong': 7345362244096970503}}, 'parent_type': 'global-system-config', 'parent_uuid': '3a60be2c-e787-4ec9-894b-c0c6d7944765', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '65eff80b-29f7-4f07-ac74-14a7caa45d89', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-dfca0089-6bd0-43e0-98ec-594f0352e4e0', 'type': 'alarm', 'uuid': '65eff80b-29f7-4f07-ac74-14a7caa45d89'} 02/13/2025 02:17:13.118 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-disk-usage-critical'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-disk-usage-critical 02/13/2025 02:17:13.125 7f0bbddb0f60 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-disk-usage-critical url = https://10.0.0.17:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.17 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-disk-usage-critical"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["analytics-node", "analytics-alarm-node", "analytics-snmp-node", "config-node", "control-node", "database-node", "config-database-node", "vrouter"]}, "alarm_severity": 0, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "NodeStatus.disk_usage_info.*.percentage_partition_space_used", "operation": ">", "operand2": {"json_value": "90"}, "variables": ["NodeStatus.disk_usage_info.__key"]}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Disk usage crosses critical threshold limit.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-disk-usage-critical"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:13.166 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-disk-usage-critical'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'NodeStatus.disk_usage_info.*.percentage_partition_space_used', 'operand2': {'json_value': '90'}, 'operation': '>', 'variables': ['NodeStatus.disk_usage_info.__key']}]}]}, 'alarm_severity': 0, 'display_name': 'system-defined-disk-usage-critical', 'fq_name': ['default-global-system-config', 'system-defined-disk-usage-critical'], 'id_perms': {'created': '2025-02-13T02:17:13.145285', 'creator': None, 'description': 'Disk usage crosses critical ' 'threshold limit.', 'enable': True, 'last_modified': '2025-02-13T02:17:13.145285', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13510054994606794717, 'uuid_mslong': 8746423696792699480}}, 'parent_type': 'global-system-config', 'parent_uuid': '3a60be2c-e787-4ec9-894b-c0c6d7944765', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '79618a02-fe63-4258-bb7d-5b703187a7dd', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-81fb9d90-8e95-4a4c-846e-ad078546784a', 'type': 'alarm', 'uuid': '79618a02-fe63-4258-bb7d-5b703187a7dd'} 02/13/2025 02:17:13.210 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-node-status'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-node-status 02/13/2025 02:17:13.224 7f0bbe5ad630 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-node-status url = https://10.0.0.17:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.17 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-node-status"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["analytics-node", "analytics-alarm-node", "analytics-snmp-node", "config-node", "control-node", "database-node", "config-database-node", "vrouter"]}, "alarm_severity": 0, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "NodeStatus", "operation": "==", "operand2": {"json_value": "null"}}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Node Failure. NodeStatus UVE not present.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-node-status"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:13.259 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-node-status'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'NodeStatus', 'operand2': {'json_value': 'null'}, 'operation': '=='}]}]}, 'alarm_severity': 0, 'display_name': 'system-defined-node-status', 'fq_name': ['default-global-system-config', 'system-defined-node-status'], 'id_perms': {'created': '2025-02-13T02:17:13.249584', 'creator': None, 'description': 'Node Failure. NodeStatus UVE not ' 'present.', 'enable': True, 'last_modified': '2025-02-13T02:17:13.249584', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9993022728145845976, 'uuid_mslong': 12958204765217246270}}, 'parent_type': 'global-system-config', 'parent_uuid': '3a60be2c-e787-4ec9-894b-c0c6d7944765', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'b3d4ca94-158c-4c3e-8aae-5939d789bed8', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-a9e8f34e-8150-4c0d-aee9-73879b289226', 'type': 'alarm', 'uuid': 'b3d4ca94-158c-4c3e-8aae-5939d789bed8'} 02/13/2025 02:17:13.305 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-partial-sysinfo'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-partial-sysinfo 02/13/2025 02:17:13.311 7f0bbe5b7240 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-partial-sysinfo url = https://10.0.0.17:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.17 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-partial-sysinfo"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["analytics-node", "analytics-alarm-node", "analytics-snmp-node", "config-node", "control-node", "database-node", "config-database-node", "vrouter"]}, "alarm_severity": 1, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "NodeStatus.build_info", "operation": "==", "operand2": {"json_value": "null"}}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "System Info Incomplete.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-partial-sysinfo"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:13.351 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-partial-sysinfo'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'NodeStatus.build_info', 'operand2': {'json_value': 'null'}, 'operation': '=='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-partial-sysinfo', 'fq_name': ['default-global-system-config', 'system-defined-partial-sysinfo'], 'id_perms': {'created': '2025-02-13T02:17:13.323417', 'creator': None, 'description': 'System Info Incomplete.', 'enable': True, 'last_modified': '2025-02-13T02:17:13.323417', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12290050023661532503, 'uuid_mslong': 8660100649126936650}}, 'parent_type': 'global-system-config', 'parent_uuid': '3a60be2c-e787-4ec9-894b-c0c6d7944765', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '782edba8-3e57-404a-aa8f-07618c207557', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-e23eba24-53e8-4107-b034-3fd9c224dbd7', 'type': 'alarm', 'uuid': '782edba8-3e57-404a-aa8f-07618c207557'} 02/13/2025 02:17:13.389 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-process-connectivity'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-process-connectivity 02/13/2025 02:17:13.396 7f0bbac35828 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-process-connectivity url = https://10.0.0.17:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.17 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-process-connectivity"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["analytics-node", "analytics-alarm-node", "analytics-snmp-node", "config-node", "control-node", "database-node", "config-database-node", "vrouter"]}, "alarm_severity": 0, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "NodeStatus.process_status", "operation": "==", "operand2": {"json_value": "null"}}]}, {"and_list": [{"operand1": "NodeStatus.process_status.state", "operation": "!=", "operand2": {"json_value": "\"Functional\""}, "variables": ["NodeStatus.process_status.module_id", "NodeStatus.process_status.instance_id"]}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Process(es) reporting as non-functional.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-process-connectivity"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:13.422 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-process-connectivity'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'NodeStatus.process_status', 'operand2': {'json_value': 'null'}, 'operation': '=='}]}, {'and_list': [{'operand1': 'NodeStatus.process_status.state', 'operand2': {'json_value': '"Functional"'}, 'operation': '!=', 'variables': ['NodeStatus.process_status.module_id', 'NodeStatus.process_status.instance_id']}]}]}, 'alarm_severity': 0, 'display_name': 'system-defined-process-connectivity', 'fq_name': ['default-global-system-config', 'system-defined-process-connectivity'], 'id_perms': {'created': '2025-02-13T02:17:13.411854', 'creator': None, 'description': 'Process(es) reporting as ' 'non-functional.', 'enable': True, 'last_modified': '2025-02-13T02:17:13.411854', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13430856302376078302, 'uuid_mslong': 9802270655640387995}}, 'parent_type': 'global-system-config', 'parent_uuid': '3a60be2c-e787-4ec9-894b-c0c6d7944765', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '8808a93a-05ae-419b-ba63-fca5b4714bde', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-a3eef1ce-bc6d-49b7-88f4-9b3681384353', 'type': 'alarm', 'uuid': '8808a93a-05ae-419b-ba63-fca5b4714bde'} 02/13/2025 02:17:13.470 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-process-status'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-process-status 02/13/2025 02:17:13.476 7f0bbddc2d68 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-process-status url = https://10.0.0.17:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.17 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-process-status"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["analytics-node", "analytics-alarm-node", "analytics-snmp-node", "config-node", "control-node", "database-node", "config-database-node", "vrouter"]}, "alarm_severity": 0, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "NodeStatus.process_info", "operation": "==", "operand2": {"json_value": "null"}}]}, {"and_list": [{"operand1": "NodeStatus.process_info.process_state", "operation": "!=", "operand2": {"json_value": "\"PROCESS_STATE_RUNNING\""}, "variables": ["NodeStatus.process_info.process_name"]}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Process Failure.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-process-status"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:13.499 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-process-status'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'NodeStatus.process_info', 'operand2': {'json_value': 'null'}, 'operation': '=='}]}, {'and_list': [{'operand1': 'NodeStatus.process_info.process_state', 'operand2': {'json_value': '"PROCESS_STATE_RUNNING"'}, 'operation': '!=', 'variables': ['NodeStatus.process_info.process_name']}]}]}, 'alarm_severity': 0, 'display_name': 'system-defined-process-status', 'fq_name': ['default-global-system-config', 'system-defined-process-status'], 'id_perms': {'created': '2025-02-13T02:17:13.489799', 'creator': None, 'description': 'Process Failure.', 'enable': True, 'last_modified': '2025-02-13T02:17:13.489799', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9469246701406104860, 'uuid_mslong': 9111074494591683405}}, 'parent_type': 'global-system-config', 'parent_uuid': '3a60be2c-e787-4ec9-894b-c0c6d7944765', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '7e7109fa-f442-4b4d-8369-85b467d9ad1c', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-b4916dc2-dcae-4c55-8d72-dc09ca542397', 'type': 'alarm', 'uuid': '7e7109fa-f442-4b4d-8369-85b467d9ad1c'} 02/13/2025 02:17:13.546 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-prouter-connectivity'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-prouter-connectivity 02/13/2025 02:17:13.554 7f0bbda058d0 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-prouter-connectivity url = https://10.0.0.17:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.17 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-prouter-connectivity"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["prouter"]}, "alarm_severity": 1, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "ContrailConfig.elements.virtual_router_refs", "operation": "!=", "operand2": {"json_value": "null"}}, {"operand1": "ProuterData.connected_agent_list", "operation": "size!=", "operand2": {"json_value": "1"}}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Prouter connectivity to controlling tor agent does not exist we look for non-empty value for connected_agent_list", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-prouter-connectivity"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:13.587 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-prouter-connectivity'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'ContrailConfig.elements.virtual_router_refs', 'operand2': {'json_value': 'null'}, 'operation': '!='}, {'operand1': 'ProuterData.connected_agent_list', 'operand2': {'json_value': '1'}, 'operation': 'size!='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-prouter-connectivity', 'fq_name': ['default-global-system-config', 'system-defined-prouter-connectivity'], 'id_perms': {'created': '2025-02-13T02:17:13.575880', 'creator': None, 'description': 'Prouter connectivity to controlling ' 'tor agent does not exist we look ' 'for non-empty value for ' 'connected_agent_list', 'enable': True, 'last_modified': '2025-02-13T02:17:13.575880', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12277215686703435698, 'uuid_mslong': 14111296427165895108}}, 'parent_type': 'global-system-config', 'parent_uuid': '3a60be2c-e787-4ec9-894b-c0c6d7944765', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'c3d56555-ed76-49c4-aa61-6e9e9ccfcfb2', 'uve_keys': {'uve_key': ['prouter']}}, 'oper': 'CREATE', 'request-id': 'req-bffd4f4c-7ab5-41f2-9f8a-1839336ef8b2', 'type': 'alarm', 'uuid': 'c3d56555-ed76-49c4-aa61-6e9e9ccfcfb2'} 02/13/2025 02:17:13.635 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-prouter-tsn-connectivity'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-prouter-tsn-connectivity 02/13/2025 02:17:13.641 7f0bbe5ad630 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-prouter-tsn-connectivity url = https://10.0.0.17:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.17 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-prouter-tsn-connectivity"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["prouter"]}, "alarm_severity": 1, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "ContrailConfig.elements.virtual_router_refs", "operation": "!=", "operand2": {"json_value": "null"}}, {"operand1": "ProuterData.tsn_agent_list", "operation": "size!=", "operand2": {"json_value": "1"}}, {"operand1": "ProuterData.gateway_mode", "operation": "!=", "operand2": {"json_value": "\"SERVER\""}}]}, {"and_list": [{"operand1": "ContrailConfig.elements.virtual_router_refs", "operation": "!=", "operand2": {"json_value": "null"}}, {"operand1": "ProuterData.tsn_agent_list", "operation": "size!=", "operand2": {"json_value": "0"}}, {"operand1": "ProuterData.gateway_mode", "operation": "==", "operand2": {"json_value": "\"SERVER\""}}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Prouter connectivity to controlling tsn agent does not exist we look for non-empty value for tsn_agent_list", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-prouter-tsn-connectivity"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:13.717 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-prouter-tsn-connectivity'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'ContrailConfig.elements.virtual_router_refs', 'operand2': {'json_value': 'null'}, 'operation': '!='}, {'operand1': 'ProuterData.tsn_agent_list', 'operand2': {'json_value': '1'}, 'operation': 'size!='}, {'operand1': 'ProuterData.gateway_mode', 'operand2': {'json_value': '"SERVER"'}, 'operation': '!='}]}, {'and_list': [{'operand1': 'ContrailConfig.elements.virtual_router_refs', 'operand2': {'json_value': 'null'}, 'operation': '!='}, {'operand1': 'ProuterData.tsn_agent_list', 'operand2': {'json_value': '0'}, 'operation': 'size!='}, {'operand1': 'ProuterData.gateway_mode', 'operand2': {'json_value': '"SERVER"'}, 'operation': '=='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-prouter-tsn-connectivity', 'fq_name': ['default-global-system-config', 'system-defined-prouter-tsn-connectivity'], 'id_perms': {'created': '2025-02-13T02:17:13.658615', 'creator': None, 'description': 'Prouter connectivity to controlling ' 'tsn agent does not exist we look ' 'for non-empty value for ' 'tsn_agent_list', 'enable': True, 'last_modified': '2025-02-13T02:17:13.658615', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13597892585072748868, 'uuid_mslong': 13822454464478858188}}, 'parent_type': 'global-system-config', 'parent_uuid': '3a60be2c-e787-4ec9-894b-c0c6d7944765', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'bfd33919-d746-4bcc-bcb5-6b42eb2de944', 'uve_keys': {'uve_key': ['prouter']}}, 'oper': 'CREATE', 'request-id': 'req-de8b2003-5fb0-4b55-85b2-cdca90ac05cd', 'type': 'alarm', 'uuid': 'bfd33919-d746-4bcc-bcb5-6b42eb2de944'} 02/13/2025 02:17:13.761 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-storage-cluster-state'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-storage-cluster-state 02/13/2025 02:17:13.768 7f0bbddb2d68 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-storage-cluster-state url = https://10.0.0.17:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.17 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-storage-cluster-state"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["storage-cluster"]}, "alarm_severity": 1, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "StorageCluster.info_stats.status", "operation": "!=", "operand2": {"json_value": "0"}, "variables": ["StorageCluster.info_stats.health_summary"]}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Storage Cluster warning/errors.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-storage-cluster-state"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:13.787 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-storage-cluster-state'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'StorageCluster.info_stats.status', 'operand2': {'json_value': '0'}, 'operation': '!=', 'variables': ['StorageCluster.info_stats.health_summary']}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-storage-cluster-state', 'fq_name': ['default-global-system-config', 'system-defined-storage-cluster-state'], 'id_perms': {'created': '2025-02-13T02:17:13.780093', 'creator': None, 'description': 'Storage Cluster warning/errors.', 'enable': True, 'last_modified': '2025-02-13T02:17:13.780093', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10608296685719007599, 'uuid_mslong': 11195172289335609109}}, 'parent_type': 'global-system-config', 'parent_uuid': '3a60be2c-e787-4ec9-894b-c0c6d7944765', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '9b5d3de1-f125-4715-9338-3da1683c316f', 'uve_keys': {'uve_key': ['storage-cluster']}}, 'oper': 'CREATE', 'request-id': 'req-463a96f9-bd98-496d-91bc-d6483f009790', 'type': 'alarm', 'uuid': '9b5d3de1-f125-4715-9338-3da1683c316f'} 02/13/2025 02:17:13.833 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-vrouter-interface'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-vrouter-interface 02/13/2025 02:17:13.839 7f0bbde69f60 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-vrouter-interface url = https://10.0.0.17:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.17 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-vrouter-interface"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["vrouter"]}, "alarm_severity": 1, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "VrouterAgent.down_interface_count", "operation": ">=", "operand2": {"json_value": "1"}, "variables": ["VrouterAgent.error_intf_list", "VrouterAgent.no_config_intf_list"]}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Vrouter interface(s) down.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-vrouter-interface"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:13.853 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:13.872 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-vrouter-interface'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'VrouterAgent.down_interface_count', 'operand2': {'json_value': '1'}, 'operation': '>=', 'variables': ['VrouterAgent.error_intf_list', 'VrouterAgent.no_config_intf_list']}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-vrouter-interface', 'fq_name': ['default-global-system-config', 'system-defined-vrouter-interface'], 'id_perms': {'created': '2025-02-13T02:17:13.852074', 'creator': None, 'description': 'Vrouter interface(s) down.', 'enable': True, 'last_modified': '2025-02-13T02:17:13.852074', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11890970496541633955, 'uuid_mslong': 12369562166647080755}}, 'parent_type': 'global-system-config', 'parent_uuid': '3a60be2c-e787-4ec9-894b-c0c6d7944765', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'aba98340-fa1c-4733-a505-36aa6103ada3', 'uve_keys': {'uve_key': ['vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-49a84a2d-b0d6-4e2d-9cc3-386819847f2e', 'type': 'alarm', 'uuid': 'aba98340-fa1c-4733-a505-36aa6103ada3'} 02/13/2025 02:17:13.907 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-xmpp-connectivity'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-xmpp-connectivity 02/13/2025 02:17:13.915 7f0bbda04828 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-xmpp-connectivity url = https://10.0.0.17:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.17 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-xmpp-connectivity"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["control-node"]}, "alarm_severity": 1, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "BgpRouterState.num_up_xmpp_peer", "operation": "==", "operand2": {"json_value": "null"}}]}, {"and_list": [{"operand1": "BgpRouterState.num_up_xmpp_peer", "operation": "!=", "operand2": {"uve_attribute": "BgpRouterState.num_xmpp_peer"}}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "XMPP peer mismatch.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-xmpp-connectivity"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:13.943 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-xmpp-connectivity'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'BgpRouterState.num_up_xmpp_peer', 'operand2': {'json_value': 'null'}, 'operation': '=='}]}, {'and_list': [{'operand1': 'BgpRouterState.num_up_xmpp_peer', 'operand2': {'uve_attribute': 'BgpRouterState.num_xmpp_peer'}, 'operation': '!='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-xmpp-connectivity', 'fq_name': ['default-global-system-config', 'system-defined-xmpp-connectivity'], 'id_perms': {'created': '2025-02-13T02:17:13.928090', 'creator': None, 'description': 'XMPP peer mismatch.', 'enable': True, 'last_modified': '2025-02-13T02:17:13.928090', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10952425501720124952, 'uuid_mslong': 16587066213226598332}}, 'parent_type': 'global-system-config', 'parent_uuid': '3a60be2c-e787-4ec9-894b-c0c6d7944765', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'e63118d6-c141-4fbc-97fe-d4f722f51e18', 'uve_keys': {'uve_key': ['control-node']}}, 'oper': 'CREATE', 'request-id': 'req-a64f3c3b-f059-44fd-a6ca-82ed21840004', 'type': 'alarm', 'uuid': 'e63118d6-c141-4fbc-97fe-d4f722f51e18'} 02/13/2025 02:17:13.981 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-xmpp-close-reason'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-xmpp-close-reason 02/13/2025 02:17:13.987 7f0bc0b8cc18 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-xmpp-close-reason url = https://10.0.0.17:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.17 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-xmpp-close-reason"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["xmpp-peer"]}, "alarm_severity": 1, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "XmppPeerInfoData.close_reason", "operation": "!=", "operand2": {"json_value": "null"}}, {"operand1": "XmppPeerInfoData.state_info.state", "operation": "!=", "operand2": {"json_value": "\"Established\""}}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "XMPP connection closed towards peer, alarm has close reason", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-xmpp-close-reason"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:14.016 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-xmpp-close-reason'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'XmppPeerInfoData.close_reason', 'operand2': {'json_value': 'null'}, 'operation': '!='}, {'operand1': 'XmppPeerInfoData.state_info.state', 'operand2': {'json_value': '"Established"'}, 'operation': '!='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-xmpp-close-reason', 'fq_name': ['default-global-system-config', 'system-defined-xmpp-close-reason'], 'id_perms': {'created': '2025-02-13T02:17:14.004666', 'creator': None, 'description': 'XMPP connection closed towards ' 'peer, ' 'alarm has close reason', 'enable': True, 'last_modified': '2025-02-13T02:17:14.004666', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10683275629020542923, 'uuid_mslong': 5181890247713574821}}, 'parent_type': 'global-system-config', 'parent_uuid': '3a60be2c-e787-4ec9-894b-c0c6d7944765', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '47e9c5b9-b807-4fa5-9442-9e95095b77cb', 'uve_keys': {'uve_key': ['xmpp-peer']}}, 'oper': 'CREATE', 'request-id': 'req-7d9d1717-d0e7-44d1-9104-fdc409508dc5', 'type': 'alarm', 'uuid': '47e9c5b9-b807-4fa5-9442-9e95095b77cb'} 02/13/2025 02:17:14.057 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-core-files'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-core-files 02/13/2025 02:17:14.063 7f0bbd9ab438 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-core-files url = https://10.0.0.17:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.17 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-core-files"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["analytics-node", "analytics-alarm-node", "analytics-snmp-node", "config-node", "control-node", "database-node", "config-database-node", "vrouter"]}, "alarm_severity": 0, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "NodeStatus.all_core_file_list", "operand2": {"json_value": "null"}, "operation": "!="}, {"operand1": "NodeStatus.all_core_file_list", "operand2": {"json_value": "0"}, "operation": "size!="}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "A core file has been generated on the node.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-core-files"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:14.091 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-core-files'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'NodeStatus.all_core_file_list', 'operand2': {'json_value': 'null'}, 'operation': '!='}, {'operand1': 'NodeStatus.all_core_file_list', 'operand2': {'json_value': '0'}, 'operation': 'size!='}]}]}, 'alarm_severity': 0, 'display_name': 'system-defined-core-files', 'fq_name': ['default-global-system-config', 'system-defined-core-files'], 'id_perms': {'created': '2025-02-13T02:17:14.077967', 'creator': None, 'description': 'A core file has been generated on ' 'the node.', 'enable': True, 'last_modified': '2025-02-13T02:17:14.077967', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12151377602753391480, 'uuid_mslong': 6018581576157580189}}, 'parent_type': 'global-system-config', 'parent_uuid': '3a60be2c-e787-4ec9-894b-c0c6d7944765', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '53864c0e-4295-4b9d-a8a2-5d8c87c0eb78', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-53b4a0df-8218-405a-9b78-4816214eed5b', 'type': 'alarm', 'uuid': '53864c0e-4295-4b9d-a8a2-5d8c87c0eb78'} 02/13/2025 02:17:14.137 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-pending-cassandra-compaction-tasks'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-pending-cassandra-compaction-tasks 02/13/2025 02:17:14.143 7f0bbd9beac8 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-pending-cassandra-compaction-tasks url = https://10.0.0.17:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.17 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-pending-cassandra-compaction-tasks"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["database-nodeconfig-database-node"]}, "alarm_severity": 1, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "CassandraStatusData.cassandra_compaction_task.pending_compaction_tasks", "operand2": {"json_value": "300"}, "operation": ">="}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Pending compaction tasks in cassandra crossed the configured threshold.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-pending-cassandra-compaction-tasks"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:14.164 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-pending-cassandra-compaction-tasks'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'CassandraStatusData.cassandra_compaction_task.pending_compaction_tasks', 'operand2': {'json_value': '300'}, 'operation': '>='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-pending-cassandra-compaction-tasks', 'fq_name': ['default-global-system-config', 'system-defined-pending-cassandra-compaction-tasks'], 'id_perms': {'created': '2025-02-13T02:17:14.156130', 'creator': None, 'description': 'Pending compaction tasks in ' 'cassandra crossed the configured ' 'threshold.', 'enable': True, 'last_modified': '2025-02-13T02:17:14.156130', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13570135860965291452, 'uuid_mslong': 7435421051241317541}}, 'parent_type': 'global-system-config', 'parent_uuid': '3a60be2c-e787-4ec9-894b-c0c6d7944765', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '672fec0d-32b3-44a5-bc52-ceaafaa035bc', 'uve_keys': {'uve_key': ['database-nodeconfig-database-node']}}, 'oper': 'CREATE', 'request-id': 'req-e3a0848e-e49d-4c7b-b48d-3e053c152145', 'type': 'alarm', 'uuid': '672fec0d-32b3-44a5-bc52-ceaafaa035bc'} 02/13/2025 02:17:14.209 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-package-version-mismatch'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-package-version-mismatch 02/13/2025 02:17:14.215 7f0bbd9ab2e8 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-package-version-mismatch url = https://10.0.0.17:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.17 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-package-version-mismatch"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["analytics-node", "analytics-alarm-node", "analytics-snmp-node", "config-node", "control-node", "database-node", "config-database-node", "vrouter"]}, "alarm_severity": 0, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "NodeStatus.running_package_version", "operation": "!=", "operand2": {"uve_attribute": "NodeStatus.installed_package_version"}}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "There is a mismatch between installed and running package version.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-package-version-mismatch"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:14.236 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-package-version-mismatch'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'NodeStatus.running_package_version', 'operand2': {'uve_attribute': 'NodeStatus.installed_package_version'}, 'operation': '!='}]}]}, 'alarm_severity': 0, 'display_name': 'system-defined-package-version-mismatch', 'fq_name': ['default-global-system-config', 'system-defined-package-version-mismatch'], 'id_perms': {'created': '2025-02-13T02:17:14.228701', 'creator': None, 'description': 'There is a mismatch between ' 'installed and running package ' 'version.', 'enable': True, 'last_modified': '2025-02-13T02:17:14.228701', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9266321713955698907, 'uuid_mslong': 4933988710328059010}}, 'parent_type': 'global-system-config', 'parent_uuid': '3a60be2c-e787-4ec9-894b-c0c6d7944765', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '44790c97-34e4-4482-8098-9680f6d178db', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-a01567af-70e6-49e9-8928-d5ab3da578e8', 'type': 'alarm', 'uuid': '44790c97-34e4-4482-8098-9680f6d178db'} 02/13/2025 02:17:14.282 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-vrouter-limit-exceeded'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-vrouter-limit-exceeded 02/13/2025 02:17:14.287 7f0bbde696d8 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-vrouter-limit-exceeded url = https://10.0.0.17:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.17 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-vrouter-limit-exceeded"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["vrouter"]}, "alarm_severity": 1, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "VrouterAgent.res_limit", "operation": "==", "operand2": {"json_value": "true"}, "variables": ["VrouterAgent.vr_limit_exceeded_list"]}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Agent resource usage exceeded configured watermark for resource.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-vrouter-limit-exceeded"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:14.311 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-vrouter-limit-exceeded'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'VrouterAgent.res_limit', 'operand2': {'json_value': 'true'}, 'operation': '==', 'variables': ['VrouterAgent.vr_limit_exceeded_list']}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-vrouter-limit-exceeded', 'fq_name': ['default-global-system-config', 'system-defined-vrouter-limit-exceeded'], 'id_perms': {'created': '2025-02-13T02:17:14.301571', 'creator': None, 'description': 'Agent resource usage exceeded ' 'configured watermark for resource.', 'enable': True, 'last_modified': '2025-02-13T02:17:14.301571', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10824398512693575283, 'uuid_mslong': 17873144619840260359}}, 'parent_type': 'global-system-config', 'parent_uuid': '3a60be2c-e787-4ec9-894b-c0c6d7944765', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'f80a2a55-f4c3-4907-9637-fd18e05dfe73', 'uve_keys': {'uve_key': ['vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-47ef2172-c30e-4cde-a4eb-4c18cda63392', 'type': 'alarm', 'uuid': 'f80a2a55-f4c3-4907-9637-fd18e05dfe73'} 02/13/2025 02:17:14.357 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-vrouter-table-limit-exceeded'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-vrouter-table-limit-exceeded 02/13/2025 02:17:14.364 7f0bbdff98d0 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-vrouter-table-limit-exceeded url = https://10.0.0.17:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.17 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-vrouter-table-limit-exceeded"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["vrouter"]}, "alarm_severity": 0, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "VrouterAgent.res_table_limit", "operation": "==", "operand2": {"json_value": "true"}, "variables": ["VrouterAgent.vr_limit_exceeded_list"]}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Agent resource usage exceeded table size for resource in vrouter.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-vrouter-table-limit-exceeded"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:14.391 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-vrouter-table-limit-exceeded'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'VrouterAgent.res_table_limit', 'operand2': {'json_value': 'true'}, 'operation': '==', 'variables': ['VrouterAgent.vr_limit_exceeded_list']}]}]}, 'alarm_severity': 0, 'display_name': 'system-defined-vrouter-table-limit-exceeded', 'fq_name': ['default-global-system-config', 'system-defined-vrouter-table-limit-exceeded'], 'id_perms': {'created': '2025-02-13T02:17:14.383190', 'creator': None, 'description': 'Agent resource usage exceeded table ' 'size for resource in vrouter.', 'enable': True, 'last_modified': '2025-02-13T02:17:14.383190', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12981143261438209537, 'uuid_mslong': 6177749889852984898}}, 'parent_type': 'global-system-config', 'parent_uuid': '3a60be2c-e787-4ec9-894b-c0c6d7944765', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '55bbc6cb-2fe5-4e42-b426-4904ec7aee01', 'uve_keys': {'uve_key': ['vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-c9c8dcea-882f-444f-99bc-c60d287ba3d3', 'type': 'alarm', 'uuid': '55bbc6cb-2fe5-4e42-b426-4904ec7aee01'} 02/13/2025 02:17:14.438 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-vrouter-l3mh-physical-interface-link-down'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-vrouter-l3mh-physical-interface-link-down 02/13/2025 02:17:14.452 7f0bbddc2240 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-vrouter-l3mh-physical-interface-link-down url = https://10.0.0.17:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.17 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-vrouter-l3mh-physical-interface-link-down"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["vrouter"]}, "alarm_severity": 0, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "VrouterAgent.l3mh_phy_interface_down", "operation": "==", "operand2": {"json_value": "true"}, "variables": ["VrouterAgent.vr_l3mh_intf_list"]}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "On l3 multihomed compute one or more physical interfaces are Inactive.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-vrouter-l3mh-physical-interface-link-down"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:14.485 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-vrouter-l3mh-physical-interface-link-down'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'VrouterAgent.l3mh_phy_interface_down', 'operand2': {'json_value': 'true'}, 'operation': '==', 'variables': ['VrouterAgent.vr_l3mh_intf_list']}]}]}, 'alarm_severity': 0, 'display_name': 'system-defined-vrouter-l3mh-physical-interface-link-down', 'fq_name': ['default-global-system-config', 'system-defined-vrouter-l3mh-physical-interface-link-down'], 'id_perms': {'created': '2025-02-13T02:17:14.469179', 'creator': None, 'description': 'On l3 multihomed compute one or ' 'more physical interfaces are ' 'Inactive.', 'enable': True, 'last_modified': '2025-02-13T02:17:14.469179', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10933236757492713019, 'uuid_mslong': 7849764968776614544}}, 'parent_type': 'global-system-config', 'parent_uuid': '3a60be2c-e787-4ec9-894b-c0c6d7944765', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '6ceff7a6-363c-4290-97ba-a8e77798c63b', 'uve_keys': {'uve_key': ['vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-fd1c4c61-aa50-4238-8ccf-c06b1dd99fc9', 'type': 'alarm', 'uuid': '6ceff7a6-363c-4290-97ba-a8e77798c63b'} 02/13/2025 02:17:14.854 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:15.407 7f0bbda04c18 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = d48c3f74-6d3e-4161-9c67-f163e4a29b37 object_type = global_vrouter_config identifier_name = default-global-system-config:default-global-vrouter-config url = https://10.0.0.17:8082/global-vrouter-config/d48c3f74-6d3e-4161-9c67-f163e4a29b37 operation = http_put useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_encap.py remote_ip = 10.0.0.17 body = {"global-vrouter-config": {"uuid": "d48c3f74-6d3e-4161-9c67-f163e4a29b37", "fq_name": ["default-global-system-config", "default-global-vrouter-config"], "parent_type": "global-system-config", "encapsulation_priorities": {"encapsulation": ["MPLSoUDP", "MPLSoGRE", "VXLAN"]}, "vxlan_network_identifier_mode": "automatic"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:15.417 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'default-global-vrouter-config'], 'oper': 'UPDATE', 'request-id': 'req-56c856fc-3b03-4958-b161-265e0107d6d4', 'type': 'global_vrouter_config', 'uuid': 'd48c3f74-6d3e-4161-9c67-f163e4a29b37'} 02/13/2025 02:17:15.857 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:16.309 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_objects in resource global-vrouter-config: 'security_logging_objects' 02/13/2025 02:17:16.309 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource global-vrouter-config: 'tag_refs' 02/13/2025 02:17:16.310 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource global-vrouter-config: 'application_policy_set_back_refs' 02/13/2025 02:17:16.411 7f0bbdddb2e8 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = d48c3f74-6d3e-4161-9c67-f163e4a29b37 object_type = global_vrouter_config identifier_name = default-global-system-config:default-global-vrouter-config url = https://10.0.0.17:8082/global-vrouter-config/d48c3f74-6d3e-4161-9c67-f163e4a29b37 operation = http_put useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_global_vrouter_config.py remote_ip = 10.0.0.17 body = {"global-vrouter-config": {"uuid": "d48c3f74-6d3e-4161-9c67-f163e4a29b37", "fq_name": ["default-global-system-config", "default-global-vrouter-config"], "parent_type": "global-system-config", "port_translation_pools": {"port_translation_pool": []}}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:16.584 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'default-global-vrouter-config'], 'oper': 'UPDATE', 'request-id': 'req-6ccf3438-bd35-48e5-a522-8db9f040522e', 'type': 'global_vrouter_config', 'uuid': 'd48c3f74-6d3e-4161-9c67-f163e4a29b37'} 02/13/2025 02:17:16.858 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:17.244 7f0bbe5bc048 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:17:17.294 7f0bbe5bc048 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:17:17.563 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource global-system-config: 'parent_type' 02/13/2025 02:17:17.563 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flags in resource global-system-config: 'feature_flags' 02/13/2025 02:17:17.563 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zones in resource global-system-config: 'control_node_zones' 02/13/2025 02:17:17.563 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_configs in resource global-system-config: 'global_vrouter_configs' 02/13/2025 02:17:17.564 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_configs in resource global-system-config: 'global_qos_configs' 02/13/2025 02:17:17.564 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_routers in resource global-system-config: 'virtual_routers' 02/13/2025 02:17:17.564 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_nodes in resource global-system-config: 'config_nodes' 02/13/2025 02:17:17.564 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_nodes in resource global-system-config: 'analytics_nodes' 02/13/2025 02:17:17.564 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_nodes in resource global-system-config: 'flow_nodes' 02/13/2025 02:17:17.564 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_nodes in resource global-system-config: 'devicemgr_nodes' 02/13/2025 02:17:17.564 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_nodes in resource global-system-config: 'database_nodes' 02/13/2025 02:17:17.564 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_nodes in resource global-system-config: 'webui_nodes' 02/13/2025 02:17:17.564 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_nodes in resource global-system-config: 'config_database_nodes' 02/13/2025 02:17:17.564 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_nodes in resource global-system-config: 'analytics_alarm_nodes' 02/13/2025 02:17:17.565 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_nodes in resource global-system-config: 'analytics_snmp_nodes' 02/13/2025 02:17:17.565 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_sets in resource global-system-config: 'service_appliance_sets' 02/13/2025 02:17:17.565 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource global-system-config: 'api_access_lists' 02/13/2025 02:17:17.565 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource global-system-config: 'alarms' 02/13/2025 02:17:17.565 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_propertiess in resource global-system-config: 'config_propertiess' 02/13/2025 02:17:17.565 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_templates in resource global-system-config: 'job_templates' 02/13/2025 02:17:17.565 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnects in resource global-system-config: 'data_center_interconnects' 02/13/2025 02:17:17.565 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_maps in resource global-system-config: 'intent_maps' 02/13/2025 02:17:17.565 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabrics in resource global-system-config: 'fabrics' 02/13/2025 02:17:17.565 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profiles in resource global-system-config: 'node_profiles' 02/13/2025 02:17:17.565 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_routers in resource global-system-config: 'physical_routers' 02/13/2025 02:17:17.566 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_images in resource global-system-config: 'device_images' 02/13/2025 02:17:17.566 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field nodes in resource global-system-config: 'nodes' 02/13/2025 02:17:17.566 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field features in resource global-system-config: 'features' 02/13/2025 02:17:17.566 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_roles in resource global-system-config: 'physical_roles' 02/13/2025 02:17:17.566 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_roles in resource global-system-config: 'overlay_roles' 02/13/2025 02:17:17.566 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definitions in resource global-system-config: 'role_definitions' 02/13/2025 02:17:17.566 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_configs in resource global-system-config: 'global_analytics_configs' 02/13/2025 02:17:17.566 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource global-system-config: 'tag_refs' 02/13/2025 02:17:17.566 7f0bbd919828 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource global-system-config: 'qos_config_back_refs' 02/13/2025 02:17:17.623 7f0bbd9d7d68 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 3a60be2c-e787-4ec9-894b-c0c6d7944765 object_type = global_system_config identifier_name = default-global-system-config url = https://10.0.0.17:8082/global-system-config/3a60be2c-e787-4ec9-894b-c0c6d7944765 operation = http_put useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_control.py remote_ip = 10.0.0.17 body = {"global-system-config": {"uuid": "3a60be2c-e787-4ec9-894b-c0c6d7944765", "autonomous_system": 64512, "ibgp_auto_mesh": true}} domain = default-domain project = admin user = admin >> 02/13/2025 02:17:17.648 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-9735f43d-2b2a-4ae0-b286-507ad8e3608c', 'type': 'global_system_config', 'uuid': '3a60be2c-e787-4ec9-894b-c0c6d7944765'} 02/13/2025 02:17:17.860 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:18.862 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:19.864 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:20.865 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:21.866 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:22.868 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:23.869 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:24.870 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:25.872 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:26.873 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:27.875 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:28.876 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:29.878 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:30.879 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:32.758 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:33.199 7f0bbde2dc18 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:17:33.252 7f0bbdd250f0 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:17:33.761 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:34.762 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:35.764 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:36.765 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:37.767 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:38.768 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:39.770 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:40.771 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:41.434 7f0bbe536b70 [contrail-api] [ERROR]: Session Event: TCP Connection Closed 02/13/2025 02:17:41.434 7f0bbdd250f0 [contrail-api] [INFO]: Processing event[EvTcpClose] in state[Established] 02/13/2025 02:17:41.773 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:42.774 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:43.775 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:44.776 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:45.778 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:46.437 7f0bbdd250f0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:17:46.437 7f0bbdd250f0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpClose] => State[Established] -> State[Connect] 02/13/2025 02:17:46.438 7f0bbdd250f0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:17:46.438 7f0bbdd250f0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Established to Connect on EvTcpClose >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Established to Connect on EvTcpClose] connection down >>, ] >> 02/13/2025 02:17:46.438 7f0bbdd250f0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:17:46.438 7f0bbdd250f0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:17:46.438 7f0bbdd250f0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:17:46.439 7f0bbdd250f0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:17:46.439 7f0bbdd250f0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:17:46.439 7f0bbdd250f0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:17:46.439 7f0bbdd250f0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:17:46.439 7f0bbdd250f0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 02/13/2025 02:17:46.439 7f0bbdd250f0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:17:46.801 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:47.802 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:48.593 7f0bbdd250f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource global-system-config: 'parent_type' 02/13/2025 02:17:48.593 7f0bbdd250f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flags in resource global-system-config: 'feature_flags' 02/13/2025 02:17:48.593 7f0bbdd250f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zones in resource global-system-config: 'control_node_zones' 02/13/2025 02:17:48.593 7f0bbdd250f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_configs in resource global-system-config: 'global_vrouter_configs' 02/13/2025 02:17:48.593 7f0bbdd250f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_configs in resource global-system-config: 'global_qos_configs' 02/13/2025 02:17:48.593 7f0bbdd250f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_routers in resource global-system-config: 'virtual_routers' 02/13/2025 02:17:48.593 7f0bbdd250f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_nodes in resource global-system-config: 'config_nodes' 02/13/2025 02:17:48.593 7f0bbdd250f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_nodes in resource global-system-config: 'analytics_nodes' 02/13/2025 02:17:48.594 7f0bbdd250f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_nodes in resource global-system-config: 'flow_nodes' 02/13/2025 02:17:48.594 7f0bbdd250f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_nodes in resource global-system-config: 'devicemgr_nodes' 02/13/2025 02:17:48.594 7f0bbdd250f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_nodes in resource global-system-config: 'database_nodes' 02/13/2025 02:17:48.594 7f0bbdd250f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_nodes in resource global-system-config: 'webui_nodes' 02/13/2025 02:17:48.594 7f0bbdd250f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_nodes in resource global-system-config: 'config_database_nodes' 02/13/2025 02:17:48.594 7f0bbdd250f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_nodes in resource global-system-config: 'analytics_alarm_nodes' 02/13/2025 02:17:48.594 7f0bbdd250f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_nodes in resource global-system-config: 'analytics_snmp_nodes' 02/13/2025 02:17:48.594 7f0bbdd250f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_sets in resource global-system-config: 'service_appliance_sets' 02/13/2025 02:17:48.594 7f0bbdd250f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource global-system-config: 'api_access_lists' 02/13/2025 02:17:48.594 7f0bbdd250f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource global-system-config: 'alarms' 02/13/2025 02:17:48.594 7f0bbdd250f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_propertiess in resource global-system-config: 'config_propertiess' 02/13/2025 02:17:48.595 7f0bbdd250f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_templates in resource global-system-config: 'job_templates' 02/13/2025 02:17:48.595 7f0bbdd250f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnects in resource global-system-config: 'data_center_interconnects' 02/13/2025 02:17:48.595 7f0bbdd250f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_maps in resource global-system-config: 'intent_maps' 02/13/2025 02:17:48.595 7f0bbdd250f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabrics in resource global-system-config: 'fabrics' 02/13/2025 02:17:48.595 7f0bbdd250f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profiles in resource global-system-config: 'node_profiles' 02/13/2025 02:17:48.595 7f0bbdd250f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_routers in resource global-system-config: 'physical_routers' 02/13/2025 02:17:48.595 7f0bbdd250f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_images in resource global-system-config: 'device_images' 02/13/2025 02:17:48.595 7f0bbdd250f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field nodes in resource global-system-config: 'nodes' 02/13/2025 02:17:48.596 7f0bbdd250f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field features in resource global-system-config: 'features' 02/13/2025 02:17:48.596 7f0bbdd250f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_roles in resource global-system-config: 'physical_roles' 02/13/2025 02:17:48.596 7f0bbdd250f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_roles in resource global-system-config: 'overlay_roles' 02/13/2025 02:17:48.596 7f0bbdd250f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definitions in resource global-system-config: 'role_definitions' 02/13/2025 02:17:48.596 7f0bbdd250f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_configs in resource global-system-config: 'global_analytics_configs' 02/13/2025 02:17:48.596 7f0bbdd250f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource global-system-config: 'tag_refs' 02/13/2025 02:17:48.596 7f0bbdd250f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource global-system-config: 'qos_config_back_refs' 02/13/2025 02:17:48.707 7f0bbdd250f0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource config-node: 'tag_refs' 02/13/2025 02:17:48.803 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:49.805 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:50.438 7f0bbda078d0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:17:50.807 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:51.808 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:52.810 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:53.811 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:54.813 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:55.441 7f0bbda078d0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:17:55.441 7f0bbda078d0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:17:55.441 7f0bbda078d0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:17:55.442 7f0bbda078d0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:17:55.442 7f0bbda078d0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:17:55.442 7f0bbda078d0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:17:55.442 7f0bbda078d0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:17:55.442 7f0bbda078d0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:17:55.442 7f0bbda078d0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:17:55.443 7f0bbda078d0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:17:55.443 7f0bbda078d0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:17:55.443 7f0bbda078d0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 02/13/2025 02:17:55.443 7f0bbda078d0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:17:55.815 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:56.816 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:57.817 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:58.820 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:17:59.448 7f0bbda078d0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:17:59.820 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:00.191 7f0bbddb0588 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_routers in resource routing-instance: 'bgp_routers' 02/13/2025 02:18:00.191 7f0bbddb0588 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_refs in resource routing-instance: 'routing_instance_refs' 02/13/2025 02:18:00.192 7f0bbddb0588 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_target_refs in resource routing-instance: 'route_target_refs' 02/13/2025 02:18:00.192 7f0bbddb0588 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource routing-instance: 'tag_refs' 02/13/2025 02:18:00.192 7f0bbddb0588 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource routing-instance: 'virtual_machine_interface_back_refs' 02/13/2025 02:18:00.192 7f0bbddb0588 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregate_back_refs in resource routing-instance: 'route_aggregate_back_refs' 02/13/2025 02:18:00.192 7f0bbddb0588 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_back_refs in resource routing-instance: 'routing_policy_back_refs' 02/13/2025 02:18:00.192 7f0bbddb0588 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_back_refs in resource routing-instance: 'routing_instance_back_refs' 02/13/2025 02:18:00.352 7f0bbddb0588 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zone_refs in resource bgp-router: 'control_node_zone_refs' 02/13/2025 02:18:00.352 7f0bbddb0588 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource bgp-router: 'sub_cluster_refs' 02/13/2025 02:18:00.352 7f0bbddb0588 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_refs in resource bgp-router: 'bgp_router_refs' 02/13/2025 02:18:00.353 7f0bbddb0588 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource bgp-router: 'tag_refs' 02/13/2025 02:18:00.353 7f0bbddb0588 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource bgp-router: 'physical_router_back_refs' 02/13/2025 02:18:00.353 7f0bbddb0588 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource bgp-router: 'virtual_machine_interface_back_refs' 02/13/2025 02:18:00.353 7f0bbddb0588 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource bgp-router: 'bgp_as_a_service_back_refs' 02/13/2025 02:18:00.354 7f0bbddb0588 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_back_refs in resource bgp-router: 'bgp_router_back_refs' 02/13/2025 02:18:00.413 7f0bbda07390 [contrail-api] [ERROR]: SANDESH: Not connected: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a22e8c6d-5dac-4501-884a-d6ff702b6d50 object_type = bgp_router identifier_name = default-domain:default-project:ip-fabric:__default__:cn-jenkins-deploy-platform-juju-os-473-1 url = https://10.0.0.17:8082/bgp-router/a22e8c6d-5dac-4501-884a-d6ff702b6d50 operation = http_put useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_control.py remote_ip = 10.0.0.17 body = {"bgp-router": {"uuid": "a22e8c6d-5dac-4501-884a-d6ff702b6d50", "bgp_router_parameters": {"admin_down": false, "vendor": "contrail", "cluster_id": null, "autonomous_system": 64512, "identifier": "10.0.0.17", "address": "10.0.0.17", "port": 179, "source_port": null, "hold_time": 0, "address_families": {"family": ["route-target", "inet-vpn", "e-vpn", "erm-vpn", "inet6-vpn"]}, "auth_data": null, "local_autonomous_system": null, "router_type": "control-node", "gateway_address": null, "ipv6_gateway_address": null}}} domain = default-domain project = admin user = admin >> 02/13/2025 02:18:00.414 7f0bbda07390 [contrail-api] [ERROR]: SANDESH: [DROP: SessionNotConnected] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a22e8c6d-5dac-4501-884a-d6ff702b6d50 object_type = bgp_router identifier_name = default-domain:default-project:ip-fabric:__default__:cn-jenkins-deploy-platform-juju-os-473-1 url = https://10.0.0.17:8082/bgp-router/a22e8c6d-5dac-4501-884a-d6ff702b6d50 operation = http_put useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_control.py remote_ip = 10.0.0.17 body = {"bgp-router": {"uuid": "a22e8c6d-5dac-4501-884a-d6ff702b6d50", "bgp_router_parameters": {"admin_down": false, "vendor": "contrail", "cluster_id": null, "autonomous_system": 64512, "identifier": "10.0.0.17", "address": "10.0.0.17", "port": 179, "source_port": null, "hold_time": 0, "address_families": {"family": ["route-target", "inet-vpn", "e-vpn", "erm-vpn", "inet6-vpn"]}, "auth_data": null, "local_autonomous_system": null, "router_type": "control-node", "gateway_address": null, "ipv6_gateway_address": null}}} domain = default-domain project = admin user = admin >> 02/13/2025 02:18:00.455 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-juju-os-473-1'], 'oper': 'UPDATE', 'request-id': 'req-4d9283d4-0bbe-4906-8efc-cc39b5012fba', 'type': 'bgp_router', 'uuid': 'a22e8c6d-5dac-4501-884a-d6ff702b6d50'} 02/13/2025 02:18:00.822 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:01.806 7f0bbddb0588 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:18:01.864 7f0bbd907198 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:18:02.758 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:03.760 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:04.453 7f0bbda078d0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:18:04.453 7f0bbda078d0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:18:04.453 7f0bbda078d0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:18:04.454 7f0bbda078d0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:18:04.454 7f0bbda078d0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:18:04.454 7f0bbda078d0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:18:04.454 7f0bbda078d0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 elements = { bgp_router_parameters : {"admin_down": false, "vendor": "contrail", "cluster_id": null, "autonomous_system": 64512, "identifier": "10.0.0.17", "address": "10.0.0.17", "port": 179, "source_port": null, "hold_time": 0, "address_families": {"family": ["route-target", "inet-vpn", "e-vpn", "erm-vpn", "inet6-vpn"]}, "auth_data": null, "local_autonomous_system": null, "router_type": "control-node", "gateway_address": null, "ipv6_gateway_address": null}display_name : "cn-jenkins-deploy-platform-juju-os-473-1"fq_name : ["default-domain", "default-project", "ip-fabric", "__default__", "cn-jenkins-deploy-platform-juju-os-473-1"]id_perms : {"permissions": {"owner": "admin", "owner_access": 7, "group": "reader", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 11686432484423910657, "uuid_lslong": 9820898330021686608}, "enable": true, "created": "2025-02-13T02:17:09.316738", "last_modified": "2025-02-13T02:18:00.441908", "description": null, "user_visible": true, "creator": null}parent_type : "routing-instance"parent_uuid : "ee7c4a6e-9c5f-4011-b493-66a7db24a1ae"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "a22e8c6d-5dac-4501-884a-d6ff702b6d50" } deleted = False >> 02/13/2025 02:18:04.454 7f0bbda078d0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:18:04.454 7f0bbda078d0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:18:04.454 7f0bbda078d0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:18:04.455 7f0bbda078d0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:18:04.455 7f0bbda078d0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:18:04.455 7f0bbda078d0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:18:04.455 7f0bbda078d0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:18:04.455 7f0bbda078d0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 02/13/2025 02:18:04.455 7f0bbda078d0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:18:04.762 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:05.763 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:06.764 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:07.766 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:08.454 7f0bbd957b70 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:18:08.768 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:09.769 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:10.770 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:11.772 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:11.876 7f0bbddbb438 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:18:11.927 7f0bbdf11240 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:18:12.774 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:13.456 7f0bbd957b70 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:18:13.457 7f0bbd957b70 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:18:13.457 7f0bbd957b70 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:18:13.457 7f0bbd957b70 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:18:13.457 7f0bbd957b70 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:18:13.457 7f0bbd957b70 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:18:13.458 7f0bbd957b70 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:18:13.458 7f0bbd957b70 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:18:13.458 7f0bbd957b70 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:18:13.458 7f0bbd957b70 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:18:13.458 7f0bbd957b70 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:18:13.459 7f0bbd957b70 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 02/13/2025 02:18:13.459 7f0bbd957b70 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:18:13.775 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:14.777 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:15.779 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:16.779 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:17.458 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:18:17.781 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:18.782 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:19.646 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource global-system-config: 'parent_type' 02/13/2025 02:18:19.646 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flags in resource global-system-config: 'feature_flags' 02/13/2025 02:18:19.646 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zones in resource global-system-config: 'control_node_zones' 02/13/2025 02:18:19.647 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_configs in resource global-system-config: 'global_vrouter_configs' 02/13/2025 02:18:19.647 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_configs in resource global-system-config: 'global_qos_configs' 02/13/2025 02:18:19.647 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_routers in resource global-system-config: 'virtual_routers' 02/13/2025 02:18:19.647 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_nodes in resource global-system-config: 'config_nodes' 02/13/2025 02:18:19.647 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_nodes in resource global-system-config: 'analytics_nodes' 02/13/2025 02:18:19.647 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_nodes in resource global-system-config: 'flow_nodes' 02/13/2025 02:18:19.647 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_nodes in resource global-system-config: 'devicemgr_nodes' 02/13/2025 02:18:19.647 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_nodes in resource global-system-config: 'database_nodes' 02/13/2025 02:18:19.648 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_nodes in resource global-system-config: 'webui_nodes' 02/13/2025 02:18:19.648 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_nodes in resource global-system-config: 'config_database_nodes' 02/13/2025 02:18:19.649 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_nodes in resource global-system-config: 'analytics_alarm_nodes' 02/13/2025 02:18:19.649 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_nodes in resource global-system-config: 'analytics_snmp_nodes' 02/13/2025 02:18:19.649 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_sets in resource global-system-config: 'service_appliance_sets' 02/13/2025 02:18:19.649 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource global-system-config: 'api_access_lists' 02/13/2025 02:18:19.649 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource global-system-config: 'alarms' 02/13/2025 02:18:19.649 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_propertiess in resource global-system-config: 'config_propertiess' 02/13/2025 02:18:19.649 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_templates in resource global-system-config: 'job_templates' 02/13/2025 02:18:19.649 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnects in resource global-system-config: 'data_center_interconnects' 02/13/2025 02:18:19.649 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_maps in resource global-system-config: 'intent_maps' 02/13/2025 02:18:19.649 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabrics in resource global-system-config: 'fabrics' 02/13/2025 02:18:19.650 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profiles in resource global-system-config: 'node_profiles' 02/13/2025 02:18:19.650 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_routers in resource global-system-config: 'physical_routers' 02/13/2025 02:18:19.650 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_images in resource global-system-config: 'device_images' 02/13/2025 02:18:19.650 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field nodes in resource global-system-config: 'nodes' 02/13/2025 02:18:19.650 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field features in resource global-system-config: 'features' 02/13/2025 02:18:19.650 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_roles in resource global-system-config: 'physical_roles' 02/13/2025 02:18:19.650 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_roles in resource global-system-config: 'overlay_roles' 02/13/2025 02:18:19.650 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definitions in resource global-system-config: 'role_definitions' 02/13/2025 02:18:19.650 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_configs in resource global-system-config: 'global_analytics_configs' 02/13/2025 02:18:19.651 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource global-system-config: 'tag_refs' 02/13/2025 02:18:19.651 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource global-system-config: 'qos_config_back_refs' 02/13/2025 02:18:19.753 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_routers in resource routing-instance: 'bgp_routers' 02/13/2025 02:18:19.758 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_refs in resource routing-instance: 'routing_instance_refs' 02/13/2025 02:18:19.758 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_target_refs in resource routing-instance: 'route_target_refs' 02/13/2025 02:18:19.758 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource routing-instance: 'tag_refs' 02/13/2025 02:18:19.758 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource routing-instance: 'virtual_machine_interface_back_refs' 02/13/2025 02:18:19.758 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregate_back_refs in resource routing-instance: 'route_aggregate_back_refs' 02/13/2025 02:18:19.758 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_back_refs in resource routing-instance: 'routing_policy_back_refs' 02/13/2025 02:18:19.758 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_back_refs in resource routing-instance: 'routing_instance_back_refs' 02/13/2025 02:18:19.783 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:19.811 7f0bbd90d438 [contrail-api] [ERROR]: SANDESH: Not connected: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 3a60be2c-e787-4ec9-894b-c0c6d7944765 object_type = global_system_config identifier_name = default-global-system-config url = https://10.0.0.17:8082/global-system-config/3a60be2c-e787-4ec9-894b-c0c6d7944765 operation = http_put useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_vrouter.py remote_ip = 10.0.0.17 body = {"global-system-config": {"uuid": "3a60be2c-e787-4ec9-894b-c0c6d7944765", "ip_fabric_subnets": {"subnet": [{"ip_prefix": "10.0.0.0", "ip_prefix_len": 24}]}}} domain = default-domain project = admin user = admin >> 02/13/2025 02:18:19.811 7f0bbd90d438 [contrail-api] [ERROR]: SANDESH: [DROP: SessionNotConnected] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 3a60be2c-e787-4ec9-894b-c0c6d7944765 object_type = global_system_config identifier_name = default-global-system-config url = https://10.0.0.17:8082/global-system-config/3a60be2c-e787-4ec9-894b-c0c6d7944765 operation = http_put useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_vrouter.py remote_ip = 10.0.0.17 body = {"global-system-config": {"uuid": "3a60be2c-e787-4ec9-894b-c0c6d7944765", "ip_fabric_subnets": {"subnet": [{"ip_prefix": "10.0.0.0", "ip_prefix_len": 24}]}}} domain = default-domain project = admin user = admin >> 02/13/2025 02:18:19.823 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-87c44341-e264-491c-9ee2-a3e408c4ab6f', 'type': 'global_system_config', 'uuid': '3a60be2c-e787-4ec9-894b-c0c6d7944765'} 02/13/2025 02:18:19.921 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-os-473-1'] not found in DB: Unknown id: virtual_router default-global-system-config:cn-jenkins-deploy-platform-juju-os-473-1 02/13/2025 02:18:19.926 7f0bbddb0f60 [contrail-api] [ERROR]: SANDESH: Not connected: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = virtual_router identifier_name = default-global-system-config:cn-jenkins-deploy-platform-juju-os-473-1 url = https://10.0.0.17:8082/virtual-routers operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_vrouter.py remote_ip = 10.0.0.17 body = {"virtual-router": {"uuid": null, "fq_name": ["default-global-system-config", "cn-jenkins-deploy-platform-juju-os-473-1"], "parent_type": "global-system-config", "virtual_router_dpdk_enabled": false, "virtual_router_ip_address": "10.0.0.17", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "cn-jenkins-deploy-platform-juju-os-473-1"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:18:19.926 7f0bbddb0f60 [contrail-api] [ERROR]: SANDESH: [DROP: SessionNotConnected] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = virtual_router identifier_name = default-global-system-config:cn-jenkins-deploy-platform-juju-os-473-1 url = https://10.0.0.17:8082/virtual-routers operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_vrouter.py remote_ip = 10.0.0.17 body = {"virtual-router": {"uuid": null, "fq_name": ["default-global-system-config", "cn-jenkins-deploy-platform-juju-os-473-1"], "parent_type": "global-system-config", "virtual_router_dpdk_enabled": false, "virtual_router_ip_address": "10.0.0.17", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "cn-jenkins-deploy-platform-juju-os-473-1"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:18:19.944 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-os-473-1'], 'obj_dict': {'display_name': 'cn-jenkins-deploy-platform-juju-os-473-1', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-os-473-1'], 'id_perms': {'created': '2025-02-13T02:18:19.937395', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:18:19.937395', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9686949667980308758, 'uuid_mslong': 7686204798981718132}}, 'parent_type': 'global-system-config', 'parent_uuid': '3a60be2c-e787-4ec9-894b-c0c6d7944765', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '6aaae28a-9a35-4074-866e-f5663d0df116', 'virtual_router_dpdk_enabled': False, 'virtual_router_ip_address': '10.0.0.17'}, 'oper': 'CREATE', 'request-id': 'req-087b9cc0-4f53-4b50-bf1b-c31a74f0d023', 'type': 'virtual_router', 'uuid': '6aaae28a-9a35-4074-866e-f5663d0df116'} 02/13/2025 02:18:19.945 7f0bbe5bcc18 [contrail-api] [DEBUG]: Add uve in the [ObjectVRouter:ContrailConfigTrace] map 02/13/2025 02:18:20.041 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 02/13/2025 02:18:20.041 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 02/13/2025 02:18:20.042 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 02/13/2025 02:18:20.042 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_refs in resource virtual-router: 'virtual_machine_refs' 02/13/2025 02:18:20.042 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 02/13/2025 02:18:20.042 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 02/13/2025 02:18:20.042 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 02/13/2025 02:18:20.042 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 02/13/2025 02:18:20.198 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_lists in resource virtual-network: 'access_control_lists' 02/13/2025 02:18:20.199 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pools in resource virtual-network: 'floating_ip_pools' 02/13/2025 02:18:20.199 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pools in resource virtual-network: 'alias_ip_pools' 02/13/2025 02:18:20.199 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instances in resource virtual-network: 'routing_instances' 02/13/2025 02:18:20.199 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domains in resource virtual-network: 'bridge_domains' 02/13/2025 02:18:20.199 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource virtual-network: 'security_logging_object_refs' 02/13/2025 02:18:20.199 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_refs in resource virtual-network: 'qos_config_refs' 02/13/2025 02:18:20.200 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-network: 'network_ipam_refs' 02/13/2025 02:18:20.200 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policy_refs in resource virtual-network: 'network_policy_refs' 02/13/2025 02:18:20.200 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource virtual-network: 'virtual_network_refs' 02/13/2025 02:18:20.200 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_refs in resource virtual-network: 'service_health_check_refs' 02/13/2025 02:18:20.200 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_refs in resource virtual-network: 'route_table_refs' 02/13/2025 02:18:20.200 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_refs in resource virtual-network: 'multicast_policy_refs' 02/13/2025 02:18:20.200 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_refs in resource virtual-network: 'bgpvpn_refs' 02/13/2025 02:18:20.200 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_refs in resource virtual-network: 'intent_map_refs' 02/13/2025 02:18:20.200 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_refs in resource virtual-network: 'routing_policy_refs' 02/13/2025 02:18:20.201 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-network: 'tag_refs' 02/13/2025 02:18:20.201 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource virtual-network: 'virtual_network_back_refs' 02/13/2025 02:18:20.201 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource virtual-network: 'virtual_machine_interface_back_refs' 02/13/2025 02:18:20.201 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-network: 'instance_ip_back_refs' 02/13/2025 02:18:20.201 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-network: 'physical_router_back_refs' 02/13/2025 02:18:20.201 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource virtual-network: 'port_tuple_back_refs' 02/13/2025 02:18:20.201 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource virtual-network: 'logical_router_back_refs' 02/13/2025 02:18:20.201 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource virtual-network: 'flow_node_back_refs' 02/13/2025 02:18:20.201 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource virtual-network: 'firewall_rule_back_refs' 02/13/2025 02:18:20.202 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource virtual-network: 'data_center_interconnect_back_refs' 02/13/2025 02:18:20.202 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource virtual-network: 'fabric_back_refs' 02/13/2025 02:18:20.202 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource virtual-network: 'host_based_service_back_refs' 02/13/2025 02:18:20.258 7f0bbddbbe10 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-os-473-1', 'vhost0'] not found in DB: Unknown id: virtual_machine_interface default-global-system-config:cn-jenkins-deploy-platform-juju-os-473-1:vhost0 02/13/2025 02:18:20.262 7f0bbd9ea978 [contrail-api] [ERROR]: SANDESH: Not connected: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = virtual_machine_interface identifier_name = default-global-system-config:cn-jenkins-deploy-platform-juju-os-473-1:vhost0 url = https://10.0.0.17:8082/virtual-machine-interfaces operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_vrouter.py remote_ip = 10.0.0.17 body = {"virtual-machine-interface": {"uuid": null, "fq_name": ["default-global-system-config", "cn-jenkins-deploy-platform-juju-os-473-1", "vhost0"], "parent_type": "virtual-router", "virtual_machine_interface_disable_policy": true, "virtual_network_refs": [{"to": ["default-domain", "default-project", "ip-fabric"], "uuid": "2d6b6d1c-978e-4fe6-9af3-aa814e83dd99"}], "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "vhost0"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:18:20.263 7f0bbd9ea978 [contrail-api] [ERROR]: SANDESH: [DROP: SessionNotConnected] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = virtual_machine_interface identifier_name = default-global-system-config:cn-jenkins-deploy-platform-juju-os-473-1:vhost0 url = https://10.0.0.17:8082/virtual-machine-interfaces operation = post useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_vrouter.py remote_ip = 10.0.0.17 body = {"virtual-machine-interface": {"uuid": null, "fq_name": ["default-global-system-config", "cn-jenkins-deploy-platform-juju-os-473-1", "vhost0"], "parent_type": "virtual-router", "virtual_machine_interface_disable_policy": true, "virtual_network_refs": [{"to": ["default-domain", "default-project", "ip-fabric"], "uuid": "2d6b6d1c-978e-4fe6-9af3-aa814e83dd99"}], "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "vhost0"}} domain = default-domain project = admin user = admin >> 02/13/2025 02:18:20.283 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-os-473-1', 'vhost0'], 'obj_dict': {'display_name': 'vhost0', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-os-473-1', 'vhost0'], 'id_perms': {'created': '2025-02-13T02:18:20.274315', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-02-13T02:18:20.274315', 'permissions': {'group': 'reader', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12751948770660362951, 'uuid_mslong': 8057723190644852743}}, 'parent_type': 'virtual-router', 'parent_uuid': '6aaae28a-9a35-4074-866e-f5663d0df116', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '6fd2c88c-dcd0-4807-b0f8-05d81d30aac7', 'virtual_machine_interface_disable_policy': True, 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:6f:d2:c8:8c:dc']}, 'virtual_network_refs': [{'to': ['default-domain', 'default-project', 'ip-fabric'], 'uuid': '2d6b6d1c-978e-4fe6-9af3-aa814e83dd99'}]}, 'oper': 'CREATE', 'request-id': 'req-7501d1af-525f-433d-a331-cf326b7658f5', 'type': 'virtual_machine_interface', 'uuid': '6fd2c88c-dcd0-4807-b0f8-05d81d30aac7'} 02/13/2025 02:18:20.283 7f0bbe5bcc18 [contrail-api] [DEBUG]: Add uve in the [ObjectVMITable:ContrailConfigTrace] map 02/13/2025 02:18:20.291 7f0bbd9ea780 [contrail-api] [ERROR]: SANDESH: Not connected: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 6fd2c88c-dcd0-4807-b0f8-05d81d30aac7 object_type = virtual_machine_interface identifier_name = default-global-system-config:cn-jenkins-deploy-platform-juju-os-473-1:vhost0 url = http://127.0.0.1/ref-update operation = ref-update domain = default-domain >> 02/13/2025 02:18:20.291 7f0bbd9ea780 [contrail-api] [ERROR]: SANDESH: [DROP: SessionNotConnected] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 6fd2c88c-dcd0-4807-b0f8-05d81d30aac7 object_type = virtual_machine_interface identifier_name = default-global-system-config:cn-jenkins-deploy-platform-juju-os-473-1:vhost0 url = http://127.0.0.1/ref-update operation = ref-update domain = default-domain >> 02/13/2025 02:18:20.306 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-os-473-1', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-7501d1af-525f-433d-a331-cf326b7658f5', 'type': 'virtual_machine_interface', 'uuid': '6fd2c88c-dcd0-4807-b0f8-05d81d30aac7'} 02/13/2025 02:18:20.785 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:20.804 7f0bbd8400f0 [contrail-api] [ERROR]: SANDESH: Not connected: __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = b1c1623a-323b-4bff-b945-1cec547ea138 object_type = access_control_list identifier_name = default-domain:default-project:ip-fabric:ip-fabric url = https://10.0.0.17:8082/access-control-list/b1c1623a-323b-4bff-b945-1cec547ea138 operation = http_put useragent = cn-jenkins-deploy-platform-juju-os-473-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.17 body = {"access-control-list": {"uuid": "b1c1623a-323b-4bff-b945-1cec547ea138", "access_control_list_entries": {"dynamic": false, "acl_rule": [{"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "default-domain:default-project:ip-fabric", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": "default-domain:default-project:ip-fabric", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": null}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "00000000-0000-0000-0000-100000000001", "direction": null}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "default-domain:default-project:ip-fabric", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": "any", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": null}, "action_list": {"simple_action": "deny", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "00000000-0000-0000-0000-100000000002", "direction": "<>"}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "any", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": "any", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": null}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "00000000-0000-0000-0000-100000000001", "direction": null}]}, "access_control_list_hash": -8618641849503496219}} domain = default-domain project = admin user = admin >> 02/13/2025 02:18:20.804 7f0bbd8400f0 [contrail-api] [ERROR]: SANDESH: [DROP: SessionNotConnected] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = b1c1623a-323b-4bff-b945-1cec547ea138 object_type = access_control_list identifier_name = default-domain:default-project:ip-fabric:ip-fabric url = https://10.0.0.17:8082/access-control-list/b1c1623a-323b-4bff-b945-1cec547ea138 operation = http_put useragent = cn-jenkins-deploy-platform-juju-os-473-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.17 body = {"access-control-list": {"uuid": "b1c1623a-323b-4bff-b945-1cec547ea138", "access_control_list_entries": {"dynamic": false, "acl_rule": [{"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "default-domain:default-project:ip-fabric", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": "default-domain:default-project:ip-fabric", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": null}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "00000000-0000-0000-0000-100000000001", "direction": null}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "default-domain:default-project:ip-fabric", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": "any", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": null}, "action_list": {"simple_action": "deny", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "00000000-0000-0000-0000-100000000002", "direction": "<>"}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "any", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": "any", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": null}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "00000000-0000-0000-0000-100000000001", "direction": null}]}, "access_control_list_hash": -8618641849503496219}} domain = default-domain project = admin user = admin >> 02/13/2025 02:18:20.815 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'oper': 'UPDATE', 'request-id': 'req-2e024e14-0dd2-4a82-9abb-925197456b3d', 'type': 'access_control_list', 'uuid': 'b1c1623a-323b-4bff-b945-1cec547ea138'} 02/13/2025 02:18:21.787 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:22.463 7f0bbd049ac8 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:18:22.463 7f0bbd049ac8 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:18:22.463 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:18:22.463 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:18:22.464 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:18:22.464 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:18:22.464 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 elements = { display_name : "cn-jenkins-deploy-platform-juju-os-473-1"fq_name : ["default-global-system-config", "cn-jenkins-deploy-platform-juju-os-473-1"]id_perms : {"permissions": {"owner": "admin", "owner_access": 7, "group": "reader", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 7686204798981718132, "uuid_lslong": 9686949667980308758}, "enable": true, "created": "2025-02-13T02:18:19.937395", "last_modified": "2025-02-13T02:18:19.937395", "description": null, "user_visible": true, "creator": null}parent_type : "global-system-config"parent_uuid : "3a60be2c-e787-4ec9-894b-c0c6d7944765"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "6aaae28a-9a35-4074-866e-f5663d0df116"virtual_router_dpdk_enabled : falsevirtual_router_ip_address : "10.0.0.17" } deleted = False >> 02/13/2025 02:18:22.464 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:18:22.464 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:18:22.464 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-global-system-config:cn-jenkins-deploy-platform-juju-os-473-1:vhost0 elements = { display_name : "vhost0"fq_name : ["default-global-system-config", "cn-jenkins-deploy-platform-juju-os-473-1", "vhost0"]id_perms : {"permissions": {"owner": "admin", "owner_access": 7, "group": "reader", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 8057723190644852743, "uuid_lslong": 12751948770660362951}, "enable": true, "created": "2025-02-13T02:18:20.274315", "last_modified": "2025-02-13T02:18:20.274315", "description": null, "user_visible": true, "creator": null}parent_type : "virtual-router"parent_uuid : "6aaae28a-9a35-4074-866e-f5663d0df116"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "6fd2c88c-dcd0-4807-b0f8-05d81d30aac7"virtual_machine_interface_disable_policy : truevirtual_machine_interface_mac_addresses : {"mac_address": ["02:6f:d2:c8:8c:dc"]}virtual_network_refs : [{"to": ["default-domain", "default-project", "ip-fabric"], "uuid": "2d6b6d1c-978e-4fe6-9af3-aa814e83dd99"}] } deleted = False >> 02/13/2025 02:18:22.464 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:18:22.464 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:18:22.464 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-global-system-config:cn-jenkins-deploy-platform-juju-os-473-1:vhost0 elements = { display_name : "vhost0"fq_name : ["default-global-system-config", "cn-jenkins-deploy-platform-juju-os-473-1", "vhost0"]id_perms : {"permissions": {"owner": "admin", "owner_access": 7, "group": "reader", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 8057723190644852743, "uuid_lslong": 12751948770660362951}, "enable": true, "created": "2025-02-13T02:18:20.274315", "last_modified": "2025-02-13T02:18:20.297624", "description": null, "user_visible": true, "creator": null}parent_type : "virtual-router"parent_uuid : "6aaae28a-9a35-4074-866e-f5663d0df116"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}routing_instance_refs : [{"to": ["default-domain", "default-project", "ip-fabric", "ip-fabric"], "attr": {"direction": "both", "vlan_tag": null, "src_mac": null, "dst_mac": null, "mpls_label": null, "service_chain_address": null, "ipv6_service_chain_address": null, "protocol": null}, "uuid": "5194c63a-dd89-4fca-aa15-43c79ab61241"}]uuid : "6fd2c88c-dcd0-4807-b0f8-05d81d30aac7"virtual_machine_interface_disable_policy : truevirtual_machine_interface_mac_addresses : {"mac_address": ["02:6f:d2:c8:8c:dc"]}virtual_network_refs : [{"to": ["default-domain", "default-project", "ip-fabric"], "attr": null, "uuid": "2d6b6d1c-978e-4fe6-9af3-aa814e83dd99"}] } deleted = False >> 02/13/2025 02:18:22.464 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:18:22.464 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:18:22.465 7f0bbd049ac8 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:18:22.465 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:18:22.465 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:18:22.465 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:18:22.465 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:18:22.465 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:18:22.465 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:18:22.789 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:23.791 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:24.792 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:25.794 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:26.464 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:18:26.795 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:27.797 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:28.798 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:29.001 7f0bbd840630 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:18:29.054 7f0bbdd03198 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:18:29.800 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:30.802 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:31.468 7f0bbd049ac8 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:18:31.468 7f0bbd049ac8 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:18:31.468 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:18:31.468 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:18:31.468 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:18:31.468 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:18:31.469 7f0bbd049ac8 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:18:31.469 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:18:31.469 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:18:31.469 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:18:31.469 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:18:31.470 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:18:31.470 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:18:32.760 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:33.761 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:34.763 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:35.468 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:18:35.765 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:36.766 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:37.768 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:38.769 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:39.771 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:39.906 7f0bbd840198 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:18:39.963 7f0bbd840198 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:18:40.472 7f0bbd049ac8 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:18:40.472 7f0bbd049ac8 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:18:40.472 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:18:40.472 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:18:40.472 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:18:40.472 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:18:40.473 7f0bbd049ac8 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:18:40.473 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:18:40.473 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:18:40.473 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:18:40.473 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:18:40.474 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:18:40.474 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:18:40.773 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:41.775 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:42.777 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:43.778 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:44.473 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:18:44.780 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:45.782 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:46.783 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:47.785 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:48.787 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:49.476 7f0bbd049ac8 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:18:49.476 7f0bbd049ac8 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:18:49.476 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:18:49.476 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:18:49.476 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:18:49.476 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:18:49.477 7f0bbd049ac8 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:18:49.477 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:18:49.477 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:18:49.477 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:18:49.477 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:18:49.477 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:18:49.477 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:18:49.789 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:50.791 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:51.793 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:52.794 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:53.477 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:18:53.796 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:54.798 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:55.799 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:56.801 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:57.803 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:58.480 7f0bbd049ac8 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:18:58.480 7f0bbd049ac8 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:18:58.480 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:18:58.480 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:18:58.481 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:18:58.481 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:18:58.481 7f0bbd049ac8 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:18:58.481 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:18:58.481 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:18:58.481 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:18:58.482 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:18:58.482 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:18:58.482 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:18:58.804 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:18:59.809 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:00.376 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource global-system-config: 'parent_type' 02/13/2025 02:19:00.376 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flags in resource global-system-config: 'feature_flags' 02/13/2025 02:19:00.376 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zones in resource global-system-config: 'control_node_zones' 02/13/2025 02:19:00.376 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_configs in resource global-system-config: 'global_vrouter_configs' 02/13/2025 02:19:00.376 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_configs in resource global-system-config: 'global_qos_configs' 02/13/2025 02:19:00.376 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_routers in resource global-system-config: 'virtual_routers' 02/13/2025 02:19:00.376 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_nodes in resource global-system-config: 'config_nodes' 02/13/2025 02:19:00.377 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_nodes in resource global-system-config: 'analytics_nodes' 02/13/2025 02:19:00.377 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_nodes in resource global-system-config: 'flow_nodes' 02/13/2025 02:19:00.377 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_nodes in resource global-system-config: 'devicemgr_nodes' 02/13/2025 02:19:00.377 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_nodes in resource global-system-config: 'database_nodes' 02/13/2025 02:19:00.377 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_nodes in resource global-system-config: 'webui_nodes' 02/13/2025 02:19:00.377 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_nodes in resource global-system-config: 'config_database_nodes' 02/13/2025 02:19:00.377 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_nodes in resource global-system-config: 'analytics_alarm_nodes' 02/13/2025 02:19:00.377 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_nodes in resource global-system-config: 'analytics_snmp_nodes' 02/13/2025 02:19:00.377 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_sets in resource global-system-config: 'service_appliance_sets' 02/13/2025 02:19:00.378 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource global-system-config: 'api_access_lists' 02/13/2025 02:19:00.378 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource global-system-config: 'alarms' 02/13/2025 02:19:00.378 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_propertiess in resource global-system-config: 'config_propertiess' 02/13/2025 02:19:00.378 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_templates in resource global-system-config: 'job_templates' 02/13/2025 02:19:00.378 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnects in resource global-system-config: 'data_center_interconnects' 02/13/2025 02:19:00.378 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_maps in resource global-system-config: 'intent_maps' 02/13/2025 02:19:00.378 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabrics in resource global-system-config: 'fabrics' 02/13/2025 02:19:00.378 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profiles in resource global-system-config: 'node_profiles' 02/13/2025 02:19:00.379 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_routers in resource global-system-config: 'physical_routers' 02/13/2025 02:19:00.379 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_images in resource global-system-config: 'device_images' 02/13/2025 02:19:00.379 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field nodes in resource global-system-config: 'nodes' 02/13/2025 02:19:00.379 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field features in resource global-system-config: 'features' 02/13/2025 02:19:00.379 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_roles in resource global-system-config: 'physical_roles' 02/13/2025 02:19:00.379 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_roles in resource global-system-config: 'overlay_roles' 02/13/2025 02:19:00.379 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definitions in resource global-system-config: 'role_definitions' 02/13/2025 02:19:00.379 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_configs in resource global-system-config: 'global_analytics_configs' 02/13/2025 02:19:00.379 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource global-system-config: 'tag_refs' 02/13/2025 02:19:00.379 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource global-system-config: 'qos_config_back_refs' 02/13/2025 02:19:00.485 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_routers in resource routing-instance: 'bgp_routers' 02/13/2025 02:19:00.485 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_refs in resource routing-instance: 'routing_instance_refs' 02/13/2025 02:19:00.485 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_target_refs in resource routing-instance: 'route_target_refs' 02/13/2025 02:19:00.486 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource routing-instance: 'tag_refs' 02/13/2025 02:19:00.486 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource routing-instance: 'virtual_machine_interface_back_refs' 02/13/2025 02:19:00.486 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregate_back_refs in resource routing-instance: 'route_aggregate_back_refs' 02/13/2025 02:19:00.486 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_back_refs in resource routing-instance: 'routing_policy_back_refs' 02/13/2025 02:19:00.486 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_back_refs in resource routing-instance: 'routing_instance_back_refs' 02/13/2025 02:19:00.601 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 02/13/2025 02:19:00.601 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 02/13/2025 02:19:00.601 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 02/13/2025 02:19:00.601 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_refs in resource virtual-router: 'virtual_machine_refs' 02/13/2025 02:19:00.601 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 02/13/2025 02:19:00.602 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 02/13/2025 02:19:00.602 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 02/13/2025 02:19:00.602 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 02/13/2025 02:19:00.655 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 6aaae28a-9a35-4074-866e-f5663d0df116 object_type = virtual_router identifier_name = default-global-system-config:cn-jenkins-deploy-platform-juju-os-473-1 url = https://10.0.0.17:8082/virtual-router/6aaae28a-9a35-4074-866e-f5663d0df116 operation = http_put useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_vrouter.py remote_ip = 10.0.0.17 body = {"virtual-router": {"uuid": "6aaae28a-9a35-4074-866e-f5663d0df116", "virtual_router_dpdk_enabled": false}} domain = default-domain project = admin user = admin >> 02/13/2025 02:19:00.670 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-os-473-1'], 'oper': 'UPDATE', 'request-id': 'req-c619c58c-6c50-468d-8d4d-3ca3a3193de6', 'type': 'virtual_router', 'uuid': '6aaae28a-9a35-4074-866e-f5663d0df116'} 02/13/2025 02:19:00.680 7f0bbddc9c18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:19:00.681 7f0bbddc9c18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 elements = { display_name : "cn-jenkins-deploy-platform-juju-os-473-1"fq_name : ["default-global-system-config", "cn-jenkins-deploy-platform-juju-os-473-1"]id_perms : {"permissions": {"owner": "admin", "owner_access": 7, "group": "reader", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 7686204798981718132, "uuid_lslong": 9686949667980308758}, "enable": true, "created": "2025-02-13T02:18:19.937395", "last_modified": "2025-02-13T02:19:00.663271", "description": null, "user_visible": true, "creator": null}parent_type : "global-system-config"parent_uuid : "3a60be2c-e787-4ec9-894b-c0c6d7944765"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "6aaae28a-9a35-4074-866e-f5663d0df116"virtual_router_dpdk_enabled : falsevirtual_router_ip_address : "10.0.0.17" } deleted = False >> 02/13/2025 02:19:00.681 7f0bbddc9c18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:19:00.765 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 02/13/2025 02:19:00.765 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 02/13/2025 02:19:00.766 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 02/13/2025 02:19:00.766 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_refs in resource virtual-router: 'virtual_machine_refs' 02/13/2025 02:19:00.766 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 02/13/2025 02:19:00.766 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 02/13/2025 02:19:00.766 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 02/13/2025 02:19:00.766 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 02/13/2025 02:19:00.810 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:00.874 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource virtual-machine-interface: 'security_logging_object_refs' 02/13/2025 02:19:00.875 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_refs in resource virtual-machine-interface: 'qos_config_refs' 02/13/2025 02:19:00.875 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_group_refs in resource virtual-machine-interface: 'security_group_refs' 02/13/2025 02:19:00.875 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_refs in resource virtual-machine-interface: 'virtual_machine_interface_refs' 02/13/2025 02:19:00.875 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_refs in resource virtual-machine-interface: 'virtual_machine_refs' 02/13/2025 02:19:00.875 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_refs in resource virtual-machine-interface: 'bgp_router_refs' 02/13/2025 02:19:00.875 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_refs in resource virtual-machine-interface: 'port_tuple_refs' 02/13/2025 02:19:00.875 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_refs in resource virtual-machine-interface: 'service_health_check_refs' 02/13/2025 02:19:00.875 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_table_refs in resource virtual-machine-interface: 'interface_route_table_refs' 02/13/2025 02:19:00.876 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_interface_refs in resource virtual-machine-interface: 'physical_interface_refs' 02/13/2025 02:19:00.876 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domain_refs in resource virtual-machine-interface: 'bridge_domain_refs' 02/13/2025 02:19:00.876 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_endpoint_refs in resource virtual-machine-interface: 'service_endpoint_refs' 02/13/2025 02:19:00.876 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_refs in resource virtual-machine-interface: 'virtual_port_group_refs' 02/13/2025 02:19:00.876 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profile_refs in resource virtual-machine-interface: 'port_profile_refs' 02/13/2025 02:19:00.876 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-machine-interface: 'tag_refs' 02/13/2025 02:19:00.876 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource virtual-machine-interface: 'virtual_machine_interface_back_refs' 02/13/2025 02:19:00.876 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-machine-interface: 'instance_ip_back_refs' 02/13/2025 02:19:00.876 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field subnet_back_refs in resource virtual-machine-interface: 'subnet_back_refs' 02/13/2025 02:19:00.876 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource virtual-machine-interface: 'floating_ip_back_refs' 02/13/2025 02:19:00.876 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource virtual-machine-interface: 'alias_ip_back_refs' 02/13/2025 02:19:00.877 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_back_refs in resource virtual-machine-interface: 'logical_interface_back_refs' 02/13/2025 02:19:00.877 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource virtual-machine-interface: 'bgp_as_a_service_back_refs' 02/13/2025 02:19:00.877 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource virtual-machine-interface: 'customer_attachment_back_refs' 02/13/2025 02:19:00.877 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource virtual-machine-interface: 'logical_router_back_refs' 02/13/2025 02:19:00.877 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource virtual-machine-interface: 'loadbalancer_pool_back_refs' 02/13/2025 02:19:00.877 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ip_back_refs in resource virtual-machine-interface: 'virtual_ip_back_refs' 02/13/2025 02:19:00.877 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_back_refs in resource virtual-machine-interface: 'loadbalancer_back_refs' 02/13/2025 02:19:00.877 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_back_refs in resource virtual-machine-interface: 'virtual_port_group_back_refs' 02/13/2025 02:19:00.877 7f0bbd049ac8 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field link_aggregation_group_back_refs in resource virtual-machine-interface: 'link_aggregation_group_back_refs' 02/13/2025 02:19:00.931 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 6fd2c88c-dcd0-4807-b0f8-05d81d30aac7 object_type = virtual_machine_interface identifier_name = default-global-system-config:cn-jenkins-deploy-platform-juju-os-473-1:vhost0 url = https://10.0.0.17:8082/virtual-machine-interface/6fd2c88c-dcd0-4807-b0f8-05d81d30aac7 operation = http_put useragent = cn-jenkins-deploy-platform-juju-os-473-1:/opt/contrail/utils/provision_vrouter.py remote_ip = 10.0.0.17 body = {"virtual-machine-interface": {"uuid": "6fd2c88c-dcd0-4807-b0f8-05d81d30aac7", "virtual_machine_interface_disable_policy": true}} domain = default-domain project = admin user = admin >> 02/13/2025 02:19:00.948 7f0bbe5bcc18 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-os-473-1', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-29629f07-bddc-4211-a036-db143ff922db', 'type': 'virtual_machine_interface', 'uuid': '6fd2c88c-dcd0-4807-b0f8-05d81d30aac7'} 02/13/2025 02:19:00.963 7f0bbd937198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:19:00.964 7f0bbd937198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-global-system-config:cn-jenkins-deploy-platform-juju-os-473-1:vhost0 elements = { display_name : "vhost0"fq_name : ["default-global-system-config", "cn-jenkins-deploy-platform-juju-os-473-1", "vhost0"]id_perms : {"permissions": {"owner": "admin", "owner_access": 7, "group": "reader", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 8057723190644852743, "uuid_lslong": 12751948770660362951}, "enable": true, "created": "2025-02-13T02:18:20.274315", "last_modified": "2025-02-13T02:19:00.939078", "description": null, "user_visible": true, "creator": null}parent_type : "virtual-router"parent_uuid : "6aaae28a-9a35-4074-866e-f5663d0df116"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}routing_instance_refs : [{"to": ["default-domain", "default-project", "ip-fabric", "ip-fabric"], "attr": {"direction": "both", "vlan_tag": null, "src_mac": null, "dst_mac": null, "mpls_label": null, "service_chain_address": null, "ipv6_service_chain_address": null, "protocol": null}, "uuid": "5194c63a-dd89-4fca-aa15-43c79ab61241"}]uuid : "6fd2c88c-dcd0-4807-b0f8-05d81d30aac7"virtual_machine_interface_disable_policy : truevirtual_machine_interface_mac_addresses : {"mac_address": ["02:6f:d2:c8:8c:dc"]}virtual_network_refs : [{"to": ["default-domain", "default-project", "ip-fabric"], "attr": null, "uuid": "2d6b6d1c-978e-4fe6-9af3-aa814e83dd99"}] } deleted = False >> 02/13/2025 02:19:00.964 7f0bbd937198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:19:02.481 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:19:02.760 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:03.762 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:04.764 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:04.955 7f0bbd854a20 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:19:05.011 7f0bcabbb0f0 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:19:05.766 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:06.767 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:07.484 7f0bbd049ac8 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:19:07.484 7f0bbd049ac8 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:19:07.484 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:19:07.484 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:19:07.485 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:19:07.485 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:19:07.485 7f0bbd049ac8 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:19:07.485 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:19:07.485 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:19:07.486 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:19:07.486 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:19:07.486 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:19:07.486 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:19:07.769 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:08.770 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:09.772 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:10.774 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:11.485 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:19:11.776 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:12.777 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:13.780 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:14.781 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:15.172 7f0bbd9be4e0 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:19:15.225 7f0bbd9ad0f0 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:19:15.783 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:16.488 7f0bbd049ac8 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:19:16.489 7f0bbd049ac8 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:19:16.489 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:19:16.489 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:19:16.490 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:19:16.490 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:19:16.491 7f0bbd049ac8 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:19:16.491 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:19:16.491 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:19:16.491 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:19:16.491 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:19:16.491 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:19:16.491 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:19:16.784 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:17.785 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:18.786 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:19.788 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:20.488 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:19:20.790 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:21.792 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:22.794 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:23.796 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:24.798 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:25.491 7f0bbd049ac8 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:19:25.491 7f0bbd049ac8 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:19:25.492 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:19:25.492 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:19:25.492 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:19:25.492 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:19:25.493 7f0bbd049ac8 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:19:25.493 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:19:25.493 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:19:25.493 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:19:25.493 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:19:25.493 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:19:25.493 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:19:25.799 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:26.800 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:27.802 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:28.804 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:29.491 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:19:29.806 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:30.807 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:32.762 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:33.763 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:34.494 7f0bbd049ac8 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:19:34.494 7f0bbd049ac8 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:19:34.494 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:19:34.495 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:19:34.495 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:19:34.495 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:19:34.496 7f0bbd049ac8 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:19:34.496 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:19:34.496 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:19:34.497 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:19:34.497 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:19:34.497 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:19:34.497 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:19:34.765 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:35.767 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:36.769 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:37.770 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:38.495 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:19:38.772 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:39.774 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:40.776 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:41.777 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:42.779 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:43.498 7f0bbd049ac8 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:19:43.498 7f0bbd049ac8 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:19:43.498 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:19:43.499 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:19:43.499 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:19:43.499 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:19:43.499 7f0bbd049ac8 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:19:43.500 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:19:43.500 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:19:43.500 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:19:43.500 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:19:43.500 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:19:43.500 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:19:43.780 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:44.782 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:45.783 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:46.785 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:47.499 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:19:47.787 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:48.788 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:49.418 7f0bbd83cc18 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:19:49.474 7f0bbd83cc18 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:19:49.790 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:50.792 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:51.793 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:52.502 7f0bbd049ac8 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:19:52.502 7f0bbd049ac8 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:19:52.502 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:19:52.503 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:19:52.503 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:19:52.503 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:19:52.503 7f0bbd049ac8 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:19:52.507 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:19:52.508 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:19:52.508 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:19:52.508 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:19:52.508 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:19:52.508 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:19:52.794 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:53.796 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:54.797 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:55.799 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:56.503 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:19:56.800 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:57.802 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:58.804 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:19:59.805 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:00.807 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:01.506 7f0bbd049ac8 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:20:01.506 7f0bbd049ac8 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:20:01.506 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:20:01.507 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:20:01.507 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:20:01.507 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:20:01.507 7f0bbd049ac8 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:20:01.507 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:20:01.507 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:20:01.508 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:20:01.508 7f0bbd049ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:20:01.508 7f0bbd049ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:20:01.508 7f0bbd049ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:20:02.763 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:03.765 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:04.766 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:05.506 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:20:05.768 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:06.769 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:07.770 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:08.771 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:09.773 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:10.510 7f0bbde2a828 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:20:10.510 7f0bbde2a828 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:20:10.510 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:20:10.510 7f0bbde2a828 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:20:10.510 7f0bbde2a828 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:20:10.511 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:20:10.511 7f0bbde2a828 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:20:10.511 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:20:10.511 7f0bbde2a828 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:20:10.511 7f0bbde2a828 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:20:10.512 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:20:10.512 7f0bbde2a828 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:20:10.512 7f0bbde2a828 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:20:10.775 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:11.777 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:12.778 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:13.779 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:14.510 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:20:14.781 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:15.782 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:16.785 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:17.786 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:18.788 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:19.513 7f0bbde2a828 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:20:19.514 7f0bbde2a828 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:20:19.514 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:20:19.514 7f0bbde2a828 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:20:19.514 7f0bbde2a828 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:20:19.514 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:20:19.515 7f0bbde2a828 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:20:19.515 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:20:19.515 7f0bbde2a828 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:20:19.515 7f0bbde2a828 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:20:19.515 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:20:19.515 7f0bbde2a828 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:20:19.516 7f0bbde2a828 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:20:19.789 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:20.792 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:21.792 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:22.794 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:23.514 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:20:23.795 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:24.797 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:25.799 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:26.801 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:27.802 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:28.517 7f0bbde2a828 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:20:28.517 7f0bbde2a828 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:20:28.517 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:20:28.518 7f0bbde2a828 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:20:28.519 7f0bbde2a828 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:20:28.519 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:20:28.520 7f0bbde2a828 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:20:28.520 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:20:28.520 7f0bbde2a828 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:20:28.520 7f0bbde2a828 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:20:28.520 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:20:28.521 7f0bbde2a828 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:20:28.521 7f0bbde2a828 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:20:28.803 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:29.805 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:30.806 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:32.518 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:20:32.764 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:33.765 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:34.767 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:35.770 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:36.775 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:37.521 7f0bbde2a828 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:20:37.521 7f0bbde2a828 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:20:37.521 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:20:37.521 7f0bbde2a828 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:20:37.521 7f0bbde2a828 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:20:37.521 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:20:37.522 7f0bbde2a828 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:20:37.522 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:20:37.522 7f0bbde2a828 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:20:37.522 7f0bbde2a828 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:20:37.522 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:20:37.522 7f0bbde2a828 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:20:37.522 7f0bbde2a828 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:20:37.777 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:38.779 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:39.782 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:40.784 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:41.521 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:20:41.785 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:42.787 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:43.789 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:44.790 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:45.794 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:46.523 7f0bbde2a828 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:20:46.524 7f0bbde2a828 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:20:46.524 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:20:46.524 7f0bbde2a828 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:20:46.524 7f0bbde2a828 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:20:46.524 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:20:46.525 7f0bbde2a828 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:20:46.525 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:20:46.525 7f0bbde2a828 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:20:46.525 7f0bbde2a828 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:20:46.525 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:20:46.525 7f0bbde2a828 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:20:46.526 7f0bbde2a828 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:20:46.795 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:47.797 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:48.799 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:49.801 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:50.524 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:20:50.802 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:51.804 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:52.806 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:53.808 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:54.809 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:55.527 7f0bbde2a828 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:20:55.527 7f0bbde2a828 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:20:55.527 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:20:55.527 7f0bbde2a828 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:20:55.527 7f0bbde2a828 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:20:55.527 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:20:55.528 7f0bbde2a828 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:20:55.528 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:20:55.528 7f0bbde2a828 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:20:55.528 7f0bbde2a828 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:20:55.528 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:20:55.528 7f0bbde2a828 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:20:55.529 7f0bbde2a828 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:20:55.810 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:56.812 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:57.814 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:58.816 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:20:59.528 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:20:59.817 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:00.819 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:02.765 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:03.767 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:04.530 7f0bbde2a828 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:21:04.530 7f0bbde2a828 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:21:04.530 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:21:04.530 7f0bbde2a828 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:21:04.531 7f0bbde2a828 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:21:04.531 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:21:04.531 7f0bbde2a828 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:21:04.531 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:21:04.532 7f0bbde2a828 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:21:04.532 7f0bbde2a828 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:21:04.532 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:21:04.532 7f0bbde2a828 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:21:04.532 7f0bbde2a828 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:21:04.768 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:05.770 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:06.771 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:07.773 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:08.531 7f0bbd83ceb8 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:21:08.775 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:09.776 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:10.778 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:11.779 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:12.780 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:13.534 7f0bbd83ceb8 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:21:13.534 7f0bbd83ceb8 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:21:13.534 7f0bbd83ceb8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:21:13.534 7f0bbd83ceb8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:21:13.535 7f0bbd83ceb8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:21:13.535 7f0bbd83ceb8 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:21:13.535 7f0bbd83ceb8 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:21:13.535 7f0bbd83ceb8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:21:13.536 7f0bbd83ceb8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:21:13.536 7f0bbd83ceb8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:21:13.536 7f0bbd83ceb8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:21:13.536 7f0bbd83ceb8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:21:13.536 7f0bbd83ceb8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:21:13.782 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:14.783 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:15.784 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:16.786 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:17.535 7f0bbd85d438 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:21:17.788 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:18.790 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:19.791 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:20.793 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:21.795 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:22.537 7f0bbd85d438 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:21:22.538 7f0bbd85d438 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:21:22.538 7f0bbd85d438 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:21:22.538 7f0bbd85d438 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:21:22.538 7f0bbd85d438 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:21:22.538 7f0bbd85d438 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:21:22.539 7f0bbd85d438 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:21:22.539 7f0bbd85d438 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:21:22.539 7f0bbd85d438 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:21:22.539 7f0bbd85d438 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:21:22.539 7f0bbd85d438 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:21:22.539 7f0bbd85d438 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:21:22.539 7f0bbd85d438 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:21:22.796 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:23.798 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:24.799 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:25.801 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:26.538 7f0bbd85d438 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:21:26.802 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:27.804 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:28.806 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:29.808 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:30.810 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:31.541 7f0bbd85d438 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:21:31.541 7f0bbd85d438 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:21:31.541 7f0bbd85d438 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:21:31.542 7f0bbd85d438 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:21:31.542 7f0bbd85d438 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:21:31.542 7f0bbd85d438 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:21:31.542 7f0bbd85d438 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:21:31.543 7f0bbd85d438 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:21:31.543 7f0bbd85d438 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:21:31.543 7f0bbd85d438 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:21:31.543 7f0bbd85d438 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:21:31.543 7f0bbd85d438 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:21:31.543 7f0bbd85d438 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:21:32.766 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:33.768 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:34.769 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:35.542 7f0bbd85d438 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:21:35.770 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:36.772 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:37.773 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:38.775 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:39.777 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:40.544 7f0bbd85d438 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:21:40.544 7f0bbd85d438 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:21:40.545 7f0bbd85d438 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:21:40.545 7f0bbd85d438 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:21:40.545 7f0bbd85d438 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:21:40.545 7f0bbd85d438 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:21:40.545 7f0bbd85d438 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:21:40.545 7f0bbd85d438 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:21:40.546 7f0bbd85d438 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:21:40.546 7f0bbd85d438 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:21:40.546 7f0bbd85d438 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:21:40.546 7f0bbd85d438 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:21:40.546 7f0bbd85d438 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:21:40.778 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:41.780 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:42.781 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:43.782 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:44.545 7f0bbd85d438 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:21:44.784 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:45.785 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:46.787 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:47.790 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:48.791 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:49.548 7f0bbd85d438 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:21:49.548 7f0bbd85d438 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:21:49.548 7f0bbd85d438 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:21:49.548 7f0bbd85d438 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:21:49.548 7f0bbd85d438 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:21:49.548 7f0bbd85d438 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:21:49.549 7f0bbd85d438 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:21:49.549 7f0bbd85d438 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:21:49.549 7f0bbd85d438 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:21:49.549 7f0bbd85d438 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:21:49.549 7f0bbd85d438 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:21:49.549 7f0bbd85d438 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:21:49.549 7f0bbd85d438 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:21:49.793 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:50.794 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:51.796 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:52.797 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:53.548 7f0bbd85d438 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:21:53.798 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:54.800 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:55.801 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:56.803 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:57.804 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:58.551 7f0bbd85d438 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:21:58.551 7f0bbd85d438 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:21:58.551 7f0bbd85d438 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:21:58.551 7f0bbd85d438 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:21:58.551 7f0bbd85d438 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:21:58.551 7f0bbd85d438 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:21:58.552 7f0bbd85d438 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:21:58.552 7f0bbd85d438 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:21:58.552 7f0bbd85d438 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:21:58.552 7f0bbd85d438 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:21:58.552 7f0bbd85d438 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:21:58.552 7f0bbd85d438 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:21:58.552 7f0bbd85d438 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:21:58.806 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:21:59.808 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:00.285 7f0bbd85d438 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:22:00.339 7f0bbd85d438 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:22:00.810 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:02.551 7f0bbd85d438 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:22:02.767 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:03.769 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:04.770 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:05.773 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:06.774 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:07.553 7f0bbd85d438 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:22:07.554 7f0bbd85d438 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:22:07.554 7f0bbd85d438 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:22:07.554 7f0bbd85d438 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:22:07.554 7f0bbd85d438 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:22:07.554 7f0bbd85d438 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:22:07.555 7f0bbd85d438 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:22:07.555 7f0bbd85d438 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:22:07.555 7f0bbd85d438 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:22:07.555 7f0bbd85d438 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:22:07.555 7f0bbd85d438 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:22:07.555 7f0bbd85d438 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:22:07.555 7f0bbd85d438 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:22:07.776 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:08.778 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:09.780 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:10.781 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:11.554 7f0bc1ea36d8 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:22:11.783 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:12.785 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:13.786 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:14.788 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:15.790 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:16.556 7f0bc1ea36d8 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:22:16.557 7f0bc1ea36d8 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:22:16.557 7f0bc1ea36d8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:22:16.557 7f0bc1ea36d8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:22:16.557 7f0bc1ea36d8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:22:16.557 7f0bc1ea36d8 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:22:16.558 7f0bc1ea36d8 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:22:16.558 7f0bc1ea36d8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:22:16.558 7f0bc1ea36d8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:22:16.558 7f0bc1ea36d8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:22:16.558 7f0bc1ea36d8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:22:16.558 7f0bc1ea36d8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:22:16.558 7f0bc1ea36d8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:22:16.790 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:17.792 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:18.794 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:19.795 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:20.557 7f0bbd8566d8 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:22:20.797 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:21.798 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:22.800 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:23.801 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:24.803 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:25.559 7f0bbd8566d8 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:22:25.559 7f0bbd8566d8 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:22:25.560 7f0bbd8566d8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:22:25.560 7f0bbd8566d8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:22:25.560 7f0bbd8566d8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:22:25.560 7f0bbd8566d8 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:22:25.560 7f0bbd8566d8 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:22:25.560 7f0bbd8566d8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:22:25.561 7f0bbd8566d8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:22:25.561 7f0bbd8566d8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:22:25.561 7f0bbd8566d8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:22:25.561 7f0bbd8566d8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:22:25.561 7f0bbd8566d8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:22:25.804 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:26.806 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:27.807 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:28.808 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:29.560 7f0bbd8566d8 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:22:29.809 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:30.811 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:32.768 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:33.770 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:34.562 7f0bbd8566d8 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:22:34.562 7f0bbd8566d8 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:22:34.562 7f0bbd8566d8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:22:34.562 7f0bbd8566d8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:22:34.562 7f0bbd8566d8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:22:34.563 7f0bbd8566d8 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:22:34.563 7f0bbd8566d8 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:22:34.563 7f0bbd8566d8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:22:34.563 7f0bbd8566d8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:22:34.563 7f0bbd8566d8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:22:34.563 7f0bbd8566d8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:22:34.564 7f0bbd8566d8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:22:34.564 7f0bbd8566d8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:22:34.771 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:35.773 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:36.775 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:37.776 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:38.563 7f0bbd8566d8 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:22:38.778 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:39.779 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:40.780 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:41.782 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:42.784 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:43.565 7f0bbd8566d8 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:22:43.565 7f0bbd8566d8 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:22:43.565 7f0bbd8566d8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:22:43.566 7f0bbd8566d8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:22:43.566 7f0bbd8566d8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:22:43.566 7f0bbd8566d8 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:22:43.566 7f0bbd8566d8 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:22:43.567 7f0bbd8566d8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:22:43.567 7f0bbd8566d8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:22:43.567 7f0bbd8566d8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:22:43.567 7f0bbd8566d8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:22:43.567 7f0bbd8566d8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:22:43.567 7f0bbd8566d8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:22:43.785 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:44.787 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:45.788 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:46.790 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:47.566 7f0bbd8566d8 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:22:47.792 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:48.793 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:49.794 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:50.796 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:51.798 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:52.568 7f0bbd8566d8 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:22:52.568 7f0bbd8566d8 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:22:52.568 7f0bbd8566d8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:22:52.568 7f0bbd8566d8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:22:52.569 7f0bbd8566d8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:22:52.569 7f0bbd8566d8 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:22:52.569 7f0bbd8566d8 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:22:52.569 7f0bbd8566d8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:22:52.569 7f0bbd8566d8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:22:52.569 7f0bbd8566d8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:22:52.570 7f0bbd8566d8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:22:52.570 7f0bbd8566d8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:22:52.570 7f0bbd8566d8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:22:52.800 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:53.802 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:54.803 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:55.805 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:56.568 7f0bbd8566d8 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:22:56.807 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:57.809 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:58.810 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:22:59.811 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:00.551 7f0bbe534978 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:23:00.606 7f0bbe534978 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:23:00.812 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:01.573 7f0bbd8566d8 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:23:01.573 7f0bbd8566d8 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:23:01.574 7f0bbd8566d8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:23:01.574 7f0bbd8566d8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:23:01.574 7f0bbd8566d8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:23:01.574 7f0bbd8566d8 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:23:01.575 7f0bbd8566d8 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:23:01.575 7f0bbd8566d8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:23:01.575 7f0bbd8566d8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:23:01.575 7f0bbd8566d8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:23:01.575 7f0bbd8566d8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:23:01.576 7f0bbd8566d8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:23:01.576 7f0bbd8566d8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:23:02.769 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:03.771 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:04.773 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:05.574 7f0bbe534978 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:23:05.774 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:06.775 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:07.777 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:08.779 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:09.781 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:10.577 7f0bbe534978 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:23:10.577 7f0bbe534978 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:23:10.577 7f0bbe534978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:23:10.577 7f0bbe534978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:23:10.577 7f0bbe534978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:23:10.578 7f0bbe534978 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:23:10.578 7f0bbe534978 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:23:10.578 7f0bbe534978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:23:10.578 7f0bbe534978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:23:10.578 7f0bbe534978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:23:10.578 7f0bbe534978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:23:10.579 7f0bbe534978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:23:10.579 7f0bbe534978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:23:10.782 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:11.669 7f0bbe534978 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:23:11.722 7f0bbe534978 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:23:11.783 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:12.785 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:13.786 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:14.578 7f0bbe534978 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:23:14.788 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:15.790 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:16.791 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:17.793 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:18.794 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:19.581 7f0bbe534978 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:23:19.582 7f0bbe534978 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:23:19.582 7f0bbe534978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:23:19.582 7f0bbe534978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:23:19.582 7f0bbe534978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:23:19.582 7f0bbe534978 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:23:19.583 7f0bbe534978 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:23:19.583 7f0bbe534978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:23:19.583 7f0bbe534978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:23:19.583 7f0bbe534978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:23:19.583 7f0bbe534978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:23:19.583 7f0bbe534978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:23:19.583 7f0bbe534978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:23:19.796 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:20.797 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:21.799 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:22.801 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:23.582 7f0bbe534978 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:23:23.803 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:24.804 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:25.806 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:26.808 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:27.809 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:28.585 7f0bbe534978 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:23:28.585 7f0bbe534978 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:23:28.585 7f0bbe534978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:23:28.585 7f0bbe534978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:23:28.585 7f0bbe534978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:23:28.586 7f0bbe534978 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:23:28.586 7f0bbe534978 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:23:28.586 7f0bbe534978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:23:28.586 7f0bbe534978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:23:28.586 7f0bbe534978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:23:28.586 7f0bbe534978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:23:28.587 7f0bbe534978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:23:28.587 7f0bbe534978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:23:28.811 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:29.813 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:30.814 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:32.586 7f0bbe534978 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:23:32.770 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:33.771 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:34.772 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:35.774 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:36.775 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:37.588 7f0bbe534978 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:23:37.588 7f0bbe534978 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:23:37.589 7f0bbe534978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:23:37.589 7f0bbe534978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:23:37.589 7f0bbe534978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:23:37.589 7f0bbe534978 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:23:37.589 7f0bbe534978 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:23:37.589 7f0bbe534978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:23:37.590 7f0bbe534978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:23:37.590 7f0bbe534978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:23:37.590 7f0bbe534978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:23:37.590 7f0bbe534978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:23:37.590 7f0bbe534978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:23:37.777 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:38.778 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:39.780 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:40.781 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:41.589 7f0bbe534978 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:23:41.783 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:42.784 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:43.786 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:44.788 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:45.788 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:46.592 7f0bbe534978 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:23:46.592 7f0bbe534978 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:23:46.592 7f0bbe534978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:23:46.592 7f0bbe534978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:23:46.593 7f0bbe534978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:23:46.593 7f0bbe534978 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:23:46.593 7f0bbe534978 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:23:46.593 7f0bbe534978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:23:46.594 7f0bbe534978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:23:46.594 7f0bbe534978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:23:46.594 7f0bbe534978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:23:46.594 7f0bbe534978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:23:46.594 7f0bbe534978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:23:46.790 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:47.792 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:48.793 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:49.795 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:50.593 7f0bbe534978 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:23:50.797 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:51.798 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:52.800 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:53.802 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:54.803 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:55.596 7f0bbe534978 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:23:55.596 7f0bbe534978 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:23:55.596 7f0bbe534978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:23:55.596 7f0bbe534978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:23:55.596 7f0bbe534978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:23:55.596 7f0bbe534978 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:23:55.597 7f0bbe534978 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:23:55.597 7f0bbe534978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:23:55.597 7f0bbe534978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:23:55.597 7f0bbe534978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:23:55.597 7f0bbe534978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:23:55.597 7f0bbe534978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:23:55.597 7f0bbe534978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:23:55.805 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:56.807 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:57.807 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:58.809 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:23:59.596 7f0bbe534978 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:23:59.810 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:00.812 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:02.772 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:03.773 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:04.599 7f0bbe534978 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:24:04.599 7f0bbe534978 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:24:04.599 7f0bbe534978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:24:04.599 7f0bbe534978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:24:04.599 7f0bbe534978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:24:04.599 7f0bbe534978 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:24:04.600 7f0bbe534978 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:24:04.600 7f0bbe534978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:24:04.600 7f0bbe534978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:24:04.600 7f0bbe534978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:24:04.600 7f0bbe534978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:24:04.600 7f0bbe534978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:24:04.600 7f0bbe534978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:24:04.774 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:04.907 7f0bbd868390 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:24:04.959 7f0bbd85d6d8 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:24:05.775 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:06.777 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:07.778 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:08.600 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:24:08.780 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:09.782 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:10.784 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:11.786 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:12.787 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:13.603 7f0bbd866a20 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:24:13.603 7f0bbd866a20 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:24:13.603 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:24:13.604 7f0bbd866a20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:24:13.604 7f0bbd866a20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:24:13.604 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:24:13.604 7f0bbd866a20 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:24:13.604 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:24:13.604 7f0bbd866a20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:24:13.605 7f0bbd866a20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:24:13.605 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:24:13.605 7f0bbd866a20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:24:13.605 7f0bbd866a20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:24:13.788 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:14.790 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:15.792 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:16.794 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:17.604 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:24:17.796 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:18.798 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:19.800 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:20.801 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:21.802 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:22.607 7f0bbd866a20 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:24:22.607 7f0bbd866a20 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:24:22.607 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:24:22.607 7f0bbd866a20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:24:22.607 7f0bbd866a20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:24:22.608 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:24:22.608 7f0bbd866a20 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:24:22.608 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:24:22.608 7f0bbd866a20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:24:22.608 7f0bbd866a20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:24:22.609 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:24:22.609 7f0bbd866a20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:24:22.609 7f0bbd866a20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:24:22.803 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:23.805 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:24.807 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:25.808 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:26.608 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:24:26.810 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:27.811 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:28.813 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:29.815 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:30.817 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:31.610 7f0bbd866a20 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:24:31.610 7f0bbd866a20 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:24:31.610 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:24:31.610 7f0bbd866a20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:24:31.610 7f0bbd866a20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:24:31.610 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:24:31.611 7f0bbd866a20 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:24:31.611 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:24:31.611 7f0bbd866a20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:24:31.611 7f0bbd866a20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:24:31.611 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:24:31.611 7f0bbd866a20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:24:31.611 7f0bbd866a20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:24:32.772 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:33.774 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:34.776 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:35.610 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:24:35.777 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:36.778 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:37.779 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:38.781 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:39.783 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:40.613 7f0bbd866a20 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:24:40.613 7f0bbd866a20 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:24:40.613 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:24:40.613 7f0bbd866a20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:24:40.613 7f0bbd866a20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:24:40.613 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:24:40.614 7f0bbd866a20 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:24:40.614 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:24:40.614 7f0bbd866a20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:24:40.614 7f0bbd866a20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:24:40.614 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:24:40.614 7f0bbd866a20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:24:40.614 7f0bbd866a20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:24:40.784 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:41.785 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:42.787 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:43.788 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:44.613 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:24:44.789 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:45.790 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:46.792 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:47.794 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:48.796 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:49.615 7f0bbd866a20 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:24:49.615 7f0bbd866a20 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:24:49.615 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:24:49.616 7f0bbd866a20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:24:49.616 7f0bbd866a20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:24:49.616 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:24:49.616 7f0bbd866a20 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:24:49.616 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:24:49.616 7f0bbd866a20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:24:49.617 7f0bbd866a20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:24:49.617 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:24:49.617 7f0bbd866a20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:24:49.617 7f0bbd866a20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:24:49.798 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:50.800 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:51.802 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:52.803 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:53.616 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:24:53.804 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:54.806 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:55.808 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:56.809 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:57.811 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:58.618 7f0bbd866a20 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:24:58.618 7f0bbd866a20 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:24:58.618 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:24:58.619 7f0bbd866a20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:24:58.619 7f0bbd866a20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:24:58.619 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:24:58.619 7f0bbd866a20 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:24:58.619 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:24:58.620 7f0bbd866a20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:24:58.620 7f0bbd866a20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:24:58.620 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:24:58.620 7f0bbd866a20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:24:58.620 7f0bbd866a20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:24:58.813 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:24:59.814 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:00.816 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:02.619 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:25:02.773 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:03.774 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:04.775 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:05.777 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:06.778 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:07.622 7f0bbd866a20 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:25:07.622 7f0bbd866a20 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:25:07.622 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:25:07.622 7f0bbd866a20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:25:07.623 7f0bbd866a20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:25:07.623 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:25:07.623 7f0bbd866a20 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:25:07.623 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:25:07.623 7f0bbd866a20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:25:07.623 7f0bbd866a20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:25:07.623 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:25:07.624 7f0bbd866a20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:25:07.624 7f0bbd866a20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:25:07.779 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:08.781 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:09.782 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:10.783 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:11.623 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:25:11.785 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:12.786 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:13.788 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:14.790 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:15.792 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:16.626 7f0bbd866a20 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:25:16.626 7f0bbd866a20 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:25:16.626 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:25:16.626 7f0bbd866a20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:25:16.626 7f0bbd866a20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:25:16.626 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:25:16.627 7f0bbd866a20 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:25:16.627 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:25:16.627 7f0bbd866a20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:25:16.627 7f0bbd866a20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:25:16.627 7f0bbd866a20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:25:16.627 7f0bbd866a20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:25:16.627 7f0bbd866a20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:25:16.793 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:17.795 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:18.797 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:19.798 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:20.626 7f0bbd950630 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:25:20.799 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:21.801 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:22.802 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:23.804 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:24.806 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:25.628 7f0bbd950630 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:25:25.628 7f0bbd950630 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:25:25.628 7f0bbd950630 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:25:25.629 7f0bbd950630 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:25:25.629 7f0bbd950630 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:25:25.629 7f0bbd950630 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:25:25.629 7f0bbd950630 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:25:25.629 7f0bbd950630 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:25:25.630 7f0bbd950630 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:25:25.630 7f0bbd950630 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:25:25.630 7f0bbd950630 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:25:25.630 7f0bbd950630 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:25:25.630 7f0bbd950630 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:25:25.807 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:26.808 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:27.810 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:28.811 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:29.629 7f0bbd950630 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:25:29.813 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:30.814 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:32.774 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:33.775 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:34.632 7f0bbd950630 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:25:34.632 7f0bbd950630 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:25:34.632 7f0bbd950630 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:25:34.632 7f0bbd950630 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:25:34.632 7f0bbd950630 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:25:34.633 7f0bbd950630 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:25:34.633 7f0bbd950630 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:25:34.633 7f0bbd950630 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:25:34.633 7f0bbd950630 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:25:34.633 7f0bbd950630 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:25:34.634 7f0bbd950630 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:25:34.634 7f0bbd950630 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:25:34.634 7f0bbd950630 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:25:34.776 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:35.778 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:36.780 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:37.782 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:38.632 7f0bbe5bcac8 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:25:38.783 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:39.785 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:40.786 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:41.788 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:42.793 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:43.636 7f0bbe5bcac8 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:25:43.636 7f0bbe5bcac8 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:25:43.636 7f0bbe5bcac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:25:43.636 7f0bbe5bcac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:25:43.636 7f0bbe5bcac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:25:43.636 7f0bbe5bcac8 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:25:43.637 7f0bbe5bcac8 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:25:43.637 7f0bbe5bcac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:25:43.637 7f0bbe5bcac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:25:43.637 7f0bbe5bcac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:25:43.637 7f0bbe5bcac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:25:43.637 7f0bbe5bcac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:25:43.637 7f0bbe5bcac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:25:43.794 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:44.796 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:45.800 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:46.801 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:47.637 7f0bbe5bcac8 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:25:47.803 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:48.804 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:49.807 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:50.809 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:51.810 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:52.640 7f0bbe5bcac8 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:25:52.641 7f0bbe5bcac8 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:25:52.641 7f0bbe5bcac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:25:52.641 7f0bbe5bcac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:25:52.641 7f0bbe5bcac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:25:52.641 7f0bbe5bcac8 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:25:52.642 7f0bbe5bcac8 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:25:52.642 7f0bbe5bcac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:25:52.642 7f0bbe5bcac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:25:52.642 7f0bbe5bcac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:25:52.642 7f0bbe5bcac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:25:52.642 7f0bbe5bcac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:25:52.642 7f0bbe5bcac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:25:52.811 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:53.813 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:54.814 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:55.815 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:56.641 7f0bbe5bcac8 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:25:56.817 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:57.819 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:58.820 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:25:59.821 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:00.823 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:01.643 7f0bbe5bcac8 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:26:01.643 7f0bbe5bcac8 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:26:01.643 7f0bbe5bcac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:26:01.644 7f0bbe5bcac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:26:01.644 7f0bbe5bcac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:26:01.644 7f0bbe5bcac8 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:26:01.644 7f0bbe5bcac8 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:26:01.645 7f0bbe5bcac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:26:01.645 7f0bbe5bcac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:26:01.645 7f0bbe5bcac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:26:01.645 7f0bbe5bcac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:26:01.645 7f0bbe5bcac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:26:01.645 7f0bbe5bcac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:26:02.775 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:03.777 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:04.778 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:05.644 7f0bbe5a04e0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:26:05.779 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:06.781 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:07.783 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:08.784 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:09.785 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:10.647 7f0bbe5a04e0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:26:10.647 7f0bbe5a04e0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:26:10.647 7f0bbe5a04e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:26:10.647 7f0bbe5a04e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:26:10.647 7f0bbe5a04e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:26:10.648 7f0bbe5a04e0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:26:10.648 7f0bbe5a04e0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:26:10.648 7f0bbe5a04e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:26:10.648 7f0bbe5a04e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:26:10.648 7f0bbe5a04e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:26:10.648 7f0bbe5a04e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:26:10.649 7f0bbe5a04e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:26:10.649 7f0bbe5a04e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:26:10.786 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:11.788 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:12.790 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:13.792 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:14.648 7f0bbe5a04e0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:26:14.793 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:15.794 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:16.796 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:17.798 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:18.799 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:19.651 7f0bbe5a04e0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:26:19.651 7f0bbe5a04e0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:26:19.651 7f0bbe5a04e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:26:19.651 7f0bbe5a04e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:26:19.652 7f0bbe5a04e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:26:19.652 7f0bbe5a04e0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:26:19.652 7f0bbe5a04e0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:26:19.652 7f0bbe5a04e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:26:19.653 7f0bbe5a04e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:26:19.653 7f0bbe5a04e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:26:19.653 7f0bbe5a04e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:26:19.653 7f0bbe5a04e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:26:19.653 7f0bbe5a04e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:26:19.801 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:20.803 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:21.804 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:22.806 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:23.651 7f0bbd8400f0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:26:23.807 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:24.809 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:25.811 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:26.812 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:27.814 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:28.654 7f0bbd8400f0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:26:28.654 7f0bbd8400f0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:26:28.654 7f0bbd8400f0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:26:28.655 7f0bbd8400f0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:26:28.655 7f0bbd8400f0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:26:28.655 7f0bbd8400f0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:26:28.656 7f0bbd8400f0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:26:28.656 7f0bbd8400f0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:26:28.656 7f0bbd8400f0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:26:28.656 7f0bbd8400f0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:26:28.656 7f0bbd8400f0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:26:28.656 7f0bbd8400f0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:26:28.656 7f0bbd8400f0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:26:28.815 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:29.816 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:30.818 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:32.655 7f0bbd8400f0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:26:32.776 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:33.778 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:34.779 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:35.781 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:36.783 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:37.658 7f0bbd8400f0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:26:37.658 7f0bbd8400f0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:26:37.658 7f0bbd8400f0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:26:37.658 7f0bbd8400f0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:26:37.658 7f0bbd8400f0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:26:37.658 7f0bbd8400f0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:26:37.659 7f0bbd8400f0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:26:37.659 7f0bbd8400f0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:26:37.659 7f0bbd8400f0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:26:37.659 7f0bbd8400f0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:26:37.659 7f0bbd8400f0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:26:37.660 7f0bbd8400f0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:26:37.660 7f0bbd8400f0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:26:37.784 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:38.786 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:39.787 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:40.788 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:41.658 7f0bbd8400f0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:26:41.789 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:42.790 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:43.791 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:44.793 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:45.795 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:46.660 7f0bbd8400f0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:26:46.661 7f0bbd8400f0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:26:46.661 7f0bbd8400f0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:26:46.661 7f0bbd8400f0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:26:46.661 7f0bbd8400f0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:26:46.661 7f0bbd8400f0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:26:46.661 7f0bbd8400f0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:26:46.662 7f0bbd8400f0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:26:46.662 7f0bbd8400f0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:26:46.662 7f0bbd8400f0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:26:46.662 7f0bbd8400f0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:26:46.662 7f0bbd8400f0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:26:46.662 7f0bbd8400f0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:26:46.796 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:47.798 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:48.799 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:49.801 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:50.662 7f0bbd8400f0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:26:50.802 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:51.804 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:52.806 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:53.808 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:54.810 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:55.664 7f0bbd8400f0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:26:55.665 7f0bbd8400f0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:26:55.665 7f0bbd8400f0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:26:55.665 7f0bbd8400f0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:26:55.665 7f0bbd8400f0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:26:55.665 7f0bbd8400f0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:26:55.666 7f0bbd8400f0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:26:55.666 7f0bbd8400f0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:26:55.666 7f0bbd8400f0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:26:55.666 7f0bbd8400f0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:26:55.666 7f0bbd8400f0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:26:55.666 7f0bbd8400f0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:26:55.666 7f0bbd8400f0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:26:55.811 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:56.813 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:57.814 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:58.019 7f0bbd8400f0 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:26:58.070 7f0bbd8400f0 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:26:58.816 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:26:59.666 7f0bbd8400f0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:26:59.817 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:00.819 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:02.777 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:03.779 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:04.668 7f0bbd8400f0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:27:04.669 7f0bbd8400f0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:27:04.669 7f0bbd8400f0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:27:04.669 7f0bbd8400f0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:27:04.669 7f0bbd8400f0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:27:04.670 7f0bbd8400f0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:27:04.670 7f0bbd8400f0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:27:04.670 7f0bbd8400f0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:27:04.671 7f0bbd8400f0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:27:04.671 7f0bbd8400f0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:27:04.671 7f0bbd8400f0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:27:04.671 7f0bbd8400f0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:27:04.671 7f0bbd8400f0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:27:04.781 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:05.783 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:06.784 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:07.785 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:08.669 7f0bbe5b7ac8 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:27:08.787 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:09.789 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:10.791 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:11.793 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:12.794 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:13.672 7f0bbe5b7ac8 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:27:13.672 7f0bbe5b7ac8 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:27:13.672 7f0bbe5b7ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:27:13.673 7f0bbe5b7ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:27:13.673 7f0bbe5b7ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:27:13.673 7f0bbe5b7ac8 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:27:13.673 7f0bbe5b7ac8 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:27:13.673 7f0bbe5b7ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:27:13.674 7f0bbe5b7ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:27:13.674 7f0bbe5b7ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:27:13.674 7f0bbe5b7ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:27:13.674 7f0bbe5b7ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:27:13.674 7f0bbe5b7ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:27:13.796 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:14.797 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:15.799 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:16.800 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:17.673 7f0bbe5b7ac8 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:27:17.802 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:18.804 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:19.806 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:20.808 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:21.810 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:22.675 7f0bbe5b7ac8 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:27:22.675 7f0bbe5b7ac8 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:27:22.675 7f0bbe5b7ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:27:22.675 7f0bbe5b7ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:27:22.675 7f0bbe5b7ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:27:22.675 7f0bbe5b7ac8 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:27:22.676 7f0bbe5b7ac8 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:27:22.676 7f0bbe5b7ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:27:22.676 7f0bbe5b7ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:27:22.676 7f0bbe5b7ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:27:22.676 7f0bbe5b7ac8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:27:22.677 7f0bbe5b7ac8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:27:22.677 7f0bbe5b7ac8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:27:22.811 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:23.813 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:24.815 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:25.816 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:26.676 7f0bbdd25978 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:27:26.818 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:27.820 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:28.822 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:29.824 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:30.825 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:31.678 7f0bbdd25978 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:27:31.678 7f0bbdd25978 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:27:31.679 7f0bbdd25978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:27:31.679 7f0bbdd25978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:27:31.679 7f0bbdd25978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:27:31.679 7f0bbdd25978 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:27:31.680 7f0bbdd25978 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:27:31.680 7f0bbdd25978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:27:31.680 7f0bbdd25978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:27:31.680 7f0bbdd25978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:27:31.680 7f0bbdd25978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:27:31.680 7f0bbdd25978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:27:31.680 7f0bbdd25978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:27:32.778 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:33.779 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:34.780 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:35.679 7f0bbdd25978 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:27:35.782 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:36.783 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:37.784 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:38.786 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:39.787 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:40.682 7f0bbdd25978 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:27:40.682 7f0bbdd25978 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:27:40.682 7f0bbdd25978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:27:40.682 7f0bbdd25978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:27:40.682 7f0bbdd25978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:27:40.682 7f0bbdd25978 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:27:40.683 7f0bbdd25978 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:27:40.683 7f0bbdd25978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:27:40.683 7f0bbdd25978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:27:40.683 7f0bbdd25978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:27:40.683 7f0bbdd25978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:27:40.684 7f0bbdd25978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:27:40.684 7f0bbdd25978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:27:40.788 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:41.790 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:42.791 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:43.792 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:44.682 7f0bbdd25978 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:27:44.793 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:45.795 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:46.797 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:47.799 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:48.801 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:49.684 7f0bbdd25978 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:27:49.685 7f0bbdd25978 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:27:49.685 7f0bbdd25978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:27:49.685 7f0bbdd25978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:27:49.685 7f0bbdd25978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:27:49.685 7f0bbdd25978 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:27:49.686 7f0bbdd25978 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:27:49.686 7f0bbdd25978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:27:49.686 7f0bbdd25978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:27:49.686 7f0bbdd25978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:27:49.686 7f0bbdd25978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:27:49.686 7f0bbdd25978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:27:49.686 7f0bbdd25978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:27:49.802 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:50.804 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:51.806 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:52.808 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:53.685 7f0bbdd25978 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:27:53.809 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:54.810 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:55.812 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:56.814 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:57.815 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:58.688 7f0bbdd25978 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:27:58.688 7f0bbdd25978 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:27:58.688 7f0bbdd25978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:27:58.688 7f0bbdd25978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:27:58.688 7f0bbdd25978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:27:58.688 7f0bbdd25978 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:27:58.689 7f0bbdd25978 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:27:58.689 7f0bbdd25978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:27:58.689 7f0bbdd25978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:27:58.689 7f0bbdd25978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:27:58.689 7f0bbdd25978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:27:58.689 7f0bbdd25978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:27:58.690 7f0bbdd25978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:27:58.817 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:27:59.819 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:00.821 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:02.688 7f0bbd8392e8 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:28:02.779 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:03.781 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:04.782 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:05.784 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:06.785 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:07.691 7f0bbd8392e8 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:28:07.691 7f0bbd8392e8 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:28:07.691 7f0bbd8392e8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:28:07.692 7f0bbd8392e8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:28:07.692 7f0bbd8392e8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:28:07.692 7f0bbd8392e8 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:28:07.692 7f0bbd8392e8 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:28:07.692 7f0bbd8392e8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:28:07.693 7f0bbd8392e8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:28:07.693 7f0bbd8392e8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:28:07.693 7f0bbd8392e8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:28:07.693 7f0bbd8392e8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:28:07.693 7f0bbd8392e8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:28:07.787 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:08.789 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:09.793 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:10.795 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:11.692 7f0bbd8392e8 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:28:11.796 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:12.450 7f0bbd8522e8 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:28:12.503 7f0bbd8522e8 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:28:12.797 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:13.799 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:14.801 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:15.802 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:16.695 7f0bbd8392e8 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:28:16.695 7f0bbd8392e8 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:28:16.695 7f0bbd8392e8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:28:16.696 7f0bbd8392e8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:28:16.696 7f0bbd8392e8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:28:16.696 7f0bbd8392e8 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:28:16.696 7f0bbd8392e8 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:28:16.696 7f0bbd8392e8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:28:16.697 7f0bbd8392e8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:28:16.697 7f0bbd8392e8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:28:16.697 7f0bbd8392e8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:28:16.697 7f0bbd8392e8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:28:16.697 7f0bbd8392e8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:28:16.807 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:17.808 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:18.810 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:19.812 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:20.696 7f0bbd8392e8 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:28:20.813 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:21.815 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:22.816 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:23.615 7f0bbda02588 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:28:23.667 7f0bbd947e10 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:28:23.818 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:24.819 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:25.699 7f0bbd8392e8 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:28:25.699 7f0bbd8392e8 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:28:25.699 7f0bbd8392e8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:28:25.699 7f0bbd8392e8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:28:25.699 7f0bbd8392e8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:28:25.699 7f0bbd8392e8 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:28:25.700 7f0bbd8392e8 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:28:25.700 7f0bbd8392e8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:28:25.700 7f0bbd8392e8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:28:25.700 7f0bbd8392e8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:28:25.700 7f0bbd8392e8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:28:25.700 7f0bbd8392e8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:28:25.700 7f0bbd8392e8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:28:25.821 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:26.823 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:27.824 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:28.826 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:29.700 7f0bbd8392e8 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:28:29.827 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:30.829 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:32.780 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:33.782 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:34.711 7f0bbd8392e8 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:28:34.712 7f0bbd8392e8 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:28:34.712 7f0bbd8392e8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:28:34.712 7f0bbd8392e8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:28:34.712 7f0bbd8392e8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:28:34.712 7f0bbd8392e8 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:28:34.713 7f0bbd8392e8 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:28:34.713 7f0bbd8392e8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:28:34.713 7f0bbd8392e8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:28:34.713 7f0bbd8392e8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:28:34.713 7f0bbd8392e8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:28:34.713 7f0bbd8392e8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:28:34.713 7f0bbd8392e8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:28:34.783 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:35.785 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:36.786 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:37.787 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:38.713 7f0bbda02d68 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:28:38.789 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:39.790 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:40.791 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:41.793 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:42.794 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:43.716 7f0bbda02d68 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:28:43.716 7f0bbda02d68 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:28:43.716 7f0bbda02d68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:28:43.716 7f0bbda02d68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:28:43.716 7f0bbda02d68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:28:43.716 7f0bbda02d68 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:28:43.717 7f0bbda02d68 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:28:43.717 7f0bbda02d68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:28:43.717 7f0bbda02d68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:28:43.717 7f0bbda02d68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:28:43.717 7f0bbda02d68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:28:43.718 7f0bbda02d68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:28:43.718 7f0bbda02d68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:28:43.795 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:44.797 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:45.799 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:46.801 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:47.716 7f0bbda02d68 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:28:47.802 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:48.803 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:49.805 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:50.807 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:51.809 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:52.719 7f0bbda02d68 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:28:52.719 7f0bbda02d68 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:28:52.719 7f0bbda02d68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:28:52.720 7f0bbda02d68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:28:52.720 7f0bbda02d68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:28:52.720 7f0bbda02d68 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:28:52.720 7f0bbda02d68 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:28:52.720 7f0bbda02d68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:28:52.720 7f0bbda02d68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:28:52.721 7f0bbda02d68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:28:52.721 7f0bbda02d68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:28:52.721 7f0bbda02d68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:28:52.721 7f0bbda02d68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:28:52.810 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:53.812 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:54.814 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:55.815 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:56.720 7f0bbda02d68 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:28:56.816 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:57.818 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:58.820 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:28:59.821 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:00.823 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:01.723 7f0bbda02d68 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:29:01.723 7f0bbda02d68 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:29:01.724 7f0bbda02d68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:29:01.724 7f0bbda02d68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:29:01.724 7f0bbda02d68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:29:01.724 7f0bbda02d68 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:29:01.725 7f0bbda02d68 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:29:01.725 7f0bbda02d68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:29:01.725 7f0bbda02d68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:29:01.725 7f0bbda02d68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:29:01.725 7f0bbda02d68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:29:01.725 7f0bbda02d68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:29:01.725 7f0bbda02d68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:29:02.781 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:03.783 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:04.785 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:05.723 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:29:05.787 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:06.789 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:07.791 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:07.835 7f0bbddbdd68 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:29:07.890 7f0bbddbdd68 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:29:08.792 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:09.794 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:10.725 7f0bbd8fa588 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:29:10.726 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:29:10.726 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:29:10.726 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:29:10.726 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:29:10.726 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:29:10.727 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:29:10.727 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:29:10.727 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:29:10.727 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:29:10.727 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:29:10.727 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:29:10.727 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:29:10.795 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:11.797 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:12.799 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:13.800 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:14.726 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:29:14.804 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:15.805 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:16.807 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:17.808 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:18.810 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:19.729 7f0bbd8fa588 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:29:19.729 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:29:19.729 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:29:19.729 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:29:19.729 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:29:19.729 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:29:19.730 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:29:19.730 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:29:19.730 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:29:19.731 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:29:19.731 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:29:19.731 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:29:19.731 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:29:19.812 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:20.813 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:21.814 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:22.816 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:23.730 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:29:23.817 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:24.818 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:25.819 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:26.821 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:27.822 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:28.733 7f0bbd8fa588 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:29:28.733 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:29:28.733 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:29:28.733 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:29:28.734 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:29:28.734 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:29:28.734 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:29:28.734 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:29:28.734 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:29:28.734 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:29:28.735 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:29:28.735 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:29:28.735 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:29:28.823 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:29.825 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:30.826 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:32.734 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:29:32.781 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:33.784 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:34.785 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:35.787 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:36.787 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:37.737 7f0bbd8fa588 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:29:37.737 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:29:37.737 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:29:37.737 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:29:37.738 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:29:37.738 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:29:37.738 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:29:37.738 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:29:37.738 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:29:37.738 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:29:37.738 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:29:37.739 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:29:37.739 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:29:37.788 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:38.790 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:39.791 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:40.793 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:41.738 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:29:41.794 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:42.796 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:43.798 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:44.799 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:45.801 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:46.740 7f0bbd8fa588 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:29:46.740 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:29:46.740 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:29:46.740 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:29:46.741 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:29:46.741 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:29:46.741 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:29:46.742 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:29:46.742 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:29:46.742 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:29:46.742 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:29:46.742 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:29:46.742 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:29:46.803 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:47.805 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:48.806 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:49.808 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:50.740 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:29:50.810 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:51.811 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:52.812 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:53.815 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:54.816 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:55.743 7f0bbd8fa588 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:29:55.743 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:29:55.743 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:29:55.744 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:29:55.744 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:29:55.744 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:29:55.744 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:29:55.744 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:29:55.745 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:29:55.745 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:29:55.745 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:29:55.745 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:29:55.745 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:29:55.817 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:56.819 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:57.821 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:58.823 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:29:59.744 7f0bbd840630 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:29:59.824 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:00.825 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:02.783 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:03.785 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:04.746 7f0bbd840630 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:30:04.746 7f0bbd840630 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:30:04.746 7f0bbd840630 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:30:04.746 7f0bbd840630 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:30:04.747 7f0bbd840630 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:30:04.747 7f0bbd840630 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:30:04.747 7f0bbd840630 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:30:04.747 7f0bbd840630 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:30:04.747 7f0bbd840630 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:30:04.747 7f0bbd840630 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:30:04.748 7f0bbd840630 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:30:04.748 7f0bbd840630 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:30:04.748 7f0bbd840630 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:30:04.786 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:05.788 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:06.789 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:07.790 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:08.754 7f0bbd958588 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:30:08.799 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:09.802 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:10.804 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:11.808 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:12.810 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:13.758 7f0bbd958588 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:30:13.758 7f0bbd958588 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:30:13.759 7f0bbd958588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:30:13.759 7f0bbd958588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:30:13.759 7f0bbd958588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:30:13.759 7f0bbd958588 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:30:13.760 7f0bbd958588 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:30:13.760 7f0bbd958588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:30:13.760 7f0bbd958588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:30:13.760 7f0bbd958588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:30:13.760 7f0bbd958588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:30:13.760 7f0bbd958588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:30:13.760 7f0bbd958588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:30:13.814 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:14.816 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:15.818 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:16.819 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:17.759 7f0bbd958588 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:30:17.822 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:18.823 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:19.825 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:20.827 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:21.828 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:22.761 7f0bbd958588 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:30:22.762 7f0bbd958588 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:30:22.762 7f0bbd958588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:30:22.762 7f0bbd958588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:30:22.762 7f0bbd958588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:30:22.762 7f0bbd958588 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:30:22.763 7f0bbd958588 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:30:22.763 7f0bbd958588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:30:22.763 7f0bbd958588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:30:22.763 7f0bbd958588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:30:22.763 7f0bbd958588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:30:22.764 7f0bbd958588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:30:22.764 7f0bbd958588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:30:22.829 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:23.831 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:24.832 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:25.834 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:26.763 7f0bbd810780 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:30:26.835 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:27.836 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:28.838 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:29.840 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:30.842 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:31.765 7f0bbd810780 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:30:31.765 7f0bbd810780 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:30:31.765 7f0bbd810780 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:30:31.765 7f0bbd810780 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:30:31.765 7f0bbd810780 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:30:31.765 7f0bbd810780 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:30:31.766 7f0bbd810780 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:30:31.766 7f0bbd810780 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:30:31.766 7f0bbd810780 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:30:31.766 7f0bbd810780 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:30:31.766 7f0bbd810780 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:30:31.766 7f0bbd810780 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:30:31.766 7f0bbd810780 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:30:32.784 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:33.786 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:34.788 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:35.766 7f0bbd810780 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:30:35.789 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:36.789 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:37.791 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:38.793 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:39.795 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:40.769 7f0bbd810780 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:30:40.769 7f0bbd810780 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:30:40.769 7f0bbd810780 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:30:40.770 7f0bbd810780 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:30:40.770 7f0bbd810780 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:30:40.770 7f0bbd810780 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:30:40.770 7f0bbd810780 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:30:40.770 7f0bbd810780 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:30:40.770 7f0bbd810780 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:30:40.771 7f0bbd810780 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:30:40.771 7f0bbd810780 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:30:40.771 7f0bbd810780 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:30:40.771 7f0bbd810780 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:30:40.796 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:41.799 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:42.800 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:43.802 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:44.770 7f0bbd810780 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:30:44.803 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:45.804 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:46.806 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:47.807 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:48.808 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:49.773 7f0bbd810780 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:30:49.773 7f0bbd810780 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:30:49.773 7f0bbd810780 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:30:49.774 7f0bbd810780 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:30:49.774 7f0bbd810780 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:30:49.774 7f0bbd810780 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:30:49.774 7f0bbd810780 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:30:49.774 7f0bbd810780 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:30:49.774 7f0bbd810780 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:30:49.775 7f0bbd810780 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:30:49.775 7f0bbd810780 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:30:49.775 7f0bbd810780 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:30:49.775 7f0bbd810780 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:30:49.809 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:50.812 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:51.813 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:52.815 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:53.774 7f0bbd81e6d8 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:30:53.816 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:54.818 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:55.819 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:56.821 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:57.822 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:58.776 7f0bbd81e6d8 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:30:58.777 7f0bbd81e6d8 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:30:58.777 7f0bbd81e6d8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:30:58.777 7f0bbd81e6d8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:30:58.777 7f0bbd81e6d8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:30:58.777 7f0bbd81e6d8 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:30:58.777 7f0bbd81e6d8 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:30:58.778 7f0bbd81e6d8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:30:58.778 7f0bbd81e6d8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:30:58.778 7f0bbd81e6d8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:30:58.778 7f0bbd81e6d8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:30:58.778 7f0bbd81e6d8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:30:58.778 7f0bbd81e6d8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:30:58.823 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:30:59.824 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:00.826 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:02.777 7f0bbd81e6d8 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:31:02.785 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:03.787 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:04.789 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:05.790 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:06.791 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:07.780 7f0bbd81e6d8 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:31:07.780 7f0bbd81e6d8 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:31:07.780 7f0bbd81e6d8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:31:07.780 7f0bbd81e6d8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:31:07.781 7f0bbd81e6d8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:31:07.781 7f0bbd81e6d8 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:31:07.781 7f0bbd81e6d8 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:31:07.781 7f0bbd81e6d8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:31:07.781 7f0bbd81e6d8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:31:07.781 7f0bbd81e6d8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:31:07.782 7f0bbd81e6d8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:31:07.782 7f0bbd81e6d8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:31:07.782 7f0bbd81e6d8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:31:07.792 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:08.794 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:09.796 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:10.797 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:11.781 7f0bbe5a04e0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:31:11.798 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:12.800 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:13.802 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:14.804 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:15.805 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:16.783 7f0bbe5a04e0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:31:16.784 7f0bbe5a04e0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:31:16.784 7f0bbe5a04e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:31:16.784 7f0bbe5a04e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:31:16.784 7f0bbe5a04e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:31:16.784 7f0bbe5a04e0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:31:16.785 7f0bbe5a04e0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:31:16.785 7f0bbe5a04e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:31:16.785 7f0bbe5a04e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:31:16.785 7f0bbe5a04e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:31:16.785 7f0bbe5a04e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:31:16.785 7f0bbe5a04e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:31:16.785 7f0bbe5a04e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:31:16.807 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:17.809 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:18.811 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:19.812 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:20.784 7f0bbe5a04e0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:31:20.814 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:21.815 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:22.817 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:23.819 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:24.820 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:25.787 7f0bbe5a04e0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:31:25.787 7f0bbe5a04e0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:31:25.787 7f0bbe5a04e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:31:25.787 7f0bbe5a04e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:31:25.787 7f0bbe5a04e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:31:25.787 7f0bbe5a04e0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:31:25.788 7f0bbe5a04e0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:31:25.788 7f0bbe5a04e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:31:25.788 7f0bbe5a04e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:31:25.788 7f0bbe5a04e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:31:25.788 7f0bbe5a04e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:31:25.788 7f0bbe5a04e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:31:25.789 7f0bbe5a04e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:31:25.821 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:26.822 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:27.824 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:28.825 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:29.788 7f0bbd8190f0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:31:29.827 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:30.828 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:32.786 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:33.787 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:34.788 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:34.790 7f0bbd8190f0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:31:34.790 7f0bbd8190f0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:31:34.790 7f0bbd8190f0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:31:34.790 7f0bbd8190f0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:31:34.790 7f0bbd8190f0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:31:34.790 7f0bbd8190f0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:31:34.791 7f0bbd8190f0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:31:34.791 7f0bbd8190f0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:31:34.791 7f0bbd8190f0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:31:34.791 7f0bbd8190f0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:31:34.791 7f0bbd8190f0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:31:34.792 7f0bbd8190f0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:31:34.792 7f0bbd8190f0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:31:35.790 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:36.791 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:37.792 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:38.791 7f0bbd926f60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:31:38.793 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:39.794 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:40.795 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:41.797 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:42.799 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:43.793 7f0bbd926f60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:31:43.793 7f0bbd926f60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:31:43.794 7f0bbd926f60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:31:43.794 7f0bbd926f60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:31:43.794 7f0bbd926f60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:31:43.794 7f0bbd926f60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:31:43.794 7f0bbd926f60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:31:43.795 7f0bbd926f60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:31:43.795 7f0bbd926f60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:31:43.795 7f0bbd926f60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:31:43.795 7f0bbd926f60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:31:43.795 7f0bbd926f60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:31:43.795 7f0bbd926f60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:31:43.801 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:44.801 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:45.803 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:46.804 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:47.794 7f0bbe5e6f60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:31:47.806 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:48.807 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:49.809 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:50.810 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:51.812 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:52.797 7f0bbe5e6f60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:31:52.798 7f0bbe5e6f60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:31:52.798 7f0bbe5e6f60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:31:52.798 7f0bbe5e6f60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:31:52.798 7f0bbe5e6f60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:31:52.798 7f0bbe5e6f60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:31:52.799 7f0bbe5e6f60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:31:52.799 7f0bbe5e6f60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:31:52.799 7f0bbe5e6f60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:31:52.799 7f0bbe5e6f60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:31:52.799 7f0bbe5e6f60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:31:52.800 7f0bbe5e6f60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:31:52.800 7f0bbe5e6f60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:31:52.813 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:53.262 7f0bbe5e6f60 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:31:53.314 7f0bbe5e6f60 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:31:53.814 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:54.816 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:55.818 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:56.799 7f0bbe5e6f60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:31:56.819 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:57.820 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:58.822 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:31:59.824 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:00.825 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:01.801 7f0bbe5e6f60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:32:01.801 7f0bbe5e6f60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:32:01.801 7f0bbe5e6f60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:32:01.802 7f0bbe5e6f60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:32:01.802 7f0bbe5e6f60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:32:01.802 7f0bbe5e6f60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:32:01.802 7f0bbe5e6f60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:32:01.802 7f0bbe5e6f60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:32:01.803 7f0bbe5e6f60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:32:01.803 7f0bbe5e6f60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:32:01.803 7f0bbe5e6f60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:32:01.803 7f0bbe5e6f60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:32:01.803 7f0bbe5e6f60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:32:02.787 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:03.788 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:04.789 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:05.792 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:05.801 7f0bbdf11240 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:32:06.792 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:07.794 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:08.796 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:09.797 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:10.799 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:10.803 7f0bbdf11240 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:32:10.804 7f0bbdf11240 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:32:10.804 7f0bbdf11240 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:32:10.804 7f0bbdf11240 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:32:10.804 7f0bbdf11240 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:32:10.804 7f0bbdf11240 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:32:10.805 7f0bbdf11240 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:32:10.805 7f0bbdf11240 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:32:10.805 7f0bbdf11240 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:32:10.805 7f0bbdf11240 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:32:10.805 7f0bbdf11240 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:32:10.805 7f0bbdf11240 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:32:10.805 7f0bbdf11240 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:32:11.800 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:12.802 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:13.804 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:14.805 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:14.805 7f0bbdf11240 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:32:15.807 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:16.808 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:17.810 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:18.812 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:19.808 7f0bbdf11240 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:32:19.808 7f0bbdf11240 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:32:19.809 7f0bbdf11240 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:32:19.809 7f0bbdf11240 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:32:19.809 7f0bbdf11240 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:32:19.809 7f0bbdf11240 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:32:19.809 7f0bbdf11240 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:32:19.810 7f0bbdf11240 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:32:19.810 7f0bbdf11240 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:32:19.810 7f0bbdf11240 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:32:19.810 7f0bbdf11240 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:32:19.810 7f0bbdf11240 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:32:19.810 7f0bbdf11240 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:32:19.812 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:20.815 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:21.816 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:22.818 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:23.809 7f0bbdf11240 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:32:23.820 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:24.821 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:25.822 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:26.824 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:27.825 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:28.812 7f0bbdf11240 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:32:28.812 7f0bbdf11240 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:32:28.812 7f0bbdf11240 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:32:28.813 7f0bbdf11240 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:32:28.813 7f0bbdf11240 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:32:28.813 7f0bbdf11240 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:32:28.813 7f0bbdf11240 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:32:28.813 7f0bbdf11240 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:32:28.813 7f0bbdf11240 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:32:28.813 7f0bbdf11240 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:32:28.814 7f0bbdf11240 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:32:28.814 7f0bbdf11240 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:32:28.814 7f0bbdf11240 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:32:28.826 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:29.828 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:30.830 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:32.789 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:32.812 7f0bbdf11240 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:32:33.791 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:34.792 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:35.794 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:36.795 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:37.797 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:37.815 7f0bbdf11240 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:32:37.815 7f0bbdf11240 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:32:37.815 7f0bbdf11240 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:32:37.815 7f0bbdf11240 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:32:37.815 7f0bbdf11240 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:32:37.815 7f0bbdf11240 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:32:37.816 7f0bbdf11240 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:32:37.816 7f0bbdf11240 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:32:37.816 7f0bbdf11240 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:32:37.816 7f0bbdf11240 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:32:37.816 7f0bbdf11240 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:32:37.816 7f0bbdf11240 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:32:37.816 7f0bbdf11240 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:32:38.798 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:39.800 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:40.802 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:41.804 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:41.815 7f0bbdf11240 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:32:42.806 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:43.808 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:44.809 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:45.811 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:46.813 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:46.817 7f0bbdf11240 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:32:46.818 7f0bbdf11240 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:32:46.818 7f0bbdf11240 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:32:46.818 7f0bbdf11240 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:32:46.818 7f0bbdf11240 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:32:46.818 7f0bbdf11240 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:32:46.819 7f0bbdf11240 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:32:46.819 7f0bbdf11240 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:32:46.819 7f0bbdf11240 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:32:46.819 7f0bbdf11240 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:32:46.819 7f0bbdf11240 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:32:46.819 7f0bbdf11240 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:32:46.819 7f0bbdf11240 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:32:47.814 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:48.815 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:49.817 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:50.819 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:50.819 7f0bbdf11240 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:32:51.820 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:52.174 7f0bbdcff0f0 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:32:52.226 7f0bbd958ac8 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:32:52.822 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:53.824 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:54.825 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:55.822 7f0bbdf11240 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:32:55.822 7f0bbdf11240 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:32:55.822 7f0bbdf11240 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:32:55.822 7f0bbdf11240 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:32:55.823 7f0bbdf11240 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:32:55.823 7f0bbdf11240 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:32:55.823 7f0bbdf11240 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:32:55.823 7f0bbdf11240 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:32:55.823 7f0bbdf11240 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:32:55.823 7f0bbdf11240 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:32:55.823 7f0bbdf11240 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:32:55.824 7f0bbdf11240 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:32:55.824 7f0bbdf11240 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:32:55.826 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:56.828 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:57.829 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:58.831 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:32:59.822 7f0bbdf11240 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:32:59.832 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:00.833 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:02.789 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:03.791 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:04.076 7f0bbddbb390 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:33:04.126 7f0bbddbb390 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:33:04.792 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:04.824 7f0bbdf11240 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:33:04.825 7f0bbdf11240 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:33:04.825 7f0bbdf11240 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:33:04.825 7f0bbdf11240 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:33:04.825 7f0bbdf11240 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:33:04.825 7f0bbdf11240 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:33:04.826 7f0bbdf11240 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:33:04.826 7f0bbdf11240 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:33:04.826 7f0bbdf11240 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:33:04.826 7f0bbdf11240 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:33:04.826 7f0bbdf11240 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:33:04.826 7f0bbdf11240 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:33:04.827 7f0bbdf11240 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:33:05.794 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:06.795 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:07.796 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:08.798 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:08.825 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:33:09.800 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:10.802 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:11.804 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:12.805 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:13.807 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:13.827 7f0bbd819e10 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:33:13.827 7f0bbd819e10 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:33:13.827 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:33:13.828 7f0bbd819e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:33:13.828 7f0bbd819e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:33:13.828 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:33:13.828 7f0bbd819e10 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:33:13.829 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:33:13.829 7f0bbd819e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:33:13.829 7f0bbd819e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:33:13.829 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:33:13.829 7f0bbd819e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:33:13.829 7f0bbd819e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:33:14.809 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:15.810 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:16.812 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:17.814 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:17.828 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:33:18.816 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:19.818 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:20.820 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:21.821 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:22.822 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:22.830 7f0bbd819e10 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:33:22.830 7f0bbd819e10 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:33:22.830 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:33:22.830 7f0bbd819e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:33:22.830 7f0bbd819e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:33:22.830 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:33:22.831 7f0bbd819e10 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:33:22.831 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:33:22.831 7f0bbd819e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:33:22.831 7f0bbd819e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:33:22.831 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:33:22.831 7f0bbd819e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:33:22.832 7f0bbd819e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:33:23.823 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:24.825 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:25.826 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:26.828 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:26.830 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:33:27.831 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:28.832 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:29.834 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:30.835 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:31.832 7f0bbd819e10 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:33:31.832 7f0bbd819e10 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:33:31.832 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:33:31.833 7f0bbd819e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:33:31.833 7f0bbd819e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:33:31.833 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:33:31.833 7f0bbd819e10 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:33:31.833 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:33:31.834 7f0bbd819e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:33:31.834 7f0bbd819e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:33:31.834 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:33:31.834 7f0bbd819e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:33:31.834 7f0bbd819e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:33:32.790 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:33.791 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:34.793 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:35.795 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:35.832 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:33:36.796 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:37.798 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:38.800 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:39.801 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:40.803 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:40.834 7f0bbd819e10 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:33:40.834 7f0bbd819e10 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:33:40.834 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:33:40.834 7f0bbd819e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:33:40.834 7f0bbd819e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:33:40.834 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:33:40.835 7f0bbd819e10 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:33:40.835 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:33:40.835 7f0bbd819e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:33:40.835 7f0bbd819e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:33:40.835 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:33:40.836 7f0bbd819e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:33:40.836 7f0bbd819e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:33:41.805 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:42.807 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:43.808 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:44.810 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:44.834 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:33:45.812 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:46.813 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:47.815 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:48.816 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:49.817 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:49.836 7f0bbd819e10 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:33:49.836 7f0bbd819e10 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:33:49.836 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:33:49.837 7f0bbd819e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:33:49.837 7f0bbd819e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:33:49.837 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:33:49.837 7f0bbd819e10 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:33:49.837 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:33:49.838 7f0bbd819e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:33:49.838 7f0bbd819e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:33:49.838 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:33:49.838 7f0bbd819e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:33:49.838 7f0bbd819e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:33:50.819 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:51.820 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:52.822 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:53.823 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:53.836 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:33:54.046 7f0bbdd00588 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:33:54.102 7f0bbdd00588 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:33:54.824 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:55.826 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:56.828 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:57.829 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:58.830 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:33:58.839 7f0bbd819e10 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:33:58.839 7f0bbd819e10 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:33:58.839 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:33:58.839 7f0bbd819e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:33:58.839 7f0bbd819e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:33:58.839 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:33:58.840 7f0bbd819e10 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:33:58.840 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:33:58.840 7f0bbd819e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:33:58.840 7f0bbd819e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:33:58.840 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:33:58.840 7f0bbd819e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:33:58.840 7f0bbd819e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:33:59.832 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:00.833 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:02.791 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:02.839 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:34:03.793 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:04.795 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:05.797 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:06.798 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:07.799 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:07.841 7f0bbd819e10 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:34:07.841 7f0bbd819e10 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:34:07.841 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:34:07.842 7f0bbd819e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:34:07.842 7f0bbd819e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:34:07.842 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:34:07.842 7f0bbd819e10 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:34:07.842 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:34:07.843 7f0bbd819e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:34:07.843 7f0bbd819e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:34:07.843 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:34:07.843 7f0bbd819e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:34:07.843 7f0bbd819e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:34:08.802 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:09.804 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:10.805 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:11.807 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:11.842 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:34:12.809 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:13.811 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:14.813 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:15.815 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:16.817 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:16.844 7f0bbd819e10 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:34:16.845 7f0bbd819e10 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:34:16.845 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:34:16.845 7f0bbd819e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:34:16.845 7f0bbd819e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:34:16.845 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:34:16.846 7f0bbd819e10 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:34:16.846 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:34:16.846 7f0bbd819e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:34:16.846 7f0bbd819e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:34:16.846 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:34:16.846 7f0bbd819e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:34:16.847 7f0bbd819e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:34:17.819 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:18.820 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:19.822 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:20.823 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:20.845 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:34:21.825 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:22.827 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:23.829 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:24.830 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:25.831 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:25.847 7f0bbd819e10 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:34:25.847 7f0bbd819e10 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:34:25.847 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:34:25.848 7f0bbd819e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:34:25.848 7f0bbd819e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:34:25.848 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:34:25.848 7f0bbd819e10 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:34:25.849 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:34:25.849 7f0bbd819e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:34:25.849 7f0bbd819e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:34:25.849 7f0bbd819e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:34:25.849 7f0bbd819e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:34:25.849 7f0bbd819e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:34:26.833 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:27.834 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:28.836 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:29.838 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:29.848 7f0bbd83cd68 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:34:30.840 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:32.793 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:33.794 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:34.796 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:34.849 7f0bbd83cd68 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:34:34.850 7f0bbd83cd68 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:34:34.850 7f0bbd83cd68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:34:34.850 7f0bbd83cd68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:34:34.850 7f0bbd83cd68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:34:34.850 7f0bbd83cd68 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:34:34.851 7f0bbd83cd68 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:34:34.851 7f0bbd83cd68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:34:34.851 7f0bbd83cd68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:34:34.851 7f0bbd83cd68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:34:34.851 7f0bbd83cd68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:34:34.851 7f0bbd83cd68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:34:34.851 7f0bbd83cd68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:34:35.797 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:36.798 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:37.799 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:38.801 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:38.850 7f0bbd83cd68 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:34:39.804 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:40.805 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:41.807 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:42.808 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:43.810 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:43.852 7f0bbd83cd68 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:34:43.853 7f0bbd83cd68 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:34:43.853 7f0bbd83cd68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:34:43.853 7f0bbd83cd68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:34:43.853 7f0bbd83cd68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:34:43.853 7f0bbd83cd68 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:34:43.854 7f0bbd83cd68 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:34:43.854 7f0bbd83cd68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:34:43.854 7f0bbd83cd68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:34:43.854 7f0bbd83cd68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:34:43.854 7f0bbd83cd68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:34:43.854 7f0bbd83cd68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:34:43.854 7f0bbd83cd68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:34:44.811 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:45.813 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:46.814 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:47.816 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:47.853 7f0bbd83cd68 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:34:48.818 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:49.819 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:50.820 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:51.822 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:52.823 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:52.855 7f0bbd83cd68 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:34:52.855 7f0bbd83cd68 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:34:52.855 7f0bbd83cd68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:34:52.856 7f0bbd83cd68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:34:52.856 7f0bbd83cd68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:34:52.856 7f0bbd83cd68 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:34:52.856 7f0bbd83cd68 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:34:52.856 7f0bbd83cd68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:34:52.856 7f0bbd83cd68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:34:52.856 7f0bbd83cd68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:34:52.857 7f0bbd83cd68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:34:52.857 7f0bbd83cd68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:34:52.857 7f0bbd83cd68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:34:53.825 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:54.827 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:55.830 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:56.832 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:56.858 7f0bbd83cd68 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:34:57.835 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:58.838 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:34:59.839 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:00.841 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:01.860 7f0bbd83cd68 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:35:01.860 7f0bbd83cd68 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:35:01.860 7f0bbd83cd68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:35:01.861 7f0bbd83cd68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:35:01.861 7f0bbd83cd68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:35:01.861 7f0bbd83cd68 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:35:01.862 7f0bbd83cd68 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:35:01.862 7f0bbd83cd68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:35:01.862 7f0bbd83cd68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:35:01.863 7f0bbd83cd68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:35:01.864 7f0bbd83cd68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:35:01.865 7f0bbd83cd68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:35:01.865 7f0bbd83cd68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:35:02.796 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:03.797 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:04.799 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:05.801 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:05.860 7f0bbdd00a20 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:35:06.802 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:07.804 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:08.806 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:09.808 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:10.809 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:10.863 7f0bbdd00a20 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:35:10.863 7f0bbdd00a20 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:35:10.863 7f0bbdd00a20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:35:10.863 7f0bbdd00a20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:35:10.863 7f0bbdd00a20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:35:10.863 7f0bbdd00a20 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:35:10.864 7f0bbdd00a20 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:35:10.864 7f0bbdd00a20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:35:10.864 7f0bbdd00a20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:35:10.864 7f0bbdd00a20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:35:10.864 7f0bbdd00a20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:35:10.864 7f0bbdd00a20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:35:10.864 7f0bbdd00a20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:35:11.811 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:12.812 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:13.814 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:14.816 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:14.863 7f0bbdd00a20 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:35:15.818 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:16.820 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:17.822 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:18.824 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:19.825 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:19.866 7f0bbdd00a20 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:35:19.866 7f0bbdd00a20 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:35:19.866 7f0bbdd00a20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:35:19.867 7f0bbdd00a20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:35:19.867 7f0bbdd00a20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:35:19.867 7f0bbdd00a20 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:35:19.868 7f0bbdd00a20 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:35:19.868 7f0bbdd00a20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:35:19.868 7f0bbdd00a20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:35:19.868 7f0bbdd00a20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:35:19.868 7f0bbdd00a20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:35:19.868 7f0bbdd00a20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:35:19.868 7f0bbdd00a20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:35:20.827 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:21.828 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:22.830 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:23.831 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:23.866 7f0bbdd00a20 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:35:24.834 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:25.836 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:26.837 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:27.838 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:28.839 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:28.867 7f0bbdd00a20 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:35:28.868 7f0bbdd00a20 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:35:28.868 7f0bbdd00a20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:35:28.868 7f0bbdd00a20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:35:28.868 7f0bbdd00a20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:35:28.868 7f0bbdd00a20 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:35:28.869 7f0bbdd00a20 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:35:28.869 7f0bbdd00a20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:35:28.869 7f0bbdd00a20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:35:28.869 7f0bbdd00a20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:35:28.869 7f0bbdd00a20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:35:28.869 7f0bbdd00a20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:35:28.869 7f0bbdd00a20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:35:29.841 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:30.842 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:32.798 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:32.868 7f0bbd9c7eb8 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:35:33.800 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:34.801 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:35.803 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:36.805 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:37.807 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:37.870 7f0bbd9c7eb8 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:35:37.871 7f0bbd9c7eb8 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:35:37.871 7f0bbd9c7eb8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:35:37.871 7f0bbd9c7eb8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:35:37.871 7f0bbd9c7eb8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:35:37.871 7f0bbd9c7eb8 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:35:37.872 7f0bbd9c7eb8 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:35:37.872 7f0bbd9c7eb8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:35:37.872 7f0bbd9c7eb8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:35:37.872 7f0bbd9c7eb8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:35:37.872 7f0bbd9c7eb8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:35:37.872 7f0bbd9c7eb8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:35:37.872 7f0bbd9c7eb8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:35:38.808 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:39.809 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:40.810 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:41.812 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:41.871 7f0bbd9c7eb8 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:35:42.814 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:43.816 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:44.818 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:45.819 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:46.821 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:46.873 7f0bbd9c7eb8 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:35:46.873 7f0bbd9c7eb8 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:35:46.873 7f0bbd9c7eb8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:35:46.874 7f0bbd9c7eb8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:35:46.874 7f0bbd9c7eb8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:35:46.874 7f0bbd9c7eb8 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:35:46.874 7f0bbd9c7eb8 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:35:46.874 7f0bbd9c7eb8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:35:46.875 7f0bbd9c7eb8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:35:46.875 7f0bbd9c7eb8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:35:46.875 7f0bbd9c7eb8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:35:46.875 7f0bbd9c7eb8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:35:46.875 7f0bbd9c7eb8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:35:47.823 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:48.824 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:49.826 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:50.828 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:50.874 7f0bbd9c7eb8 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:35:51.831 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:52.833 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:53.834 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:54.836 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:55.838 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:55.876 7f0bbd9c7eb8 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:35:55.876 7f0bbd9c7eb8 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:35:55.877 7f0bbd9c7eb8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:35:55.877 7f0bbd9c7eb8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:35:55.877 7f0bbd9c7eb8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:35:55.877 7f0bbd9c7eb8 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:35:55.877 7f0bbd9c7eb8 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:35:55.878 7f0bbd9c7eb8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:35:55.878 7f0bbd9c7eb8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:35:55.878 7f0bbd9c7eb8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:35:55.878 7f0bbd9c7eb8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:35:55.878 7f0bbd9c7eb8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:35:55.878 7f0bbd9c7eb8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:35:56.839 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:57.840 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:58.841 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:59.842 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:35:59.877 7f0bbd9c7eb8 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:36:00.845 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:02.798 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:03.799 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:04.801 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:04.880 7f0bbd9c7eb8 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:36:04.880 7f0bbd9c7eb8 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:36:04.880 7f0bbd9c7eb8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:36:04.880 7f0bbd9c7eb8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:36:04.880 7f0bbd9c7eb8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:36:04.880 7f0bbd9c7eb8 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:36:04.881 7f0bbd9c7eb8 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:36:04.881 7f0bbd9c7eb8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:36:04.881 7f0bbd9c7eb8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:36:04.881 7f0bbd9c7eb8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:36:04.881 7f0bbd9c7eb8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:36:04.881 7f0bbd9c7eb8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:36:04.881 7f0bbd9c7eb8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:36:05.802 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:06.803 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:07.805 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:08.806 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:08.880 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:36:09.808 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:10.809 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:11.201 7f0bbd9bef60 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:36:11.259 7f0bbd9bef60 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:36:11.811 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:12.813 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:13.814 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:13.882 7f0bbd8f0e10 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:36:13.882 7f0bbd8f0e10 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:36:13.882 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:36:13.883 7f0bbd8f0e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:36:13.883 7f0bbd8f0e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:36:13.883 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:36:13.883 7f0bbd8f0e10 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:36:13.884 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:36:13.884 7f0bbd8f0e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:36:13.884 7f0bbd8f0e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:36:13.884 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:36:13.884 7f0bbd8f0e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:36:13.884 7f0bbd8f0e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:36:14.815 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:15.817 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:16.818 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:17.821 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:17.882 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:36:18.822 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:19.824 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:20.824 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:21.826 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:22.828 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:22.885 7f0bbd8f0e10 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:36:22.885 7f0bbd8f0e10 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:36:22.885 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:36:22.886 7f0bbd8f0e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:36:22.886 7f0bbd8f0e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:36:22.886 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:36:22.887 7f0bbd8f0e10 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:36:22.887 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:36:22.887 7f0bbd8f0e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:36:22.887 7f0bbd8f0e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:36:22.887 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:36:22.887 7f0bbd8f0e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:36:22.887 7f0bbd8f0e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:36:23.830 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:24.832 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:25.833 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:26.834 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:26.885 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:36:27.835 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:28.837 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:29.839 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:30.841 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:31.888 7f0bbd8f0e10 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:36:31.888 7f0bbd8f0e10 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:36:31.888 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:36:31.889 7f0bbd8f0e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:36:31.889 7f0bbd8f0e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:36:31.889 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:36:31.889 7f0bbd8f0e10 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:36:31.889 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:36:31.889 7f0bbd8f0e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:36:31.890 7f0bbd8f0e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:36:31.890 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:36:31.890 7f0bbd8f0e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:36:31.890 7f0bbd8f0e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:36:32.799 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:33.801 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:34.803 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:35.805 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:35.888 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:36:36.806 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:37.807 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:38.809 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:39.810 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:40.811 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:40.891 7f0bbd8f0e10 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:36:40.891 7f0bbd8f0e10 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:36:40.891 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:36:40.891 7f0bbd8f0e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:36:40.891 7f0bbd8f0e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:36:40.892 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:36:40.892 7f0bbd8f0e10 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:36:40.892 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:36:40.892 7f0bbd8f0e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:36:40.892 7f0bbd8f0e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:36:40.892 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:36:40.893 7f0bbd8f0e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:36:40.893 7f0bbd8f0e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:36:41.813 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:42.815 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:43.816 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:44.817 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:44.891 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:36:45.819 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:46.821 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:47.822 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:48.824 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:49.825 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:49.893 7f0bbd8f0e10 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:36:49.893 7f0bbd8f0e10 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:36:49.894 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:36:49.894 7f0bbd8f0e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:36:49.894 7f0bbd8f0e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:36:49.894 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:36:49.895 7f0bbd8f0e10 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:36:49.895 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:36:49.895 7f0bbd8f0e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:36:49.895 7f0bbd8f0e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:36:49.895 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:36:49.895 7f0bbd8f0e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:36:49.895 7f0bbd8f0e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:36:50.827 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:51.828 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:52.830 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:53.832 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:53.894 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:36:54.834 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:55.835 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:56.837 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:57.838 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:58.839 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:36:58.896 7f0bbd8f0e10 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:36:58.896 7f0bbd8f0e10 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:36:58.896 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:36:58.897 7f0bbd8f0e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:36:58.897 7f0bbd8f0e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:36:58.897 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:36:58.897 7f0bbd8f0e10 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:36:58.897 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:36:58.898 7f0bbd8f0e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:36:58.898 7f0bbd8f0e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:36:58.898 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:36:58.898 7f0bbd8f0e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:36:58.898 7f0bbd8f0e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:36:59.841 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:00.842 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:02.802 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:02.896 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:37:03.804 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:04.128 7f0bbd8542e8 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:37:04.182 7f0bbd8542e8 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:37:04.806 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:05.808 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:06.809 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:07.810 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:07.899 7f0bbd8f0e10 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:37:07.899 7f0bbd8f0e10 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:37:07.899 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:37:07.900 7f0bbd8f0e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:37:07.900 7f0bbd8f0e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:37:07.900 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:37:07.901 7f0bbd8f0e10 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:37:07.901 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:37:07.901 7f0bbd8f0e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:37:07.901 7f0bbd8f0e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:37:07.901 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:37:07.901 7f0bbd8f0e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:37:07.901 7f0bbd8f0e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:37:08.812 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:09.814 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:10.816 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:11.818 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:11.900 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:37:12.819 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:13.820 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:14.822 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:15.535 7f0bbd9ff6d8 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:37:15.586 7f0bbd90da20 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:37:15.823 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:16.825 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:16.902 7f0bbd8f0e10 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:37:16.902 7f0bbd8f0e10 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:37:16.902 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:37:16.902 7f0bbd8f0e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:37:16.902 7f0bbd8f0e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:37:16.903 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:37:16.903 7f0bbd8f0e10 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:37:16.903 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:37:16.903 7f0bbd8f0e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:37:16.903 7f0bbd8f0e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:37:16.903 7f0bbd8f0e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:37:16.904 7f0bbd8f0e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:37:16.904 7f0bbd8f0e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:37:17.826 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:18.828 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:19.830 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:20.832 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:20.902 7f0bbddc9198 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:37:21.834 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:22.836 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:23.838 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:24.839 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:25.841 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:25.904 7f0bbddc9198 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:37:25.905 7f0bbddc9198 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:37:25.905 7f0bbddc9198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:37:25.905 7f0bbddc9198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:37:25.905 7f0bbddc9198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:37:25.905 7f0bbddc9198 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:37:25.906 7f0bbddc9198 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:37:25.906 7f0bbddc9198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:37:25.906 7f0bbddc9198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:37:25.906 7f0bbddc9198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:37:25.906 7f0bbddc9198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:37:25.906 7f0bbddc9198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:37:25.907 7f0bbddc9198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:37:26.842 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:27.844 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:28.846 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:29.848 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:29.905 7f0bbddc9198 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:37:30.850 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:32.801 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:33.803 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:34.804 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:34.907 7f0bbddc9198 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:37:34.907 7f0bbddc9198 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:37:34.907 7f0bbddc9198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:37:34.908 7f0bbddc9198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:37:34.908 7f0bbddc9198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:37:34.908 7f0bbddc9198 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:37:34.908 7f0bbddc9198 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:37:34.908 7f0bbddc9198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:37:34.909 7f0bbddc9198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:37:34.909 7f0bbddc9198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:37:34.909 7f0bbddc9198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:37:34.909 7f0bbddc9198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:37:34.909 7f0bbddc9198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:37:35.806 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:36.808 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:37.810 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:38.812 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:38.907 7f0bbd9ad4e0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:37:39.813 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:40.814 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:41.816 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:42.818 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:43.819 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:43.909 7f0bbd9ad4e0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:37:43.910 7f0bbd9ad4e0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:37:43.910 7f0bbd9ad4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:37:43.910 7f0bbd9ad4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:37:43.910 7f0bbd9ad4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:37:43.910 7f0bbd9ad4e0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:37:43.911 7f0bbd9ad4e0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:37:43.911 7f0bbd9ad4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:37:43.911 7f0bbd9ad4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:37:43.911 7f0bbd9ad4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:37:43.911 7f0bbd9ad4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:37:43.911 7f0bbd9ad4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:37:43.911 7f0bbd9ad4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:37:44.820 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:45.822 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:46.824 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:47.825 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:47.910 7f0bbd9ad4e0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:37:48.827 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:49.829 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:50.830 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:51.831 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:52.832 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:52.912 7f0bbd9ad4e0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:37:52.912 7f0bbd9ad4e0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:37:52.913 7f0bbd9ad4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:37:52.913 7f0bbd9ad4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:37:52.914 7f0bbd9ad4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:37:52.914 7f0bbd9ad4e0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:37:52.915 7f0bbd9ad4e0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:37:52.915 7f0bbd9ad4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:37:52.915 7f0bbd9ad4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:37:52.915 7f0bbd9ad4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:37:52.915 7f0bbd9ad4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:37:52.915 7f0bbd9ad4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:37:52.916 7f0bbd9ad4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:37:53.834 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:54.836 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:55.837 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:56.839 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:56.912 7f0bbd9ad4e0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:37:57.841 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:58.842 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:37:59.843 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:00.845 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:01.914 7f0bbd9ad4e0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:38:01.914 7f0bbd9ad4e0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:38:01.914 7f0bbd9ad4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:38:01.915 7f0bbd9ad4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:38:01.915 7f0bbd9ad4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:38:01.915 7f0bbd9ad4e0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:38:01.915 7f0bbd9ad4e0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:38:01.916 7f0bbd9ad4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:38:01.916 7f0bbd9ad4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:38:01.916 7f0bbd9ad4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:38:01.916 7f0bbd9ad4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:38:01.916 7f0bbd9ad4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:38:01.916 7f0bbd9ad4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:38:02.802 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:03.803 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:04.805 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:05.806 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:05.915 7f0bbd9ad4e0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:38:06.808 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:07.808 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:08.810 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:09.811 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:10.813 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:10.917 7f0bbd9ad4e0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:38:10.917 7f0bbd9ad4e0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:38:10.917 7f0bbd9ad4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:38:10.917 7f0bbd9ad4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:38:10.917 7f0bbd9ad4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:38:10.917 7f0bbd9ad4e0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:38:10.918 7f0bbd9ad4e0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:38:10.918 7f0bbd9ad4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:38:10.918 7f0bbd9ad4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:38:10.918 7f0bbd9ad4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:38:10.919 7f0bbd9ad4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:38:10.919 7f0bbd9ad4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:38:10.919 7f0bbd9ad4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:38:11.814 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:12.816 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:13.818 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:14.819 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:14.917 7f0bbd9ad4e0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:38:15.822 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:16.823 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:17.825 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:18.827 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:19.829 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:19.920 7f0bbd9ad4e0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:38:19.920 7f0bbd9ad4e0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:38:19.920 7f0bbd9ad4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:38:19.920 7f0bbd9ad4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:38:19.920 7f0bbd9ad4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:38:19.921 7f0bbd9ad4e0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:38:19.921 7f0bbd9ad4e0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:38:19.921 7f0bbd9ad4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:38:19.921 7f0bbd9ad4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:38:19.921 7f0bbd9ad4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:38:19.922 7f0bbd9ad4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:38:19.922 7f0bbd9ad4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:38:19.922 7f0bbd9ad4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:38:20.830 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:21.831 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:22.832 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:23.834 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:23.921 7f0bbd9ad4e0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:38:24.836 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:25.837 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:26.839 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:27.841 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:28.842 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:28.923 7f0bbd9ad4e0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:38:28.923 7f0bbd9ad4e0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:38:28.923 7f0bbd9ad4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:38:28.923 7f0bbd9ad4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:38:28.924 7f0bbd9ad4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:38:28.924 7f0bbd9ad4e0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:38:28.924 7f0bbd9ad4e0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:38:28.924 7f0bbd9ad4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:38:28.924 7f0bbd9ad4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:38:28.924 7f0bbd9ad4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:38:28.924 7f0bbd9ad4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:38:28.925 7f0bbd9ad4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:38:28.925 7f0bbd9ad4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:38:29.844 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:30.846 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:32.803 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:32.925 7f0bbd91e438 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:38:33.804 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:34.806 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:35.807 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:36.808 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:37.810 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:37.927 7f0bbd91e438 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:38:37.927 7f0bbd91e438 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:38:37.927 7f0bbd91e438 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:38:37.928 7f0bbd91e438 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:38:37.928 7f0bbd91e438 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:38:37.928 7f0bbd91e438 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:38:37.928 7f0bbd91e438 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:38:37.928 7f0bbd91e438 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:38:37.929 7f0bbd91e438 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:38:37.929 7f0bbd91e438 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:38:37.929 7f0bbd91e438 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:38:37.929 7f0bbd91e438 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:38:37.929 7f0bbd91e438 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:38:38.812 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:39.813 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:40.814 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:41.816 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:41.928 7f0bbd91e438 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:38:42.818 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:43.820 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:44.821 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:45.823 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:46.824 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:46.930 7f0bbd91e438 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:38:46.930 7f0bbd91e438 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:38:46.930 7f0bbd91e438 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:38:46.931 7f0bbd91e438 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:38:46.931 7f0bbd91e438 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:38:46.931 7f0bbd91e438 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:38:46.931 7f0bbd91e438 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:38:46.931 7f0bbd91e438 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:38:46.932 7f0bbd91e438 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:38:46.932 7f0bbd91e438 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:38:46.932 7f0bbd91e438 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:38:46.932 7f0bbd91e438 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:38:46.932 7f0bbd91e438 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:38:47.825 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:48.827 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:49.829 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:50.831 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:50.931 7f0bbd91e438 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:38:51.834 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:52.835 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:53.837 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:54.838 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:55.840 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:55.933 7f0bbd91e438 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:38:55.933 7f0bbd91e438 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:38:55.934 7f0bbd91e438 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:38:55.934 7f0bbd91e438 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:38:55.934 7f0bbd91e438 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:38:55.934 7f0bbd91e438 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:38:55.935 7f0bbd91e438 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:38:55.935 7f0bbd91e438 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:38:55.935 7f0bbd91e438 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:38:55.935 7f0bbd91e438 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:38:55.935 7f0bbd91e438 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:38:55.935 7f0bbd91e438 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:38:55.935 7f0bbd91e438 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:38:56.841 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:57.843 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:58.844 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:59.846 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:38:59.933 7f0bbd91e438 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:39:00.848 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:02.804 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:03.187 7f0bbd7f82e8 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:39:03.238 7f0bbd7f82e8 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:39:03.806 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:04.807 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:04.936 7f0bbd91e438 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:39:04.936 7f0bbd91e438 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:39:04.936 7f0bbd91e438 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:39:04.936 7f0bbd91e438 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:39:04.936 7f0bbd91e438 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:39:04.936 7f0bbd91e438 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:39:04.937 7f0bbd91e438 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:39:04.937 7f0bbd91e438 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:39:04.937 7f0bbd91e438 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:39:04.937 7f0bbd91e438 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:39:04.937 7f0bbd91e438 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:39:04.937 7f0bbd91e438 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:39:04.937 7f0bbd91e438 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:39:05.808 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:06.809 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:07.810 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:08.812 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:08.936 7f0bbd9be198 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:39:09.814 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:10.816 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:11.818 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:12.819 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:13.821 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:13.938 7f0bbd9be198 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:39:13.939 7f0bbd9be198 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:39:13.939 7f0bbd9be198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:39:13.939 7f0bbd9be198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:39:13.939 7f0bbd9be198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:39:13.939 7f0bbd9be198 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:39:13.940 7f0bbd9be198 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:39:13.940 7f0bbd9be198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:39:13.940 7f0bbd9be198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:39:13.940 7f0bbd9be198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:39:13.940 7f0bbd9be198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:39:13.940 7f0bbd9be198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:39:13.940 7f0bbd9be198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:39:14.822 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:15.823 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:16.824 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:17.826 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:17.939 7f0bbd9be198 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:39:18.828 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:19.830 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:20.832 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:21.833 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:22.835 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:22.941 7f0bbd9be198 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:39:22.941 7f0bbd9be198 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:39:22.941 7f0bbd9be198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:39:22.941 7f0bbd9be198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:39:22.942 7f0bbd9be198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:39:22.942 7f0bbd9be198 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:39:22.942 7f0bbd9be198 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:39:22.942 7f0bbd9be198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:39:22.943 7f0bbd9be198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:39:22.943 7f0bbd9be198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:39:22.943 7f0bbd9be198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:39:22.943 7f0bbd9be198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:39:22.943 7f0bbd9be198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:39:23.837 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:24.842 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:25.844 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:26.846 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:26.947 7f0bbd9be198 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:39:27.848 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:28.849 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:29.851 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:30.852 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:31.953 7f0bbd9be198 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:39:31.953 7f0bbd9be198 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:39:31.953 7f0bbd9be198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:39:31.954 7f0bbd9be198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:39:31.954 7f0bbd9be198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:39:31.954 7f0bbd9be198 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:39:31.955 7f0bbd9be198 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:39:31.955 7f0bbd9be198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:39:31.955 7f0bbd9be198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:39:31.955 7f0bbd9be198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:39:31.955 7f0bbd9be198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:39:31.955 7f0bbd9be198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:39:31.956 7f0bbd9be198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:39:32.806 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:33.808 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:34.810 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:35.813 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:35.953 7f0bbd7b4d68 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:39:36.814 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:37.822 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:38.824 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:39.825 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:40.826 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:40.956 7f0bbd7b4d68 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:39:40.956 7f0bbd7b4d68 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:39:40.956 7f0bbd7b4d68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:39:40.956 7f0bbd7b4d68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:39:40.956 7f0bbd7b4d68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:39:40.956 7f0bbd7b4d68 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:39:40.957 7f0bbd7b4d68 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:39:40.957 7f0bbd7b4d68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:39:40.957 7f0bbd7b4d68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:39:40.957 7f0bbd7b4d68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:39:40.957 7f0bbd7b4d68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:39:40.957 7f0bbd7b4d68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:39:40.958 7f0bbd7b4d68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:39:41.828 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:42.830 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:43.831 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:44.833 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:44.957 7f0bbd7b4d68 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:39:45.835 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:46.837 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:47.839 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:48.840 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:49.841 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:49.959 7f0bbd7b4d68 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:39:49.959 7f0bbd7b4d68 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:39:49.960 7f0bbd7b4d68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:39:49.960 7f0bbd7b4d68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:39:49.960 7f0bbd7b4d68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:39:49.960 7f0bbd7b4d68 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:39:49.961 7f0bbd7b4d68 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:39:49.961 7f0bbd7b4d68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:39:49.961 7f0bbd7b4d68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:39:49.961 7f0bbd7b4d68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:39:49.961 7f0bbd7b4d68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:39:49.961 7f0bbd7b4d68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:39:49.961 7f0bbd7b4d68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:39:50.844 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:51.845 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:52.847 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:53.849 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:53.960 7f0bbd7b4d68 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:39:54.851 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:55.853 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:56.855 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:57.857 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:58.858 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:39:58.961 7f0bbd7b4d68 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:39:58.962 7f0bbd7b4d68 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:39:58.962 7f0bbd7b4d68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:39:58.962 7f0bbd7b4d68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:39:58.962 7f0bbd7b4d68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:39:58.962 7f0bbd7b4d68 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:39:58.963 7f0bbd7b4d68 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:39:58.963 7f0bbd7b4d68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:39:58.963 7f0bbd7b4d68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:39:58.963 7f0bbd7b4d68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:39:58.963 7f0bbd7b4d68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:39:58.963 7f0bbd7b4d68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:39:58.963 7f0bbd7b4d68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:39:59.860 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:00.861 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:02.806 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:02.962 7f0bbd7b4d68 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:40:03.809 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:04.810 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:05.812 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:06.814 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:07.815 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:07.965 7f0bbd7b4d68 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:40:07.965 7f0bbd7b4d68 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:40:07.965 7f0bbd7b4d68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:40:07.965 7f0bbd7b4d68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:40:07.965 7f0bbd7b4d68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:40:07.965 7f0bbd7b4d68 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:40:07.966 7f0bbd7b4d68 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:40:07.966 7f0bbd7b4d68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:40:07.966 7f0bbd7b4d68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:40:07.966 7f0bbd7b4d68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:40:07.966 7f0bbd7b4d68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:40:07.966 7f0bbd7b4d68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:40:07.966 7f0bbd7b4d68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:40:08.817 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:09.819 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:10.821 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:11.823 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:11.965 7f0bbd7b4d68 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:40:12.825 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:13.827 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:14.829 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:15.831 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:16.833 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:16.968 7f0bbd7b4d68 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:40:16.968 7f0bbd7b4d68 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:40:16.968 7f0bbd7b4d68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:40:16.968 7f0bbd7b4d68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:40:16.969 7f0bbd7b4d68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:40:16.969 7f0bbd7b4d68 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:40:16.969 7f0bbd7b4d68 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:40:16.969 7f0bbd7b4d68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:40:16.970 7f0bbd7b4d68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:40:16.970 7f0bbd7b4d68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:40:16.970 7f0bbd7b4d68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:40:16.970 7f0bbd7b4d68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:40:16.970 7f0bbd7b4d68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:40:17.835 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:18.837 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:19.838 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:20.840 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:20.968 7f0bbd7b4d68 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:40:21.843 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:22.844 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:23.846 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:24.847 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:25.849 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:25.970 7f0bbd7b4d68 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:40:25.972 7f0bbd7b4d68 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:40:25.972 7f0bbd7b4d68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:40:25.972 7f0bbd7b4d68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:40:25.972 7f0bbd7b4d68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:40:25.972 7f0bbd7b4d68 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:40:25.973 7f0bbd7b4d68 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:40:25.973 7f0bbd7b4d68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:40:25.973 7f0bbd7b4d68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:40:25.973 7f0bbd7b4d68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:40:25.973 7f0bbd7b4d68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:40:25.973 7f0bbd7b4d68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:40:25.974 7f0bbd7b4d68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:40:26.851 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:27.853 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:28.854 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:29.856 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:29.970 7f0bbde2deb8 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:40:30.857 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:32.807 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:33.809 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:34.811 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:34.973 7f0bbde2deb8 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:40:34.973 7f0bbde2deb8 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:40:34.973 7f0bbde2deb8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:40:34.973 7f0bbde2deb8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:40:34.973 7f0bbde2deb8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:40:34.973 7f0bbde2deb8 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:40:34.974 7f0bbde2deb8 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:40:34.974 7f0bbde2deb8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:40:34.974 7f0bbde2deb8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:40:34.974 7f0bbde2deb8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:40:34.974 7f0bbde2deb8 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:40:34.975 7f0bbde2deb8 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:40:34.975 7f0bbde2deb8 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:40:35.812 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:36.814 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:37.815 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:38.817 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:38.973 7f0bbd7b4978 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:40:39.819 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:40.820 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:41.822 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:42.823 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:43.825 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:43.975 7f0bbd7b4978 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:40:43.976 7f0bbd7b4978 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:40:43.976 7f0bbd7b4978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:40:43.976 7f0bbd7b4978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:40:43.976 7f0bbd7b4978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:40:43.976 7f0bbd7b4978 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:40:43.977 7f0bbd7b4978 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:40:43.977 7f0bbd7b4978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:40:43.977 7f0bbd7b4978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:40:43.977 7f0bbd7b4978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:40:43.977 7f0bbd7b4978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:40:43.977 7f0bbd7b4978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:40:43.977 7f0bbd7b4978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:40:44.827 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:45.829 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:46.830 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:47.832 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:47.976 7f0bbd7b4978 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:40:48.834 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:49.836 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:50.837 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:51.839 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:52.841 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:52.978 7f0bbd7b4978 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:40:52.979 7f0bbd7b4978 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:40:52.979 7f0bbd7b4978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:40:52.979 7f0bbd7b4978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:40:52.979 7f0bbd7b4978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:40:52.979 7f0bbd7b4978 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:40:52.980 7f0bbd7b4978 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:40:52.980 7f0bbd7b4978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:40:52.980 7f0bbd7b4978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:40:52.980 7f0bbd7b4978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:40:52.980 7f0bbd7b4978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:40:52.980 7f0bbd7b4978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:40:52.980 7f0bbd7b4978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:40:53.842 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:54.843 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:55.845 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:56.848 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:56.980 7f0bbd7b4978 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:40:57.849 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:58.851 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:40:59.853 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:00.854 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:01.982 7f0bbd7b4978 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:41:01.982 7f0bbd7b4978 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:41:01.982 7f0bbd7b4978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:41:01.983 7f0bbd7b4978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:41:01.983 7f0bbd7b4978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:41:01.983 7f0bbd7b4978 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:41:01.983 7f0bbd7b4978 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:41:01.984 7f0bbd7b4978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:41:01.984 7f0bbd7b4978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:41:01.984 7f0bbd7b4978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:41:01.984 7f0bbd7b4978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:41:01.984 7f0bbd7b4978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:41:01.984 7f0bbd7b4978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:41:02.807 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:03.809 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:04.810 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:05.813 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:05.982 7f0bbd7b4978 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:41:06.814 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:07.815 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:08.817 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:09.818 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:10.820 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:10.985 7f0bbd7b4978 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:41:10.985 7f0bbd7b4978 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:41:10.985 7f0bbd7b4978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:41:10.985 7f0bbd7b4978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:41:10.985 7f0bbd7b4978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:41:10.985 7f0bbd7b4978 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:41:10.986 7f0bbd7b4978 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:41:10.986 7f0bbd7b4978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:41:10.986 7f0bbd7b4978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:41:10.986 7f0bbd7b4978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:41:10.987 7f0bbd7b4978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:41:10.987 7f0bbd7b4978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:41:10.987 7f0bbd7b4978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:41:11.821 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:12.823 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:13.825 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:14.827 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:14.985 7f0bbde2dc18 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:41:15.829 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:16.831 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:17.832 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:18.834 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:19.835 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:19.987 7f0bbde2dc18 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:41:19.987 7f0bbde2dc18 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:41:19.987 7f0bbde2dc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:41:19.988 7f0bbde2dc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:41:19.988 7f0bbde2dc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:41:19.988 7f0bbde2dc18 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:41:19.988 7f0bbde2dc18 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:41:19.988 7f0bbde2dc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:41:19.989 7f0bbde2dc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:41:19.989 7f0bbde2dc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:41:19.989 7f0bbde2dc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:41:19.989 7f0bbde2dc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:41:19.989 7f0bbde2dc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:41:20.837 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:21.838 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:22.840 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:23.842 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:23.988 7f0bbde2dc18 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:41:24.256 7f0bbd7b4eb8 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:41:24.312 7f0bbde2deb8 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:41:24.844 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:25.845 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:26.847 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:27.849 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:28.850 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:28.990 7f0bbde2dc18 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:41:28.990 7f0bbde2dc18 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:41:28.990 7f0bbde2dc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:41:28.990 7f0bbde2dc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:41:28.991 7f0bbde2dc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:41:28.991 7f0bbde2dc18 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:41:28.991 7f0bbde2dc18 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:41:28.991 7f0bbde2dc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:41:28.992 7f0bbde2dc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:41:28.992 7f0bbde2dc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:41:28.992 7f0bbde2dc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:41:28.992 7f0bbde2dc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:41:28.992 7f0bbde2dc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:41:29.852 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:30.854 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:32.809 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:32.990 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:41:33.811 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:34.812 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:35.814 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:36.815 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:37.817 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:37.993 7f0bbde2a828 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:41:37.993 7f0bbde2a828 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:41:37.993 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:41:37.993 7f0bbde2a828 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:41:37.993 7f0bbde2a828 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:41:37.993 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:41:37.994 7f0bbde2a828 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:41:37.994 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:41:37.994 7f0bbde2a828 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:41:37.994 7f0bbde2a828 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:41:37.994 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:41:37.994 7f0bbde2a828 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:41:37.994 7f0bbde2a828 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:41:38.819 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:39.821 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:40.822 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:41.824 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:41.993 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:41:42.826 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:43.828 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:44.830 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:45.831 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:46.833 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:46.995 7f0bbde2a828 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:41:46.995 7f0bbde2a828 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:41:46.995 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:41:46.996 7f0bbde2a828 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:41:46.996 7f0bbde2a828 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:41:46.996 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:41:46.996 7f0bbde2a828 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:41:46.996 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:41:46.996 7f0bbde2a828 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:41:46.997 7f0bbde2a828 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:41:46.997 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:41:46.997 7f0bbde2a828 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:41:46.997 7f0bbde2a828 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:41:47.835 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:48.837 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:49.838 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:50.840 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:50.996 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:41:51.842 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:52.844 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:53.845 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:54.847 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:55.849 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:55.998 7f0bbde2a828 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:41:55.998 7f0bbde2a828 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:41:55.998 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:41:55.998 7f0bbde2a828 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:41:55.998 7f0bbde2a828 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:41:55.998 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:41:55.999 7f0bbde2a828 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:41:55.999 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:41:55.999 7f0bbde2a828 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:41:55.999 7f0bbde2a828 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:41:55.999 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:41:55.999 7f0bbde2a828 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:41:56.000 7f0bbde2a828 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:41:56.850 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:57.851 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:58.853 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:59.855 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:41:59.998 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:42:00.857 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:02.811 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:03.812 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:04.814 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:05.000 7f0bbde2a828 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:42:05.000 7f0bbde2a828 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:42:05.000 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:42:05.000 7f0bbde2a828 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:42:05.000 7f0bbde2a828 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:42:05.000 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:42:05.001 7f0bbde2a828 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:42:05.001 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:42:05.001 7f0bbde2a828 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:42:05.001 7f0bbde2a828 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:42:05.001 7f0bbde2a828 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:42:05.001 7f0bbde2a828 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:42:05.002 7f0bbde2a828 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:42:05.815 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:06.817 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:07.818 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:08.820 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:09.000 7f0bc0b8cc18 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:42:09.822 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:10.823 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:11.825 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:12.826 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:13.827 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:14.002 7f0bc0b8cc18 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:42:14.002 7f0bc0b8cc18 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:42:14.002 7f0bc0b8cc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:42:14.002 7f0bc0b8cc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:42:14.003 7f0bc0b8cc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:42:14.003 7f0bc0b8cc18 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:42:14.003 7f0bc0b8cc18 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:42:14.003 7f0bc0b8cc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:42:14.003 7f0bc0b8cc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:42:14.003 7f0bc0b8cc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:42:14.004 7f0bc0b8cc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:42:14.004 7f0bc0b8cc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:42:14.004 7f0bc0b8cc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:42:14.829 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:15.830 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:16.832 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:17.834 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:18.002 7f0bc0b8cc18 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:42:18.836 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:19.837 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:20.839 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:21.841 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:22.842 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:23.005 7f0bc0b8cc18 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:42:23.005 7f0bc0b8cc18 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:42:23.005 7f0bc0b8cc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:42:23.006 7f0bc0b8cc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:42:23.006 7f0bc0b8cc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:42:23.006 7f0bc0b8cc18 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:42:23.006 7f0bc0b8cc18 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:42:23.006 7f0bc0b8cc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:42:23.007 7f0bc0b8cc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:42:23.007 7f0bc0b8cc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:42:23.007 7f0bc0b8cc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:42:23.007 7f0bc0b8cc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:42:23.007 7f0bc0b8cc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:42:23.844 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:24.845 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:25.847 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:26.849 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:27.006 7f0bc0b8cc18 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:42:27.850 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:28.852 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:29.854 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:30.856 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:32.008 7f0bc0b8cc18 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:42:32.008 7f0bc0b8cc18 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:42:32.009 7f0bc0b8cc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:42:32.009 7f0bc0b8cc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:42:32.009 7f0bc0b8cc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:42:32.010 7f0bc0b8cc18 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:42:32.011 7f0bc0b8cc18 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:42:32.011 7f0bc0b8cc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:42:32.011 7f0bc0b8cc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:42:32.011 7f0bc0b8cc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:42:32.011 7f0bc0b8cc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:42:32.011 7f0bc0b8cc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:42:32.011 7f0bc0b8cc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:42:32.811 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:33.812 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:34.814 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:35.816 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:36.008 7f0bc0b8cc18 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:42:36.817 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:37.818 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:38.820 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:38.982 7f0bbe547cc0 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:42:39.038 7f0bbe547cc0 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:42:39.821 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:40.823 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:41.010 7f0bc0b8cc18 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:42:41.010 7f0bc0b8cc18 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:42:41.010 7f0bc0b8cc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:42:41.011 7f0bc0b8cc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:42:41.011 7f0bc0b8cc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:42:41.011 7f0bc0b8cc18 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:42:41.011 7f0bc0b8cc18 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:42:41.011 7f0bc0b8cc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:42:41.011 7f0bc0b8cc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:42:41.012 7f0bc0b8cc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:42:41.012 7f0bc0b8cc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:42:41.012 7f0bc0b8cc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:42:41.012 7f0bc0b8cc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:42:41.826 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:42.827 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:43.829 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:44.831 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:45.011 7f0bc0b8cc18 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:42:45.833 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:46.835 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:47.837 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:48.838 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:49.840 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:49.919 7f0bbdf11e10 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:42:49.970 7f0bc1ea36d8 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:42:50.013 7f0bc0b8cc18 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:42:50.013 7f0bc0b8cc18 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:42:50.013 7f0bc0b8cc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:42:50.014 7f0bc0b8cc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:42:50.014 7f0bc0b8cc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:42:50.014 7f0bc0b8cc18 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:42:50.014 7f0bc0b8cc18 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:42:50.014 7f0bc0b8cc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:42:50.015 7f0bc0b8cc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:42:50.015 7f0bc0b8cc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:42:50.015 7f0bc0b8cc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:42:50.015 7f0bc0b8cc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:42:50.015 7f0bc0b8cc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:42:50.842 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:51.843 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:52.845 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:53.847 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:54.014 7f0bbddb5f60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:42:54.849 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:55.850 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:56.852 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:57.854 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:58.855 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:42:59.016 7f0bbddb5f60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:42:59.016 7f0bbddb5f60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:42:59.016 7f0bbddb5f60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:42:59.016 7f0bbddb5f60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:42:59.016 7f0bbddb5f60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:42:59.017 7f0bbddb5f60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:42:59.017 7f0bbddb5f60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:42:59.017 7f0bbddb5f60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:42:59.017 7f0bbddb5f60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:42:59.017 7f0bbddb5f60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:42:59.017 7f0bbddb5f60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:42:59.018 7f0bbddb5f60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:42:59.018 7f0bbddb5f60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:42:59.857 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:00.859 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:02.812 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:03.017 7f0bbddb5f60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:43:03.814 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:04.816 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:05.818 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:06.820 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:07.822 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:08.019 7f0bbddb5f60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:43:08.020 7f0bbddb5f60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:43:08.020 7f0bbddb5f60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:43:08.020 7f0bbddb5f60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:43:08.020 7f0bbddb5f60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:43:08.020 7f0bbddb5f60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:43:08.021 7f0bbddb5f60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:43:08.021 7f0bbddb5f60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:43:08.021 7f0bbddb5f60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:43:08.021 7f0bbddb5f60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:43:08.021 7f0bbddb5f60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:43:08.022 7f0bbddb5f60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:43:08.022 7f0bbddb5f60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:43:08.823 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:09.825 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:10.826 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:11.828 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:12.020 7f0bbddb5f60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:43:12.830 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:13.831 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:14.833 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:15.835 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:16.837 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:17.022 7f0bbddb5f60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:43:17.022 7f0bbddb5f60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:43:17.022 7f0bbddb5f60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:43:17.023 7f0bbddb5f60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:43:17.023 7f0bbddb5f60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:43:17.023 7f0bbddb5f60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:43:17.023 7f0bbddb5f60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:43:17.024 7f0bbddb5f60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:43:17.024 7f0bbddb5f60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:43:17.024 7f0bbddb5f60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:43:17.024 7f0bbddb5f60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:43:17.024 7f0bbddb5f60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:43:17.024 7f0bbddb5f60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:43:17.839 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:18.841 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:19.842 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:20.844 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:21.022 7f0bbddb5f60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:43:21.846 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:22.848 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:23.850 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:24.852 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:25.853 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:26.024 7f0bbddb5f60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:43:26.024 7f0bbddb5f60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:43:26.024 7f0bbddb5f60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:43:26.025 7f0bbddb5f60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:43:26.025 7f0bbddb5f60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:43:26.025 7f0bbddb5f60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:43:26.025 7f0bbddb5f60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:43:26.025 7f0bbddb5f60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:43:26.025 7f0bbddb5f60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:43:26.026 7f0bbddb5f60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:43:26.026 7f0bbddb5f60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:43:26.026 7f0bbddb5f60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:43:26.026 7f0bbddb5f60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:43:26.855 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:27.856 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:28.858 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:29.859 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:30.024 7f0bbddb5f60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:43:30.862 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:32.813 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:33.815 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:34.816 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:35.027 7f0bbddb5f60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:43:35.027 7f0bbddb5f60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:43:35.028 7f0bbddb5f60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:43:35.028 7f0bbddb5f60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:43:35.028 7f0bbddb5f60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:43:35.028 7f0bbddb5f60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:43:35.029 7f0bbddb5f60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:43:35.029 7f0bbddb5f60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:43:35.029 7f0bbddb5f60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:43:35.029 7f0bbddb5f60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:43:35.029 7f0bbddb5f60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:43:35.029 7f0bbddb5f60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:43:35.030 7f0bbddb5f60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:43:35.818 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:36.819 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:37.821 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:38.822 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:39.027 7f0bbd7aea20 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:43:39.825 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:40.825 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:41.828 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:42.829 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:43.831 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:44.029 7f0bbd7aea20 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:43:44.029 7f0bbd7aea20 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:43:44.029 7f0bbd7aea20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:43:44.029 7f0bbd7aea20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:43:44.029 7f0bbd7aea20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:43:44.029 7f0bbd7aea20 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:43:44.030 7f0bbd7aea20 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:43:44.030 7f0bbd7aea20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:43:44.030 7f0bbd7aea20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:43:44.031 7f0bbd7aea20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:43:44.031 7f0bbd7aea20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:43:44.031 7f0bbd7aea20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:43:44.031 7f0bbd7aea20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:43:44.832 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:45.834 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:46.835 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:47.836 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:48.029 7f0bbd7aea20 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:43:48.838 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:49.840 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:50.315 7f0bbd9bc198 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:43:50.365 7f0bbd826a20 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:43:50.842 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:51.842 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:52.844 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:53.032 7f0bbd7aea20 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:43:53.032 7f0bbd7aea20 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:43:53.032 7f0bbd7aea20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:43:53.033 7f0bbd7aea20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:43:53.033 7f0bbd7aea20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:43:53.033 7f0bbd7aea20 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:43:53.033 7f0bbd7aea20 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:43:53.033 7f0bbd7aea20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:43:53.034 7f0bbd7aea20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:43:53.034 7f0bbd7aea20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:43:53.034 7f0bbd7aea20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:43:53.034 7f0bbd7aea20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:43:53.034 7f0bbd7aea20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:43:53.846 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:54.847 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:55.848 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:56.850 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:57.033 7f0bbde3b978 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:43:57.852 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:58.853 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:43:59.855 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:00.856 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:02.035 7f0bbde3b978 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:44:02.035 7f0bbde3b978 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:44:02.036 7f0bbde3b978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:44:02.036 7f0bbde3b978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:44:02.036 7f0bbde3b978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:44:02.036 7f0bbde3b978 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:44:02.037 7f0bbde3b978 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:44:02.037 7f0bbde3b978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:44:02.037 7f0bbde3b978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:44:02.037 7f0bbde3b978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:44:02.037 7f0bbde3b978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:44:02.037 7f0bbde3b978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:44:02.037 7f0bbde3b978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:44:02.814 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:03.815 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:04.826 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:05.828 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:06.036 7f0bbde3b978 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:44:06.831 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:07.831 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:08.833 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:09.835 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:10.838 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:11.039 7f0bbde3b978 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:44:11.039 7f0bbde3b978 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:44:11.039 7f0bbde3b978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:44:11.041 7f0bbde3b978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:44:11.041 7f0bbde3b978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:44:11.041 7f0bbde3b978 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:44:11.042 7f0bbde3b978 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:44:11.042 7f0bbde3b978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:44:11.042 7f0bbde3b978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:44:11.042 7f0bbde3b978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:44:11.042 7f0bbde3b978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:44:11.042 7f0bbde3b978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:44:11.042 7f0bbde3b978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:44:11.849 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:12.851 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:13.854 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:14.856 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:15.041 7f0bbde3b978 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:44:15.858 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:16.859 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:17.861 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:18.863 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:19.864 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:20.043 7f0bbde3b978 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:44:20.043 7f0bbde3b978 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:44:20.043 7f0bbde3b978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:44:20.043 7f0bbde3b978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:44:20.044 7f0bbde3b978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:44:20.044 7f0bbde3b978 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:44:20.044 7f0bbde3b978 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:44:20.044 7f0bbde3b978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:44:20.044 7f0bbde3b978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:44:20.045 7f0bbde3b978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:44:20.045 7f0bbde3b978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:44:20.045 7f0bbde3b978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:44:20.045 7f0bbde3b978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:44:20.866 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:21.867 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:22.869 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:23.871 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:24.044 7f0bbde3b978 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:44:24.872 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:25.873 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:26.874 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:27.875 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:28.877 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:29.046 7f0bbde3b978 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:44:29.046 7f0bbde3b978 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:44:29.046 7f0bbde3b978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:44:29.046 7f0bbde3b978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:44:29.047 7f0bbde3b978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:44:29.047 7f0bbde3b978 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:44:29.047 7f0bbde3b978 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:44:29.047 7f0bbde3b978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:44:29.048 7f0bbde3b978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:44:29.048 7f0bbde3b978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:44:29.048 7f0bbde3b978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:44:29.048 7f0bbde3b978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:44:29.048 7f0bbde3b978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:44:29.879 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:30.880 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:32.815 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:33.046 7f0bbde3b978 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:44:33.817 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:34.818 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:35.820 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:36.822 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:37.824 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:38.049 7f0bbde3b978 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:44:38.049 7f0bbde3b978 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:44:38.049 7f0bbde3b978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:44:38.049 7f0bbde3b978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:44:38.049 7f0bbde3b978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:44:38.049 7f0bbde3b978 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:44:38.050 7f0bbde3b978 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:44:38.050 7f0bbde3b978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:44:38.050 7f0bbde3b978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:44:38.050 7f0bbde3b978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:44:38.050 7f0bbde3b978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:44:38.051 7f0bbde3b978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:44:38.051 7f0bbde3b978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:44:38.825 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:39.826 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:40.828 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:41.830 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:42.050 7f0bbdd00e10 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:44:42.831 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:43.832 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:44.834 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:45.836 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:46.837 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:47.052 7f0bbdd00e10 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:44:47.052 7f0bbdd00e10 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:44:47.052 7f0bbdd00e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:44:47.052 7f0bbdd00e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:44:47.052 7f0bbdd00e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:44:47.053 7f0bbdd00e10 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:44:47.053 7f0bbdd00e10 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:44:47.053 7f0bbdd00e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:44:47.053 7f0bbdd00e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:44:47.053 7f0bbdd00e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:44:47.053 7f0bbdd00e10 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:44:47.054 7f0bbdd00e10 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:44:47.054 7f0bbdd00e10 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:44:47.838 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:48.840 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:49.841 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:50.843 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:51.053 7f0bbd9ffd68 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:44:51.845 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:52.846 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:53.848 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:54.849 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:55.851 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:56.055 7f0bbd9ffd68 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:44:56.055 7f0bbd9ffd68 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:44:56.055 7f0bbd9ffd68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:44:56.055 7f0bbd9ffd68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:44:56.056 7f0bbd9ffd68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:44:56.056 7f0bbd9ffd68 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:44:56.056 7f0bbd9ffd68 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:44:56.056 7f0bbd9ffd68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:44:56.057 7f0bbd9ffd68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:44:56.057 7f0bbd9ffd68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:44:56.057 7f0bbd9ffd68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:44:56.057 7f0bbd9ffd68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:44:56.057 7f0bbd9ffd68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:44:56.853 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:57.855 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:58.856 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:44:59.858 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:00.056 7f0bbd9ffd68 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:45:00.860 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:02.816 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:03.817 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:04.819 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:05.058 7f0bbd9ffd68 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:45:05.058 7f0bbd9ffd68 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:45:05.058 7f0bbd9ffd68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:45:05.059 7f0bbd9ffd68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:45:05.059 7f0bbd9ffd68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:45:05.059 7f0bbd9ffd68 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:45:05.059 7f0bbd9ffd68 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:45:05.059 7f0bbd9ffd68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:45:05.060 7f0bbd9ffd68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:45:05.060 7f0bbd9ffd68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:45:05.060 7f0bbd9ffd68 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:45:05.060 7f0bbd9ffd68 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:45:05.060 7f0bbd9ffd68 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:45:05.820 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:06.822 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:07.824 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:08.826 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:09.058 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:45:09.827 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:10.829 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:11.831 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:12.832 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:13.833 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:14.061 7f0bbd8fa588 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:45:14.061 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:45:14.062 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:45:14.062 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:45:14.062 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:45:14.062 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:45:14.063 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:45:14.063 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:45:14.063 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:45:14.063 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:45:14.063 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:45:14.063 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:45:14.064 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:45:14.835 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:15.837 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:16.839 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:17.841 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:18.062 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:45:18.843 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:19.845 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:20.846 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:21.848 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:22.849 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:23.064 7f0bbd8fa588 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:45:23.064 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:45:23.065 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:45:23.065 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:45:23.065 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:45:23.065 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:45:23.065 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:45:23.067 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:45:23.067 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:45:23.067 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:45:23.067 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:45:23.067 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:45:23.067 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:45:23.851 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:24.852 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:25.854 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:26.856 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:27.065 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:45:27.858 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:28.859 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:29.861 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:30.343 7f0bbddbbeb8 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:45:30.401 7f0bbd9ada20 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:45:30.863 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:32.067 7f0bbd8fa588 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:45:32.068 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:45:32.068 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:45:32.068 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:45:32.068 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:45:32.068 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:45:32.069 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:45:32.069 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:45:32.069 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:45:32.070 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:45:32.070 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:45:32.070 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:45:32.070 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:45:32.817 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:33.818 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:34.819 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:35.821 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:36.068 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:45:36.822 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:37.823 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:38.825 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:39.827 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:40.829 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:41.070 7f0bbd8fa588 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:45:41.070 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:45:41.070 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:45:41.070 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:45:41.070 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:45:41.070 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:45:41.071 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:45:41.071 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:45:41.071 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:45:41.071 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:45:41.071 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:45:41.071 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:45:41.071 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:45:41.831 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:42.833 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:43.835 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:44.837 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:45.070 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:45:45.838 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:46.840 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:47.842 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:48.843 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:49.845 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:50.073 7f0bbd8fa588 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:45:50.073 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:45:50.073 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:45:50.073 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:45:50.074 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:45:50.074 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:45:50.074 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:45:50.074 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:45:50.074 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:45:50.075 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:45:50.075 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:45:50.075 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:45:50.075 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:45:50.846 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:51.848 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:52.850 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:53.852 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:54.073 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:45:54.853 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:55.854 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:56.855 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:57.857 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:58.858 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:45:59.076 7f0bbd8fa588 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:45:59.076 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:45:59.076 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:45:59.076 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:45:59.076 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:45:59.077 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:45:59.077 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:45:59.077 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:45:59.077 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:45:59.077 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:45:59.078 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:45:59.078 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:45:59.078 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:45:59.860 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:00.862 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:02.818 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:03.076 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:46:03.820 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:04.822 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:05.824 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:06.825 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:07.827 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:08.079 7f0bbd8fa588 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:46:08.079 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:46:08.079 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:46:08.079 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:46:08.079 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:46:08.079 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:46:08.080 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:46:08.080 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:46:08.080 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:46:08.080 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:46:08.080 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:46:08.080 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:46:08.080 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:46:08.829 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:09.831 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:10.832 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:11.834 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:12.079 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:46:12.836 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:13.838 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:14.840 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:15.842 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:16.844 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:17.081 7f0bbd8fa588 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:46:17.082 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:46:17.082 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:46:17.082 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:46:17.082 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:46:17.082 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:46:17.083 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:46:17.083 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:46:17.083 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:46:17.083 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:46:17.083 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:46:17.083 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:46:17.083 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:46:17.845 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:18.846 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:19.847 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:20.849 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:21.082 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:46:21.852 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:22.852 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:23.855 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:24.857 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:25.858 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:26.085 7f0bbd8fa588 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:46:26.085 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:46:26.085 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:46:26.085 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:46:26.085 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:46:26.086 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:46:26.086 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:46:26.086 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:46:26.086 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:46:26.086 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:46:26.087 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:46:26.087 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:46:26.087 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:46:26.859 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:27.861 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:28.863 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:29.865 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:30.086 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:46:30.867 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:32.820 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:33.821 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:34.823 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:35.088 7f0bbd8fa588 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:46:35.088 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:46:35.088 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:46:35.088 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:46:35.088 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:46:35.088 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:46:35.089 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:46:35.089 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:46:35.089 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:46:35.089 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:46:35.089 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:46:35.089 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:46:35.089 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:46:35.825 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:36.826 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:37.828 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:38.830 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:39.088 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:46:39.832 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:40.834 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:41.835 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:42.837 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:43.606 7f0bbd9adf60 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:46:43.662 7f0bbd9adf60 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:46:43.838 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:44.090 7f0bbd8fa588 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:46:44.090 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:46:44.091 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:46:44.091 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:46:44.091 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:46:44.091 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:46:44.091 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:46:44.092 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:46:44.092 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:46:44.092 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:46:44.092 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:46:44.092 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:46:44.092 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:46:44.839 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:45.841 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:46.843 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:47.845 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:48.091 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:46:48.847 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:49.848 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:50.851 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:51.852 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:52.854 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:53.094 7f0bbd8fa588 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:46:53.094 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:46:53.094 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:46:53.094 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:46:53.094 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:46:53.095 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:46:53.095 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:46:53.095 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:46:53.095 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:46:53.095 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:46:53.095 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:46:53.095 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:46:53.096 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:46:53.856 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:54.426 7f0bbd8fa588 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:46:54.477 7f0bbd8fa588 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:46:54.858 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:55.858 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:56.860 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:57.095 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:46:57.862 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:58.863 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:46:59.865 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:00.866 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:02.097 7f0bbd8fa588 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:47:02.097 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:47:02.098 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:47:02.098 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:47:02.098 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:47:02.098 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:47:02.098 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:47:02.099 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:47:02.099 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:47:02.099 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:47:02.099 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:47:02.099 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:47:02.099 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:47:02.820 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:03.822 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:04.824 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:05.826 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:06.098 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:47:06.828 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:07.829 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:08.831 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:09.832 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:10.834 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:11.100 7f0bbd8fa588 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:47:11.100 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:47:11.100 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:47:11.100 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:47:11.100 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:47:11.100 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:47:11.101 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:47:11.101 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:47:11.101 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:47:11.101 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:47:11.101 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:47:11.102 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:47:11.102 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:47:11.835 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:12.837 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:13.838 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:14.840 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:15.100 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:47:15.842 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:16.844 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:17.845 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:18.846 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:19.848 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:20.103 7f0bbd8fa588 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:47:20.103 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:47:20.103 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:47:20.103 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:47:20.103 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:47:20.103 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:47:20.104 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:47:20.104 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:47:20.104 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:47:20.104 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:47:20.104 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:47:20.104 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:47:20.104 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:47:20.849 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:21.850 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:22.852 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:23.854 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:24.104 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:47:24.856 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:25.857 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:26.860 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:27.861 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:28.863 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:29.106 7f0bbd8fa588 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:47:29.106 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:47:29.106 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:47:29.107 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:47:29.107 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:47:29.107 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:47:29.107 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:47:29.107 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:47:29.108 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:47:29.108 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:47:29.108 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:47:29.108 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:47:29.108 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:47:29.865 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:30.867 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:32.821 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:33.106 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:47:33.824 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:34.825 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:35.828 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:36.829 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:37.831 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:38.109 7f0bbd8fa588 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:47:38.109 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:47:38.109 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:47:38.109 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:47:38.109 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:47:38.110 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:47:38.110 7f0bbd8fa588 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:47:38.110 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:47:38.110 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:47:38.110 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:47:38.111 7f0bbd8fa588 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:47:38.111 7f0bbd8fa588 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:47:38.111 7f0bbd8fa588 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:47:38.832 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:39.834 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:40.834 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:41.836 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:42.109 7f0bbd808198 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:47:42.837 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:43.839 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:44.841 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:45.842 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:46.844 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:47.111 7f0bbd808198 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:47:47.111 7f0bbd808198 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:47:47.112 7f0bbd808198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:47:47.112 7f0bbd808198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:47:47.112 7f0bbd808198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:47:47.112 7f0bbd808198 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:47:47.112 7f0bbd808198 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:47:47.112 7f0bbd808198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:47:47.113 7f0bbd808198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:47:47.113 7f0bbd808198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:47:47.113 7f0bbd808198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:47:47.113 7f0bbd808198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:47:47.113 7f0bbd808198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:47:47.846 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:48.847 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:49.849 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:50.851 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:51.111 7f0bbd808198 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:47:51.852 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:52.854 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:53.856 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:54.858 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:55.859 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:56.114 7f0bbd808198 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:47:56.114 7f0bbd808198 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:47:56.114 7f0bbd808198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:47:56.114 7f0bbd808198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:47:56.114 7f0bbd808198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:47:56.115 7f0bbd808198 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:47:56.115 7f0bbd808198 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:47:56.115 7f0bbd808198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:47:56.115 7f0bbd808198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:47:56.115 7f0bbd808198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:47:56.115 7f0bbd808198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:47:56.115 7f0bbd808198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:47:56.116 7f0bbd808198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:47:56.860 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:57.862 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:58.863 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:47:59.865 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:00.115 7f0bbd808198 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:48:00.867 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:02.822 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:03.824 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:04.826 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:05.117 7f0bbd808198 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:48:05.117 7f0bbd808198 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:48:05.117 7f0bbd808198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:48:05.117 7f0bbd808198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:48:05.118 7f0bbd808198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:48:05.118 7f0bbd808198 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:48:05.118 7f0bbd808198 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:48:05.118 7f0bbd808198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:48:05.119 7f0bbd808198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:48:05.119 7f0bbd808198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:48:05.119 7f0bbd808198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:48:05.119 7f0bbd808198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:48:05.119 7f0bbd808198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:48:05.828 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:06.829 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:07.831 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:08.833 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:09.118 7f0bbd937198 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:48:09.834 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:10.835 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:11.837 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:12.838 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:13.840 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:14.120 7f0bbd937198 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:48:14.120 7f0bbd937198 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:48:14.120 7f0bbd937198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:48:14.120 7f0bbd937198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:48:14.120 7f0bbd937198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:48:14.120 7f0bbd937198 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:48:14.121 7f0bbd937198 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:48:14.121 7f0bbd937198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:48:14.121 7f0bbd937198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:48:14.121 7f0bbd937198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:48:14.122 7f0bbd937198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:48:14.122 7f0bbd937198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:48:14.122 7f0bbd937198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:48:14.841 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:15.843 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:16.845 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:17.846 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:18.120 7f0bbd937198 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:48:18.849 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:19.850 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:20.852 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:21.854 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:22.855 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:23.122 7f0bbd937198 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:48:23.123 7f0bbd937198 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:48:23.123 7f0bbd937198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:48:23.123 7f0bbd937198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:48:23.123 7f0bbd937198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:48:23.123 7f0bbd937198 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:48:23.124 7f0bbd937198 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:48:23.124 7f0bbd937198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:48:23.124 7f0bbd937198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:48:23.124 7f0bbd937198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:48:23.124 7f0bbd937198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:48:23.124 7f0bbd937198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:48:23.124 7f0bbd937198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:48:23.857 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:24.858 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:25.860 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:26.862 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:27.123 7f0bbd937198 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:48:27.864 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:28.865 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:29.867 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:30.869 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:32.125 7f0bbd937198 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:48:32.125 7f0bbd937198 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:48:32.125 7f0bbd937198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:48:32.126 7f0bbd937198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:48:32.126 7f0bbd937198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:48:32.126 7f0bbd937198 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:48:32.126 7f0bbd937198 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:48:32.126 7f0bbd937198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:48:32.127 7f0bbd937198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:48:32.127 7f0bbd937198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:48:32.127 7f0bbd937198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:48:32.127 7f0bbd937198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:48:32.127 7f0bbd937198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:48:32.824 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:33.825 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:34.827 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:35.828 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:36.126 7f0bbd937198 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:48:36.830 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:37.831 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:38.833 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:39.834 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:40.835 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:41.128 7f0bbd937198 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:48:41.129 7f0bbd937198 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:48:41.129 7f0bbd937198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:48:41.129 7f0bbd937198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:48:41.129 7f0bbd937198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:48:41.129 7f0bbd937198 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:48:41.130 7f0bbd937198 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:48:41.130 7f0bbd937198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:48:41.130 7f0bbd937198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:48:41.130 7f0bbd937198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:48:41.130 7f0bbd937198 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:48:41.130 7f0bbd937198 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:48:41.130 7f0bbd937198 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:48:41.837 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:42.839 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:43.840 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:44.842 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:45.130 7f0bbd7cd630 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:48:45.844 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:46.845 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:47.846 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:48.847 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:49.849 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:50.132 7f0bbd7cd630 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:48:50.132 7f0bbd7cd630 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:48:50.132 7f0bbd7cd630 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:48:50.132 7f0bbd7cd630 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:48:50.132 7f0bbd7cd630 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:48:50.133 7f0bbd7cd630 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:48:50.133 7f0bbd7cd630 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:48:50.133 7f0bbd7cd630 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:48:50.133 7f0bbd7cd630 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:48:50.133 7f0bbd7cd630 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:48:50.133 7f0bbd7cd630 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:48:50.134 7f0bbd7cd630 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:48:50.134 7f0bbd7cd630 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:48:50.850 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:51.852 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:52.854 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:53.856 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:54.133 7f0bbd7cd630 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:48:54.858 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:55.860 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:56.862 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:57.863 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:58.865 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:48:59.134 7f0bbd7cd630 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:48:59.135 7f0bbd7cd630 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:48:59.135 7f0bbd7cd630 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:48:59.135 7f0bbd7cd630 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:48:59.135 7f0bbd7cd630 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:48:59.135 7f0bbd7cd630 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:48:59.136 7f0bbd7cd630 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:48:59.136 7f0bbd7cd630 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:48:59.136 7f0bbd7cd630 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:48:59.136 7f0bbd7cd630 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:48:59.136 7f0bbd7cd630 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:48:59.136 7f0bbd7cd630 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:48:59.136 7f0bbd7cd630 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:48:59.866 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:00.868 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:02.825 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:03.136 7f0bbd7cd630 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:49:03.827 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:04.829 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:05.831 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:06.832 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:07.834 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:08.138 7f0bbd7cd630 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:49:08.138 7f0bbd7cd630 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:49:08.138 7f0bbd7cd630 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:49:08.139 7f0bbd7cd630 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:49:08.139 7f0bbd7cd630 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:49:08.139 7f0bbd7cd630 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:49:08.139 7f0bbd7cd630 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:49:08.139 7f0bbd7cd630 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:49:08.140 7f0bbd7cd630 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:49:08.140 7f0bbd7cd630 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:49:08.140 7f0bbd7cd630 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:49:08.140 7f0bbd7cd630 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:49:08.140 7f0bbd7cd630 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:49:08.836 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:09.837 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:10.839 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:11.841 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:12.139 7f0bbd7cd630 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:49:12.843 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:13.845 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:14.847 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:15.849 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:16.850 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:17.141 7f0bbd7cd630 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:49:17.141 7f0bbd7cd630 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:49:17.141 7f0bbd7cd630 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:49:17.142 7f0bbd7cd630 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:49:17.142 7f0bbd7cd630 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:49:17.142 7f0bbd7cd630 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:49:17.142 7f0bbd7cd630 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:49:17.143 7f0bbd7cd630 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:49:17.143 7f0bbd7cd630 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:49:17.143 7f0bbd7cd630 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:49:17.143 7f0bbd7cd630 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:49:17.143 7f0bbd7cd630 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:49:17.143 7f0bbd7cd630 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:49:17.852 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:18.854 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:19.855 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:20.857 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:21.142 7f0bbd7cd630 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:49:21.859 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:22.861 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:23.863 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:24.865 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:25.866 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:26.144 7f0bbd7cd630 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:49:26.144 7f0bbd7cd630 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:49:26.144 7f0bbd7cd630 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:49:26.144 7f0bbd7cd630 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:49:26.145 7f0bbd7cd630 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:49:26.145 7f0bbd7cd630 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:49:26.145 7f0bbd7cd630 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:49:26.145 7f0bbd7cd630 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:49:26.145 7f0bbd7cd630 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:49:26.146 7f0bbd7cd630 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:49:26.146 7f0bbd7cd630 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:49:26.146 7f0bbd7cd630 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:49:26.146 7f0bbd7cd630 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:49:26.867 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:27.771 7f0bbd7cd630 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:49:27.822 7f0bbd7cd630 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:49:27.869 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:28.871 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:29.873 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:30.145 7f0bbd7cd630 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:49:30.875 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:32.825 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:33.827 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:34.829 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:35.147 7f0bbd7cd630 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:49:35.148 7f0bbd7cd630 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:49:35.148 7f0bbd7cd630 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:49:35.148 7f0bbd7cd630 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:49:35.148 7f0bbd7cd630 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:49:35.148 7f0bbd7cd630 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:49:35.149 7f0bbd7cd630 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:49:35.149 7f0bbd7cd630 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:49:35.149 7f0bbd7cd630 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:49:35.149 7f0bbd7cd630 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:49:35.149 7f0bbd7cd630 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:49:35.149 7f0bbd7cd630 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:49:35.149 7f0bbd7cd630 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:49:35.830 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:36.832 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:37.834 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:38.502 7f0bbd926978 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:49:38.553 7f0bbd926978 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:49:38.835 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:39.148 7f0bbd926978 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:49:39.837 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:40.839 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:41.840 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:42.842 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:43.843 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:44.150 7f0bbd926978 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:49:44.151 7f0bbd926978 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:49:44.151 7f0bbd926978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:49:44.151 7f0bbd926978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:49:44.151 7f0bbd926978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:49:44.151 7f0bbd926978 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:49:44.152 7f0bbd926978 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:49:44.152 7f0bbd926978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:49:44.152 7f0bbd926978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:49:44.152 7f0bbd926978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:49:44.152 7f0bbd926978 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:49:44.152 7f0bbd926978 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:49:44.152 7f0bbd926978 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:49:44.845 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:45.846 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:46.848 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:47.850 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:48.151 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:49:48.852 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:49.855 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:50.862 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:51.864 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:52.865 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:53.161 7f0bbe54f4e0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:49:53.161 7f0bbe54f4e0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:49:53.161 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:49:53.166 7f0bbe54f4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:49:53.167 7f0bbe54f4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:49:53.167 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:49:53.180 7f0bbe54f4e0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:49:53.181 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:49:53.181 7f0bbe54f4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:49:53.181 7f0bbe54f4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:49:53.181 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:49:53.181 7f0bbe54f4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:49:53.181 7f0bbe54f4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:49:53.868 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:54.874 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:55.876 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:56.877 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:57.161 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:49:57.882 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:58.883 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:49:59.885 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:00.886 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:02.164 7f0bbe54f4e0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:50:02.165 7f0bbe54f4e0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:50:02.165 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:50:02.165 7f0bbe54f4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:50:02.165 7f0bbe54f4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:50:02.165 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:50:02.166 7f0bbe54f4e0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:50:02.166 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:50:02.167 7f0bbe54f4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:50:02.167 7f0bbe54f4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:50:02.167 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:50:02.167 7f0bbe54f4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:50:02.167 7f0bbe54f4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:50:02.826 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:03.827 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:04.828 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:05.830 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:06.165 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:50:06.831 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:07.832 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:08.834 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:09.834 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:10.836 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:11.167 7f0bbe54f4e0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:50:11.168 7f0bbe54f4e0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:50:11.168 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:50:11.168 7f0bbe54f4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:50:11.168 7f0bbe54f4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:50:11.168 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:50:11.169 7f0bbe54f4e0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:50:11.169 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:50:11.169 7f0bbe54f4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:50:11.169 7f0bbe54f4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:50:11.169 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:50:11.169 7f0bbe54f4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:50:11.169 7f0bbe54f4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:50:11.837 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:12.838 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:13.840 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:14.842 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:15.168 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:50:15.844 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:16.846 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:17.848 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:18.849 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:19.851 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:20.170 7f0bbe54f4e0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:50:20.171 7f0bbe54f4e0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:50:20.171 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:50:20.171 7f0bbe54f4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:50:20.171 7f0bbe54f4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:50:20.171 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:50:20.172 7f0bbe54f4e0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:50:20.172 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:50:20.172 7f0bbe54f4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:50:20.172 7f0bbe54f4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:50:20.172 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:50:20.173 7f0bbe54f4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:50:20.173 7f0bbe54f4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:50:20.853 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:21.855 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:22.857 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:23.859 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:24.171 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:50:24.861 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:25.862 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:26.864 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:27.866 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:28.868 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:29.174 7f0bbe54f4e0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:50:29.174 7f0bbe54f4e0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:50:29.174 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:50:29.174 7f0bbe54f4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:50:29.175 7f0bbe54f4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:50:29.175 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:50:29.175 7f0bbe54f4e0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:50:29.175 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:50:29.175 7f0bbe54f4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:50:29.176 7f0bbe54f4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:50:29.176 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:50:29.176 7f0bbe54f4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:50:29.176 7f0bbe54f4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:50:29.870 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:30.872 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:32.828 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:33.175 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:50:33.829 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:34.831 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:35.833 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:36.835 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:37.836 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:38.177 7f0bbe54f4e0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:50:38.178 7f0bbe54f4e0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:50:38.178 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:50:38.178 7f0bbe54f4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:50:38.178 7f0bbe54f4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:50:38.178 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:50:38.179 7f0bbe54f4e0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:50:38.179 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:50:38.179 7f0bbe54f4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:50:38.179 7f0bbe54f4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:50:38.179 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:50:38.179 7f0bbe54f4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:50:38.179 7f0bbe54f4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:50:38.837 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:39.839 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:40.840 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:41.842 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:42.178 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:50:42.844 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:43.846 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:44.847 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:45.849 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:46.851 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:47.180 7f0bbe54f4e0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:50:47.180 7f0bbe54f4e0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:50:47.180 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:50:47.180 7f0bbe54f4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:50:47.180 7f0bbe54f4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:50:47.181 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:50:47.181 7f0bbe54f4e0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:50:47.181 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:50:47.181 7f0bbe54f4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:50:47.181 7f0bbe54f4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:50:47.181 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:50:47.182 7f0bbe54f4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:50:47.182 7f0bbe54f4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:50:47.852 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:48.853 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:49.854 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:50.856 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:51.181 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:50:51.858 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:52.859 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:53.861 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:54.862 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:55.864 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:56.183 7f0bbe54f4e0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:50:56.183 7f0bbe54f4e0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:50:56.183 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:50:56.184 7f0bbe54f4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:50:56.184 7f0bbe54f4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:50:56.184 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:50:56.184 7f0bbe54f4e0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:50:56.184 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:50:56.184 7f0bbe54f4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:50:56.185 7f0bbe54f4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:50:56.185 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:50:56.185 7f0bbe54f4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:50:56.185 7f0bbe54f4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:50:56.866 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:57.868 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:58.869 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:50:59.871 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:00.184 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:51:00.873 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:02.829 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:03.830 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:04.832 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:05.186 7f0bbe54f4e0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:51:05.186 7f0bbe54f4e0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:51:05.186 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:51:05.186 7f0bbe54f4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:51:05.186 7f0bbe54f4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:51:05.186 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:51:05.187 7f0bbe54f4e0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:51:05.187 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:51:05.187 7f0bbe54f4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:51:05.187 7f0bbe54f4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:51:05.187 7f0bbe54f4e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:51:05.188 7f0bbe54f4e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:51:05.188 7f0bbe54f4e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:51:05.834 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:06.835 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:07.837 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:08.839 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:09.186 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:51:09.841 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:10.842 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:11.844 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:12.846 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:13.847 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:14.188 7f0bbd810048 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:51:14.189 7f0bbd810048 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:51:14.189 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:51:14.189 7f0bbd810048 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:51:14.189 7f0bbd810048 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:51:14.189 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:51:14.190 7f0bbd810048 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:51:14.190 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:51:14.190 7f0bbd810048 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:51:14.190 7f0bbd810048 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:51:14.190 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:51:14.191 7f0bbd810048 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:51:14.191 7f0bbd810048 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:51:14.849 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:15.850 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:16.851 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:17.853 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:18.190 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:51:18.854 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:19.856 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:20.858 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:21.860 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:22.862 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:23.193 7f0bbd810048 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:51:23.193 7f0bbd810048 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:51:23.193 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:51:23.193 7f0bbd810048 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:51:23.194 7f0bbd810048 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:51:23.194 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:51:23.195 7f0bbd810048 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:51:23.195 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:51:23.196 7f0bbd810048 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:51:23.196 7f0bbd810048 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:51:23.196 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:51:23.196 7f0bbd810048 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:51:23.196 7f0bbd810048 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:51:23.864 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:24.866 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:25.868 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:26.870 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:27.193 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:51:27.872 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:28.751 7f0bbd850588 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:51:28.807 7f0bbd8506d8 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:51:28.873 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:29.875 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:30.877 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:32.195 7f0bbd810048 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:51:32.196 7f0bbd810048 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:51:32.196 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:51:32.196 7f0bbd810048 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:51:32.196 7f0bbd810048 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:51:32.196 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:51:32.197 7f0bbd810048 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:51:32.197 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:51:32.197 7f0bbd810048 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:51:32.197 7f0bbd810048 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:51:32.197 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:51:32.197 7f0bbd810048 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:51:32.198 7f0bbd810048 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:51:32.829 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:33.830 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:34.832 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:35.838 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:36.196 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:51:36.839 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:37.841 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:38.843 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:39.844 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:39.846 7f0bbd9ab2e8 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:51:39.898 7f0bbdcffd68 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:51:40.846 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:41.198 7f0bbd810048 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:51:41.198 7f0bbd810048 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:51:41.198 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:51:41.199 7f0bbd810048 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:51:41.199 7f0bbd810048 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:51:41.199 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:51:41.199 7f0bbd810048 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:51:41.199 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:51:41.199 7f0bbd810048 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:51:41.200 7f0bbd810048 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:51:41.200 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:51:41.200 7f0bbd810048 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:51:41.200 7f0bbd810048 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:51:41.848 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:42.850 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:43.852 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:44.854 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:45.199 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:51:45.854 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:46.856 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:47.858 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:48.860 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:49.862 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:50.201 7f0bbd810048 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:51:50.201 7f0bbd810048 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:51:50.201 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:51:50.201 7f0bbd810048 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:51:50.201 7f0bbd810048 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:51:50.202 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:51:50.202 7f0bbd810048 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:51:50.202 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:51:50.202 7f0bbd810048 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:51:50.202 7f0bbd810048 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:51:50.202 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:51:50.203 7f0bbd810048 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:51:50.203 7f0bbd810048 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:51:50.863 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:51.865 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:52.866 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:53.868 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:54.202 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:51:54.870 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:55.871 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:56.873 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:57.874 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:58.876 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:51:59.204 7f0bbd810048 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:51:59.204 7f0bbd810048 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:51:59.204 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:51:59.204 7f0bbd810048 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:51:59.205 7f0bbd810048 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:51:59.205 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:51:59.205 7f0bbd810048 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:51:59.205 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:51:59.205 7f0bbd810048 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:51:59.205 7f0bbd810048 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:51:59.205 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:51:59.206 7f0bbd810048 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:51:59.206 7f0bbd810048 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:51:59.877 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:00.878 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:02.829 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:03.204 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:52:03.832 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:04.834 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:05.836 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:06.837 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:07.839 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:08.206 7f0bbd810048 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:52:08.206 7f0bbd810048 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:52:08.207 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:52:08.207 7f0bbd810048 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:52:08.207 7f0bbd810048 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:52:08.207 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:52:08.207 7f0bbd810048 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:52:08.208 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:52:08.208 7f0bbd810048 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:52:08.208 7f0bbd810048 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:52:08.208 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:52:08.208 7f0bbd810048 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:52:08.208 7f0bbd810048 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:52:08.840 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:09.842 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:10.844 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:11.846 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:12.207 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:52:12.847 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:13.848 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:14.850 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:15.852 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:16.853 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:17.209 7f0bbd810048 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:52:17.209 7f0bbd810048 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:52:17.209 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:52:17.210 7f0bbd810048 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:52:17.210 7f0bbd810048 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:52:17.210 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:52:17.210 7f0bbd810048 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:52:17.210 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:52:17.210 7f0bbd810048 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:52:17.211 7f0bbd810048 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:52:17.211 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:52:17.211 7f0bbd810048 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:52:17.211 7f0bbd810048 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:52:17.855 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:18.857 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:19.858 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:20.860 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:21.210 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:52:21.861 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:22.863 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:23.865 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:24.867 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:25.869 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:26.211 7f0bbd810048 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:52:26.212 7f0bbd810048 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:52:26.212 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:52:26.212 7f0bbd810048 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:52:26.212 7f0bbd810048 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:52:26.212 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:52:26.213 7f0bbd810048 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:52:26.213 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:52:26.213 7f0bbd810048 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:52:26.213 7f0bbd810048 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:52:26.213 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:52:26.213 7f0bbd810048 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:52:26.213 7f0bbd810048 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:52:26.871 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:27.873 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:28.875 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:29.877 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:30.212 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:52:30.879 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:32.831 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:33.832 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:34.833 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:35.214 7f0bbd810048 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:52:35.214 7f0bbd810048 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:52:35.214 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:52:35.214 7f0bbd810048 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:52:35.214 7f0bbd810048 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:52:35.215 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:52:35.215 7f0bbd810048 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:52:35.215 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:52:35.215 7f0bbd810048 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:52:35.215 7f0bbd810048 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:52:35.215 7f0bbd810048 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:52:35.216 7f0bbd810048 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:52:35.216 7f0bbd810048 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:52:35.835 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:36.837 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:37.838 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:38.840 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:39.215 7f0bbd810630 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:52:39.842 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:40.843 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:41.846 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:42.847 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:43.848 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:44.216 7f0bbd810630 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:52:44.217 7f0bbd810630 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:52:44.217 7f0bbd810630 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:52:44.217 7f0bbd810630 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:52:44.217 7f0bbd810630 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:52:44.217 7f0bbd810630 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:52:44.218 7f0bbd810630 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:52:44.218 7f0bbd810630 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:52:44.218 7f0bbd810630 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:52:44.218 7f0bbd810630 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:52:44.218 7f0bbd810630 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:52:44.218 7f0bbd810630 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:52:44.219 7f0bbd810630 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:52:44.850 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:45.851 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:46.853 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:47.855 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:48.217 7f0bbda05b70 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:52:48.857 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:49.859 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:50.861 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:51.862 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:52.864 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:53.219 7f0bbda05b70 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:52:53.219 7f0bbda05b70 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:52:53.219 7f0bbda05b70 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:52:53.219 7f0bbda05b70 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:52:53.219 7f0bbda05b70 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:52:53.219 7f0bbda05b70 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:52:53.220 7f0bbda05b70 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:52:53.220 7f0bbda05b70 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:52:53.220 7f0bbda05b70 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:52:53.220 7f0bbda05b70 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:52:53.220 7f0bbda05b70 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:52:53.220 7f0bbda05b70 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:52:53.220 7f0bbda05b70 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:52:53.866 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:54.867 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:55.869 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:56.871 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:57.220 7f0bbda05b70 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:52:57.873 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:58.874 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:52:59.876 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:00.877 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:02.222 7f0bbda05b70 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:53:02.222 7f0bbda05b70 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:53:02.222 7f0bbda05b70 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:53:02.222 7f0bbda05b70 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:53:02.223 7f0bbda05b70 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:53:02.223 7f0bbda05b70 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:53:02.223 7f0bbda05b70 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:53:02.223 7f0bbda05b70 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:53:02.223 7f0bbda05b70 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:53:02.223 7f0bbda05b70 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:53:02.224 7f0bbda05b70 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:53:02.224 7f0bbda05b70 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:53:02.224 7f0bbda05b70 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:53:02.832 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:03.834 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:04.836 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:05.837 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:06.222 7f0bbda05b70 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:53:06.838 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:07.840 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:08.842 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:09.844 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:10.846 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:11.224 7f0bbda05b70 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:53:11.225 7f0bbda05b70 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:53:11.225 7f0bbda05b70 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:53:11.225 7f0bbda05b70 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:53:11.225 7f0bbda05b70 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:53:11.225 7f0bbda05b70 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:53:11.225 7f0bbda05b70 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:53:11.226 7f0bbda05b70 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:53:11.226 7f0bbda05b70 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:53:11.226 7f0bbda05b70 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:53:11.226 7f0bbda05b70 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:53:11.226 7f0bbda05b70 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:53:11.226 7f0bbda05b70 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:53:11.847 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:12.849 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:13.850 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:14.852 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:15.225 7f0bbda05b70 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:53:15.854 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:16.855 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:17.857 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:18.859 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:19.860 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:20.227 7f0bbda05b70 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:53:20.228 7f0bbda05b70 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:53:20.228 7f0bbda05b70 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:53:20.228 7f0bbda05b70 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:53:20.228 7f0bbda05b70 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:53:20.228 7f0bbda05b70 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:53:20.229 7f0bbda05b70 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:53:20.229 7f0bbda05b70 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:53:20.229 7f0bbda05b70 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:53:20.229 7f0bbda05b70 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:53:20.229 7f0bbda05b70 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:53:20.229 7f0bbda05b70 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:53:20.229 7f0bbda05b70 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:53:20.862 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:21.864 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:22.866 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:23.867 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:24.229 7f0bbda05b70 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:53:24.867 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:25.869 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:26.871 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:27.872 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:28.874 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:29.231 7f0bbda05b70 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:53:29.231 7f0bbda05b70 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:53:29.231 7f0bbda05b70 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:53:29.232 7f0bbda05b70 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:53:29.232 7f0bbda05b70 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:53:29.232 7f0bbda05b70 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:53:29.233 7f0bbda05b70 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:53:29.233 7f0bbda05b70 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:53:29.233 7f0bbda05b70 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:53:29.233 7f0bbda05b70 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:53:29.233 7f0bbda05b70 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:53:29.233 7f0bbda05b70 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:53:29.233 7f0bbda05b70 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:53:29.876 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:30.878 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:32.833 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:33.232 7f0bbda05b70 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:53:33.835 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:34.836 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:35.838 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:36.839 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:37.840 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:38.234 7f0bbda05b70 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:53:38.234 7f0bbda05b70 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:53:38.234 7f0bbda05b70 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:53:38.235 7f0bbda05b70 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:53:38.235 7f0bbda05b70 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:53:38.235 7f0bbda05b70 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:53:38.235 7f0bbda05b70 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:53:38.235 7f0bbda05b70 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:53:38.235 7f0bbda05b70 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:53:38.236 7f0bbda05b70 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:53:38.236 7f0bbda05b70 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:53:38.236 7f0bbda05b70 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:53:38.236 7f0bbda05b70 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:53:38.842 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:39.844 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:40.845 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:41.846 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:42.235 7f0bbda05b70 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:53:42.848 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:43.849 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:44.851 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:45.853 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:46.855 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:47.237 7f0bbda05b70 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:53:47.237 7f0bbda05b70 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:53:47.237 7f0bbda05b70 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:53:47.237 7f0bbda05b70 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:53:47.237 7f0bbda05b70 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:53:47.237 7f0bbda05b70 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:53:47.238 7f0bbda05b70 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:53:47.238 7f0bbda05b70 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:53:47.238 7f0bbda05b70 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:53:47.238 7f0bbda05b70 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:53:47.238 7f0bbda05b70 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:53:47.238 7f0bbda05b70 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:53:47.238 7f0bbda05b70 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:53:47.856 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:48.858 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:49.859 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:50.861 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:51.238 7f0bbd9c0048 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:53:51.862 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:52.863 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:53.865 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:54.522 7f0bbd811a20 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:53:54.573 7f0bbd811a20 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:53:54.866 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:55.868 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:56.240 7f0bbd9c0048 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:53:56.240 7f0bbd9c0048 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:53:56.240 7f0bbd9c0048 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:53:56.240 7f0bbd9c0048 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:53:56.240 7f0bbd9c0048 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:53:56.240 7f0bbd9c0048 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:53:56.241 7f0bbd9c0048 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:53:56.241 7f0bbd9c0048 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:53:56.241 7f0bbd9c0048 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:53:56.241 7f0bbd9c0048 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:53:56.241 7f0bbd9c0048 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:53:56.242 7f0bbd9c0048 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:53:56.242 7f0bbd9c0048 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:53:56.870 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:57.871 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:58.871 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:53:59.873 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:00.241 7f0bbd9c0048 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:54:00.875 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:02.834 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:03.836 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:04.837 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:05.243 7f0bbd9c0048 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:54:05.243 7f0bbd9c0048 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:54:05.243 7f0bbd9c0048 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:54:05.243 7f0bbd9c0048 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:54:05.243 7f0bbd9c0048 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:54:05.244 7f0bbd9c0048 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:54:05.244 7f0bbd9c0048 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:54:05.244 7f0bbd9c0048 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:54:05.244 7f0bbd9c0048 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:54:05.244 7f0bbd9c0048 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:54:05.244 7f0bbd9c0048 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:54:05.245 7f0bbd9c0048 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:54:05.245 7f0bbd9c0048 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:54:05.839 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:06.840 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:07.841 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:08.843 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:09.243 7f0bbd854390 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:54:09.844 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:10.846 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:11.848 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:12.849 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:13.850 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:14.246 7f0bbd854390 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:54:14.246 7f0bbd854390 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:54:14.246 7f0bbd854390 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:54:14.247 7f0bbd854390 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:54:14.247 7f0bbd854390 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:54:14.247 7f0bbd854390 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:54:14.248 7f0bbd854390 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:54:14.248 7f0bbd854390 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:54:14.248 7f0bbd854390 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:54:14.248 7f0bbd854390 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:54:14.248 7f0bbd854390 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:54:14.248 7f0bbd854390 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:54:14.249 7f0bbd854390 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:54:14.853 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:15.854 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:16.855 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:17.857 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:18.246 7f0bbd854390 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:54:18.859 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:19.861 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:20.863 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:21.865 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:22.867 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:23.248 7f0bbd854390 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:54:23.248 7f0bbd854390 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:54:23.248 7f0bbd854390 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:54:23.249 7f0bbd854390 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:54:23.249 7f0bbd854390 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:54:23.249 7f0bbd854390 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:54:23.249 7f0bbd854390 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:54:23.249 7f0bbd854390 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:54:23.250 7f0bbd854390 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:54:23.250 7f0bbd854390 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:54:23.250 7f0bbd854390 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:54:23.250 7f0bbd854390 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:54:23.250 7f0bbd854390 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:54:23.868 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:24.870 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:25.871 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:26.873 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:27.249 7f0bbd854390 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:54:27.874 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:28.876 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:29.878 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:30.880 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:32.251 7f0bbd854390 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:54:32.251 7f0bbd854390 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:54:32.251 7f0bbd854390 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:54:32.251 7f0bbd854390 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:54:32.252 7f0bbd854390 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:54:32.252 7f0bbd854390 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:54:32.252 7f0bbd854390 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:54:32.252 7f0bbd854390 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:54:32.252 7f0bbd854390 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:54:32.252 7f0bbd854390 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:54:32.253 7f0bbd854390 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:54:32.253 7f0bbd854390 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:54:32.253 7f0bbd854390 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:54:32.835 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:33.837 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:34.838 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:35.840 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:36.251 7f0bbd854390 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:54:36.841 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:37.842 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:38.844 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:39.846 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:40.847 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:41.253 7f0bbd854390 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:54:41.253 7f0bbd854390 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:54:41.253 7f0bbd854390 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:54:41.254 7f0bbd854390 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:54:41.254 7f0bbd854390 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:54:41.254 7f0bbd854390 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:54:41.254 7f0bbd854390 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:54:41.255 7f0bbd854390 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:54:41.255 7f0bbd854390 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:54:41.255 7f0bbd854390 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:54:41.255 7f0bbd854390 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:54:41.255 7f0bbd854390 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:54:41.255 7f0bbd854390 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:54:41.849 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:42.851 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:43.853 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:44.854 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:45.254 7f0bbd854390 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:54:45.856 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:46.857 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:47.859 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:48.860 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:49.862 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:50.256 7f0bbd854390 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:54:50.256 7f0bbd854390 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:54:50.257 7f0bbd854390 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:54:50.257 7f0bbd854390 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:54:50.257 7f0bbd854390 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:54:50.257 7f0bbd854390 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:54:50.257 7f0bbd854390 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:54:50.258 7f0bbd854390 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:54:50.258 7f0bbd854390 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:54:50.258 7f0bbd854390 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:54:50.258 7f0bbd854390 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:54:50.258 7f0bbd854390 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:54:50.258 7f0bbd854390 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:54:50.864 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:51.865 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:52.867 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:53.869 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:54.257 7f0bbdcffa20 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:54:54.870 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:55.871 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:56.873 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:57.874 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:58.875 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:54:59.259 7f0bbdcffa20 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:54:59.260 7f0bbdcffa20 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:54:59.260 7f0bbdcffa20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:54:59.260 7f0bbdcffa20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:54:59.260 7f0bbdcffa20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:54:59.260 7f0bbdcffa20 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:54:59.261 7f0bbdcffa20 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:54:59.261 7f0bbdcffa20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:54:59.261 7f0bbdcffa20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:54:59.261 7f0bbdcffa20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:54:59.261 7f0bbdcffa20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:54:59.261 7f0bbdcffa20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:54:59.261 7f0bbdcffa20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:54:59.877 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:00.879 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:01.214 7f0bbdcffa20 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:55:01.273 7f0bbdcffa20 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:55:02.836 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:03.260 7f0bbdcffa20 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:55:03.838 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:04.840 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:05.842 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:06.843 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:07.845 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:08.262 7f0bbdcffa20 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:55:08.262 7f0bbdcffa20 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:55:08.262 7f0bbdcffa20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:55:08.262 7f0bbdcffa20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:55:08.263 7f0bbdcffa20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:55:08.263 7f0bbdcffa20 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:55:08.263 7f0bbdcffa20 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:55:08.263 7f0bbdcffa20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:55:08.263 7f0bbdcffa20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:55:08.264 7f0bbdcffa20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:55:08.264 7f0bbdcffa20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:55:08.264 7f0bbdcffa20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:55:08.264 7f0bbdcffa20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:55:08.847 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:09.849 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:10.850 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:11.852 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:12.263 7f0bbdcffa20 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:55:12.854 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:13.855 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:14.857 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:15.858 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:16.859 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:17.265 7f0bbdcffa20 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:55:17.265 7f0bbdcffa20 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:55:17.265 7f0bbdcffa20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:55:17.265 7f0bbdcffa20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:55:17.265 7f0bbdcffa20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:55:17.265 7f0bbdcffa20 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:55:17.266 7f0bbdcffa20 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:55:17.266 7f0bbdcffa20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:55:17.266 7f0bbdcffa20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:55:17.266 7f0bbdcffa20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:55:17.266 7f0bbdcffa20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:55:17.266 7f0bbdcffa20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:55:17.267 7f0bbdcffa20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:55:17.860 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:18.861 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:19.862 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:20.864 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:21.266 7f0bbdcffa20 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:55:21.866 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:22.868 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:23.870 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:24.872 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:25.873 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:26.268 7f0bbdcffa20 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:55:26.268 7f0bbdcffa20 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:55:26.268 7f0bbdcffa20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:55:26.268 7f0bbdcffa20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:55:26.269 7f0bbdcffa20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:55:26.269 7f0bbdcffa20 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:55:26.269 7f0bbdcffa20 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:55:26.269 7f0bbdcffa20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:55:26.269 7f0bbdcffa20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:55:26.270 7f0bbdcffa20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:55:26.270 7f0bbdcffa20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:55:26.270 7f0bbdcffa20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:55:26.270 7f0bbdcffa20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:55:26.875 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:27.876 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:28.878 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:29.880 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:30.268 7f0bbdcffa20 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:55:30.882 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:32.837 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:33.839 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:34.841 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:35.270 7f0bbdcffa20 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:55:35.271 7f0bbdcffa20 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:55:35.271 7f0bbdcffa20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:55:35.271 7f0bbdcffa20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:55:35.271 7f0bbdcffa20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:55:35.271 7f0bbdcffa20 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:55:35.272 7f0bbdcffa20 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:55:35.272 7f0bbdcffa20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:55:35.272 7f0bbdcffa20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:55:35.272 7f0bbdcffa20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:55:35.272 7f0bbdcffa20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:55:35.272 7f0bbdcffa20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:55:35.272 7f0bbdcffa20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:55:35.843 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:36.844 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:37.845 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:38.848 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:39.272 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:55:39.850 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:40.851 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:41.853 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:42.855 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:43.856 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:44.273 7f0bbd9abc18 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:55:44.273 7f0bbd9abc18 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:55:44.274 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:55:44.274 7f0bbd9abc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:55:44.274 7f0bbd9abc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:55:44.274 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:55:44.274 7f0bbd9abc18 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:55:44.275 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:55:44.275 7f0bbd9abc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:55:44.275 7f0bbd9abc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:55:44.275 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:55:44.275 7f0bbd9abc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:55:44.275 7f0bbd9abc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:55:44.857 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:45.859 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:46.860 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:47.862 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:48.275 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:55:48.864 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:49.866 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:50.869 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:51.870 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:52.874 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:53.278 7f0bbd9abc18 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:55:53.278 7f0bbd9abc18 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:55:53.278 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:55:53.278 7f0bbd9abc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:55:53.279 7f0bbd9abc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:55:53.279 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:55:53.279 7f0bbd9abc18 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:55:53.279 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:55:53.279 7f0bbd9abc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:55:53.279 7f0bbd9abc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:55:53.279 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:55:53.280 7f0bbd9abc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:55:53.280 7f0bbd9abc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:55:53.876 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:54.877 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:55.886 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:56.888 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:57.279 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:55:57.890 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:58.891 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:55:59.892 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:00.894 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:00.946 7f0bbddbb390 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:56:01.001 7f0bbddaa048 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:56:02.281 7f0bbd9abc18 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:56:02.281 7f0bbd9abc18 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:56:02.281 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:56:02.282 7f0bbd9abc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:56:02.282 7f0bbd9abc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:56:02.282 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:56:02.283 7f0bbd9abc18 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:56:02.283 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:56:02.283 7f0bbd9abc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:56:02.283 7f0bbd9abc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:56:02.283 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:56:02.283 7f0bbd9abc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:56:02.283 7f0bbd9abc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:56:02.838 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:03.839 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:04.841 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:05.843 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:06.282 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:56:06.844 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:07.845 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:08.848 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:09.850 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:10.851 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:11.283 7f0bbd9abc18 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:56:11.284 7f0bbd9abc18 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:56:11.284 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:56:11.284 7f0bbd9abc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:56:11.284 7f0bbd9abc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:56:11.284 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:56:11.285 7f0bbd9abc18 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:56:11.285 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:56:11.285 7f0bbd9abc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:56:11.285 7f0bbd9abc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:56:11.285 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:56:11.285 7f0bbd9abc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:56:11.285 7f0bbd9abc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:56:11.846 7f0bbd9abc18 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:56:11.853 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:11.900 7f0bbd9abc18 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:56:12.854 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:13.856 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:14.857 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:15.284 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:56:15.859 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:16.861 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:17.862 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:18.867 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:19.868 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:20.286 7f0bbd9abc18 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:56:20.286 7f0bbd9abc18 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:56:20.286 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:56:20.287 7f0bbd9abc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:56:20.287 7f0bbd9abc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:56:20.287 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:56:20.287 7f0bbd9abc18 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:56:20.287 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:56:20.287 7f0bbd9abc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:56:20.288 7f0bbd9abc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:56:20.288 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:56:20.288 7f0bbd9abc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:56:20.288 7f0bbd9abc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:56:20.870 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:21.871 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:22.873 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:23.875 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:24.286 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:56:24.877 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:25.879 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:26.880 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:27.882 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:28.884 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:29.288 7f0bbd9abc18 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:56:29.288 7f0bbd9abc18 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:56:29.289 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:56:29.289 7f0bbd9abc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:56:29.289 7f0bbd9abc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:56:29.289 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:56:29.290 7f0bbd9abc18 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:56:29.290 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:56:29.290 7f0bbd9abc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:56:29.290 7f0bbd9abc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:56:29.290 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:56:29.291 7f0bbd9abc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:56:29.291 7f0bbd9abc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:56:29.886 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:30.888 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:32.839 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:33.289 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:56:33.840 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:34.841 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:35.843 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:36.845 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:37.846 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:38.291 7f0bbd9abc18 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:56:38.292 7f0bbd9abc18 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:56:38.292 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:56:38.292 7f0bbd9abc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:56:38.292 7f0bbd9abc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:56:38.292 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:56:38.293 7f0bbd9abc18 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:56:38.293 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:56:38.293 7f0bbd9abc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:56:38.293 7f0bbd9abc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:56:38.293 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:56:38.294 7f0bbd9abc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:56:38.294 7f0bbd9abc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:56:38.848 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:39.849 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:40.851 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:41.852 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:42.292 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:56:42.853 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:43.854 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:44.856 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:45.858 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:46.860 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:47.294 7f0bbd9abc18 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:56:47.295 7f0bbd9abc18 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:56:47.295 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:56:47.295 7f0bbd9abc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:56:47.295 7f0bbd9abc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:56:47.295 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:56:47.296 7f0bbd9abc18 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:56:47.296 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:56:47.296 7f0bbd9abc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:56:47.296 7f0bbd9abc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:56:47.296 7f0bbd9abc18 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:56:47.296 7f0bbd9abc18 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:56:47.296 7f0bbd9abc18 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:56:47.861 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:48.862 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:49.864 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:50.866 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:51.295 7f0bbd957438 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:56:51.868 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:52.870 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:53.872 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:54.874 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:55.875 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:56.296 7f0bbd957438 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:56:56.296 7f0bbd957438 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:56:56.297 7f0bbd957438 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:56:56.297 7f0bbd957438 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:56:56.297 7f0bbd957438 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:56:56.297 7f0bbd957438 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:56:56.297 7f0bbd957438 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:56:56.297 7f0bbd957438 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:56:56.298 7f0bbd957438 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:56:56.298 7f0bbd957438 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:56:56.298 7f0bbd957438 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:56:56.298 7f0bbd957438 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:56:56.298 7f0bbd957438 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:56:56.877 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:57.878 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:58.880 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:56:59.882 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:00.297 7f0bbd957438 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:57:00.884 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:02.840 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:03.842 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:04.844 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:05.299 7f0bbd957438 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:57:05.299 7f0bbd957438 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:57:05.299 7f0bbd957438 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:57:05.300 7f0bbd957438 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:57:05.300 7f0bbd957438 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:57:05.300 7f0bbd957438 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:57:05.300 7f0bbd957438 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:57:05.300 7f0bbd957438 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:57:05.301 7f0bbd957438 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:57:05.301 7f0bbd957438 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:57:05.301 7f0bbd957438 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:57:05.302 7f0bbd957438 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:57:05.302 7f0bbd957438 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:57:05.846 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:06.847 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:07.848 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:08.849 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:09.300 7f0bbd7eb240 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:57:09.851 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:10.852 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:11.854 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:12.856 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:13.857 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:14.302 7f0bbd7eb240 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:57:14.302 7f0bbd7eb240 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:57:14.302 7f0bbd7eb240 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:57:14.303 7f0bbd7eb240 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:57:14.303 7f0bbd7eb240 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:57:14.303 7f0bbd7eb240 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:57:14.303 7f0bbd7eb240 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:57:14.303 7f0bbd7eb240 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:57:14.303 7f0bbd7eb240 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:57:14.303 7f0bbd7eb240 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:57:14.304 7f0bbd7eb240 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:57:14.304 7f0bbd7eb240 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:57:14.304 7f0bbd7eb240 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:57:14.858 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:15.860 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:16.862 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:17.864 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:18.303 7f0bbd7eb240 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:57:18.866 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:19.868 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:20.870 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:21.871 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:22.872 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:23.305 7f0bbd7eb240 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:57:23.305 7f0bbd7eb240 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:57:23.306 7f0bbd7eb240 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:57:23.306 7f0bbd7eb240 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:57:23.306 7f0bbd7eb240 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:57:23.306 7f0bbd7eb240 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:57:23.306 7f0bbd7eb240 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:57:23.307 7f0bbd7eb240 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:57:23.307 7f0bbd7eb240 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:57:23.307 7f0bbd7eb240 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:57:23.307 7f0bbd7eb240 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:57:23.307 7f0bbd7eb240 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:57:23.307 7f0bbd7eb240 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:57:23.874 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:24.875 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:25.876 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:26.879 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:27.306 7f0bbd7eb240 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:57:27.880 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:28.882 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:29.884 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:30.886 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:32.308 7f0bbd7eb240 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:57:32.308 7f0bbd7eb240 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:57:32.308 7f0bbd7eb240 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:57:32.309 7f0bbd7eb240 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:57:32.309 7f0bbd7eb240 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:57:32.309 7f0bbd7eb240 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:57:32.309 7f0bbd7eb240 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:57:32.310 7f0bbd7eb240 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:57:32.310 7f0bbd7eb240 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:57:32.310 7f0bbd7eb240 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:57:32.310 7f0bbd7eb240 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:57:32.310 7f0bbd7eb240 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:57:32.310 7f0bbd7eb240 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:57:32.841 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:33.842 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:34.844 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:35.846 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:36.309 7f0bbd7eb240 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:57:36.847 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:37.849 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:38.851 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:39.852 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:40.854 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:41.311 7f0bbd7eb240 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:57:41.311 7f0bbd7eb240 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:57:41.311 7f0bbd7eb240 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:57:41.312 7f0bbd7eb240 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:57:41.312 7f0bbd7eb240 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:57:41.312 7f0bbd7eb240 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:57:41.312 7f0bbd7eb240 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:57:41.312 7f0bbd7eb240 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:57:41.313 7f0bbd7eb240 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:57:41.313 7f0bbd7eb240 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:57:41.313 7f0bbd7eb240 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:57:41.313 7f0bbd7eb240 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:57:41.313 7f0bbd7eb240 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:57:41.856 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:42.857 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:43.859 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:44.861 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:45.312 7f0bbd7eb240 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:57:45.863 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:46.864 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:47.866 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:48.868 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:49.869 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:50.314 7f0bbd7eb240 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:57:50.314 7f0bbd7eb240 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:57:50.314 7f0bbd7eb240 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:57:50.315 7f0bbd7eb240 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:57:50.315 7f0bbd7eb240 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:57:50.315 7f0bbd7eb240 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:57:50.315 7f0bbd7eb240 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:57:50.315 7f0bbd7eb240 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:57:50.316 7f0bbd7eb240 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:57:50.316 7f0bbd7eb240 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:57:50.316 7f0bbd7eb240 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:57:50.316 7f0bbd7eb240 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:57:50.316 7f0bbd7eb240 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:57:50.871 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:51.872 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:52.874 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:53.876 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:54.315 7f0bbd9dc780 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:57:54.877 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:55.878 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:56.880 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:57.882 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:58.883 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:57:59.317 7f0bbd9dc780 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:57:59.318 7f0bbd9dc780 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:57:59.318 7f0bbd9dc780 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:57:59.318 7f0bbd9dc780 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:57:59.318 7f0bbd9dc780 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:57:59.318 7f0bbd9dc780 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:57:59.319 7f0bbd9dc780 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:57:59.319 7f0bbd9dc780 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:57:59.319 7f0bbd9dc780 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:57:59.319 7f0bbd9dc780 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:57:59.319 7f0bbd9dc780 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:57:59.319 7f0bbd9dc780 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:57:59.319 7f0bbd9dc780 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:57:59.884 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:00.886 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:02.842 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:03.319 7f0bbd9dc780 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:58:03.845 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:04.846 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:05.848 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:06.850 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:07.851 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:08.320 7f0bbd9dc780 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:58:08.321 7f0bbd9dc780 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:58:08.321 7f0bbd9dc780 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:58:08.321 7f0bbd9dc780 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:58:08.321 7f0bbd9dc780 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:58:08.321 7f0bbd9dc780 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:58:08.322 7f0bbd9dc780 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:58:08.322 7f0bbd9dc780 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:58:08.322 7f0bbd9dc780 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:58:08.322 7f0bbd9dc780 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:58:08.322 7f0bbd9dc780 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:58:08.322 7f0bbd9dc780 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:58:08.322 7f0bbd9dc780 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:58:08.852 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:09.854 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:10.856 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:11.858 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:12.322 7f0bbd9dc780 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:58:12.859 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:13.861 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:14.862 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:15.864 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:16.865 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:17.323 7f0bbd9dc780 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:58:17.324 7f0bbd9dc780 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:58:17.324 7f0bbd9dc780 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:58:17.324 7f0bbd9dc780 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:58:17.324 7f0bbd9dc780 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:58:17.324 7f0bbd9dc780 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:58:17.325 7f0bbd9dc780 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:58:17.325 7f0bbd9dc780 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:58:17.325 7f0bbd9dc780 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:58:17.325 7f0bbd9dc780 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:58:17.325 7f0bbd9dc780 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:58:17.325 7f0bbd9dc780 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:58:17.325 7f0bbd9dc780 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:58:17.867 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:18.868 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:19.869 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:20.871 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:21.324 7f0bbd9dc780 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:58:21.873 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:22.874 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:23.876 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:24.878 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:25.880 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:26.326 7f0bbd9dc780 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:58:26.326 7f0bbd9dc780 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:58:26.326 7f0bbd9dc780 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:58:26.327 7f0bbd9dc780 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:58:26.327 7f0bbd9dc780 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:58:26.327 7f0bbd9dc780 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:58:26.327 7f0bbd9dc780 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:58:26.327 7f0bbd9dc780 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:58:26.328 7f0bbd9dc780 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:58:26.328 7f0bbd9dc780 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:58:26.328 7f0bbd9dc780 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:58:26.328 7f0bbd9dc780 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:58:26.328 7f0bbd9dc780 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:58:26.881 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:27.882 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:28.884 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:29.886 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:30.327 7f0bbd9dc780 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:58:30.887 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:32.843 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:33.844 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:34.142 7f0bbd7e3eb8 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 02/13/2025 02:58:34.194 7f0bbd7e3eb8 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 02/13/2025 02:58:34.847 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:35.329 7f0bbd9dc780 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 02/13/2025 02:58:35.329 7f0bbd9dc780 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/13/2025 02:58:35.329 7f0bbd9dc780 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:58:35.330 7f0bbd9dc780 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/13/2025 02:58:35.330 7f0bbd9dc780 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/13/2025 02:58:35.330 7f0bbd9dc780 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/13/2025 02:58:35.330 7f0bbd9dc780 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/13/2025 02:58:35.330 7f0bbd9dc780 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:58:35.331 7f0bbd9dc780 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-473-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.17:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.17:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.17:5673, ] status = Up description = >>, << type = Generic Connection name = Keystone server_addrs = [ http://252.17.40.195:5050/v3, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/13/2025 02:58:35.331 7f0bbd9dc780 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:58:35.331 7f0bbd9dc780 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:58:35.331 7f0bbd9dc780 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-473-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8084 start_time = 1739412996184383 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/13/2025 02:58:35.331 7f0bbd9dc780 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/13/2025 02:58:35.848 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:36.849 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:37.850 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:38.852 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:39.330 7f0bbd937d68 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/13/2025 02:58:39.853 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 02/13/2025 02:58:40.855 7f0bbdb298d0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process.