2026-01-30 18:12:44,145 - DEBUG - [10.0.0.38]: Running cmd : hostname
2026-01-30 18:12:44,354 - DEBUG - Output : cn-jenkins-deploy-platform-ansible-os-5150-1
2026-01-30 18:12:44,354 - DEBUG - [10.0.0.38]: Running cmd : hostname -f
2026-01-30 18:12:44,413 - DEBUG - Output : cn-jenkins-deploy-platform-ansible-os-5150-1.
2026-01-30 18:12:44,413 - DEBUG - [10.0.0.38]: Running cmd : docker ps 2>/dev/null | grep -v "/pause\|/usr/bin/pod\|nova_api_\|contrail.*init\|init.*contrail\|provisioner\|placement" | awk '{print $NF}'
2026-01-30 18:12:44,519 - DEBUG - Output : NAMES
contrail_test_wxV1YPIlg
vrouter_vrouter-agent_1
vrouter_nodemgr_1
rsyslogd_rsyslogd_1
analytics_snmp_snmp-collector_1
analytics_snmp_topology_1
analytics_snmp_nodemgr_1
analytics_alarm_nodemgr_1
analytics_alarm_alarm-gen_1
analytics_alarm_kafka_1
analytics_collector_1
analytics_api_1
analytics_nodemgr_1
analytics_database_nodemgr_1
analytics_database_query-engine_1
analytics_database_cassandra_1
control_named_1
control_nodemgr_1
control_dns_1
control_control_1
webui_web_1
webui_job_1
config_devicemgr_1
config_svcmonitor_1
config_nodemgr_1
config_dnsmasq_1
config_schema_1
config_api_1
config_database_nodemgr_1
config_database_rabbitmq_1
config_database_zookeeper_1
config_database_cassandra_1
redis_redis_1
barbican_worker
barbican_keystone_listener
barbican_api
horizon
heat_engine
heat_api_cfn
heat_api
neutron_server
nova_compute
nova_libvirt
nova_ssh
nova_novncproxy
nova_conductor
nova_api
nova_scheduler
glance_api
keystone
keystone_fernet
keystone_ssh
rabbitmq
memcached
mariadb
cron
kolla_toolbox
fluentd
2026-01-30 18:12:44,519 - DEBUG - [10.0.0.38]: Running cmd : getent hosts 10.0.0.38 | head -n 1 | awk '{print $2}'
2026-01-30 18:12:44,580 - DEBUG - Output : cn-jenkins-deploy-platform-ansible-os-5150-1.
2026-01-30 18:12:44,580 - DEBUG - [10.0.0.38]: Running cmd : ip -4 -o addr show | awk '{print $4}'
2026-01-30 18:12:44,652 - DEBUG - Output : 127.0.0.1/8
10.0.0.38/24
172.17.0.1/16
10.20.0.17/24
2026-01-30 18:12:44,652 - DEBUG - [10.0.0.38]: Running cmd : ip -4 -o addr show dev vhost0 | awk '{print $4}'
2026-01-30 18:12:44,709 - DEBUG - Output : 10.20.0.17/24
2026-01-30 18:12:44,709 - DEBUG - [10.0.0.38]: Running cmd : getent hosts 10.20.0.17 | head -n 1 | awk '{print $2}'
2026-01-30 18:12:44,773 - DEBUG - Output :
2026-01-30 18:12:46,533 - INFO - Using existing project ['default-domain', 'admin'](367b3493-15db-4f0a-931f-f7427769564a)
2026-01-30 18:12:46,665 - INFO - ================================================================================
2026-01-30 18:12:46,665 - INFO - STARTING TEST : test_contrail_status
2026-01-30 18:12:46,666 - INFO - TEST DESCRIPTION : Test to verify that all services are running and active
2026-01-30 18:12:46,787 - WARNING - Test is running with cores: {'10.0.0.38': ['core.isc-loop-0001.70298.cn-jenkins-deploy-platform-ansible-os-5150-1.1769796495']}
2026-01-30 18:12:46,787 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:12:46,794 - DEBUG - Nothing to compare xmpp stats {'10.0.0.38': {'10.20.0.17': '0'}} with
2026-01-30 18:12:46,794 - INFO - Initial checks done. Running the testcase now
2026-01-30 18:12:46,794 - INFO -
2026-01-30 18:12:46,794 - DEBUG - [10.0.0.38]: Running cmd : docker ps -f status=running --format {{.Names}} 2>/dev/null
2026-01-30 18:12:46,885 - DEBUG - Output : contrail_test_wxV1YPIlg
vrouter_vrouter-agent_1
vrouter_nodemgr_1
vrouter_provisioner_1
rsyslogd_rsyslogd_1
analytics_snmp_snmp-collector_1
analytics_snmp_topology_1
analytics_snmp_nodemgr_1
analytics_snmp_provisioner_1
analytics_alarm_nodemgr_1
analytics_alarm_alarm-gen_1
analytics_alarm_provisioner_1
analytics_alarm_kafka_1
analytics_collector_1
analytics_api_1
analytics_provisioner_1
analytics_nodemgr_1
analytics_database_nodemgr_1
analytics_database_provisioner_1
analytics_database_query-engine_1
analytics_database_cassandra_1
control_named_1
control_nodemgr_1
control_dns_1
control_provisioner_1
control_control_1
webui_web_1
webui_job_1
config_devicemgr_1
config_svcmonitor_1
config_nodemgr_1
config_dnsmasq_1
config_schema_1
config_provisioner_1
config_api_1
config_database_nodemgr_1
config_database_rabbitmq_1
config_database_provisioner_1
config_database_zookeeper_1
config_database_cassandra_1
redis_redis_1
barbican_worker
barbican_keystone_listener
barbican_api
horizon
heat_engine
heat_api_cfn
heat_api
neutron_server
nova_compute
nova_libvirt
nova_ssh
nova_novncproxy
nova_conductor
nova_api
nova_scheduler
placement_api
glance_api
keystone
keystone_fernet
keystone_ssh
rabbitmq
memcached
mariadb
cron
kolla_toolbox
fluentd
2026-01-30 18:12:46,885 - INFO - 10.0.0.38
2026-01-30 18:12:46,885 - INFO - config
2026-01-30 18:12:46,890 - INFO - config-nodemgr:active
2026-01-30 18:12:46,895 - INFO - api-server:active
2026-01-30 18:12:46,900 - INFO - schema:active
2026-01-30 18:12:46,905 - INFO - svc-monitor:active
2026-01-30 18:12:46,909 - INFO - device-manager:active
2026-01-30 18:12:46,909 - INFO - config-database
2026-01-30 18:12:46,910 - INFO - config-cassandra:active
2026-01-30 18:12:46,910 - INFO - config-zookeeper:active
2026-01-30 18:12:46,910 - INFO - config-rabbitmq:active
2026-01-30 18:12:46,910 - INFO - vrouter
2026-01-30 18:12:46,915 - INFO - vrouter-nodemgr:active
2026-01-30 18:12:46,920 - INFO - agent:active
2026-01-30 18:12:46,920 - INFO - control
2026-01-30 18:12:46,925 - INFO - control-nodemgr:active
2026-01-30 18:12:46,930 - INFO - control:active
2026-01-30 18:12:46,930 - INFO - named:active
2026-01-30 18:12:46,935 - INFO - dns:active
2026-01-30 18:12:46,935 - INFO - analytics
2026-01-30 18:12:46,941 - INFO - analytics-nodemgr:active
2026-01-30 18:12:46,945 - INFO - analytics-api:active
2026-01-30 18:12:46,949 - INFO - collector:active
2026-01-30 18:12:46,949 - INFO - analytics-database
2026-01-30 18:12:46,949 - INFO - analytics-cassandra:active
2026-01-30 18:12:46,954 - INFO - analyticsdb-nodemgr:active
2026-01-30 18:12:46,959 - INFO - query-engine:active
2026-01-30 18:12:46,959 - INFO - webui
2026-01-30 18:12:46,959 - INFO - webui:active
2026-01-30 18:12:46,959 - INFO - webui-middleware:active
2026-01-30 18:12:46,959 - INFO - redis:active
2026-01-30 18:12:46,959 - INFO - analytics_snmp
2026-01-30 18:12:46,963 - INFO - snmp-collector:active
2026-01-30 18:12:46,963 - INFO - snmp-topology:active
2026-01-30 18:12:46,963 - INFO - analytics_alarm
2026-01-30 18:12:46,963 - INFO - alarmgen:active
2026-01-30 18:12:46,963 - INFO - All the contrail services are up on all nodes
2026-01-30 18:12:46,963 - INFO - contrail-status passed
2026-01-30 18:12:47,095 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:12:47,101 - DEBUG - No XMPP flaps were noticed during the test
2026-01-30 18:12:47,101 - INFO - END TEST : test_contrail_status : PASSED[0:00:01]
2026-01-30 18:12:47,102 - INFO - --------------------------------------------------------------------------------
2026-01-30 18:12:49,375 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:12:51,591 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-AnalyticsBasicTestSanity-64147306:ctest-vn22-07386657:ctest-vn22-07386657
2026-01-30 18:12:51,597 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-AnalyticsBasicTestSanity-64147306:ctest-vn22-07386657:ctest-vn22-07386657
2026-01-30 18:12:51,601 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-AnalyticsBasicTestSanity-64147306:ctest-vn22-07386657:ctest-vn22-07386657
2026-01-30 18:13:07,356 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VnListReq?name=default-domain:ctest-AnalyticsBasicTestSanity-64147306:ctest-vn22-07386657
2026-01-30 18:13:07,362 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-AnalyticsBasicTestSanity-64147306:ctest-vn22-07386657:ctest-vn22-07386657
2026-01-30 18:13:07,366 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-AnalyticsBasicTestSanity-64147306:ctest-vn22-07386657:ctest-vn22-07386657
2026-01-30 18:13:07,494 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:13:08,519 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:13:08,524 - DEBUG - [10.0.0.38]: Running cmd : docker exec --privileged -it config_api_1 /bin/bash -c 'db-manage check'
2026-01-30 18:13:11,816 - DEBUG - Output : 2026-01-30 18:13:11,059 INFO: (v1.41) Checker check_zk_mode_and_node_count: Success
2026-01-30 18:13:11,086 INFO: (v1.41) Checker check_cassandra_keyspace_replication: Success
2026-01-30 18:13:11,182 INFO: (v1.41) Checker check_obj_mandatory_fields: Success
2026-01-30 18:13:11,367 INFO: (v1.41) Checker check_orphan_resources: Success
2026-01-30 18:13:11,548 INFO: (v1.41) Checker check_fq_name_uuid_match: Success
2026-01-30 18:13:11,548 WARNING: Be careful, that check can return false positive errors if stale FQ names and stale resources were not cleaned before. Run at least commands 'clean_obj_missing_mandatory_fields', 'clean_orphan_resources' and 'clean_stale_fq_names' before.
2026-01-30 18:13:11,641 INFO: (v1.41) Checker check_duplicate_fq_name: Success
2026-01-30 18:13:11,653 INFO: (v1.41) Checker check_route_targets_routing_instance_backrefs: Success
2026-01-30 18:13:11,658 INFO: (v1.41) Checker check_subnet_uuid: Success
2026-01-30 18:13:11,666 ERROR: (v1.41) Checker check_subnet_addr_alloc: Failed:
ZK subnet path '/api-server/subnets' does ends with a valid IP network
2026-01-30 18:13:11,690 INFO: (v1.41) Checker check_route_targets_id: Success
2026-01-30 18:13:11,699 INFO: (v1.41) Checker check_virtual_networks_id: Success
2026-01-30 18:13:11,706 INFO: (v1.41) Checker check_security_groups_id: Success
2026-01-30 18:13:11,709 INFO: (v1.41) Checker check_ipam_subnet_method: Success
2026-01-30 18:13:11,711 INFO: (v1.41) Checker check_aggregated_ethernet_id: Success
2026-01-30 18:13:11,987 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:13:12,315 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:13:12,474 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:13:12,631 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:13:12,787 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:13:15,429 - INFO - admin user already present
2026-01-30 18:13:16,239 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:13:18,555 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-FloatingipBasicTestSanity-60887965:ctest-fvn-73151199:ctest-fvn-73151199
2026-01-30 18:13:18,560 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-FloatingipBasicTestSanity-60887965:ctest-fvn-73151199:ctest-fvn-73151199
2026-01-30 18:13:18,564 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-FloatingipBasicTestSanity-60887965:ctest-fvn-73151199:ctest-fvn-73151199
2026-01-30 18:13:19,415 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-FloatingipBasicTestSanity-60887965:ctest-vn1-51748135:ctest-vn1-51748135
2026-01-30 18:13:19,419 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-FloatingipBasicTestSanity-60887965:ctest-vn1-51748135:ctest-vn1-51748135
2026-01-30 18:13:19,424 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-FloatingipBasicTestSanity-60887965:ctest-vn1-51748135:ctest-vn1-51748135
2026-01-30 18:13:22,808 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:247323e8-8bee-4b64-a65a-6a4ca1b016d4
2026-01-30 18:13:22,814 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-FloatingipBasicTestSanity-60887965:ctest-vn1-51748135:ctest-vn1-51748135
2026-01-30 18:13:22,820 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:247323e8-8bee-4b64-a65a-6a4ca1b016d4
2026-01-30 18:13:37,851 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:b2667265-2ae0-4b4d-be1b-153510f211d7
2026-01-30 18:13:37,857 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-FloatingipBasicTestSanity-60887965:ctest-fvn-73151199:ctest-fvn-73151199
2026-01-30 18:13:37,862 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:b2667265-2ae0-4b4d-be1b-153510f211d7
2026-01-30 18:13:39,375 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:247323e8-8bee-4b64-a65a-6a4ca1b016d4
2026-01-30 18:13:39,381 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VnListReq?name=default-domain:ctest-FloatingipBasicTestSanity-60887965:ctest-fvn-73151199
2026-01-30 18:13:39,385 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-FloatingipBasicTestSanity-60887965:ctest-fvn-73151199:ctest-fvn-73151199
2026-01-30 18:13:39,390 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_Inet4UcRouteReq?x=3
2026-01-30 18:13:44,983 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VnListReq?name=default-domain:ctest-FloatingipBasicTestSanity-60887965:ctest-vn1-51748135
2026-01-30 18:13:44,990 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-FloatingipBasicTestSanity-60887965:ctest-vn1-51748135:ctest-vn1-51748135
2026-01-30 18:13:44,997 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-FloatingipBasicTestSanity-60887965:ctest-vn1-51748135:ctest-vn1-51748135
2026-01-30 18:13:45,274 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VnListReq?name=default-domain:ctest-FloatingipBasicTestSanity-60887965:ctest-fvn-73151199
2026-01-30 18:13:45,278 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-FloatingipBasicTestSanity-60887965:ctest-fvn-73151199:ctest-fvn-73151199
2026-01-30 18:13:45,283 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-FloatingipBasicTestSanity-60887965:ctest-fvn-73151199:ctest-fvn-73151199
2026-01-30 18:13:45,546 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:13:48,822 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:13:49,206 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:13:55,541 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:13:57,709 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:14:00,889 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:14:02,925 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:14:03,213 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:14:10,306 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:a44dae91-6e5f-4a2f-ade7-f64dc4acb5c4
2026-01-30 18:14:10,311 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestPorts-12360373:ctest-vn1-70384125:ctest-vn1-70384125
2026-01-30 18:14:10,316 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:a44dae91-6e5f-4a2f-ade7-f64dc4acb5c4
2026-01-30 18:14:21,348 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:6be944c7-ab2d-459b-8c87-90577c11ea25
2026-01-30 18:14:21,356 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestPorts-12360373:ctest-vn1-70384125:ctest-vn1-70384125
2026-01-30 18:14:21,361 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:6be944c7-ab2d-459b-8c87-90577c11ea25
2026-01-30 18:14:26,601 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:14:29,982 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:14:39,874 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:4b6dec55-2241-40f0-bb71-c3bcdf3a7891
2026-01-30 18:14:39,881 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestRouters-31138334:ctest-vn1-07093697:ctest-vn1-07093697
2026-01-30 18:14:39,886 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:4b6dec55-2241-40f0-bb71-c3bcdf3a7891
2026-01-30 18:14:45,810 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:e6b4ff64-e407-4337-a0ee-82d38a2cb48e
2026-01-30 18:14:45,817 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestRouters-31138334:ctest-vn2-32746196:ctest-vn2-32746196
2026-01-30 18:14:45,823 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:e6b4ff64-e407-4337-a0ee-82d38a2cb48e
2026-01-30 18:15:07,646 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:15:10,878 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:15:21,678 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:47635a87-4267-4025-bcfa-e58fe6be774c
2026-01-30 18:15:21,685 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestRoutersBasic-61038947:ctest-vn-63786617:ctest-vn-63786617
2026-01-30 18:15:21,691 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:47635a87-4267-4025-bcfa-e58fe6be774c
2026-01-30 18:15:27,566 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:30b4a5f3-eaca-42e3-911c-fa14e327f9f9
2026-01-30 18:15:27,572 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestRoutersBasic-61038947:ctest-ext_vn-42972216:ctest-ext_vn-42972216
2026-01-30 18:15:27,576 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:30b4a5f3-eaca-42e3-911c-fa14e327f9f9
2026-01-30 18:15:35,669 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:15:38,829 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:15:42,385 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicyIpv4v6-24720825:ctest-vn-55964379:ctest-vn-55964379
2026-01-30 18:15:42,389 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicyIpv4v6-24720825:ctest-vn-55964379:ctest-vn-55964379
2026-01-30 18:15:42,394 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicyIpv4v6-24720825:ctest-vn-55964379:ctest-vn-55964379
2026-01-30 18:15:42,680 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicyIpv4v6-24720825:ctest-vn-79933855:ctest-vn-79933855
2026-01-30 18:15:42,684 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicyIpv4v6-24720825:ctest-vn-79933855:ctest-vn-79933855
2026-01-30 18:15:42,688 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicyIpv4v6-24720825:ctest-vn-79933855:ctest-vn-79933855
2026-01-30 18:15:43,229 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VnListReq?name=default-domain:ctest-TestBasicPolicyIpv4v6-24720825:ctest-vn-79933855
2026-01-30 18:15:43,233 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicyIpv4v6-24720825:ctest-vn-79933855:ctest-vn-79933855
2026-01-30 18:15:43,238 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicyIpv4v6-24720825:ctest-vn-79933855:ctest-vn-79933855
2026-01-30 18:15:43,547 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VnListReq?name=default-domain:ctest-TestBasicPolicyIpv4v6-24720825:ctest-vn-55964379
2026-01-30 18:15:43,552 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicyIpv4v6-24720825:ctest-vn-55964379:ctest-vn-55964379
2026-01-30 18:15:43,557 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicyIpv4v6-24720825:ctest-vn-55964379:ctest-vn-55964379
2026-01-30 18:15:43,822 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:15:46,994 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:15:47,282 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:15:49,941 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicyNegativeIpv4v6-25630737:ctest-vn4-12180297:ctest-vn4-12180297
2026-01-30 18:15:49,946 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicyNegativeIpv4v6-25630737:ctest-vn4-12180297:ctest-vn4-12180297
2026-01-30 18:15:49,951 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicyNegativeIpv4v6-25630737:ctest-vn4-12180297:ctest-vn4-12180297
2026-01-30 18:15:50,308 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VnListReq?name=default-domain:ctest-TestBasicPolicyNegativeIpv4v6-25630737:ctest-vn4-12180297
2026-01-30 18:15:50,314 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicyNegativeIpv4v6-25630737:ctest-vn4-12180297:ctest-vn4-12180297
2026-01-30 18:15:50,319 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicyNegativeIpv4v6-25630737:ctest-vn4-12180297:ctest-vn4-12180297
2026-01-30 18:15:50,769 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:15:51,058 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:15:54,221 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:15:57,356 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicy-16623861:ctest-vn-10789457:ctest-vn-10789457
2026-01-30 18:15:57,362 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicy-16623861:ctest-vn-10789457:ctest-vn-10789457
2026-01-30 18:15:57,366 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicy-16623861:ctest-vn-10789457:ctest-vn-10789457
2026-01-30 18:15:57,648 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicy-16623861:ctest-vn-33469601:ctest-vn-33469601
2026-01-30 18:15:57,652 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicy-16623861:ctest-vn-33469601:ctest-vn-33469601
2026-01-30 18:15:57,656 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicy-16623861:ctest-vn-33469601:ctest-vn-33469601
2026-01-30 18:21:24,785 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VnListReq?name=default-domain:ctest-TestBasicPolicy-16623861:ctest-vn-33469601
2026-01-30 18:21:24,789 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicy-16623861:ctest-vn-33469601:ctest-vn-33469601
2026-01-30 18:21:24,794 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicy-16623861:ctest-vn-33469601:ctest-vn-33469601
2026-01-30 18:21:25,016 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VnListReq?name=default-domain:ctest-TestBasicPolicy-16623861:ctest-vn-10789457
2026-01-30 18:21:25,021 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicy-16623861:ctest-vn-10789457:ctest-vn-10789457
2026-01-30 18:21:25,026 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicPolicy-16623861:ctest-vn-10789457:ctest-vn-10789457
2026-01-30 18:21:25,298 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:21:28,598 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:21:33,366 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:21:36,554 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:21:38,402 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-22268631:ctest-mgmt_vn-01213623:ctest-mgmt_vn-01213623
2026-01-30 18:21:38,407 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-22268631:ctest-mgmt_vn-01213623:ctest-mgmt_vn-01213623
2026-01-30 18:21:38,411 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-22268631:ctest-mgmt_vn-01213623:ctest-mgmt_vn-01213623
2026-01-30 18:21:39,218 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-22268631:ctest-left_vn-62233570:ctest-left_vn-62233570
2026-01-30 18:21:39,222 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-22268631:ctest-left_vn-62233570:ctest-left_vn-62233570
2026-01-30 18:21:39,227 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-22268631:ctest-left_vn-62233570:ctest-left_vn-62233570
2026-01-30 18:21:40,070 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-22268631:ctest-right_vn-14540017:ctest-right_vn-14540017
2026-01-30 18:21:40,075 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-22268631:ctest-right_vn-14540017:ctest-right_vn-14540017
2026-01-30 18:21:40,079 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-22268631:ctest-right_vn-14540017:ctest-right_vn-14540017
2026-01-30 18:21:45,273 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VnListReq?name=default-domain:ctest-TestSvcRegr-22268631:ctest-right_vn-14540017
2026-01-30 18:21:45,278 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-22268631:ctest-right_vn-14540017:ctest-right_vn-14540017
2026-01-30 18:21:45,289 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-22268631:ctest-right_vn-14540017:ctest-right_vn-14540017
2026-01-30 18:21:45,529 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VnListReq?name=default-domain:ctest-TestSvcRegr-22268631:ctest-left_vn-62233570
2026-01-30 18:21:45,533 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-22268631:ctest-left_vn-62233570:ctest-left_vn-62233570
2026-01-30 18:21:45,539 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-22268631:ctest-left_vn-62233570:ctest-left_vn-62233570
2026-01-30 18:21:45,781 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VnListReq?name=default-domain:ctest-TestSvcRegr-22268631:ctest-mgmt_vn-01213623
2026-01-30 18:21:45,785 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-22268631:ctest-mgmt_vn-01213623:ctest-mgmt_vn-01213623
2026-01-30 18:21:45,790 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestSvcRegr-22268631:ctest-mgmt_vn-01213623:ctest-mgmt_vn-01213623
2026-01-30 18:21:46,050 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:21:49,258 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:21:58,286 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:22:01,426 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:22:05,338 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:22:08,537 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:22:11,318 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:22:11,610 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:22:14,082 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:22:14,382 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:22:18,698 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicVMVN-87615662:ctest-vn-48174147:ctest-vn-48174147
2026-01-30 18:22:18,702 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicVMVN-87615662:ctest-vn-48174147:ctest-vn-48174147
2026-01-30 18:22:18,707 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicVMVN-87615662:ctest-vn-48174147:ctest-vn-48174147
2026-01-30 18:22:19,606 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VnListReq?name=default-domain:ctest-TestBasicVMVN-87615662:ctest-vn-48174147
2026-01-30 18:22:19,611 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicVMVN-87615662:ctest-vn-48174147:ctest-vn-48174147
2026-01-30 18:22:19,615 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_VrfListReq?name=default-domain:ctest-TestBasicVMVN-87615662:ctest-vn-48174147:ctest-vn-48174147
2026-01-30 18:22:19,958 - DEBUG - Requesting: http://10.0.0.38:8085/Snh_AgentXmppConnectionStatusReq?
2026-01-30 18:24:06,689 - INFO - Using existing project ['default-domain', 'admin'](367b3493-15db-4f0a-931f-f7427769564a)
2026-01-30 18:24:06,697 - INFO - ================================================================================
2026-01-30 18:24:06,697 - INFO - STARTING TEST : test_contrail_status
2026-01-30 18:24:06,698 - INFO - TEST DESCRIPTION : Test to verify that all services are running and active
2026-01-30 18:24:06,819 - WARNING - Test is running with cores: {'10.0.0.38': ['core.isc-loop-0001.70298.cn-jenkins-deploy-platform-ansible-os-5150-1.1769796495']}
2026-01-30 18:24:06,819 - DEBUG - Skipping xmpp flap check
2026-01-30 18:24:06,819 - INFO - Initial checks done. Running the testcase now
2026-01-30 18:24:06,819 - INFO -
2026-01-30 18:24:06,926 - INFO - 10.0.0.38
2026-01-30 18:24:06,926 - INFO - config
2026-01-30 18:24:06,932 - INFO - config-nodemgr:active
2026-01-30 18:24:06,935 - INFO - api-server:active
2026-01-30 18:24:06,939 - INFO - schema:active
2026-01-30 18:24:06,943 - INFO - svc-monitor:active
2026-01-30 18:24:06,947 - INFO - device-manager:active
2026-01-30 18:24:06,947 - INFO - config-database
2026-01-30 18:24:06,947 - INFO - config-cassandra:active
2026-01-30 18:24:06,947 - INFO - config-zookeeper:active
2026-01-30 18:24:06,947 - INFO - config-rabbitmq:active
2026-01-30 18:24:06,947 - INFO - vrouter
2026-01-30 18:24:06,951 - INFO - vrouter-nodemgr:active
2026-01-30 18:24:06,955 - INFO - agent:active
2026-01-30 18:24:06,955 - INFO - control
2026-01-30 18:24:06,959 - INFO - control-nodemgr:active
2026-01-30 18:24:06,965 - INFO - control:active
2026-01-30 18:24:06,965 - INFO - named:active
2026-01-30 18:24:06,970 - INFO - dns:active
2026-01-30 18:24:06,970 - INFO - analytics
2026-01-30 18:24:06,974 - INFO - analytics-nodemgr:active
2026-01-30 18:24:06,978 - INFO - analytics-api:active
2026-01-30 18:24:06,982 - INFO - collector:active
2026-01-30 18:24:06,982 - INFO - analytics-database
2026-01-30 18:24:06,982 - INFO - analytics-cassandra:active
2026-01-30 18:24:06,987 - INFO - analyticsdb-nodemgr:active
2026-01-30 18:24:06,991 - INFO - query-engine:active
2026-01-30 18:24:06,991 - INFO - webui
2026-01-30 18:24:06,991 - INFO - webui:active
2026-01-30 18:24:06,992 - INFO - webui-middleware:active
2026-01-30 18:24:06,992 - INFO - redis:active
2026-01-30 18:24:06,992 - INFO - analytics_snmp
2026-01-30 18:24:06,995 - INFO - snmp-collector:active
2026-01-30 18:24:06,996 - INFO - snmp-topology:active
2026-01-30 18:24:06,996 - INFO - analytics_alarm
2026-01-30 18:24:06,996 - INFO - alarmgen:active
2026-01-30 18:24:06,996 - INFO - All the contrail services are up on all nodes
2026-01-30 18:24:06,996 - INFO - contrail-status passed
2026-01-30 18:24:07,119 - DEBUG - Skipping xmpp flap check
2026-01-30 18:24:07,119 - INFO - END TEST : test_contrail_status : PASSED[0:00:01]
2026-01-30 18:24:07,119 - INFO - --------------------------------------------------------------------------------