2025-07-21 11:56:06,227 - INFO - Using existing project ['default-domain', 'admin'](847418d9-7445-491b-a497-5d2cf09a838c) 2025-07-21 11:56:06,242 - INFO - ================================================================================ 2025-07-21 11:56:06,243 - INFO - STARTING TEST : test_db_nodemgr_status 2025-07-21 11:56:06,243 - INFO - TEST DESCRIPTION : Test to verify db nodemgr status 2025-07-21 11:56:06,406 - DEBUG - Nothing to compare xmpp stats {'10.0.0.254': {'10.20.0.17': '0'}} with 2025-07-21 11:56:06,406 - INFO - Initial checks done. Running the testcase now 2025-07-21 11:56:06,406 - INFO - 2025-07-21 11:56:06,406 - DEBUG - Verifying if db node_mgr running... 2025-07-21 11:56:06,406 - INFO - Verifying through collector 10.0.0.254 for db node cn-jenkins-deploy-platform-ansible-os-3677-1. 2025-07-21 11:56:06,625 - DEBUG - No XMPP flaps were noticed during the test
2025-07-21 11:56:06,626 - INFO - END TEST : test_db_nodemgr_status : PASSED[0:00:00]
2025-07-21 11:56:06,626 - INFO - -------------------------------------------------------------------------------- 2025-07-21 11:56:06,634 - INFO - ================================================================================ 2025-07-21 11:56:06,634 - INFO - STARTING TEST : test_verify_generator_collector_connections 2025-07-21 11:56:06,634 - INFO - TEST DESCRIPTION : Description: Verify generator:module connections to collector 1.Verify all generators connected to collector - fails otherwise 2.Get the xmpp peers in vrouter uve and get the active xmpp peer out of it 3.Verify from agent introspect that active xmpp matches with step 2 - fails otherwise 4.Get bgp peers from bgp-peer uve and verify from control node introspect that that matches - fails otherwise Maintainer: sandipd@juniper.net 2025-07-21 11:56:06,841 - DEBUG - Nothing to compare xmpp stats {'10.0.0.254': {'10.20.0.17': '0'}} with 2025-07-21 11:56:06,841 - INFO - Initial checks done. Running the testcase now 2025-07-21 11:56:06,841 - INFO - 2025-07-21 11:56:06,861 - INFO - IP 10.0.0.254 port 8083 connected to collector 10.0.0.254 2025-07-21 11:56:06,875 - INFO - IP 10.0.0.254 port 8084 connected to collector 10.0.0.254 2025-07-21 11:56:06,886 - INFO - IP 10.0.0.254 port 8087 connected to collector 10.0.0.254 2025-07-21 11:56:06,898 - INFO - IP 10.0.0.254 port 8088 connected to collector 10.0.0.254 2025-07-21 11:56:06,906 - INFO - IP 10.0.0.254 port 8090 connected to collector 10.0.0.254 2025-07-21 11:56:06,923 - INFO - IP 10.0.0.254 port 8091 connected to collector 10.0.0.254 2025-07-21 11:56:06,923 - DEBUG - Verifying through opserver in 10.0.0.254 2025-07-21 11:56:06,947 - WARNING - ops generator uve analytics/uves/generator/cn-jenkins-deploy-platform-ansible-os-3677-1-test:Control:contrail-control?flat not returned 2025-07-21 11:56:06,947 - ERROR - query returned none 2025-07-21 11:56:06,947 - WARNING - cn-jenkins-deploy-platform-ansible-os-3677-1-test:Control:contrail-control:0 is NOT connected to collector 10.0.0.254 2025-07-21 11:56:08,948 - DEBUG - Verifying through opserver in 10.0.0.254 2025-07-21 11:56:08,976 - WARNING - ops generator uve analytics/uves/generator/cn-jenkins-deploy-platform-ansible-os-3677-1-test:Control:contrail-control?flat not returned 2025-07-21 11:56:08,976 - ERROR - query returned none 2025-07-21 11:56:08,977 - WARNING - cn-jenkins-deploy-platform-ansible-os-3677-1-test:Control:contrail-control:0 is NOT connected to collector 10.0.0.254 2025-07-21 11:56:10,978 - DEBUG - Verifying through opserver in 10.0.0.254 2025-07-21 11:56:10,995 - WARNING - ops generator uve analytics/uves/generator/cn-jenkins-deploy-platform-ansible-os-3677-1-test:Control:contrail-control?flat not returned 2025-07-21 11:56:10,995 - ERROR - query returned none 2025-07-21 11:56:10,995 - WARNING - cn-jenkins-deploy-platform-ansible-os-3677-1-test:Control:contrail-control:0 is NOT connected to collector 10.0.0.254 2025-07-21 11:56:10,995 - ERROR - Unable to find generator for module contrail-control 2025-07-21 11:56:11,170 - ERROR - AssertionError Python 3.6.8: /usr/bin/python3 Mon Jul 21 11:56:10 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={}) 78 log.info('Initial checks done. Running the testcase now') 79 log.info('') 80 result = function(self, *args, **kwargs) 81 if self.inputs.upgrade: 82 pid = os.getpid() result = None function = self = args = () kwargs = {} /contrail-test/scripts/analytics/test_analytics.py in test_verify_generator_collector_connections(self=) 309 ''' 310 # check collector-generator connections through uves. 311 assert self.analytics_obj.verify_collector_uve() 312 # Verify vrouter uve active xmpp connections 313 assert self.analytics_obj.verify_active_xmpp_peer_in_vrouter_uve() self = self.analytics_obj = self.analytics_obj.verify_collector_uve = > AssertionError: __cause__ = None __class__ = __context__ = None __delattr__ = __dict__ = {} __dir__ = __doc__ = 'Assertion failed.' __eq__ = __format__ = __ge__ = __getattribute__ = __gt__ = __hash__ = __init__ = __init_subclass__ = __le__ = __lt__ = __ne__ = __new__ = __reduce__ = __reduce_ex__ = __repr__ = __setattr__ = __setstate__ = __sizeof__ = __str__ = __subclasshook__ = __suppress_context__ = False __traceback__ = args = () with_traceback = The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper result = function(self, *args, **kwargs) File "/contrail-test/scripts/analytics/test_analytics.py", line 311, in test_verify_generator_collector_connections assert self.analytics_obj.verify_collector_uve() AssertionError 2025-07-21 11:56:11,190 - DEBUG - No XMPP flaps were noticed during the test 2025-07-21 11:56:11,190 - INFO -
2025-07-21 11:56:11,190 - INFO - END TEST : test_verify_generator_collector_connections : FAILED[0:00:05]
2025-07-21 11:56:11,191 - INFO - -------------------------------------------------------------------------------- 2025-07-21 11:56:11,202 - INFO - ================================================================================ 2025-07-21 11:56:11,202 - INFO - STARTING TEST : test_verify_generator_connections_to_collector_node 2025-07-21 11:56:11,203 - INFO - TEST DESCRIPTION : Test to validate generator connections 2025-07-21 11:56:11,410 - DEBUG - Nothing to compare xmpp stats {'10.0.0.254': {'10.20.0.17': '0'}} with 2025-07-21 11:56:11,410 - INFO - Initial checks done. Running the testcase now 2025-07-21 11:56:11,410 - INFO - 2025-07-21 11:56:11,410 - DEBUG - Verifying through opserver in 10.0.0.254 2025-07-21 11:56:11,438 - WARNING - ops generator uve analytics/uves/generator/cn-jenkins-deploy-platform-ansible-os-3677-1-test:Compute:contrail-vrouter-agent?flat not returned 2025-07-21 11:56:11,438 - ERROR - query returned none 2025-07-21 11:56:11,438 - WARNING - cn-jenkins-deploy-platform-ansible-os-3677-1-test:Compute:contrail-vrouter-agent:0 is NOT connected to collector 10.0.0.254 2025-07-21 11:56:13,439 - DEBUG - Verifying through opserver in 10.0.0.254 2025-07-21 11:56:13,474 - WARNING - ops generator uve analytics/uves/generator/cn-jenkins-deploy-platform-ansible-os-3677-1-test:Compute:contrail-vrouter-agent?flat not returned 2025-07-21 11:56:13,475 - ERROR - query returned none 2025-07-21 11:56:13,475 - WARNING - cn-jenkins-deploy-platform-ansible-os-3677-1-test:Compute:contrail-vrouter-agent:0 is NOT connected to collector 10.0.0.254 2025-07-21 11:56:15,477 - DEBUG - Verifying through opserver in 10.0.0.254 2025-07-21 11:56:15,512 - WARNING - ops generator uve analytics/uves/generator/cn-jenkins-deploy-platform-ansible-os-3677-1-test:Compute:contrail-vrouter-agent?flat not returned 2025-07-21 11:56:15,512 - ERROR - query returned none 2025-07-21 11:56:15,512 - WARNING - cn-jenkins-deploy-platform-ansible-os-3677-1-test:Compute:contrail-vrouter-agent:0 is NOT connected to collector 10.0.0.254 2025-07-21 11:56:15,711 - ERROR - AssertionError Python 3.6.8: /usr/bin/python3 Mon Jul 21 11:56:15 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={}) 78 log.info('Initial checks done. Running the testcase now') 79 log.info('') 80 result = function(self, *args, **kwargs) 81 if self.inputs.upgrade: 82 pid = os.getpid() result = None function = self = args = () kwargs = {} /contrail-test/scripts/analytics/test_analytics.py in test_verify_generator_connections_to_collector_node(self=) 357 358 ''' 359 self.analytics_obj.verify_generator_connection_to_collector() 360 361 @test.attr(type=['sanity', 'dev_sanity_dpdk']) self = self.analytics_obj = self.analytics_obj.verify_generator_connection_to_collector = > /contrail-test/tcutils/collector/analytics_tests.py in verify_generator_connection_to_collector(self=) 201 for elem in v: 202 assert self.verify_connection_status( 203 name,elem,k) 204 if (k == 'Analytics'): 205 for name in self.inputs.collector_names: name = 'cn-jenkins-deploy-platform-ansible-os-3677-1-test' elem = 'contrail-vrouter-agent' k = 'Compute' AssertionError: __cause__ = None __class__ = __context__ = None __delattr__ = __dict__ = {} __dir__ = __doc__ = 'Assertion failed.' __eq__ = __format__ = __ge__ = __getattribute__ = __gt__ = __hash__ = __init__ = __init_subclass__ = __le__ = __lt__ = __ne__ = __new__ = __reduce__ = __reduce_ex__ = __repr__ = __setattr__ = __setstate__ = __sizeof__ = __str__ = __subclasshook__ = __suppress_context__ = False __traceback__ = args = () with_traceback = The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper result = function(self, *args, **kwargs) File "/contrail-test/scripts/analytics/test_analytics.py", line 359, in test_verify_generator_connections_to_collector_node self.analytics_obj.verify_generator_connection_to_collector() File "/contrail-test/tcutils/collector/analytics_tests.py", line 203, in verify_generator_connection_to_collector name,elem,k) AssertionError 2025-07-21 11:56:15,726 - DEBUG - No XMPP flaps were noticed during the test 2025-07-21 11:56:15,726 - INFO -
2025-07-21 11:56:15,726 - INFO - END TEST : test_verify_generator_connections_to_collector_node : FAILED[0:00:04]
2025-07-21 11:56:15,727 - INFO - -------------------------------------------------------------------------------- 2025-07-21 11:56:15,731 - INFO - ================================================================================ 2025-07-21 11:56:15,731 - INFO - STARTING TEST : test_verify_process_status_agent 2025-07-21 11:56:15,731 - INFO - TEST DESCRIPTION : Test to validate process_status 2025-07-21 11:56:15,908 - DEBUG - Nothing to compare xmpp stats {'10.0.0.254': {'10.20.0.17': '0'}} with 2025-07-21 11:56:15,909 - INFO - Initial checks done. Running the testcase now 2025-07-21 11:56:15,909 - INFO - 2025-07-21 11:56:15,936 - WARNING - No object found for module contrail-vrouter-agent 2025-07-21 11:56:16,098 - ERROR - AssertionError Python 3.6.8: /usr/bin/python3 Mon Jul 21 11:56:15 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={}) 78 log.info('Initial checks done. Running the testcase now') 79 log.info('') 80 result = function(self, *args, **kwargs) 81 if self.inputs.upgrade: 82 pid = os.getpid() result = None function = self = args = () kwargs = {} /contrail-test/scripts/analytics/test_analytics.py in test_verify_process_status_agent(self=) 325 326 ''' 327 self.analytics_obj.verify_process_and_connection_infos_agent() 328 329 @test.attr(type=['cb_sanity', 'sanity', 'dev_sanity_dpdk']) self = self.analytics_obj = self.analytics_obj.verify_process_and_connection_infos_agent = > /contrail-test/tcutils/collector/analytics_tests.py in verify_process_and_connection_infos_agent(self=) 4076 collector_ips[0]].get_ops_vrouter(vrouter) 4077 assert self.verify_process_status(ops_inspect,\ 4078 'contrail-vrouter-agent') 4079 xmpp_servers = len(self.inputs.bgp_control_ips) 4080 count = 0 AssertionError: __cause__ = None __class__ = __context__ = None __delattr__ = __dict__ = {} __dir__ = __doc__ = 'Assertion failed.' __eq__ = __format__ = __ge__ = __getattribute__ = __gt__ = __hash__ = __init__ = __init_subclass__ = __le__ = __lt__ = __ne__ = __new__ = __reduce__ = __reduce_ex__ = __repr__ = __setattr__ = __setstate__ = __sizeof__ = __str__ = __subclasshook__ = __suppress_context__ = False __traceback__ = args = () with_traceback = The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper result = function(self, *args, **kwargs) File "/contrail-test/scripts/analytics/test_analytics.py", line 327, in test_verify_process_status_agent self.analytics_obj.verify_process_and_connection_infos_agent() File "/contrail-test/tcutils/collector/analytics_tests.py", line 4078, in verify_process_and_connection_infos_agent 'contrail-vrouter-agent') AssertionError 2025-07-21 11:56:16,118 - DEBUG - No XMPP flaps were noticed during the test 2025-07-21 11:56:16,119 - INFO -
2025-07-21 11:56:16,119 - INFO - END TEST : test_verify_process_status_agent : FAILED[0:00:01]
2025-07-21 11:56:16,119 - INFO - -------------------------------------------------------------------------------- 2025-07-21 11:56:16,129 - INFO - ================================================================================ 2025-07-21 11:56:16,129 - INFO - STARTING TEST : test_verify_process_status_analytics_node 2025-07-21 11:56:16,129 - INFO - TEST DESCRIPTION : Test to validate process_status-Analytics-Node 2025-07-21 11:56:16,311 - DEBUG - Nothing to compare xmpp stats {'10.0.0.254': {'10.20.0.17': '0'}} with 2025-07-21 11:56:16,311 - INFO - Initial checks done. Running the testcase now 2025-07-21 11:56:16,311 - INFO - 2025-07-21 11:56:16,441 - INFO - cn-jenkins-deploy-platform-ansible-os-3677-1.:contrail-collector module connection to ['10.0.0.254:8086'] servers UP 2025-07-21 11:56:16,441 - WARNING - cn-jenkins-deploy-platform-ansible-os-3677-1.:contrail-collector module connection to ['10.0.0.254:6379'] servers NOT UP 2025-07-21 11:56:16,441 - INFO - cn-jenkins-deploy-platform-ansible-os-3677-1.:contrail-collector module connection to ['127.0.0.1:6379'] servers UP 2025-07-21 11:56:16,441 - INFO - cn-jenkins-deploy-platform-ansible-os-3677-1.:contrail-collector module connection to ['10.0.0.254:9042'] servers UP 2025-07-21 11:56:16,496 - INFO - cn-jenkins-deploy-platform-ansible-os-3677-1.:contrail-analytics-api module connection to ['10.0.0.254:8086'] servers UP 2025-07-21 11:56:16,560 - INFO - cn-jenkins-deploy-platform-ansible-os-3677-1.:contrail-analytics-nodemgr module connection to ['10.0.0.254:8086'] servers UP 2025-07-21 11:56:16,715 - DEBUG - No XMPP flaps were noticed during the test
2025-07-21 11:56:16,716 - INFO - END TEST : test_verify_process_status_analytics_node : PASSED[0:00:00]
2025-07-21 11:56:16,716 - INFO - -------------------------------------------------------------------------------- 2025-07-21 11:56:16,719 - INFO - ================================================================================ 2025-07-21 11:56:16,719 - INFO - STARTING TEST : test_verify_process_status_config 2025-07-21 11:56:16,719 - INFO - TEST DESCRIPTION : Test to validate process_status-Config 2025-07-21 11:56:16,926 - DEBUG - Nothing to compare xmpp stats {'10.0.0.254': {'10.20.0.17': '0'}} with 2025-07-21 11:56:16,927 - INFO - Initial checks done. Running the testcase now 2025-07-21 11:56:16,927 - INFO - 2025-07-21 11:56:17,047 - INFO - cn-jenkins-deploy-platform-ansible-os-3677-1.:contrail-api module connection to 10.0.0.254:8086 servers UP 2025-07-21 11:56:17,047 - INFO - cn-jenkins-deploy-platform-ansible-os-3677-1.:contrail-api module connection to 10.0.0.254:2181 servers UP 2025-07-21 11:56:17,048 - WARNING - cn-jenkins-deploy-platform-ansible-os-3677-1.:contrail-api module connection to 10.0.0.254:9161 servers NOT UP 2025-07-21 11:56:17,048 - WARNING - cn-jenkins-deploy-platform-ansible-os-3677-1.:contrail-api module connection to cn-jenkins-deploy-platform-ansible-os-3677-1.*:9161 servers NOT UP 2025-07-21 11:56:17,201 - ERROR - AssertionError Python 3.6.8: /usr/bin/python3 Mon Jul 21 11:56:17 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={}) 78 log.info('Initial checks done. Running the testcase now') 79 log.info('') 80 result = function(self, *args, **kwargs) 81 if self.inputs.upgrade: 82 pid = os.getpid() result = None function = self = args = () kwargs = {} /contrail-test/scripts/analytics/test_analytics.py in test_verify_process_status_config(self=) 333 334 ''' 335 self.analytics_obj.verify_process_and_connection_infos_config() 336 337 @test.attr(type=['cb_sanity', 'sanity', 'dev_sanity_dpdk']) self = self.analytics_obj = self.analytics_obj.verify_process_and_connection_infos_config = > /contrail-test/tcutils/collector/analytics_tests.py in verify_process_and_connection_infos_config(self=) 4218 'contrail-api',\ 4219 server,node = cfgm) 4220 assert result 4221 result = False 4222 result = False AssertionError: __cause__ = None __class__ = __context__ = None __delattr__ = __dict__ = {} __dir__ = __doc__ = 'Assertion failed.' __eq__ = __format__ = __ge__ = __getattribute__ = __gt__ = __hash__ = __init__ = __init_subclass__ = __le__ = __lt__ = __ne__ = __new__ = __reduce__ = __reduce_ex__ = __repr__ = __setattr__ = __setstate__ = __sizeof__ = __str__ = __subclasshook__ = __suppress_context__ = False __traceback__ = args = () with_traceback = The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper result = function(self, *args, **kwargs) File "/contrail-test/scripts/analytics/test_analytics.py", line 335, in test_verify_process_status_config self.analytics_obj.verify_process_and_connection_infos_config() File "/contrail-test/tcutils/collector/analytics_tests.py", line 4220, in verify_process_and_connection_infos_config assert result AssertionError 2025-07-21 11:56:17,216 - DEBUG - No XMPP flaps were noticed during the test 2025-07-21 11:56:17,217 - INFO -
2025-07-21 11:56:17,217 - INFO - END TEST : test_verify_process_status_config : FAILED[0:00:01]
2025-07-21 11:56:17,217 - INFO - -------------------------------------------------------------------------------- 2025-07-21 11:56:17,226 - INFO - ================================================================================ 2025-07-21 11:56:17,226 - INFO - STARTING TEST : test_verify_process_status_control_node 2025-07-21 11:56:17,227 - INFO - TEST DESCRIPTION : Test to validate process_status-Control-Node 2025-07-21 11:56:17,413 - DEBUG - Nothing to compare xmpp stats {'10.0.0.254': {'10.20.0.17': '0'}} with 2025-07-21 11:56:17,413 - INFO - Initial checks done. Running the testcase now 2025-07-21 11:56:17,413 - INFO - 2025-07-21 11:56:17,441 - WARNING - No object found for module contrail-control 2025-07-21 11:56:17,601 - ERROR - AssertionError Python 3.6.8: /usr/bin/python3 Mon Jul 21 11:56:17 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={}) 78 log.info('Initial checks done. Running the testcase now') 79 log.info('') 80 result = function(self, *args, **kwargs) 81 if self.inputs.upgrade: 82 pid = os.getpid() result = None function = self = args = () kwargs = {} /contrail-test/scripts/analytics/test_analytics.py in test_verify_process_status_control_node(self=) 341 342 ''' 343 self.analytics_obj.verify_process_and_connection_infos_control_node() 344 345 @test.attr(type=['cb_sanity', 'sanity', 'dev_sanity_dpdk']) self = self.analytics_obj = self.analytics_obj.verify_process_and_connection_infos_control_node = > /contrail-test/tcutils/collector/analytics_tests.py in verify_process_and_connection_infos_control_node(self=) 4439 collector_ips[0]].get_ops_bgprouter(bgp) 4440 assert self.verify_process_status(ops_inspect,\ 4441 'contrail-control') 4442 result = False 4443 AssertionError: __cause__ = None __class__ = __context__ = None __delattr__ = __dict__ = {} __dir__ = __doc__ = 'Assertion failed.' __eq__ = __format__ = __ge__ = __getattribute__ = __gt__ = __hash__ = __init__ = __init_subclass__ = __le__ = __lt__ = __ne__ = __new__ = __reduce__ = __reduce_ex__ = __repr__ = __setattr__ = __setstate__ = __sizeof__ = __str__ = __subclasshook__ = __suppress_context__ = False __traceback__ = args = () with_traceback = The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper result = function(self, *args, **kwargs) File "/contrail-test/scripts/analytics/test_analytics.py", line 343, in test_verify_process_status_control_node self.analytics_obj.verify_process_and_connection_infos_control_node() File "/contrail-test/tcutils/collector/analytics_tests.py", line 4441, in verify_process_and_connection_infos_control_node 'contrail-control') AssertionError 2025-07-21 11:56:17,610 - DEBUG - No XMPP flaps were noticed during the test 2025-07-21 11:56:17,610 - INFO -
2025-07-21 11:56:17,611 - INFO - END TEST : test_verify_process_status_control_node : FAILED[0:00:00]
2025-07-21 11:56:17,611 - INFO - --------------------------------------------------------------------------------