2025-10-17 06:26:18,613 - INFO - Domain Default found not creating 2025-10-17 06:26:18,760 - INFO - Project ctest-FloatingipBasicTestSanity-60631480 not found, creating it 2025-10-17 06:26:19,228 - INFO - Created Project:ctest-FloatingipBasicTestSanity-60631480, ID : 60605973-58ee-4168-b56c-b9a42ef16b90 2025-10-17 06:26:20,620 - INFO - Using existing project ['default-domain', 'admin'](2892f460-4d8f-4b48-8195-b1d1486e2b08) 2025-10-17 06:26:21,342 - INFO - Adding rules to the default security group in Project admin 2025-10-17 06:26:21,667 - INFO - ================================================================================ 2025-10-17 06:26:21,667 - INFO - STARTING TEST : test_floating_ip 2025-10-17 06:26:21,667 - INFO - TEST DESCRIPTION : Test to validate floating-ip Assignment to a VM. It creates a VM, assigns a FIP to it and pings to a IP in the FIP VN. 2025-10-17 06:26:21,793 - DEBUG - Nothing to compare xmpp stats {'10.0.0.38': {'10.20.0.17': '0'}} with 2025-10-17 06:26:21,793 - INFO - Initial checks done. Running the testcase now 2025-10-17 06:26:21,793 - INFO - 2025-10-17 06:26:21,816 - DEBUG - Services list from nova: [, , ] 2025-10-17 06:26:21,817 - DEBUG - Hosts: {'nova': ['cn-jenkins-deploy-platform-ansible-os-4352-1']} 2025-10-17 06:26:22,371 - DEBUG - Response for create_network : {'network': {'id': '8f9d0750-db82-4e5b-87a3-cd10a05ca66d', 'name': 'ctest-fvn-38381134', 'tenant_id': '6060597358ee4168b56cb9a42ef16b90', 'project_id': '6060597358ee4168b56cb9a42ef16b90', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-FloatingipBasicTestSanity-60631480', 'ctest-fvn-38381134'], 'port_security_enabled': True, 'description': ''}} 2025-10-17 06:26:22,568 - DEBUG - Response for create_subnet : {'subnet': {'id': '895de4ee-27ad-408e-93f5-adaa1b69ed42', 'name': '', 'tenant_id': '6060597358ee4168b56cb9a42ef16b90', 'network_id': '8f9d0750-db82-4e5b-87a3-cd10a05ca66d', 'ip_version': 4, 'cidr': '138.115.91.0/26', 'allocation_pools': [{'start': '138.115.91.2', 'end': '138.115.91.62'}], 'gateway_ip': '138.115.91.1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '138.115.91.2', 'tags': [], 'project_id': '6060597358ee4168b56cb9a42ef16b90'}} 2025-10-17 06:26:22,586 - INFO - Created VN ctest-fvn-38381134 2025-10-17 06:26:22,637 - DEBUG - VN ctest-fvn-38381134 UUID is 8f9d0750-db82-4e5b-87a3-cd10a05ca66d 2025-10-17 06:26:22,742 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-10-17 06:26:23,031 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-10-17 06:26:23,059 - DEBUG - Requesting: http://10.0.0.38:8082/domain/a7cd34c9-7c21-46dd-a95f-bb2683ef8bbb 2025-10-17 06:26:23,072 - DEBUG - Requesting: http://10.0.0.38:8082/project/60605973-58ee-4168-b56c-b9a42ef16b90 2025-10-17 06:26:23,143 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/8f9d0750-db82-4e5b-87a3-cd10a05ca66d 2025-10-17 06:26:23,153 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/8f9d0750-db82-4e5b-87a3-cd10a05ca66d 2025-10-17 06:26:23,162 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/7bdbf71e-55b6-493b-b907-f5f6c5237481 2025-10-17 06:26:23,169 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/7bdbf71e-55b6-493b-b907-f5f6c5237481 2025-10-17 06:26:23,176 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/2557ca13-b316-4de1-837f-dadb95ab9637 2025-10-17 06:26:23,182 - DEBUG - Route Targets: ['target:64512:8000004'] 2025-10-17 06:26:23,182 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/8f9d0750-db82-4e5b-87a3-cd10a05ca66d 2025-10-17 06:26:23,191 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/7bdbf71e-55b6-493b-b907-f5f6c5237481 2025-10-17 06:26:23,206 - INFO - Verified VN network id 6 for VN 8f9d0750-db82-4e5b-87a3-cd10a05ca66d 2025-10-17 06:26:23,206 - INFO - Verifications in API Server for VN ctest-fvn-38381134 passed 2025-10-17 06:26:23,206 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/8f9d0750-db82-4e5b-87a3-cd10a05ca66d 2025-10-17 06:26:23,215 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/7bdbf71e-55b6-493b-b907-f5f6c5237481 2025-10-17 06:26:23,221 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/7bdbf71e-55b6-493b-b907-f5f6c5237481 2025-10-17 06:26:23,228 - DEBUG - Requesting: http://10.0.0.38:8082/route-target/2557ca13-b316-4de1-837f-dadb95ab9637 2025-10-17 06:26:23,242 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-FloatingipBasicTestSanity-60631480:ctest-fvn-38381134', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-FloatingipBasicTestSanity-60631480', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '10348435562628795995', 'uuid-lslong': '9773881087617705581'}, 'enable': 'true', 'created': '2025-10-17T06:26:22', 'last-modified': '2025-10-17T06:26:22', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.705579'} 2025-10-17 06:26:23,249 - DEBUG - Route Targets: ['target:64512:8000004'] 2025-10-17 06:26:23,249 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-fvn-38381134 passed 2025-10-17 06:26:23,250 - DEBUG - ====Verifying policy data for ctest-fvn-38381134 in API_Server ====== 2025-10-17 06:26:23,250 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-10-17 06:26:23,257 - DEBUG - Requesting: http://10.0.0.38:8082/domain/a7cd34c9-7c21-46dd-a95f-bb2683ef8bbb 2025-10-17 06:26:23,268 - DEBUG - Requesting: http://10.0.0.38:8082/project/60605973-58ee-4168-b56c-b9a42ef16b90 2025-10-17 06:26:23,341 - DEBUG - Requesting: http://10.0.0.38:8082/virtual-network/8f9d0750-db82-4e5b-87a3-cd10a05ca66d 2025-10-17 06:26:23,351 - DEBUG - =>VN ctest-fvn-38381134 has no policy to be verified 2025-10-17 06:26:23,351 - DEBUG - Verifying the vn in opserver 2025-10-17 06:26:23,351 - DEBUG - Verifying the default-domain:ctest-FloatingipBasicTestSanity-60631480:ctest-fvn-38381134 virtual network link through opserver 10.0.0.38 2025-10-17 06:26:23,351 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-10-17 06:26:23,642 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-10-17 06:26:23,696 - DEBUG - vn link and name as {'name': 'default-domain:ctest-FloatingipBasicTestSanity-60631480:ctest-fvn-38381134', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-FloatingipBasicTestSanity-60631480:ctest-fvn-38381134?flat'} 2025-10-17 06:26:23,696 - INFO - Validated that VN default-domain:ctest-FloatingipBasicTestSanity-60631480:ctest-fvn-38381134 is found in opserver 2025-10-17 06:26:23,707 - DEBUG - Do not have enough data to verify VN in agent 2025-10-17 06:26:23,713 - DEBUG - VRF ids for VN ctest-fvn-38381134: {} 2025-10-17 06:26:23,914 - ERROR - Some exception while doing Neutron net-list Traceback (most recent call last): File "/contrail-test/fixtures/quantum_test.py", line 276, in get_vn_obj_if_present net_rsp = self.obj.list_networks(**query_dict)['networks'] File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 817, in list_networks return self.list('networks', self.networks_path, retrieve_all, File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 369, in list for r in self._pagination(collection, path, **params): File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 384, in _pagination res = self.get(path, params=params) File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 353, in get return self.retry_request("GET", action, body=body, File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 330, in retry_request return self.do_request(method, action, body=body, File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 294, in do_request self._handle_fault_response(status_code, replybody, resp) File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 269, in _handle_fault_response exception_handler_v20(status_code, error_body) File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 91, in exception_handler_v20 raise client_exc(message=error_message, neutronclient.common.exceptions.InternalServerError: The server has either erred or is incapable of performing the requested operation.

Neutron server returns request_ids: ['req-409ab27f-4d89-4e34-9557-5ad493065ddd'] 2025-10-17 06:26:23,915 - ERROR - Got exception as "The server has either erred or is incapable of performing the requested operation.

\n\n\n\nNeutron server returns request_ids: ['req-409ab27f-4d89-4e34-9557-5ad493065ddd']" while creating ctest-vn1-84015476 Traceback (most recent call last): File "/contrail-test/fixtures/quantum_test.py", line 276, in get_vn_obj_if_present net_rsp = self.obj.list_networks(**query_dict)['networks'] File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 817, in list_networks return self.list('networks', self.networks_path, retrieve_all, File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 369, in list for r in self._pagination(collection, path, **params): File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 384, in _pagination res = self.get(path, params=params) File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 353, in get return self.retry_request("GET", action, body=body, File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 330, in retry_request return self.do_request(method, action, body=body, File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 294, in do_request self._handle_fault_response(status_code, replybody, resp) File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 269, in _handle_fault_response exception_handler_v20(status_code, error_body) File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 91, in exception_handler_v20 raise client_exc(message=error_message, neutronclient.common.exceptions.InternalServerError: The server has either erred or is incapable of performing the requested operation.

Neutron server returns request_ids: ['req-409ab27f-4d89-4e34-9557-5ad493065ddd'] During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/contrail-test/fixtures/vn_test.py", line 295, in _create_vn_orch self.obj = self.orchestrator.get_vn_obj_if_present(self.vn_name, File "/contrail-test/fixtures/openstack.py", line 184, in get_vn_obj_if_present return self.quantum_h.get_vn_obj_if_present(vn_name, **kwargs) File "/contrail-test/fixtures/quantum_test.py", line 282, in get_vn_obj_if_present raise NetworkClientException(message=str(e)) quantum_test.NetworkClientException: "The server has either erred or is incapable of performing the requested operation.

\n\n\n\nNeutron server returns request_ids: ['req-409ab27f-4d89-4e34-9557-5ad493065ddd']" 2025-10-17 06:26:23,921 - INFO - Deleting VN ctest-fvn-38381134 2025-10-17 06:26:24,037 - DEBUG - Response for deleting network () 2025-10-17 06:26:24,037 - DEBUG - Requesting: http://10.0.0.38:8082/routing-instance/7bdbf71e-55b6-493b-b907-f5f6c5237481 2025-10-17 06:26:24,047 - DEBUG - Response Code: 404 2025-10-17 06:26:24,047 - DEBUG - Requesting: http://10.0.0.38:8082/domains 2025-10-17 06:26:24,052 - DEBUG - Requesting: http://10.0.0.38:8082/domain/a7cd34c9-7c21-46dd-a95f-bb2683ef8bbb 2025-10-17 06:26:24,106 - DEBUG - Requesting: http://10.0.0.38:8082/project/60605973-58ee-4168-b56c-b9a42ef16b90 2025-10-17 06:26:24,180 - INFO - Validated that VN ctest-fvn-38381134 is not found in API Server 2025-10-17 06:26:24,189 - DEBUG - VN ctest-fvn-38381134 is not present in Agent 10.0.0.38 2025-10-17 06:26:24,189 - INFO - Validated that VN ctest-fvn-38381134 is not in any agent 2025-10-17 06:26:24,201 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-fvn-38381134 info 2025-10-17 06:26:24,322 - ERROR - NetworkClientException Python 3.9.21: /usr/bin/python3 Fri Oct 17 06:26:23 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={}) 78 log.info('Initial checks done. Running the testcase now') 79 log.info('') 80 result = function(self, *args, **kwargs) 81 if self.inputs.upgrade: 82 pid = os.getpid() result = None function = self = args = () kwargs = {} /contrail-test/scripts/floatingip/test_floatingip_basic.py in test_floating_ip(self=) 58 assert fvn_fixture.verify_on_setup() 59 60 vn1_fixture = self.useFixture( 61 VNFixture( 62 project_name=self.inputs.project_name, vn1_fixture undefined self = self.useFixture = > global VNFixture = project_name undefined self.inputs = self.inputs.project_name = 'ctest-FloatingipBasicTestSanity-60631480' connections undefined self.connections = inputs undefined vn_name undefined vn1_name = 'ctest-vn1-84015476' subnets undefined vn1_subnets = ['168.18.118.128/26'] orch undefined self.orchestrator = /usr/local/lib/python3.9/site-packages/testtools/testcase.py in useFixture(self=, fixture=) 754 # Gather_details worked, so raise the exception setUp 755 # encountered. 756 reraise(*exc_info) 757 else: 758 self.addCleanup(fixture.cleanUp) global reraise = exc_info = (, NetworkClientException(), ) /usr/local/lib/python3.9/site-packages/testtools/_compat3x.py in reraise(exc_class=, exc_obj=NetworkClientException(), exc_tb=, _marker=) 13 14 def reraise(exc_class, exc_obj, exc_tb, _marker=object()): 15 """Re-raise an exception received from sys.exc_info() or similar.""" 16 raise exc_obj.with_traceback(exc_tb) 17 exc_obj = NetworkClientException() exc_obj.with_traceback = exc_tb = /usr/local/lib/python3.9/site-packages/testtools/testcase.py in useFixture(self=, fixture=) 729 """ 730 try: 731 fixture.setUp() 732 except MultipleExceptions as e: 733 if (fixtures is not None and fixture = fixture.setUp = > /contrail-test/fixtures/vn_test.py in setUp(self=) 437 def setUp(self): 438 super(VNFixture, self).setUp() 439 self.create() 440 441 def create(self): self = self.create = > /contrail-test/fixtures/vn_test.py in create(self=) 450 self._create_vn_api(self.vn_name, self.project_obj) 451 else: 452 self._create_vn_orch() 453 if not self.obj: 454 self.logger.debug('VN %s not present' % (self.vn_name)) self = self._create_vn_orch = > /contrail-test/fixtures/vn_test.py in _create_vn_orch(self=) 335 if '503' in str(e) or '504' in str(e): 336 return False 337 raise NetworkClientException(message=str(e)) 338 339 def get_vn_list_in_project(self, project_uuid): global NetworkClientException = message undefined builtinstr = e undefined NetworkClientException: '"The server has either erred or is incapable of performing the requested operation.

\\n\\n\\n\\nNeutron server returns request_ids: [\'req-409ab27f-4d89-4e34-9557-5ad493065ddd\']"' __cause__ = None __class__ = __context__ = NetworkClientException() __delattr__ = __dict__ = {'_error_string': '"The server has either erred or is incapable of ...ds: [\'req-409ab27f-4d89-4e34-9557-5ad493065ddd\']"', 'message': '"The server has either erred or is incapable of ...ds: [\'req-409ab27f-4d89-4e34-9557-5ad493065ddd\']"', 'request_ids': None, 'status_code': 0} __dir__ = __doc__ = None __eq__ = __format__ = __ge__ = __getattribute__ = __gt__ = __hash__ = __init__ = __init_subclass__ = __le__ = __lt__ = __module__ = 'quantum_test' __ne__ = __new__ = __reduce__ = __reduce_ex__ = __repr__ = __setattr__ = __setstate__ = __sizeof__ = __str__ = __subclasshook__ = __suppress_context__ = False __traceback__ = __weakref__ = None _error_string = '"The server has either erred or is incapable of ...ds: [\'req-409ab27f-4d89-4e34-9557-5ad493065ddd\']"' args = () message = '"The server has either erred or is incapable of ...ds: [\'req-409ab27f-4d89-4e34-9557-5ad493065ddd\']"' req_ids_msg = 'Neutron server returns request_ids: %s' request_ids = None status_code = 0 with_traceback = The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/contrail-test/fixtures/quantum_test.py", line 276, in get_vn_obj_if_present net_rsp = self.obj.list_networks(**query_dict)['networks'] File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 817, in list_networks return self.list('networks', self.networks_path, retrieve_all, File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 369, in list for r in self._pagination(collection, path, **params): File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 384, in _pagination res = self.get(path, params=params) File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 353, in get return self.retry_request("GET", action, body=body, File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 330, in retry_request return self.do_request(method, action, body=body, File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 294, in do_request self._handle_fault_response(status_code, replybody, resp) File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 269, in _handle_fault_response exception_handler_v20(status_code, error_body) File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 91, in exception_handler_v20 raise client_exc(message=error_message, neutronclient.common.exceptions.InternalServerError: The server has either erred or is incapable of performing the requested operation.

Neutron server returns request_ids: ['req-409ab27f-4d89-4e34-9557-5ad493065ddd'] During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/contrail-test/fixtures/vn_test.py", line 295, in _create_vn_orch self.obj = self.orchestrator.get_vn_obj_if_present(self.vn_name, File "/contrail-test/fixtures/openstack.py", line 184, in get_vn_obj_if_present return self.quantum_h.get_vn_obj_if_present(vn_name, **kwargs) File "/contrail-test/fixtures/quantum_test.py", line 282, in get_vn_obj_if_present raise NetworkClientException(message=str(e)) quantum_test.NetworkClientException: "The server has either erred or is incapable of performing the requested operation.

\n\n\n\nNeutron server returns request_ids: ['req-409ab27f-4d89-4e34-9557-5ad493065ddd']" During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper result = function(self, *args, **kwargs) File "/contrail-test/scripts/floatingip/test_floatingip_basic.py", line 60, in test_floating_ip vn1_fixture = self.useFixture( File "/usr/local/lib/python3.9/site-packages/testtools/testcase.py", line 756, in useFixture reraise(*exc_info) File "/usr/local/lib/python3.9/site-packages/testtools/_compat3x.py", line 16, in reraise raise exc_obj.with_traceback(exc_tb) File "/usr/local/lib/python3.9/site-packages/testtools/testcase.py", line 731, in useFixture fixture.setUp() File "/contrail-test/fixtures/vn_test.py", line 439, in setUp self.create() File "/contrail-test/fixtures/vn_test.py", line 452, in create self._create_vn_orch() File "/contrail-test/fixtures/vn_test.py", line 337, in _create_vn_orch raise NetworkClientException(message=str(e)) quantum_test.NetworkClientException: '"The server has either erred or is incapable of performing the requested operation.

\\n\\n\\n\\nNeutron server returns request_ids: [\'req-409ab27f-4d89-4e34-9557-5ad493065ddd\']"' 2025-10-17 06:26:24,328 - DEBUG - No XMPP flaps were noticed during the test 2025-10-17 06:26:24,328 - INFO -
2025-10-17 06:26:24,328 - INFO - END TEST : test_floating_ip : FAILED[0:00:03]
2025-10-17 06:26:24,328 - INFO - -------------------------------------------------------------------------------- 2025-10-17 06:26:25,009 - INFO - Deleted project: ctest-FloatingipBasicTestSanity-60631480, ID : 60605973-58ee-4168-b56c-b9a42ef16b90