INFO: =================== Mon Sep 22 01:31:35 AM UTC 2025 =================== /contrail-functions.sh: line 167: /etc/contrail/vnc_api_lib.ini.tmp: No such file or directory /contrail-functions.sh: line 202: /etc/contrail/vnc_api_lib.ini.tmp: No such file or directory mv: cannot stat '/etc/contrail/vnc_api_lib.ini.tmp': No such file or directory [DEFAULTS] http_server_ip=0.0.0.0 log_file=/var/log/contrail/database-nodemgr/contrail-database-nodemgr.log log_level=SYS_NOTICE log_local=1 hostip=10.0.0.50 db_port=9042 db_jmx_port=7200 db_use_ssl=False [COLLECTOR] server_list=10.0.0.50:8086 [SANDESH] introspect_ssl_enable=False sandesh_ssl_enable=False [DEFAULTS] minimum_diskGB = 2 09/22/2025 01:31:36.509 7f65254be8c0 [contrail-database-nodemgr] [INFO]: SANDESH: CONNECT TO COLLECTOR: True 09/22/2025 01:31:36.637 7f65254be8c0 [contrail-database-nodemgr] [ERROR]: Failed to import package "database.sandesh" 09/22/2025 01:31:36.648 7f65254be8c0 [contrail-database-nodemgr] [INFO]: SANDESH: INTROSPECT IS ON: 0.0.0.0:8103 09/22/2025 01:31:36.655 7f65254be8c0 [contrail-database-nodemgr] [ERROR]: Failed to import package "database.sandesh" 09/22/2025 01:31:36.662 7f65254be8c0 [contrail-database-nodemgr] [INFO]: SANDESH: Logging: LEVEL: [SYS_INFO] -> [SYS_NOTICE] CRITICAL:root:Exit data: {'ID': '06a80a6c6b10bc7d1a57f7a831bfe01d55f1b58928914587f7087f20fe09b913', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'cqlsh', 'arguments': ['10.0.0.50', '9042', '-e', 'quit'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'f5bbad9e78520e1919741289e6bc8ef7338d6603883ce44b1288aa6952c5c3cc', 'DetachKeys': '', 'Pid': 74079} ERROR:contrail-database-nodemgr:Failed to connect to database by CQL and socket: Result: 1 CRITICAL:root:Exit data: {'ID': '25d2d2f6e6ad1c048fa667a903ec1e4d84ba1b8dbcbab005a28887e9d11269d9', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'nodetool', 'arguments': ['-p', '7200', 'compactionstats'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'f5bbad9e78520e1919741289e6bc8ef7338d6603883ce44b1288aa6952c5c3cc', 'DetachKeys': '', 'Pid': 74099} ERROR:contrail-database-nodemgr:Failed to get nodetool compactionstats: Result: 1 ERROR:contrail-database-nodemgr:Failed to get core files: [Errno 2] No such file or directory: '/var/crashes' CRITICAL:root:Exit data: {'ID': 'fa169ef7a2e5d97ca51c27fb955e31f938e3b78a95c50ceb8715e5ea3f7a9389', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'cqlsh', 'arguments': ['10.0.0.50', '9042', '-e', 'quit'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'f5bbad9e78520e1919741289e6bc8ef7338d6603883ce44b1288aa6952c5c3cc', 'DetachKeys': '', 'Pid': 75794} WARNING:contrail-database-nodemgr:CQLSH failed, but Cassandra native transport port is open [Introspect:8103]10.0.0.50 - - [2025-09-22 01:33:38] "GET /Snh_SandeshUVECacheReq?x=NodeStatus HTTP/1.1" 200 6712 0.001365 CRITICAL:root:Exit data: {'ID': '6d421c57bd6fb0ec3892d0d85871d13f43e252c87fce3a2ad3d79aed5d773029', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'nodetool', 'arguments': ['-p', '7200', 'compactionstats'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'f5bbad9e78520e1919741289e6bc8ef7338d6603883ce44b1288aa6952c5c3cc', 'DetachKeys': '', 'Pid': 75843} ERROR:contrail-database-nodemgr:Failed to get nodetool compactionstats: Result: 1 ERROR:contrail-database-nodemgr:Failed to get core files: [Errno 2] No such file or directory: '/var/crashes' [Introspect:8103]10.0.0.50 - - [2025-09-22 01:33:40] "GET /Snh_SandeshUVECacheReq?x=NodeStatus HTTP/1.1" 200 6712 0.001435 CRITICAL:root:Exit data: {'ID': '4386fd5a17ed351088316295dcfeaede7ea525608c7df3d234e5dc3b55a30a25', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'cqlsh', 'arguments': ['10.0.0.50', '9042', '-e', 'quit'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'f5bbad9e78520e1919741289e6bc8ef7338d6603883ce44b1288aa6952c5c3cc', 'DetachKeys': '', 'Pid': 77644} WARNING:contrail-database-nodemgr:CQLSH failed, but Cassandra native transport port is open CRITICAL:root:Exit data: {'ID': '2c30cebd11bf3a7a49df8f78ad75feaf305a57322eb028b3be34c1b11bec6a2c', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'nodetool', 'arguments': ['-p', '7200', 'compactionstats'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'f5bbad9e78520e1919741289e6bc8ef7338d6603883ce44b1288aa6952c5c3cc', 'DetachKeys': '', 'Pid': 77666} ERROR:contrail-database-nodemgr:Failed to get nodetool compactionstats: Result: 1 ERROR:contrail-database-nodemgr:Failed to get core files: [Errno 2] No such file or directory: '/var/crashes' CRITICAL:root:Exit data: {'ID': '3a812f693c7be35ac414a966064b8db77a6554df28b3f451dde149bcf59eabbe', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'cqlsh', 'arguments': ['10.0.0.50', '9042', '-e', 'quit'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'f5bbad9e78520e1919741289e6bc8ef7338d6603883ce44b1288aa6952c5c3cc', 'DetachKeys': '', 'Pid': 79992} WARNING:contrail-database-nodemgr:CQLSH failed, but Cassandra native transport port is open CRITICAL:root:Exit data: {'ID': 'f00c842b76f5714518f26a2edec517f006066dd0c7b6a2823029b4959276cf6c', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'nodetool', 'arguments': ['-p', '7200', 'compactionstats'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'f5bbad9e78520e1919741289e6bc8ef7338d6603883ce44b1288aa6952c5c3cc', 'DetachKeys': '', 'Pid': 80012} ERROR:contrail-database-nodemgr:Failed to get nodetool compactionstats: Result: 1 ERROR:contrail-database-nodemgr:Failed to get core files: [Errno 2] No such file or directory: '/var/crashes' CRITICAL:root:Exit data: {'ID': '16b30c3d9b1c854fa9568f7f5024e8fa987c932b9461817aa9c511e41bd354a4', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'cqlsh', 'arguments': ['10.0.0.50', '9042', '-e', 'quit'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'f5bbad9e78520e1919741289e6bc8ef7338d6603883ce44b1288aa6952c5c3cc', 'DetachKeys': '', 'Pid': 83154} WARNING:contrail-database-nodemgr:CQLSH failed, but Cassandra native transport port is open CRITICAL:root:Exit data: {'ID': '03006f3753b63057eed0c6e49624e91741ba3b96bf2546ff8571b4c71ff330a7', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'nodetool', 'arguments': ['-p', '7200', 'compactionstats'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'f5bbad9e78520e1919741289e6bc8ef7338d6603883ce44b1288aa6952c5c3cc', 'DetachKeys': '', 'Pid': 83172} ERROR:contrail-database-nodemgr:Failed to get nodetool compactionstats: Result: 1 ERROR:contrail-database-nodemgr:Failed to get core files: [Errno 2] No such file or directory: '/var/crashes' CRITICAL:root:Exit data: {'ID': '60eb9fa8b5dd80226065afebf166aa790b89f56d0a77967821e060dfc519d8c8', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'cqlsh', 'arguments': ['10.0.0.50', '9042', '-e', 'quit'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'f5bbad9e78520e1919741289e6bc8ef7338d6603883ce44b1288aa6952c5c3cc', 'DetachKeys': '', 'Pid': 84329} WARNING:contrail-database-nodemgr:CQLSH failed, but Cassandra native transport port is open CRITICAL:root:Exit data: {'ID': '592387cc30932d2158c38419bd93d523c422e2631cb7425bbefaeeca161f5bd1', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'nodetool', 'arguments': ['-p', '7200', 'compactionstats'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'f5bbad9e78520e1919741289e6bc8ef7338d6603883ce44b1288aa6952c5c3cc', 'DetachKeys': '', 'Pid': 84348} ERROR:contrail-database-nodemgr:Failed to get nodetool compactionstats: Result: 1 ERROR:contrail-database-nodemgr:Failed to get core files: [Errno 2] No such file or directory: '/var/crashes' CRITICAL:root:Exit data: {'ID': '5b056a478a16675551a3c82591657bb75c4685ddaf3c57a6f98091c61103f50c', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'cqlsh', 'arguments': ['10.0.0.50', '9042', '-e', 'quit'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'f5bbad9e78520e1919741289e6bc8ef7338d6603883ce44b1288aa6952c5c3cc', 'DetachKeys': '', 'Pid': 85408} WARNING:contrail-database-nodemgr:CQLSH failed, but Cassandra native transport port is open CRITICAL:root:Exit data: {'ID': '37c26c98ff93aec0b4ea4d92382395feba047fb2f313075def69f9a28a942c57', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'nodetool', 'arguments': ['-p', '7200', 'compactionstats'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'f5bbad9e78520e1919741289e6bc8ef7338d6603883ce44b1288aa6952c5c3cc', 'DetachKeys': '', 'Pid': 85441} ERROR:contrail-database-nodemgr:Failed to get nodetool compactionstats: Result: 1 ERROR:contrail-database-nodemgr:Failed to get core files: [Errno 2] No such file or directory: '/var/crashes' CRITICAL:root:Exit data: {'ID': '94bf05f6ceff40aa2e4ab829f52ffb2fb63849fb17022b0daad2545dcc151a5e', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'cqlsh', 'arguments': ['10.0.0.50', '9042', '-e', 'quit'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'f5bbad9e78520e1919741289e6bc8ef7338d6603883ce44b1288aa6952c5c3cc', 'DetachKeys': '', 'Pid': 86572} WARNING:contrail-database-nodemgr:CQLSH failed, but Cassandra native transport port is open CRITICAL:root:Exit data: {'ID': '67c177b5936aeeb77d929c873324388fe2a4025b3ebd1b484d471a6091b92624', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'nodetool', 'arguments': ['-p', '7200', 'compactionstats'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'f5bbad9e78520e1919741289e6bc8ef7338d6603883ce44b1288aa6952c5c3cc', 'DetachKeys': '', 'Pid': 86590} ERROR:contrail-database-nodemgr:Failed to get nodetool compactionstats: Result: 1 ERROR:contrail-database-nodemgr:Failed to get core files: [Errno 2] No such file or directory: '/var/crashes' CRITICAL:root:Exit data: {'ID': '51f100d994b4405408c361d9b731e2354090295da6da77a675e27f38383fe14a', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'cqlsh', 'arguments': ['10.0.0.50', '9042', '-e', 'quit'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'f5bbad9e78520e1919741289e6bc8ef7338d6603883ce44b1288aa6952c5c3cc', 'DetachKeys': '', 'Pid': 88835} WARNING:contrail-database-nodemgr:CQLSH failed, but Cassandra native transport port is open CRITICAL:root:Exit data: {'ID': '4e008bf2ff1dddb41eb35b3fe2afe789b8fcbd765f4dd441f5813fe6b6f53d80', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'nodetool', 'arguments': ['-p', '7200', 'compactionstats'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'f5bbad9e78520e1919741289e6bc8ef7338d6603883ce44b1288aa6952c5c3cc', 'DetachKeys': '', 'Pid': 88853} ERROR:contrail-database-nodemgr:Failed to get nodetool compactionstats: Result: 1 ERROR:contrail-database-nodemgr:Failed to get core files: [Errno 2] No such file or directory: '/var/crashes' [Introspect:8103]10.0.0.50 - - [2025-09-22 01:41:17] "GET /Snh_SandeshUVECacheReq?x=NodeStatus HTTP/1.1" 200 6715 0.001245 CRITICAL:root:Exit data: {'ID': '0e63b9cf6e20b13b7ab3045d41e9d3952fa79c2c5a6fc6b7b242ac40b55ebbc3', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'cqlsh', 'arguments': ['10.0.0.50', '9042', '-e', 'quit'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'f5bbad9e78520e1919741289e6bc8ef7338d6603883ce44b1288aa6952c5c3cc', 'DetachKeys': '', 'Pid': 93583} WARNING:contrail-database-nodemgr:CQLSH failed, but Cassandra native transport port is open CRITICAL:root:Exit data: {'ID': '42d1877c3b48630b7696bca8564665087f5126f82550c083febba556a3c48378', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'nodetool', 'arguments': ['-p', '7200', 'compactionstats'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'f5bbad9e78520e1919741289e6bc8ef7338d6603883ce44b1288aa6952c5c3cc', 'DetachKeys': '', 'Pid': 93639} ERROR:contrail-database-nodemgr:Failed to get nodetool compactionstats: Result: 1 ERROR:contrail-database-nodemgr:Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'