Importing couchbase python sdk fails for fresh Ubuntu 22.04 installation with missing libssl1.1 package

Couchbase python SDK 4.1.8 (and older versions) doesn’t run on a fresh Ubuntu 22.04 installation with python 3.10.

It produces the following error when trying to import couchbase:

Traceback (most recent call last):
  File "//main.py", line 1, in <module>
    import couchbase
  File "/usr/local/lib/python3.10/dist-packages/couchbase/__init__.py", line 66, in <module>
    from couchbase.pycbc_core import CXXCBC_METADATA, pycbc_logger  # nopep8 # isort:skip # noqa: E402
ImportError: libssl.so.1.1: cannot open shared object file: No such file or directory

I have installed all requirements mentioned for Ubuntu.

I have created a dockerfile to reproduce it:
ubuntu.zip (914 Bytes)

Same behavior when using python:3.10 docker images.

An older python:3.10 image where it is still working is: python:3.10@sha256:aa79a3d35cb9787452dad51e17e4b6e06822a1a601f8b4ac4ddf74f0babcbfd5

While you can manually install libssl1.1, is there a better way without having to manually install packages or is this something that will get fixed in upcoming SDK versions?

Thanks and best

Hi @fifteen_renditions - Thanks for taking the time to post and report the issued.
There is an existing case open for Python on Ubuntu 22.04 that mentions the openssl issue. Log in - Couchbase database

  • Mike

Hi @mreiche,
thanks for the quick reply. I think this issue is not publicly accessible - any way I can view it?

Thanks and best

Hmmm… yeah, it’s not public. I’m not sure why - I think it’s just the default setting for CBD issues. If you want status-update type information, you can open a PYCBC issue and indicate you are interested in CBD-5450.

That CBD points to another CBD where the developer’s notes for the work-around are:

As expected, installing the Python SDK (python3 -m pip install couchbase) will not work out-of-the-box on Ubuntu 22.04. The two paths forward (as of now) are below:

Build the Python SDK from source (uses the default OpenSSL version on Ubuntu 22.04). Need some extra dependencies (only cmake) and is slower than installing with the wheel. See attached build log (couchbase_install_ubuntu22.txt) from steps below.

Steps:


# extra installs from what we already list:  cmake, python3-venv (optional)
ubuntu@ip-172-31-4-14:~$ sudo apt-get install git python3-dev python3-pip python3-setuptools build-essential libssl-dev cmake python3-venv
 
ubuntu@ip-172-31-4-14:~$ cd /tmp
 
ubuntu@ip-172-31-4-14:/tmp$ python3 -m venv /tmp/cb_build_sdk
 
ubuntu@ip-172-31-4-14:/tmp$ source /tmp/cb_build_sdk/bin/activate
 
(cb_build_sdk) ubuntu@ip-172-31-4-14:/tmp$ python -m pip install --upgrade pip setuptools wheel
 
(cb_build_sdk) ubuntu@ip-172-31-4-14:/tmp$ python -m pip install couchbase --no-binary couchbase --log couchbase_install.txt
 
(cb_build_sdk) ubuntu@ip-172-31-4-14:/tmp$ python -m pip list
Package    Version
---------- -------
couchbase  4.1.6
pip        23.2.1
setuptools 68.0.0
wheel      0.41.0
 
(cb_build_sdk) ubuntu@ip-172-31-4-14:/tmp$ python -c "from couchbase import get_metadata; print(get_metadata(detailed=True))"
{'__cplusplus': '201703', 'asio': '1.24.0', 'build_timestamp': '2023-08-01 22:56:44', 'cc': 'GNU 11.4.0', 'cmake_build_type': 'Release', 'cmake_version': '3.27.0', 'compile_definitions': 'HAVE_BACKTRACE=1;HAVE_DLADDR=1;_GNU_SOURCE=1;SPDLOG_COMPILED_LIB;SPDLOG_FMT_EXTERNAL', 'compile_features': 'cxx_std_17;cxx_variadic_templates', 'compile_flags': '', 'compile_options': '-fdiagnostics-color=always;-ggdb3;-static-libgcc;-static-libstdc++', 'cpu': 'x86_64', 'cxx': 'GNU 11.4.0', 'fmt': '8.800.1', 'http_parser': '2.9.4', 'libc': 'glibc 2.35', 'link_depends': '', 'link_flags': '', 'link_libraries': 'project_options;project_warnings;fmt::fmt;spdlog::spdlog;snappy;jsonsl;hdr_histogram_static;couchbase_backtrace;couchbase_logger;couchbase_platform;couchbase_meta;couchbase_crypto;couchbase_sasl;couchbase_tracing;couchbase_metrics;http_parser', 'link_options': '', 'mozilla_ca_bundle_date': 'Tue May 30 03:12:04 2023 GMT', 'mozilla_ca_bundle_embedded': True, 'mozilla_ca_bundle_sha256': '5fadcae90aa4ae041150f8e2d26c37d980522cdb49f923fc1e1b5eb8d74e71ad', 'mozilla_ca_bundle_size': 137, 'openssl_config_dir': '/usr/lib/ssl', 'openssl_crypto_interface_imported_location': '/usr/lib/x86_64-linux-gnu/libcrypto.so', 'openssl_crypto_interface_include_directories': '/usr/include', 'openssl_crypto_interface_link_libraries': '', 'openssl_default_cert_dir': '/usr/lib/ssl/certs', 'openssl_default_cert_dir_env': 'SSL_CERT_DIR', 'openssl_default_cert_file': '/usr/lib/ssl/cert.pem', 'openssl_default_cert_file_env': 'SSL_CERT_FILE', 'openssl_headers': 'OpenSSL 3.0.2 15 Mar 2022', 'openssl_pkg_config_interface_include_directories': '', 'openssl_pkg_config_interface_link_libraries': '', 'openssl_runtime': 'OpenSSL 3.0.2 15 Mar 2022', 'openssl_ssl_imported_location': '/usr/lib/x86_64-linux-gnu/libssl.so', 'openssl_ssl_interface_include_directories': '/usr/include;/usr/include', 'openssl_ssl_interface_link_libraries': 'OpenSSL::Crypto', 'platform': 'Linux-5.19.0-1025-aws', 'platform_name': 'Linux', 'platform_version': '5.19.0-1025-aws', 'post_linked_openssl': 'ON', 'revision': '', 'semver': '1.0.0+', 'snappy': '1.1.10', 'snapshot': False, 'spdlog': '1.9.2', 'static_openssl': False, 'static_stdlib': True, 'txns_forward_compat_extensions': 'TI,MO,BM,QU,SD,BF3787,BF3705,BF3838,RC,UA,CO,BF3791,CM,SI,QC,IX,TS', 'txns_forward_compat_protocol_version': '2.0', 'version': '1.0.0', 'version_build': 0, 'version_major': 1, 'version_minor': 0, 'version_patch': 0}

Build OpenSSL 1.1.1 and point the Python SDK to the libs. Need some extra dependencies (build-essential and checkinstall) and need to make sure the SDK knows where to find the libs (LD_LIBRARY_PATH).

Steps:


ubuntu@ip-172-31-4-14:~$ cd /usr/src
 
ubuntu@ip-172-31-4-14:/usr/src$ sudo apt-get install -y build-essential checkinstall
 
ubuntu@ip-172-31-4-14:/usr/src$ export OPENSSL_VERSION=1.1.1t
 
ubuntu@ip-172-31-4-14:/usr/src$ sudo wget https://www.openssl.org/source/old/1.1.1/openssl-$OPENSSL_VERSION.tar.gz
 
ubuntu@ip-172-31-4-14:/usr/src$ sudo tar -xvf openssl-$OPENSSL_VERSION.tar.gz
 
ubuntu@ip-172-31-4-14:/usr/src$ sudo mv openssl-$OPENSSL_VERSION openssl
 
ubuntu@ip-172-31-4-14:/usr/src$ cd openssl
 
ubuntu@ip-172-31-4-14:/usr/src/openssl$ sudo ./config && sudo make -j4 && sudo make test
 
ubuntu@ip-172-31-4-14:/usr/src/openssl$ mkdir $HOME/opt && mkdri $HOME/opt/lib
 
ubuntu@ip-172-31-4-14:/usr/src/openssl$ cp libcrypto.so.1.1 $HOME/opt/lib
 
ubuntu@ip-172-31-4-14:/usr/src/openssl$ cp libssl.so.1.1 $HOME/opt/lib
 
ubuntu@ip-172-31-4-14:~$ cd /tmp
 
ubuntu@ip-172-31-4-14:/tmp$ python3 -m venv /tmp/cb_build_sdk
 
ubuntu@ip-172-31-4-14:/tmp$ source /tmp/cb_build_openssl11/bin/activate
 
(cb_build_openssl11) ubuntu@ip-172-31-4-14:/tmp$ python -m pip install --upgrade pip setuptools wheel
 
(cb_build_openssl11) ubuntu@ip-172-31-4-14:/tmp$ python -m pip install couchbase
 
(cb_build_openssl11) ubuntu@ip-172-31-4-14:/tmp$ python -m pip list
Package    Version
---------- -------
couchbase  4.1.6
pip        23.2.1
setuptools 68.0.0
wheel      0.41.0
 
# this will fail, SDK does not know where to find the SSL libs
(cb_build_openssl11) ubuntu@ip-172-31-4-14:/tmp$ python -c "from couchbase import get_metadata; print(get_metadata(detailed=True))"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/cb_build_openssl11/lib/python3.10/site-packages/couchbase/__init__.py", line 66, in <module>
    from couchbase.pycbc_core import CXXCBC_METADATA, pycbc_logger  # nopep8 # isort:skip # noqa: E402
ImportError: libssl.so.1.1: cannot open shared object file: No such file or directory
 
# this will succeed as the SDK knows where to pick up the SSL libs
(cb_build_openssl11) ubuntu@ip-172-31-4-14:/tmp$ LD_LIBRARY_PATH=$HOME/opt/lib:$LD_LIBRARY_PATH python -c "from couchbase import get_metadata; print(get_metadata(detailed=True))"
{'__cplusplus': '201703', 'asio': '1.24.0', 'build_timestamp': '2023-07-13 19:47:02', 'cc': 'GNU 9.3.0', 'cmake_build_type': 'RelWithDebInfo', 'cmake_version': '3.26.4', 'compile_definitions': 'HAVE_BACKTRACE=1;HAVE_DLADDR=1;_GNU_SOURCE=1;SPDLOG_COMPILED_LIB;SPDLOG_FMT_EXTERNAL', 'compile_features': 'cxx_std_17;cxx_variadic_templates', 'compile_flags': '', 'compile_options': '-fdiagnostics-color=always;-ggdb3;-static-libgcc;-static-libstdc++', 'cpu': 'x86_64', 'cxx': 'GNU 9.3.0', 'fmt': '8.800.1', 'http_parser': '2.9.4', 'libc': 'glibc 2.24', 'link_depends': '', 'link_flags': '', 'link_libraries': 'project_options;project_warnings;fmt::fmt;spdlog::spdlog;snappy;jsonsl;hdr_histogram_static;couchbase_backtrace;couchbase_logger;couchbase_platform;couchbase_meta;couchbase_crypto;couchbase_sasl;couchbase_tracing;couchbase_metrics;http_parser', 'link_options': '', 'mozilla_ca_bundle_date': 'Tue May 30 03:12:04 2023 GMT', 'mozilla_ca_bundle_embedded': True, 'mozilla_ca_bundle_sha256': '5fadcae90aa4ae041150f8e2d26c37d980522cdb49f923fc1e1b5eb8d74e71ad', 'mozilla_ca_bundle_size': 137, 'openssl_config_dir': '/usr/local/ssl', 'openssl_crypto_interface_imported_location': '/usr/local/openssl/lib/libcrypto.so', 'openssl_crypto_interface_include_directories': '/usr/local/openssl/include', 'openssl_crypto_interface_link_libraries': '', 'openssl_default_cert_dir': '/usr/local/ssl/certs', 'openssl_default_cert_dir_env': 'SSL_CERT_DIR', 'openssl_default_cert_file': '/usr/local/ssl/cert.pem', 'openssl_default_cert_file_env': 'SSL_CERT_FILE', 'openssl_headers': 'OpenSSL 1.1.1l  24 Aug 2021', 'openssl_pkg_config_interface_include_directories': '', 'openssl_pkg_config_interface_link_libraries': '', 'openssl_runtime': 'OpenSSL 1.1.1t  7 Feb 2023', 'openssl_ssl_imported_location': '/usr/local/openssl/lib/libssl.so', 'openssl_ssl_interface_include_directories': '/usr/local/openssl/include;/usr/local/openssl/include', 'openssl_ssl_interface_link_libraries': 'OpenSSL::Crypto', 'platform': 'Linux-4.14.291-218.527.amzn2.x86_64', 'platform_name': 'Linux', 'platform_version': '4.14.291-218.527.amzn2.x86_64', 'post_linked_openssl': 'ON', 'revision': 'adc416db2449283ccf33ed5cd525262900949441', 'semver': '1.0.0-dp.6', 'snappy': '1.1.10', 'snapshot': False, 'spdlog': '1.9.2', 'static_openssl': False, 'static_stdlib': True, 'txns_forward_compat_extensions': 'TI,MO,BM,QU,SD,BF3787,BF3705,BF3838,RC,UA,CO,BF3791,CM,SI,QC,IX,TS', 'txns_forward_compat_protocol_version': '2.0', 'version': '1.0.0', 'version_build': 155, 'version_major': 1, 'version_minor': 0, 'version_patch': 0}
1 Like

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.