Trying to use Python to connect via TLS to Couchbase Server

Python SDK
1

Hi,

I’m trying to use the Python SDK (2.5.1) to try to connect to Couchbase 5.5.2 using TLS. As there isn’t a lot of information online, I found bits and pieces here and there. So I want to ask some questions in hopes to get a better understanding.

Using the example found here (https://docs.couchbase.com/python-sdk/2.5/sdk-authentication-overview.html), I created all of my certificates (client side - client.pem, trust.pem, client.key; plus all server side). I get the following error:

Traceback (most recent call last):
  File "test2.py", line XX, in <module>
    authenticator = CertAuthenticator(cluster_username="Administrator", cluster_password="password", **options)
TypeError: __init__() got an unexpected keyword argument 'certpath'

Looking at the couchbase/cluster.pycode, it seems that CertAuthenticator should be taking in cert_path, trust_store_path, and key_path instead. Using those I get warnings where auth_credential and CertAuthenticator options overlap on keys.

I then looked at another example from /couchbaselabs/devguide-examples/python/connecting-ssl.py where I can use the Bucket object (http://docs.couchbase.com/sdk-api/couchbase-python-client-2.5.1/api/couchbase.html) to access the bucket directly by passing in the certpath, truststorepath, and keypath.

cb = Bucket("couchbases://172.17.0.1/MYBUCKET?certpath=/SSLCA/clientdir/client.pem&truststorepath=/SSLCA/clientdir/trust.pem&keypath=/SSLCA/clientdir/client.key")

However, I then get the error of:

couchbase.exceptions.LCB_0x37 (generated, catch: CouchbaseFatalError): <RC=0x37[Client could not verify server's certificate], There was a problem while trying to send/receive your request over the network. This may be a result of a bad network or a misconfigured client or server, C Source=(src/bucket.c,1066)>

Background info:

  • Using Python 3.6 in CentOS7.3
  • Installed libcouchbase-devel libcouchbase2-bin libcouchbase2-libevent libcouchbase2-libev libcouchbase2-core gcc gcc-c++
  • Pip installed couchbase
  • I read somewhere that there is supposed to be a compiled option for libcouchbase to use SSL? Not sure if that was n old version of the C SDK. I did not do anything beyond yum installing it.
  • Certificates was generated from provided code in /devguide-examples/etc/x509-cert (from GitHub). No error in generating the certificates.
  • Not using CertAuthenticator and just regular couchbase:// context works fine. It’s CertAuthenticator plus couchbases:// (or using the Bucket object to access directly) is creating the challenge.

Any idea or sample code that works?

Thank you.
Regards, Steve

2

Hi, apologies about the broken example code, will get that fixed ASAP. The second example you used with direct Bucket creation seems to be correct, but it may be that you haven’t uploaded and registered your certificates to the server. The scripts included are designed to work on a machine where the server is running on the same machine as the client. I did some work to try to make the uploads automatic but hadn’t got around to completing it. I’ll take a closer look on Monday but in the meantime, look at where the script copies certificates into the ‘inbox’ on the server and then calls a REST API command to register them.

Thanks,

Ellis

3

Thanks Ellis!

I did upload the root certificate, activate it, and enable client certificate with the REST endpoints:

/controller/uploadClusterCA
/node/controller/reloadCertificate
/settings/clientCertAuth

All succeeded without any problem.

I’m going to do another test with the Server running on the same environment and let you know. Thanks!

4

Just an update - I installed Couchbase 5.5.2 directly in the same environment as my Python dev environment. To test that everything worked, I tried it with PasswordAuthenticator and it works:

from couchbase.cluster import *
cb_cluster = Cluster("couchbases://127.0.0.1")
authenticator = PasswordAuthenticator(username="MYBUCKET", password="password")
cb_cluster.authenticate(authenticator)
cb = cb_cluster.open_bucket("MYBUCKET")

I then switched it to CertAuthenticator using the updated cert_path, trust_store_path, and key_path. I get the same authentication error message from previous post.

I tried it with both 127.0.0.1 and the actual physical IP of the server since the example script uses Subject Alternative Names and uses the IP defined (e.g., script uses 127.0.0.1 but I also modified that to the actual physical IP to test if that woudl change anything).

Since nothing worked. I regenerated the “out-of-the-box” Couchbase self-signed certificate and saved that as a file (e.g., cb-ss-cert.pem). File was chmod a+x. There is no trust or key file.

Then I retested everything:

from couchbase.cluster import *
cb_cluster = Cluster(connection_string="couchbases://127.0.0.1")
authenticator = CertAuthenticator(cert_path="/home/python/cb-ss-cert.pem")
cb_cluster.authenticate(authenticator)
cb = cb_cluster.open_bucket("MYBUCKET")

Again same authentication error message. I was hoping that even for this simplified scenario, at least it should have worked.

Edit:
The example scripts I’m referring to generate the certificates are from devguide-examples/etc/x509-cert/.

5

I ran a couple more tests with simplified code to give you more concrete example. All certificates were created from the devguide-examples/etc/x509-cert/ scripts without any errors. I created both the server and the client ones. I’ve upload the root certificate, activate it, and enable the client certificate as per previous reply.

This is using the Bucket object:

from couchbase.bucket import Bucket
cb = Bucket("couchbases://10.183.34.194/MYBUCKET?certpath=/home/me/SSLCA/clientdir/client.pem&truststorepath=/home/me/SSLCA/clientdir/trust.pem&keypath=/home/me/SSLCA/clientdir/client.key")
print(cb.server_nodes)

This is the error:

Traceback (most recent call last):
  File "test2.py", line 2, in <module>
    cb = Bucket("couchbases://10.183.34.194/MYBUCKET?certpath=/home/me/SSLCA/clientdir/client.pem&truststorepath=/home/me/SSLCA/clientdir/trust.pem&keypath=/home/me/SSLCA/clientdir/client.key")
  File "/home/me/python/cert/lib64/python3.6/site-packages/couchbase/bucket.py", line 259, in __init__
    self._do_ctor_connect()
  File "/home/me/python/cert/lib64/python3.6/site-packages/couchbase/bucket.py", line 268, in _do_ctor_connect
    self._connect()
couchbase.exceptions._AuthError_0x2 (generated, catch AuthError): <RC=0x2[Authentication failed. You may have provided an invalid username/password combination], There was a problem while trying to send/receive your request over the network. This may be a result of a bad network or a misconfigured client or server, C Source=(src/bucket.c,1066)>

This is using the Cluster object:

from couchbase.cluster import *
cb_cluster = Cluster(connection_string="couchbases://10.183.34.194")
authenticator = CertAuthenticator(cert_path="/home/me/SSLCA/clientdir/client.pem", trust_store_path="/home/ne/SSLCA/clientdir/trust.pem", key_path="/home/me/SSLCA/clientdir/client.key")
cb_cluster.authenticate(authenticator)
cb = cb_cluster.open_bucket("MYBUCKET")

Here is the error:

/home/me/python/cert/lib64/python3.6/site-packages/couchbase/cluster.py:230: UserWarning: auth_credential and CertAuthenticator options overlap on keys {'truststorepath', 'certpath', 'keypath'}
  warnings.warn("{} and {} options overlap on keys {}".format(item[0][0], item[1][0], clashes))
Traceback (most recent call last):
  File "test2.py", line 5, in <module>
    cb = cb_cluster.open_bucket("MYBUCKET")
  File "/home/me/python/cert/lib64/python3.6/site-packages/couchbase/cluster.py", line 144, in open_bucket
    rv = self.bucket_class(str(connstr), **kwargs)
  File "/home/me/python/cert/lib64/python3.6/site-packages/couchbase/bucket.py", line 259, in __init__
    self._do_ctor_connect()
  File "/home/me/python/cert/lib64/python3.6/site-packages/couchbase/bucket.py", line 268, in _do_ctor_connect
    self._connect()
couchbase.exceptions.LCB_0x37 (generated, catch: CouchbaseFatalError): <RC=0x37[Client could not verify server's certificate], There was a problem while trying to send/receive your request over the network. This may be a result of a bad network or a misconfigured client or server, C Source=(src/bucket.c,1066)>

Any help would be greatly appreciated! Thank you.

Edit: Testing without TLS works perfectly fine. Cluster object would just take the PasswordAuthenticator instead of CertificateAuthenticator.

Edit2: I read in the documentation the client-side certificate authentication was not supported in 5.0EE. I was using 5.5.2EE, which I believe did include the feature. However, since my earlier test did not work with 5.5.2EE, I installed 6.0EE just to double check.

6

@ellis.breen Happy Holidays. Checking if you had any luck. Thanks!

7

Hi, apologies, I haven’t looked into this as I was off sick this last week. Will take a look tomorrow.

Thanks,

Ellis

8

Ellis,

I have same problem. I could not connect to couchbase serwer via cert and password.

I tried this configuration but i got error (couchbase.exceptions._ClientTemporaryFailError_0x1B (generated, catch ClientTemporaryFailError): <RC=0x1B[Client not bootstrapped. Ensure bootstrap/connect was attempted and was successful], There was a problem scheduling your request, or determining the appropriate server or vBucket for the key(s) requested. This may also be a bug in the SDK if there are no network issues, C Source=(src/store.c,198)>):

    authenticator = CertAuthenticator(cert_path=certpath)
    cluster.authenticate(authenticator, username=username, password=password, )
9

Having the exact same issue. I just recently added SSL to my cluster. The nodes appear to be communicating fine over the encrypted ports but now the python app wont connect.

10

I suspect the only way to connect to a Couchbase cluster behind TLS with a username and password is via:

bucket = Bucket(connection_string='couchbases://hostname.example.internal/my_bucket_name?certpath=/path/to/ca.crt',
   username='USERNAME', password='PASSWORD')

Establishing the connection to the Cluster like it’s shown in the docs at https://docs.couchbase.com/python-sdk/current/managing-connections.html doesn’t seem to work resulting in the exception around mixed CertAuthenticator and PasswordAuthenticator.

11

Hi All,

I have a similar issue.

from couchbase.cluster import Cluster
from couchbase.cluster import PasswordAuthenticator

cluster = Cluster(‘couchbase://172.16.70.28’)
authenticator = PasswordAuthenticator(‘Administrator’, ‘Sea$$$$$$’)
cluster.authenticate(authenticator)
cb = cluster.open_bucket(‘ssiConfig’)

I tested the above code in two scenarios.

  1. Local Couchbase server. This worked perfectly fine.

  2. Remote server. This didn’t work.
    The consoles leaves the following error.

Traceback (most recent call last):
File “C:/Users/105067399/PycharmProjects/calculator/couchbasecon.py”, line 7, in
cb = cluster.open_bucket(‘ssiConfig’)
File “C:\Users\105067399\calculator\lib\site-packages\couchbase\cluster.py”, line 147, in open_bucket
*rv = self.bucket_class(str(connstr), *kwargs)
File “C:\Users\105067399\calculator\lib\site-packages\couchbase\bucket.py”, line 263, in init
self._do_ctor_connect()
File “C:\Users\105067399\calculator\lib\site-packages\couchbase\bucket.py”, line 272, in _do_ctor_connect
self._connect()
couchbase.exceptions._AuthError_0x2 (generated, catch AuthError): <RC=0x2[Authentication failed. You may have provided an invalid username/password combination], There was a problem while trying to send/receive your request over the network. This may be a result of a bad network or a misconfigured client or server, C Source=(src\bucket.c,1071)>
Process finished with exit code 1

Not sure if this helps you.

In terms of firewall….i did a TELNET to .28 server on port 8091 which passed. So I am not so sure of I am doing something wrong.

Any help here please.

Regards,
Agniz