3 Couchbase nodes, each running on a separate container, all in the same cluster
Python application running in another container (querying, inserting, deleting data from the Couchbase cluster)
What I managed to do: Set up a cluster, bucket, query the bucket via UI (accessed by localhost:8091 )
What I didn’t manage to do: Create a connection between a Python application ( which would at the end be Dockerized, for now for the sake of simplicity, let’s treat it as local ) and the (already working) cluster. Unfortunately, I cannot access it via Docker containers IP’s with 8091 port, via localhost too. Unfortunately, the Couchbase documentation is either severely lacking here, or I just don’t understand it. I tried to even use the setting-alternate-address option, but without much success (maybe I used it wrongly, so if you have any “how-to’s” explaining the process, I’d still be grateful)
The connection works if there is one node, but throws Timeout if I set up 3 nodes.
I would really appreciate any tips leading to solving this problemm.
You state you are using containers, but a lot more information about the network setup is needed to troubleshoot this. For example, if I was running 3 nodes in Docker on my local laptop, I would need bridging, etc set up for them to communicate with each other. A more detailed description of your node setup (Docker vs K8s, etc), networking, DNS setup, etc will help us better understand the configuration.
Make sure you note the paragraph:
“Clients are aware of which pod a data item should reside on and perform client-side load balancing. By performing the load balancing in the client, this avoids unnecessary network hops and improves performance. For this reason, Couchbase Server cannot be accessed using normal Kubernetes Service or Ingress resources.”
I will fully admit that setting up networking can be complex if it’s the first time doing it. Because of things like TLS, there isn’t a simple button you can click to make it all work together, however, I tend to try and use the Autonomous Operator when possible as it greatly reduces the amount of time I spend doing this.
That is a fairly complicated, but very interesting setup you’ve got there!
Got some questions about it before I can help:
What networking mode are you using for containers that run couchbase nodes?
What command/parameter/configuration are you using to proxy/forward the localhost:8091 port at which you get couchbase UI?
I expect that this is more of a problem with Docker networking setup, but can’t confirm or disprove this theory without knowing the answers.
Hi @dmitrii.chechetkin@biozal , thank you both for your answers. Unfortunately I need to say that I am really terrible at networking stuff. Therefore I am not sure if I can answer your questions. For me, it seems like I just followed the Docker multi-node setup instructions from the Couchbase documentation so it should work
I tried tinkering with Docker network configuration, but to no avail. So assume everything is as in documentation (if not specified, then as default in Docker).
And I can, like, create a bucket and query it using my browser. Why is it then, that my SDK can’t do this? Could you, in simplest possible words, explain what I would need to achieve it?
Regarding K8S, I made another topic to keep the discussion on point, I am currently fighting to set the same thing up using K8S (the ideal situation would be that both of those solutions are working, but even if one of them works, that’d be awesome).
Could you please share the link to the tutorial you used and the commands you are using to launch couchbase containers? Is it this one?
I would also recommend trying to diagnose the connection with our SDK doctor: SDK Doctor | Couchbase Docs
The output of it would be very helpful here.
And, finally, could you provide some additional diagnostic info? I need to know the LAN IP address of your computer and each container to understand the network setup. You can get your IPs on linux/mac via ip addr show or ifconfig, and on windows via ipconfig.
And I can, like, create a bucket and query it using my browser. Why is it then, that my SDK can’t do this? Could you, in simplest possible words, explain what I would need to achieve it?
I think that this is because there’s no connectivity from your host to all nodes in the cluster. It is really not possible to answer these questions without seeing the actual network setup, though.
@dmitrii.chechetkin Sorry for the delay, let me try to answer all your questions. One important thing - I am using Ubuntu on WSL. Not sure if that changes anything, but wanted to let you know:
Could you please share the link to the tutorial you used and the commands you are using to launch couchbase containers? Is it this one?
Yes, it is this one
I would also recommend trying to diagnose the connection with our SDK doctor: SDK Doctor | Couchbase Docs
The output of it would be very helpful here.
And, finally, could you provide some additional diagnostic info? I need to know the LAN IP address of your computer and each container to understand the network setup. You can get your IPs on linux/mac via ip addr show or ifconfig , and on windows via ipconfig .
With nomenclature as in standard documentation, container addresses:
db1 172.17.0.2
db2 172.17.0.3
db3 172.17.0.4
The inet value under eth0 interface after running ip addr show on WSL:
172.30.16.54/20
I think that this is because there’s no connectivity from your host to all nodes in the cluster. It is really not possible to answer these questions without seeing the actual network setup, though.
Yeah what surprises me is that I can easily do that over browser and connect to multiple nodes, but can’t do that over SDK seemingly…
Hi @Latkos,
Are you running the python app on Ubuntu WSL or on the windows host itself?
Could you also share any SDK logs you are getting from the app?
@dmitrii.chechetkin I am running the Python app on WSL. But if it impacted the connections somehow, I guess that Couchbase wouldn’t work in the browser.
Logs:
Traceback (most recent call last):
File “C:/Users/michal.latkowski/Desktop/Couchbase_Connector/main.py”, line 32, in
main()
File “C:/Users/michal.latkowski/Desktop/Couchbase_Connector/main.py”, line 26, in main
result = coll.get(‘airline_10’)
File “C:\Users\michal.latkowski\Desktop\Couchbase_Connector\venv\lib\site-packages\couchbase\collection.py”, line 161, in get
return self._get_internal(key, **final_args)
File “C:\Users\michal.latkowski\Desktop\Couchbase_Connector\venv\lib\site-packages\couchbase\logic\wrappers.py”, line 111, in wrapped_fn
raise e
File “C:\Users\michal.latkowski\Desktop\Couchbase_Connector\venv\lib\site-packages\couchbase\logic\wrappers.py”, line 97, in wrapped_fn
raise ErrorMapper.build_exception(ret)
couchbase.exceptions.UnAmbiguousTimeoutException: <ec=14, category=couchbase.common, message=unambiguous_timeout, context=KeyValueErrorContext:{‘key’: ‘airline_10’, ‘bucket_name’: ‘travel-sample’, ‘scope_name’: ‘_default’, ‘collection_name’: ‘_default’, ‘opaque’: 0}, C Source=C:\Jenkins\workspace\python\sdk\python-scripted-build-pipeline\py-client\src\kv_ops.cxx:209>
In case you’re wondering, the bucket exists and works, so it’s purely connection issue IMO.
Hi @Latkos
It looks like your windows machine doesn’t bridge the WSL network with the docker network. There are several things I think that can be done here:
Alternatively, (or if enabling WSL2 backend and integration doesn’t work), you can try using docker inside WSL, run the python app on the windows host, or use ipvlan/macvlan docker networking modes, which should make your docker containers appear as if they were physical computers connected to your LAN.
We do not officially support WSL afaik, but I hope this helps.
