Trying to get CNG (cloud native gateway) to work (from Cloud Native Gateway | Couchbase Docs ), with no luck. It’s a great addition, but can’t seem to get it to work.
Does anyone have a simple example?
And is the TLS definition mandatory, or can it just be open as http?
And it mentions that it should get it’s own container, but there’s not a lot of docs on that…is supposed to be a sidecar according to the docs…tried posting in the beta channel with no responses I weeks, can anyone help?
1 Like
Hi @ksully - can you show the errors you are getting?
sure, and thanks @mreiche . Here’s the errors I’m seeing in the couchbasedev-000 pod, and helm chart I’m using:
couchbase-server Starting Couchbase Server -- Web UI available at http://<ip>:8091 │
│ couchbase-server and logs available in /opt/couchbase/var/lib/couchbase/logs │
│ couchbase-server chown: changing ownership of 'var/lib/couchbase': Operation not permitted │
│ stream logs failed container "cloud-native-gateway" in pod "couchbasedev-0000" is waiting to start: trying and failing to pull image for couchbase/couchbasedev-0000 (cloud-native-gateway) │
│ stream logs failed container "cloud-native-gateway" in pod "couchbasedev-0000" is waiting to start: trying and failing to pull image for couchbase/couchbasedev-0000 (cloud-native-gateway) │
if you have something (the examples all seem very stale) that runs the native gateway, I have an on -prem cluster (kubeadm) that I need to expose externally (for ports 18091 and 18093). Sounds like the native gateway is the way to go (eliminating the need to write a custom client with the SDK), so excited to try that
It looks there are some fundamental issues getting the image. What example/documentation are you following?
“that I need to expose externally (for ports 18091 and 18093).”
There are a lot more ports than that. At least 11207 needs to be exposed for the cluster configuration.
“(eliminating the need to write a custom client with the SDK)”
The same clients should work with both couchbases:// and couchbases2://
All ports are open, those 2 are the only ones I need for the app for now. Couchbase clusters come up ok without this native gateway stuff.
Don’t have any trouble getting images for lots of other apps and couchbase except for this native gateway. There are no restrictions in my env.
Is it really 1.0.0 for the native gateway? Can’t seem to find it anywhere in github…
What example/documentation are you following?
This? Cloud Native Gateway | Couchbase Docs.
I believe it is published as couchbase/cloud-native-gateway
https://hub.docker.com/r/couchbase/cloud-native-gateway
docker pull couchbase/cloud-native-gateway:1.0.0-dockerhub
1 Like
yes, that’s the only doc I can find,https://docs.couchbase.com/operator/current/concept-cloud-native-gateway.html
So you’re saying that it is image 1.0.0, ok, that’s what i have in the helm values file, but it’s not working. Any thoughts on that?
and it pulls fine by hand from the cluster machines, but helm doesn’t like it. Is there some other debug I can get? I think I turned on debug in the helm but I don’t get much…
I’m pointing to what I see on docker.
yes, and I agree, but there’s something not working such that the image can’t be obtained. And again, this isn’t a problem with any of the other couchbase images, nor any other images (there are many) from other products loaded and regularly wiped and installed on this cluster.
So any thoughts on the yaml I sent, or examples of a sample that works fully with the native gateway for on on-prem cluster?
@ksully It looks like your values for the cloud native gateway is incorrect in your values.yaml
it should be couchbase/cloud-native-gateway:1.0.0 instead of cloud-native-gateway:1.0.0
1 Like
Ah, that did find the image (thanks @justin.ashworth), the official doc just needs updating (link posted prior).
But, now a new problem is showing up, where the couchbase instance (e.g. couchbasedev-0000) shows 1/2 containers running…the native gateway seems to not be able to access the cluster config, getting a 401…is there additional config that’s no documented to give the gateway access? Or am I missing a step?
│
│ cloud-native-gateway {"level":"warn","ts":"2024-06-03T16:39:55.908Z","logger":"gateway","caller":"gateway/gateway.go:193","msg":"failed to ping cluster","error":"failed to get cluster config: server error │
│ : access denied (status: 401, body: ``)","errorVerbose":"server error: access denied (status: 401, body: ``)\nfailed to get cluster config\ngithub.com/couchbase/stellar-gateway/gateway.pingCouchbaseCluste │
│ r\n\t/home/couchbase/jenkins/workspace/couchbase-k8s-microservice-build/couchbase-cloud-native-gateway/gateway/gateway.go:138\ngithub.com/couchbase/stellar-gateway/gateway.(*Gateway).Run\n\t/home/couchbas │
│ e/jenkins/workspace/couchbase-k8s-microservice-build/couchbase-cloud-native-gateway/gateway/gateway.go:191\nmain.startGateway\n\t/home/couchbase/jenkins/workspace/couchbase-k8s-microservice-build/couchbas │
│ e-cloud-native-gateway/cmd/gateway/main.go:399\nmain.glob..func1\n\t/home/couchbase/jenkins/workspace/couchbase-k8s-microservice-build/couchbase-cloud-native-gateway/cmd/gateway/main.go:53\ngithub.com/spf │
│ 13/cobra.(*Command).execute\n\t/home/couchbase/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:987\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\t/home/couchbase/go/pkg/mod/github.com/spf13/cobra@v1.8 │
│ .0/command.go:1115\ngithub.com/spf13/cobra.(*Command).Execute\n\t/home/couchbase/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:1039\nmain.main\n\t/home/couchbase/jenkins/workspace/couchbase-k8s-micr │
│ oservice-build/couchbase-cloud-native-gateway/cmd/gateway/main.go:463\nruntime.main\n\t/home/couchbase/jenkins/workspace/couchbase-k8s-microservice-build/toolstjyY1/go1.21.6/src/runtime/proc.go:267\nrunti │
│ me.goexit\n\t/home/couchbase/jenkins/workspace/couchbase-k8s-microservice-build/toolstjyY1/go1.21.6/src/runtime/asm_amd64.s:1650"}
The helm values file I’m using is attached, just a single instance of the services at first just to get the thing running. I also notice (maybe because of the error) that port 18098 is not exposed from the pods…
couchbase-values-1instance.support.zip (2.7 KB)
With Cloud Native Gateway, we don’t need any more than a single port @mreiche. That’s the feature!
I was referring to the comment “I have an on -prem cluster (kubeadm) that I need to expose externally (for ports 18091 and 18093)” - other ports would need to be exposed (from kubernetes) to be accessible by CNG.
Yes - that’s the name shown at https://hub.docker.com/r/couchbase/cloud-native-gateway
@ksully - there is a ticket open to fix the example in the documentation.
great, thanks for that (on the docs)
I’ve got nothing restricted, no firewalls or security in k8s in place (yet). i just mentioned those ports because that’s what my API’s use to do their job.
so if that’s the case, what do you think is causing the 401? Anything I can try (or might you have a config that does fully work?)
bad username/password?
Looks the same as I get in my dev environment with a wrong username/password:
{"level":"info","ts":"2024-06-03T11:06:20.909-0700","caller":"gateway/main.go:194","msg":"starting stellar-gateway","version":"devel"}
{"level":"info","ts":"2024-06-03T11:06:20.910-0700","caller":"gateway/main.go:196","msg":"parsed launch configuration","config":"","watch-config":false,"daemon":false}
{"level":"info","ts":"2024-06-03T11:06:20.910-0700","caller":"gateway/main.go:225","msg":"parsed gateway configuration","logLevelStr":"info","cbHost":"localhost","cbUser":"Administrator","bindAddress":"0.0.0.0","dataPort":18098,"sdPort":18099,"webPort":9091,"selfSign":true,"certPath":"","keyPath":"","cacertPath":"","otlpEndpoint":"","debug":false,"cpuprofile":""}
{"level":"info","ts":"2024-06-03T11:06:20.910-0700","caller":"webapi/webapi.go:88","msg":"starting health/metrics server","listenAddress":"0.0.0.0:9091"}
{"level":"info","ts":"2024-06-03T11:06:20.917-0700","logger":"gateway","caller":"gateway/gateway.go:176","msg":"linking to couchbase cluster","connectionString":"localhost","User":"Administrator"}
{"level":"info","ts":"2024-06-03T11:06:20.917-0700","logger":"gateway","caller":"gateway/gateway.go:184","msg":"identified couchbase server address","address":"localhost:8091"}
{"level":"info","ts":"2024-06-03T11:06:20.917-0700","logger":"gateway","caller":"gateway/gateway.go:187","msg":"waiting for couchbase server to become available","address":"localhost:8091"}
{"level":"warn","ts":"2024-06-03T11:06:20.928-0700","logger":"gateway","caller":"gateway/gateway.go:193","msg":"failed to ping cluster","error":"failed to get cluster config: server error: access denied (status: 401, body: ``)","errorVerbose":"server error: access denied (status: 401, body: ``)\nfailed to get cluster config\ngithub.com/couchbase/stellar-gateway/gateway.pingCouchbaseCluster\n\t/Users/michaelreiche/sn4/stellar-gateway/gateway/gateway.go:138\ngithub.com/couchbase/stellar-gateway/gateway.(*Gateway).Run\n\t/Users/michaelreiche/sn4/stellar-gateway/gateway/gateway.go:191\nmain.startGateway\n\t/Users/michaelreiche/sn4/stellar-gateway/cmd/gateway/main.go:399\nmain.glob..func1\n\t/Users/michaelreiche/sn4/stellar-gateway/cmd/gateway/main.go:53\ngithub.com/spf13/cobra.(*Command).execute\n\t/Users/michaelreiche/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:987\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\t/Users/michaelreiche/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:1115\ngithub.com/spf13/cobra.(*Command).Execute\n\t/Users/michaelreiche/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:1039\nmain.main\n\t/Users/michaelreiche/sn4/stellar-gateway/cmd/gateway/main.go:463\nruntime.main\n\t/Users/michaelreiche/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.4.darwin-amd64/src/runtime/proc.go:267\nruntime.goexit\n\t/Users/michaelreiche/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.4.darwin-amd64/src/runtime/asm_amd64.s:1650"}
{"level":"error","ts":"2024-06-03T11:06:20.929-0700","caller":"gateway/main.go:401","msg":"failed to run the gateway","error":"failed to get cluster config: server error: access denied (status: 401, body: ``)","errorVerbose":"server error: access denied (status: 401, body: ``)\nfailed to get cluster config\ngithub.com/couchbase/stellar-gateway/gateway.pingCouchbaseCluster\n\t/Users/michaelreiche/sn4/stellar-gateway/gateway/gateway.go:138\ngithub.com/couchbase/stellar-gateway/gateway.(*Gateway).Run\n\t/Users/michaelreiche/sn4/stellar-gateway/gateway/gateway.go:191\nmain.startGateway\n\t/Users/michaelreiche/sn4/stellar-gateway/cmd/gateway/main.go:399\nmain.glob..func1\n\t/Users/michaelreiche/sn4/stellar-gateway/cmd/gateway/main.go:53\ngithub.com/spf13/cobra.(*Command).execute\n\t/Users/michaelreiche/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:987\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\t/Users/michaelreiche/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:1115\ngithub.com/spf13/cobra.(*Command).Execute\n\t/Users/michaelreiche/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:1039\nmain.main\n\t/Users/michaelreiche/sn4/stellar-gateway/cmd/gateway/main.go:463\nruntime.main\n\t/Users/michaelreiche/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.4.darwin-amd64/src/runtime/proc.go:267\nruntime.goexit\n\t/Users/michaelreiche/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.4.darwin-amd64/src/runtime/asm_amd64.s:1650","stacktrace":"main.startGateway\n\t/Users/michaelreiche/sn4/stellar-gateway/cmd/gateway/main.go:401\nmain.glob..func1\n\t/Users/michaelreiche/sn4/stellar-gateway/cmd/gateway/main.go:53\ngithub.com/spf13/cobra.(*Command).execute\n\t/Users/michaelreiche/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:987\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\t/Users/michaelreiche/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:1115\ngithub.com/spf13/cobra.(*Command).Execute\n\t/Users/michaelreiche/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:1039\nmain.main\n\t/Users/michaelreiche/sn4/stellar-gateway/cmd/gateway/main.go:463\nruntime.main\n\t/Users/michaelreiche/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.4.darwin-amd64/src/runtime/proc.go:267"}
Can you do a kubectl get pod couchbasedev-0000 -o yaml and post that? It shows how we’re configuring the sidecar container, and it should reference a secret, that contains the username/password combination for CNG to work appropriately.