Fluent bit has no access to read log files

Couchbase Server Kubernetes
1

I’m following exactly the same thing as a tutorial to add a fluent bit. But I’m getting an error that says “cannot open /opt/couchbase/var/lib/couchbase/logs/XXX.log”

https://docs.couchbase.com/operator/current/tutorial-couchbase-log-forwarding.html#create-a-custom-configuration

And this is the configuration i used
https://github.com/couchbase/couchbase-fluent-bit/blob/main/conf/fluent-bit-simple.conf

# Cut down version with no real work in it to use for testing/debug
@include /fluent-bit/etc/couchbase/service.conf

# Just pull in all logs with no parsing at all or Multiline support
[INPUT]
    Name tail
    Alias simple_tail
    Path ${COUCHBASE_LOGS}/*.log
    Refresh_Interval 10
    Skip_Long_Lines On
    Path_Key filename
    Tag couchbase.log.<logname>
    Tag_Regex ${COUCHBASE_LOGS}/(?<logname>[^.]+).log$

@include /fluent-bit/etc/couchbase/out-stdout.conf

error-logging
error-logging1400×445 204 KB

2

What sort of storage is backing the Persistent Volumes here? Perhaps there is some sort of security context in place with the provisioned storage.

3

Hi, tommie, Thanks for the reply, I’m just using docker desktop (WSL ubuntu), and I’m following the same thing as the example provided in the documentation. As I understand security context can be a case if I put this on something like openshift. But here I’m in my local docker-desktop
So this is how i mounted volume

apiVersion: v1
kind: Secret
metadata:
  name: cb-example-auth
type: Opaque
data:
  username: QWRtaW5pc3RyYXRvcg== # Administrator
  password: cGFzc3dvcmQ=         # password
---
apiVersion: couchbase.com/v2
kind: CouchbaseBucket
metadata:
  name: default
---
apiVersion: couchbase.com/v2
kind: CouchbaseCluster
metadata:
  name: cb-example
spec:
  logging:
    server:
      enabled: true 
      manageConfiguration: true
      configurationName: "fluent-bit-config"
      sidecar:
        image: "couchbase/fluent-bit:1.0.4"
    audit:
      enabled: true 
      garbageCollection:
        sidecar:
          enabled: true 
  image: couchbase/server:6.6.2
  security:
    adminSecret: cb-example-auth
  buckets:
    managed: true
  servers:
  - size: 3
    name: all_services
    services:
    - data
    - index
    - query
    - search
    - eventing
    - analytics
    volumeMounts:
      default: couchbase
  volumeClaimTemplates: 
  - metadata:
      name: couchbase
    spec:
      resources:
        requests:
          storage: 1Gi

And this a fluent bit config

apiVersion: v1
kind: Secret
metadata:
  name: fluent-bit-config
stringData:
  fluent-bit.conf: |
    [SERVICE]
        flush        1
        daemon       Off
        log_level    Warning
        parsers_file /fluent-bit/etc/parsers-couchbase.conf
    # Include audit log only
    @include /fluent-bit/etc/couchbase/in-audit-log.conf
    # Send to the standard output
    [OUTPUT]
        name  stdout
        match couchbase.log.*
4

Got it, setup seems fine. It’s possible that the user of the fluent sidecar is running as a different user than the Couchbase user. Could you try explicitly setting a SecurityContext so see if this resolves the problem here?

apiVersion: couchbase.com/v2
kind: CouchbaseCluster
metadata:
  name: cb-example
spec:
  securityContext:
    runAsUser: 1000
   ...

ref: CouchbaseCluster Resource | Couchbase Docs

1 Like
5

Thank you , it works