CB Server 6.0.0 - beam.smp saturating cpu

Couchbase Server
1

I have a “sleeping” (no buckets ops) single server (n1-highmem-2 - 2 vCPU, 13 GB) installation of CB server 6.0.0 CE where CPU is saturated by beam.smp process. I’ve tried to stop sync gateway, but CPU load doesn’t change. Also restart service or machine reboot didn’t help. Generally CPU load used to be about 10%.

How can I inspect the cause of this increased load? I recently moved to sync gateway 2.6 (but can’t relate cpu load increase with it), might be a problem with new index structure?

2

Beam.smp is stil eagerly consuming resources (form 60 to over 100% cpu and 2GB ram on a 4 vCPU, 15 GB machine). Any suggestion? Is i considered a normal activity?

3

Hi @paolo.morgano,

beam.smp is responsible for monitoring and managing all other underlying server processes such as ongoing XDCR replications, cluster operations, and views. You can read more about beam.smp here: Couchbase Server Processes | Couchbase Docs

Can you describe your cluster in greater detail so we can get an idea how it is setup - (n1ql, 2i, xdcr,…) how many view indexes?

Anil Kumar

4

Hi @paolo.morgano - it can be difficult to diagnose things like CPU use remotely with scraps of information. If you’d like you can create a ticket (sign up at https://issues.couchbase.com) and attach your server logs [1] and we’ll take a look.

-dave

[1] cbcollect_info | Couchbase Docs

5

Thanks @davef and @anil for the link to docs and the suggestion of ticket creation. I’ve collected the required info and I’m going to fill a support ticket.

I resume my server config:

  • Single machine cluster for demo purposes

  • CB 6.0.0 CE + SG 2.6 CE

  • 10 buckets

  • SG is configured for shared bucket access

  • no views (may be SG has its how views? no use_view defined in its config file)

  • this is the list of indexes (most created by SG after updating to 2.6):

    keyspace_id name using
    “appversioning” “sg_access_1” “gsi”
    “appversioning” “sg_allDocs_1” “gsi”
    “appversioning” “sg_channels_1” “gsi”
    “appversioning” “sg_roleAccess_1” “gsi”
    “appversioning” “sg_syncDocs_1” “gsi”
    “building” “buildingindex” “gsi”
    “building” “sg_access_1” “gsi”
    “building” “sg_allDocs_1” “gsi”
    “building” “sg_channels_1” “gsi”
    “building” “sg_roleAccess_1” “gsi”
    “building” “sg_syncDocs_1” “gsi”
    “colnagowh” “sg_access_x1” “gsi”
    “colnagowh” “sg_allDocs_x1” “gsi”
    “colnagowh” “sg_channels_x1” “gsi”
    “colnagowh” “sg_roleAccess_x1” “gsi”
    “colnagowh” “sg_syncDocs_x1” “gsi”
    “colnagowh” “sg_tombstones_x1” “gsi”
    “dedalo” “dedalo_type” “gsi”
    “dedalo” “idx_type_appointment” “gsi”
    “dedalo” “ix_id” “gsi”
    “dedalo” “sg_access_x1” “gsi”
    “dedalo” “sg_allDocs_x1” “gsi”
    “dedalo” “sg_channels_x1” “gsi”
    “dedalo” “sg_roleAccess_x1” “gsi”
    “dedalo” “sg_syncDocs_x1” “gsi”
    “dedalo” “sg_tombstones_x1” “gsi”
    “kronoman” “#primary” “gsi”
    “kronoman” “km_race_id” “gsi”
    “kronoman” “km_type” “gsi”
    “kronoman” “sg_access_x1” “gsi”
    “kronoman” “sg_allDocs_x1” “gsi”
    “kronoman” “sg_channels_x1” “gsi”
    “kronoman” “sg_roleAccess_x1” “gsi”
    “kronoman” “sg_syncDocs_x1” “gsi”
    “kronoman” “sg_tombstones_x1” “gsi”
    “logs” “logsindex” “gsi”
    “logs” “sg_access_1” “gsi”
    “logs” “sg_allDocs_1” “gsi”
    “logs” “sg_channels_1” “gsi”
    “logs” “sg_roleAccess_1” “gsi”
    “logs” “sg_syncDocs_1” “gsi”
    “master” “#primary” “gsi”
    “master” “sg_access_1” “gsi”
    “master” “sg_allDocs_1” “gsi”
    “master” “sg_channels_1” “gsi”
    “master” “sg_roleAccess_1” “gsi”
    “master” “sg_syncDocs_1” “gsi”
    “step” “sg_access_1” “gsi”
    “step” “sg_allDocs_1” “gsi”
    “step” “sg_channels_1” “gsi”
    “step” “sg_roleAccess_1” “gsi”
    “step” “sg_syncDocs_1” “gsi”
    “step” “step_parent_index” “gsi”
    “step” “stepindex” “gsi”
    “task” “idx_owner” “gsi”
    “task” “sg_access_1” “gsi”
    “task” “sg_allDocs_1” “gsi”
    “task” “sg_channels_1” “gsi”
    “task” “sg_roleAccess_1” “gsi”
    “task” “sg_syncDocs_1” “gsi”
    “task” “task_owner_index” “gsi”
    “task” “task_parent_index” “gsi”
    “task” “taskindex” “gsi”
    “tour” “#primary” “gsi”
    “tour” “sg_access_1” “gsi”
    “tour” “sg_allDocs_1” “gsi”
    “tour” “sg_channels_1” “gsi”
    “tour” “sg_roleAccess_1” “gsi”
    “tour” “sg_syncDocs_1” “gsi”
    “tour” “tour-tenant” “gsi”

I noticed this frequent kind of error in server log:

    Compactor for view `building/_design/sync_housekeeping_2.0` (pid [{type,view},
    {name,
    <<"building/_design/sync_housekeeping_2.0">>},
    {important,
    false},
    {fa,
    {#Fun<compaction_new_daemon.25.106246998>,
    [<<"building">>,
    <<"_design/sync_housekeeping_2.0">>,
    {config,
    {30,
    undefined},
    {30,
    undefined},
    undefined,
    false,
    false,
    {daemon_config,
    30,
    131072,
    20971520}},
    false,
    {[{type,
    bucket}]}]}}]) terminated unexpectedly (ignoring this): {badmatch,
    {error,
    {{case_clause,
    {{error,
    vbucket_stream_not_found},
    {bufsocket,
    #Port<12396.13578>,
    <<>>}}},
    [{couch_dcp_client,
    init,
    1,
    [{file,
    "/home/couchbase/jenkins/workspace/couchbase-server-unix/couchdb/src/couch_dcp/src/couch_dcp_client.erl"},
    {line,
    314}]},
    {gen_server,
    init_it,
    6,
    [{file,
    "gen_server.erl"},
    {line,
    304}]},
    {proc_lib,
    init_p_do_apply,
    3,
    [{file,
    "proc_lib.erl"},
    {line,
    239}]}]}}}

This is the server history:

  • First installation with CB 5.0.0 and SG 1.5
  • Migrated to CB 6.0.0 and SG 2.0
  • Then updated SG to 2.6

Filled issue: Loading...