LCB_ESOCKSHUTDOWN error when trying to connect to couchbase with Node SDK.
I am using:
- Node version 8.10.0
- Couchbase Node SDK v2.6.5
- SSL
After successfully completing SSL verification it tries to create a connection over HTTP. This is when the LCB_ESOCKSHUTDOWN shows up.
It reties and happens again 17 times over 3 seconds.
Detailed stack strace:
2019-07-31 01:40:32 [myapp] info: The DB connection string is: couchbases://127.0.0.1?certpath=/var/tmp/myapp/lib/couchbase-cert.pem&bootstrap_on=http&detailed_errcodes=1
2019-07-31 01:40:32 [myapp] info: The DB bucket is: bucket
0ms [I34ac0890] {9749} [INFO] (instance - L:469) Version=2.9.5-njs, Changeset=2bf296af07b40e1b0163f900c4f8bf388ae4e8a3
0ms [I34ac0890] {9749} [INFO] (instance - L:470) Effective connection string: couchbases://127.0.0.1/bucket?certpath=%2Fvar%2Ftmp%2Fmyapp%2Flib%2Fcouchbase-cert.pem&bootstrap_on=http&detailed_errcodes=1&enable_errmap=true&enable_tracing=true&client_string=couchnode%2F2.6.5%20(node%2F8.10.0%3B%20v8%2F6.2.414.50%3B%20ssl%2F1.0.2n). Bucket=bucket
0ms [I34ac0890] {9749} [DEBUG] (SSL - L:285) Load verify locations from "(null)"
0ms [I34ac0890] {9749} [DEBUG] (instance - L:318) Applying initial cntl detailed_errcodes=1
0ms [I34ac0890] {9749} [DEBUG] (instance - L:318) Applying initial cntl enable_errmap=true
0ms [I34ac0890] {9749} [DEBUG] (instance - L:318) Applying initial cntl enable_tracing=true
0ms [I34ac0890] {9749} [DEBUG] (instance - L:318) Applying initial cntl client_string=couchnode/2.6.5 (node/8.10.0; v8/6.2.414.50; ssl/1.0.2n)
0ms [I34ac0890] {9749} [INFO] (instance - L:146) DNS SRV lookup failed: The requested feature is not supported by the client, either because of settings in the configured instance, or because of options disabled at the time the library was compiled. Ignore this if not relying on DNS SRV records
0ms [I34ac0890] {9749} [DEBUG] (instance - L:83) Adding host 127.0.0.1:18091 to initial HTTP bootstrap list
0ms [I34ac0890] {9749} [DEBUG] (instance - L:83) Adding host 127.0.0.1:11207 to initial CCCP bootstrap list
0ms [I34ac0890] {9749} [TRACE] (instance - L:126) Bootstrap hosts loaded (cccp:1, http:1)
3ms [I34ac0890] {9749} [INFO] (bootstrap - L:218) Requested network configuration: heuristic
3ms [I34ac0890] {9749} [DEBUG] (confmon - L:87) Preparing providers (this may be called multiple times)
3ms [I34ac0890] {9749} [DEBUG] (confmon - L:96) Provider CCCP is DISABLED
3ms [I34ac0890] {9749} [DEBUG] (confmon - L:94) Provider HTTP is ENABLED
3ms [I34ac0890] {9749} [TRACE] (confmon - L:277) Refreshing current cluster map
3ms [I34ac0890] {9749} [TRACE] (confmon - L:264) Attempting to retrieve cluster map via HTTP
3ms [I34ac0890] {9749} [TRACE] (htconfig - L:347) Starting HTTP Configuration Provider 0x3e603e0
3ms [I34ac0890] {9749} [INFO] (connection - L:474) <127.0.0.1:18091> (SOCK=0000000002e918bd) Starting. Timeout=2000000us
11ms [I34ac0890] {9749} [TRACE] (connection - L:367) <127.0.0.1:18091> (SOCK=0000000002e918bd) Received completion handler. Status=0. errno=0 [Success]
11ms [I34ac0890] {9749} [INFO] (connection - L:147) <127.0.0.1:18091> (SOCK=0000000002e918bd) Connected established
11ms [I34ac0890] {9749} [INFO] (connection - L:108) <127.0.0.1:18091> (SOCK=0000000002e918bd) Couldn't set TCP_NODELAY
11ms [I34ac0890] {9749} [DEBUG] (connection - L:106) <127.0.0.1:18091> (SOCK=0000000002e918bd) Successfully set TCP_KEEPALIVE
11ms [I34ac0890] {9749} [DEBUG] (htconfig - L:306) Successfuly connected to REST API 127.0.0.1:18091
11ms [I34ac0890] {9749} [DEBUG] (ioctx - L:101) <127.0.0.1:18091> (CTX=0x3e60a70,unknown) Pairing with SOCK=0000000002e918bd
11ms [I34ac0890] {9749} [TRACE] (SSL - L:235) sock=0x3e60350: ST(0x10). before/connect initialization. R(0x1)U
68ms [I34ac0890] {9749} [TRACE] (SSL - L:235) sock=0x3e60350: ST(0x20). SSL negotiation finished successfully. R(0x1)U
69ms [I34ac0890] {9749} [DEBUG] (SSL - L:239) sock=0x3e60350. Using SSL version TLSv1.2. Cipher=<cipher>
77ms [I34ac0890] {9749} [TRACE] (htconfig - L:210) <127.0.0.1:18091> (CTX=0x3e60a70,bc_http) Received 395 bytes on HTTP stream
81ms [I34ac0890] {9749} [TRACE] (htconfig - L:210) <127.0.0.1:18091> (CTX=0x3e60a70,bc_http) Received 8565 bytes on HTTP stream
81ms [I34ac0890] {9749} [TRACE] (htconfig - L:210) <127.0.0.1:18091> (CTX=0x3e60a70,bc_http) Received 9 bytes on HTTP stream
82ms [I34ac0890] {9749} [DEBUG] (htconfig - L:225) <127.0.0.1:18091> (CTX=0x3e60a70,bc_http) Generation 0 -> 1
82ms [I34ac0890] {9749} [INFO] (confmon - L:160) Setting new configuration. Received via HTTP
82ms [I34ac0890] {9749} [INFO] (bootstrap - L:90) Selected network configuration: "default"
got this far
133ms [I34ac0890] {9749} [DEBUG] (lcbio_mgr - L:448) <127.0.0.1:11207> (HE=0x3e6cf10) Creating new connection because none are available in the pool
133ms [I34ac0890] {9749} [TRACE] (lcbio_mgr - L:359) <127.0.0.1:11207> (HE=0x3e6cf10) New pool entry: I=0x3e50e80
133ms [I34ac0890] {9749} [INFO] (connection - L:474) <127.0.0.1:11207> (SOCK=00000000501ede3f) Starting. Timeout=2500000us
141ms [I34ac0890] {9749} [TRACE] (connection - L:367) <127.0.0.1:11207> (SOCK=00000000501ede3f) Received completion handler. Status=0. errno=0 [Success]
141ms [I34ac0890] {9749} [INFO] (connection - L:147) <127.0.0.1:11207> (SOCK=00000000501ede3f) Connected established
141ms [I34ac0890] {9749} [INFO] (connection - L:108) <127.0.0.1:11207> (SOCK=00000000501ede3f) Couldn't set TCP_NODELAY
141ms [I34ac0890] {9749} [DEBUG] (connection - L:106) <127.0.0.1:11207> (SOCK=00000000501ede3f) Successfully set TCP_KEEPALIVE
141ms [I34ac0890] {9749} [DEBUG] (lcbio_mgr - L:318) <127.0.0.1:11207> (HE=0x3e6cf10) Received result for I=0x3e50e80,C=(nil); E=0x0
141ms [I34ac0890] {9749} [DEBUG] (lcbio_mgr - L:278) <127.0.0.1:11207> (HE=0x3e6cf10) Assigning R=0x3e6d1c0 SOCKET=0x3e50f10
141ms [I34ac0890] {9749} [TRACE] (server - L:793) <127.0.0.1:11207> (SRV=0x3e63d80) Session not yet negotiated. Negotiating
141ms [I34ac0890] {9749} [DEBUG] (ioctx - L:101) <127.0.0.1:11207> (CTX=0x3e5a740,unknown) Pairing with SOCK=00000000501ede3f
141ms [I34ac0890] {9749} [DEBUG] (negotiation - L:440) <127.0.0.1:11207> (CTX=0x3e5a740,sasl,SASLREQ=0x3e66c20) HELO identificator: {"a":"libcouchbase/2.9.5-njs couchnode/2.6.5 (node/8.10.0; v8/6.2.414.50; ssl/1.0.2n)","i":"0000000034ac0890/00000000501ede3f"}, features: 0x02 (TLS), 0x06 (XATTR), 0x0b (JSON), 0x08 (Select bucket), 0x07 (XERROR), 0x03 (TCP nodelay), 0x0a (Snappy), 0x0f (Tracing)
141ms [I34ac0890] {9749} [TRACE] (SSL - L:235) sock=0x3e50f10: ST(0x10). before/connect initialization. R(0x1)U
149ms [I34ac0890] {9749} [ERROR] (negotiation - L:154) <127.0.0.1:11207> (CTX=0x3e5a740,sasl,SASLREQ=0x3e66c20) Error: 0x2d, IO Error
149ms [I34ac0890] {9749} [ERROR] (server - L:777) <NOHOST:NOPORT> (CTX=(nil),,SRV=0x3e63d80,IX=0) Connection attempt failed. Received LCB_ESOCKSHUTDOWN (0x2D) from libcouchbase, received 0 from operating system
149ms [I34ac0890] {9749} [DEBUG] (retryq - L:350) Adding PKT=0x3e50dc0 to retry queue. Try count=1
149ms [I34ac0890] {9749} [TRACE] (retryq - L:199) Next tick in 14 ms
...
2633ms [I34ac0890] {9749} [TRACE] (retryq - L:199) Next tick in 0 ms
2633ms [I34ac0890] {9749} [TRACE] (retryq - L:199) Next tick in 0 ms
2633ms [I34ac0890] {9749} [TRACE] (retryq - L:199) Next tick in 0 ms
2633ms [I34ac0890] {9749} [WARN] (retryq - L:162) Failing command (seq=0) from retry queue: LCB_ESOCKSHUTDOWN (0x2D)
events.js:183
throw er; // Unhandled 'error' event
^
CouchbaseError: The remote host closed the connection