How to include external Couchbase SDK (libcouchbase.so.2) into AWS Lambda deployment package

Node.js SDK
1

I get the error libcouchbase.so.2: cannot open shared object file: No such file or directory in my lambda when it tries to initalize

I am trying to use the Couchbase Node.JS SDK with external C SDK on AWS Lambda, however, even though I include the .so files into my deployment package, Couchbase is unable to find them.

Everything is installed and packaged for lambda on amazonlinux:latest docker container

Versions used:

  • Node.js 8.10
  • npm couchbase@2.6.5
  • serverless 1.48.2 (to deploy lambda)

These are the steps I do to package everything up:

  1. Install libcouchbase rpm package in docker container
  2. Copy across libcouchbase.so, libcouchbase.so.2 and libcouchbase.so.2.0.65 files into deployment package
  3. Install couchbase npm package pointing to the .so files.
  4. Use serverless framework to package and deploy to lambda

Here is a snippet of the commands I run to do so:

WORKDIR /lambda

RUN wget http://packages.couchbase.com/releases/couchbase-release/couchbase-release-1.0-6-x86_64.rpm
RUN rpm -iv couchbase-release-1.0-6-x86_64.rpm
RUN yum install -y libcouchbase-devel libcouchbase2-bin

RUN mkdir -p libs\
  && cp /usr/lib64/libcouchbase.so libs \
  && cp /usr/lib64/libcouchbase.so.2 libs \
  && cp /usr/lib64/libcouchbase.so.2.0.65 libs

RUN npm install couchbase --compile --couchbase-root=libs

The error I get is:

{
  "errorMessage": "libcouchbase.so.2: cannot open shared object file: No such file or directory",
  "errorType": "Error",
  "stackTrace": [
    "Module.load (module.js:565:32)",
    "tryModuleLoad (module.js:505:12)",
    "Function.Module._load (module.js:497:3)",
    "Module.require (module.js:596:17)",
    "require (internal/module.js:11:18)",
    "bindings (/var/task/node_modules/bindings/bindings.js:112:48)",
    "Object.<anonymous> (/var/task/node_modules/couchbase/lib/binding.js:213:36)",
    "Module._compile (module.js:652:30)",
    "Object.Module._extensions..js (module.js:663:10)"
  ]
}
2

Hey @Evan_Karopoulos,

I guess to get started here before we get too deep, my first question is about why you are using an external libcouchbase? Typically we highly recommend using the libcouchbase that gets built as part of your Node.js SDK, as it has known compatibility and is well integrated without any additional dynamic linking effort.

Something you might try is setting your NODE_PATH environment variable before running your application:

export NODE_PATH=/usr/lib64

Cheers, Brett

3

Hi Brett,

Thanks for replying. I actually figured it out… partly. Im not having a problem here anymore as I was able to include all of the external libraries in the lib directory of the lambda.

Ill update with the fix

Ill create a new issue for the problems im having now

4

Hi Brett,

Ultimately I was trying to use the external C SDK so I could enable SSL as per the documentation https://docs.couchbase.com/nodejs-sdk/current/start-using-sdk.html#standalone-lcb. However, turns out that the standard Node.JS Couchbase SDK version 2.6.5 has it built in anyways which means none of this needs to be done.

For those that want to still use the external SDK on a lambda for any reason regardless of the language the key is to do the following:

  1. Follow the guide to first install libcouchbase at the OS level. https://docs.couchbase.com/c-sdk/2.10/start-using-sdk.html#install-linux. You will need to follow the guide for installing on RHEL and CentOS just to make sure that you get the right library files. If you’re OS isn’t linux then create a docker image running amazonlinux:1
  2. Copy all of thelibcouchbase.so files from /usr/lib64/ into the lib directory of your lambda function. This is key because lambda has been set up to have this directory as part of your LD_LIBRARY_PATH already
  3. npm install couchbase --compile --couchbase-root=lib
1 Like