Couchbase wrong expiry timestamp

Node.js SDK
, , , ,
1

After inserting a document with expiry date document has the wrong time zone.

how document created

function create(data) {
  return new Promise((resolve, reject) => {
    const id = createObjectID();

    const body = JSON.parse(data.body);
    delete data.body;

    // const expiry = new Date(
    //   moment(new Date()).add(41, "days").utc().format()
    // ).getTime(); //  1 day

    bucket.insert(
      id,
      { ...data, ...body },
      { expiry: 60 * 60 * 24 }, // expiry in one day
      (error) => {
        if (error) reject(error);
        else resolve({ id });
      }
    );
  });
}

what document expiry got

{
  "meta": {
    "id": "5f356e1187fe6895539dd4f1",
    "rev": "1-162ae1a26b4200005f36bf9102000000",
    "expiration": 1597423505,
    "flags": 33554432,
    "type": "json"
  },
  "xattrs": {}
}

expiration timestamp date, Why?

> new Date(1597423505)
1970-01-19T11:43:43.505Z

How to set document to expiry in one day?

2

Hey @chawki,

Node.js Date object’s represent time in milliseconds as opposed to the seconds you receive in a unix timestamp. So in your case, you’re actually looking to do something like this:

> new Date(1597423505 * 1000)
2020-08-14T16:45:05.000Z

Cheers, Brett

4

thanks for reply @brett19

then how to set the current timestamp to expiry in one day?

 bucket.insert(
      id,
      { ...data, ...body },
      { expiry:  what to set here to make document expiry in 1 day },
      (error) => {
        if (error) reject(error);
        else resolve({ id });
      }
    );
5

Hey @chawki,

Your initial code does what you expect and sets the expiry to 1 day.

Cheers, Brett

6

Hey @brett19

Actually I need to expiry my document in 40 days

What I did

 const expiry = new Date(
      moment(new Date()).add(40, "days").utc().format()
    ).getTime(); //
    console.log(expiry); // 1600796215000 -> 2020-09-22T17:36:55.000Z

    bucket.insert(
      id,
      { ...data, ...body },
      { expiry: expiry  },
      (error) => {
        if (error) reject(error);
        else resolve({ id });
      }
    );

What it returns

TypeError: expiry option needs to between 0 and 2147483647.
7

https://docs.couchbase.com/java-sdk/3.0/concept-docs/documents.html#expiry

For Couchbase SDKs which accept simple integer expiry values (as opposed to a proper date or time object) allow expiration to be specified in two flavors.

  1. As an offset from the current time.
  2. As an absolute Unix time stamp

If the absolute value of the expiry is less than 30 days ( 60 * 60 * 24 * 30 seconds), it is considered an offset . If the value is greater, it is considered an absolute time stamp .

It might be preferable for applications to normalize the expiration value, such as by always converting it to an absolute time stamp. The conversion is performed to avoid issues when the intended offset is larger than 30 days, in which case it is taken to mean a Unix time stamp and, as a result, the document will expire automatically as soon as it is stored.

2 Likes
8

Hey @chawki,

Just to add to @vsr1’s answer: Node.js deals with dates almost entirely in terms of ‘milliseconds since epoch’, where as standard unix timestamps are ‘seconds since epoch’. So in your case, again you need to perform that conversion:

// Using Dates's directly:
expiry: (new Date()).getTime() / 1000

// Since I saw you are using moment:
expiry: moment().add(40, "days").unix()

Cheers, Brett

1 Like
9

Hi Brett, has the option to set absolute unix time been removed?
I have the following code which I use when testing new insertion methods, was working fine before but now the expiry time is always considered an offset in seconds using the last sdk 4.4.5

   var duration = { minutes: '1' };
   var options = { 'expiry': moment.utc().add(duration).unix() };
10

It’s quite possible. The server makes assumptions about the expiration based on the size (delta vs absolute), and the SDK tries to compensate. I can look up.the ticket and details if you like.

11

Thanks please do. I don’t know since which sdk this behaviour changed.

12

Jira (JSCBC-124) indicates that the change from Jira (CBD-3509) was not present in the nodejs 4.x SDK until 4.3.1

There is no option to pass an absolute timestamp in the nodejs sdk. There is a request to support it Jira (JSCBC-1270)

Here’s some text from earlier discussion…

if they were passing an expiry in seconds of times greater than 30 days (but not absolute times) - without JSCBC-1245 - the server would treat those as epoch+expiry, resulting in the documents being deleted immediately. With JSCBC-1245, those documents would be deleted at now()+expiry

Also - if they use now() + ttl for the expiry, it looks like the SDK will add another now() - setting the document expiry to the year 2078.

13

I cannot see CBD-3509, it doesn’t let me access it.
Anyway even without any parameter it was possible to pass absolute time before, my code is almost identical to the one brett posted above and the resulting value is higher than ( 60 * 60 * 24 * 30 seconds), so it should be treated as absolute, but it it is still treated as offset. Either the SDK is doing an unnecessary conversion or the server makes the wrong assumption.

14

That is correct - it was possible to pass as an absolute time-stamp - that was an error on the SDK. It should not have been able to pass as a timestamp and is no longer possible to pass as a timestamp. If you have a timestamp, subtract now() from it to get a delta time.

15

Ok, thanks. I will change it, luckily I have few instances of this to revise.
Anyway the other SDK, like the Java one above still mentions the possibility.

16

Yes - and JSCBC-1270 is a request to support it in the nodejs SDK.