Altoros, a renowned IT consulting organization with an extensive history in database comparison, has published their most recent independent benchmark, commissioned by Couchbase. The benchmark evaluates the DBaaS solutions offered by Couchbase, Redis, Amazon, and MongoDB. This study utilizes the widely accepted YCSB standard for benchmarking NoSQL databases. This blog presents a short summary of their report, which demonstrates that Couchbase Capella surpasses MongoDB Atlas, Redis Enterprise Cloud, and Amazon DynamoDB in terms of features, performance, and total cost of ownership (TCO) across diverse workloads and cluster sizes.
YCSB (Yahoo! Cloud Serving Benchmark) is a standardized framework used for benchmarking the performance of cloud-based database systems. It consists of a variety of workload tests. Here are the workloads chosen for this study:
-
- Workload A, 50% read and 50% update: designed to simulate a write-heavy workload scenario where the database system primarily handles read operations with occasional updates.
- Workload C, 100% read: simulates a scenario where the database system primarily performs read operations, providing insights into the system’s performance and scalability specifically for read-intensive workloads.
- Pagination workload: emulates a scenario involving paginated data retrieval. In this workload, the database system is evaluated based on its ability to efficiently fetch a subset of data from a larger dataset, typically through a combination of read and seek operations. It helps assess the system’s performance in handling pagination and data retrieval operations commonly encountered in applications that require displaying data in smaller chunks, like a leaderboard or implementing pagination mechanisms for long series of ordered results.
- Workload E, 95% read, 5% update: designed to assess the performance and scalability of a database system under a workload where the majority of operations are reads, with a smaller proportion of updates.
Benchmark Highlights
For the results, make sure to check out the detailed Altoros report. Note that the new Magma storage engine was used for this set of benchmarks. (Magma has efficiency benefits and reduced memory requirements for large data storage, but for pure speed Couchstore will often be the better choice.) Here are some highlights.
Performance
As in previous years, Couchbase does extraordinarily well in an update-heavy use case, especially as the need for large scale increases. For example, here’s Workload A results:
In these graphs, you can see that Capella’s throughput is the highest of all 4 DBaaS offerings. Due to Capella’s memory-first architecture and automatic sharding, the latency is also the lowest among the 4 competitors.
Functionality
Workload C is worth highlighting for a “pure cache” use case. Capella and Redis Enterprise Cloud both perform well for this workload due to their memory-first designs, which can cache results for faster response. MongoDB and DynamoDB need to read data from disk more often, and therefore do not perform as well. We’ve seen many designs where MongoDB is augmented with a Redis-like system to act as its cache, while Couchbase has this capability built-in.
Once again, high throughput and low latency.
However, while Redis performs very well in the case of a pure cache, it struggles with another workload: pagination.
As in previous years, DynamoDB is left out of this workload, since it does not have any pagination capabilities in its query language. Redis does have pagination capabilities, but they perform very poorly. In a typical Redis deployment, this type of processing would be pawned off on a secondary database. Capella handles this workload very well: high throughput, low latency, and easy as SQL syntax of SELECT . . . LIMIT . . . OFFSET.
On the other hand, DynamoDB performs well in Workload E. However, there are some other factors to consider with Workload E in Dynamo: see more detail in the report about how DynamoDB was provisioned, the money required, and DynamoDB’s behavior with regards to throttling and scaling up.
Total Cost of Ownership (TCO)
In the case of a “pure cache”, Redis is often used in conjunction with other databases. You will get good cache performance, but at the price of running two separate databases. (For instance, Redis and Mongo, Redis and Oracle, etc).
In regards to “bang for your buck”, there are charts included that summarize the cost of each database deployment in “dollars per billion operations”. For example, here is Workload A’s chart (shorter bars means less expensive):
When you read the full report in detail, you will see that Couchbase doesn’t always win every matchup. However, what it does win is the bigger picture: an ideal combination of low cost, sophisticated functionality, and performance (the “triple threat” as our customer Broadjump has stated).
Couchbase Capella does very well for the majority of workload+cluster combinations. In addition, Capella is a fast, extremely versatile multi-model database that includes:
-
- Key-value – direct lookup at memory speeds
- Built-in caching – low latency, fast lookup
- JSON Document – flexible data
- SQL++ – flexible querying
- Full Text Search – text, ranking, geospatial searches (without needing a separate tool like Solr)
- Time Series – stored in TS_arrays in JSON with built-in formatting functions
- Mobile Sync – automatic and offline-first with client to client sync, too!
- Automatic sharding/partitioning – easy for developers and great for growth
- Automatic replication and rebalancing – easy for ops
- Analytics with SQL++ – complex query options for BI
- Eventing – processing logic at the database level
- Change-data-capture with Kafka integration to stream document change logs
Taking all these factors into consideration, Couchbase Capella provides the best total package of high performance, sophisticated functionality, and lower total cost of ownership.
Next Steps
-
- Make sure to read the final report from Altoros
- Sign-up for a free trial of Couchbase Capella
- Got questions? Check out the Couchbase Discord
