The COVID-19 pandemic has substantially increased the pace of digitization, and for many organizations, data demands have never been higher. As enterprises accelerate their digital transformation journeys, the need to quickly and efficiently scale their database infrastructure to meet increased processing demands has driven the adoption of DBaaS solutions. This blog will summarize the comparison of three of the leading DBaaS, NoSQL solutions.
Altoros is a global IT service provider that has been comparing databases for 2 decades. In its latest NoSQL performance benchmark report, Altoros has provided a comparative analysis of three NoSQL DBaaS offerings: Couchbase Cloud, MongoDB Atlas, and Amazon DynamoDB. Its aim was to measure the relative performance in terms of latency and throughput that each database can achieve, by examining performance on three different cluster configurations – 6, 9, and 18 nodes – as well as under four different workloads.
To make the tests as comprehensive as possible, the workloads varied substantially. The first was update-heavy, invoking 50 percent reads and 50 percent updates of the data. The second performed a short-range scan, involving 95 percent scans and 5 percent updates, while the third workload represented a pagination query with a single filtering option to which an offset and limit were applied. Finally, the fourth workload was a JOIN query with grouping and ordering applied.
So how did the three DBaaS providers compare?
Head-to-head on NoSQL query workloads
While each of the three DBaaS offerings had its strengths and weaknesses, across every workload test Couchbase Cloud emerged as the most robust of the three. For the first, update-heavy workload, Couchbase Cloud demonstrated low latency times across all cluster sizes. While it posted similar results as Amazon DynamoDB when measured on a 6-node cluster, Couchbase Cloud stood out on 9-node and 18-node clusters with latency times of less than a millisecond.
When measured on a short-range scan workload, Couchbase Cloud again stood out. While MongoDB Atlas performed well here, growing efficiently as more shards were added, Couchbase Cloud demonstrated the best results with a gradual increase in operations per second, as well as the lowest and most stable latency.
The results for the third workload were more balanced between MongoDB Atlas and Couchbase Cloud. Both returned similar results on a 6-node cluster and a 9-node cluster, where throughput was low since the size of the dataset and indexes doubled. Despite the similar results, Couchbase Cloud was still able to outperform MongoDB Atlas across all cluster sizes on this workload.
Meanwhile, Amazon DynamoDB was unable to compete entirely: its indexing process works in such a way that requires complex full table scans, which took too long for this workload, returning time-out errors.
In the final workload category, the results once again favored Couchbase Cloud. Couchbase Cloud posted higher throughput with 750 ops/sec on 6-node clusters, compared with 100 ops/sec for MongoDB Atlas. As with the pagination workload, Amazon DynamoDB was ineligible to compete due to the nature of its indexing process.
Here is one of the workloads compared in the report – for the three cluster sizes, measuring throughput and latency:
Figure 4.1.3 Performance results under Workload A on 6-, 9-, and 18-node clusters
Amazon DynamoDB had unstable results because it produced a great number of failed operations. On each type of cluster, Amazon DynamoDB had 40% of failed update operations and almost 1% of failed read operations.
The results are in – best performing document store
Across the board, Altoros’ performance benchmark found Couchbase Cloud to be the best performing document store DBaaS available. MongoDB Atlas produced decent results, but fell short on JOIN operations for sharded collections out of the box:
Amazon DynamoDB also performed well on certain workloads but throws exceptions when read/write operations exceed the predetermined capacities which resulted in a large volume of failed operations in certain workloads. Comparatively, the query engine of Couchbase Cloud supports aggregation, filtering, and JOIN operations without the need to model data for each specific query, ensuring a good level of scalability across all of these operations.
So, it’s Couchbase with the win!
-
- Learn more about Couchbase Cloud NoSQL DBaaS and start testing your own workloads today.
- Read the full NoSQL DBaaS benchmarking report with many more details from Altoros.
- *Note this independent benchmark report was commissioned by Couchbase.
