The relational database has been foundational to the information economy just like the steam engine enabled the industrial revolution.

However, the past success of relational databases (RDBMS) is no measure for what’s needed in today’s economy. Digital transformation driven by mobile technology, ecommerce, IoT and personalized interactions requires a new approach to how applications interact with data stores.

In this article, we’ll cover what makes Couchbase a better alternative to rigid and costly relational databases, and why enterprises use the erwin Data Modeler to migrate their applications to Couchbase.

Why Move Your Workloads to NoSQL

The corporate world is littered with companies who failed to quickly adapt to a changing environment.

Users and consumers have gotten used to instantaneous access to the latest information when and where they need it. They not only want to consume information, but also want to share their insights and experiences for the benefit of others. Such a dynamic environment requires companies to adapt fast without the constraints of an aged IT infrastructure.

NoSQL databases have successfully been used to address the major shortcomings of relational databases, and NoSQL has achieved a track record with production deployments in the most demanding business environments. For instance:

NoSQL databases deliver the flexibility and scalability modern applications need while flexing with a fluid data structure. Couchbase is unique in combining all the advantages of a NoSQL database architecture paired with enterprise class capabilities that ensures reliability and performance companies expect from a data technology platform. In addition, Couchbase provides SQL query capabilities on JSON data with the SQL++ query language (formerly known as N1QL), so that database administrators, architects and developers can re-use their existing SQL expertise in a NoSQL environment.

But that’s not enough to be a viable alternative to legacy databases. ACID compliance for transaction-oriented workloads has been uniquely associated with relational databases and was considered incompatible with a NoSQL distributed architecture. The introduction of Couchbase 7.0 put an end to that assumption by adding ACID-compliant SQL transaction capabilities and a Dynamic Data Containment model that adds multiple logical organization layers called scopes and collections.

Legacy Modernization Made Simple with Couchbase

The document database architecture – which stores information in JSON documents – provides the schema flexibility that is lacking in relational databases.

However, pulling multiple JSON documents into buckets to set up a data model doesn’t provide the granular hierarchy needed to easily map relational tables into a NoSQL database. With the introduction of scopes and collections in Couchbase 7.0 it’s now possible to organize data in such a way it easily maps to rows and columns of tables from an RDBMS.

Multiple JSON documents can be brought together into a collection, which in turn can be grouped together into a scope, which is stored into a bucket. Using this hierarchy, you can map a relational database to a bucket, in which a JSON document represents a row, the collection represents a table, and the scope being the grouping of related collections. See the table below for a full comparison.

A table comparing the relational data model with Couchbase scopes and collections

Figure 1: Mapping scopes and collections to a relational database model

In addition, distributed SQL transactions have been added to the SQL++ query language. This makes it possible to modify the database documents inside a transaction and then commit or rollback all of the changes in the transaction depending on the application logic, making it fully ACID compliant. With all these capabilities Couchbase now makes it possible for enterprises to modernize their slow-performing, rigid applications that are relying on aging relational databases.

Start with the Right Application

Not all applications are equally suited to start your database migration journey.

First, you want to identify those current applications that no longer meet the evolving needs of the business. These applications typically make use of a dynamic and changing data model.

Initially, you also want to select those workloads that don’t rely on highly complex SQL queries with multiple JOINs which have been carefully tuned for performance optimization. For a successful proof of concept, you want to focus on those use cases with limited complexity, where the data is well understood and where domain expertise is readily available.

Good examples for database migration candidates are:

    • Catalog and inventory solutions: Companies deal with large and dynamically changing product and service offerings, which require a flexible data structure to deliver real-time information with millions of requests at low latency.
    • Customer 360: Capture and consolidate data across all channels in real time to create a single view of all data. The unique capability of Couchbase to scale linearly and without disruption sets it apart from legacy relational databases.
    • Caching for performance acceleration: Provide users with a highly responsive experience by reducing workloads on backend resources while lowering operational costs.

Finally, there have to be clear success criteria for measuring the desired outcomes. These can be performance related, ease of management, or scalability with a lower total cost of ownership (TCO). Once a successful migration initiative has been completed, other workloads can be evaluated as potential targets for migration.

Transforming RDBMS Models to NoSQL with erwin Data Modeler

The movement towards more agile and efficient DBMSs comes with a significant paradigm shift in data modeling in order to realize those benefits.

In traditional relational data modeling, the structure has been driven by what data enterprises wanted to capture and store. The overarching principle was storage optimization – store atomic data once. This is referred to as the “normalization” of the structure.

These highly normalized database designs are great at ensuring data integrity, particularly data with parent/child relationships. However, normalization and the complex relationships it spawns requires complicated queries with multiple JOINs to retrieve that data. This leads to slower performance, or expensive resource requirements to mitigate performance shortfalls.

With Couchbase, the structure is driven by how you expect to query the data and the DBMS attends to the integrity of the data through the process. When data modeling for Couchbase, the overarching principle is query optimization to provide high performance with access to all relevant data from the fewest containers. This approach is called denormalization and leads to simpler queries with minimal JOINs, which provide a much faster response with fewer resources.

Nested JSON documents compared to multiple tables in a relational database

Figure 2: Multiple tables vs. nested data with JSON documents

erwin Data Modeler by Quest is unique in the industry as it supports both normalized and denormalized design paradigms in one solution. It allows you to alternate back and forth between the two depending on the user’s profile.

Data analysts who are used to more structured querying methods will want a normalized view of the data, while other users such as the database administrator will want to see the denormalized design required to instantiate the database and support developers querying the native source.

Database migration from a legacy RDBMS to a NoSQL database

Figure 3: Denormalization of a relational model using erwin Data Modeler

This becomes even more important when modernizing and migrating legacy database applications to Couchbase. erwin Data Modeler documents, in detail, the highly normalized RDBMS schema and automates the denormalization and retargeting of schema to Couchbase.

This enables your organization to migrate mission-critical applications to a more modern and responsive platform much faster, with a high degree of accuracy while reducing the cost and risks associated with the database migration.

The Critical Importance of Query-Optimized Data Modeling

Achieving the right level of denormalization is the key to achieving the performance and response from Couchbase and ensuring the most operationally efficient (and financially efficient, especially in the cloud) use of infrastructure and resources.

When undertaking the denormalization of a DBMS structure previously deployed on relational technology, there are some key considerations:

    • Are your people familiar with query-optimized design techniques?
    • Do you have a clear understanding of the details and complexity of the existing normalized structure?
    • How will you ensure that nothing slipped through the cracks in terms of the completeness and integrity of the structure when going through the design transformation process?
    • How will you share and communicate the nature and impact of these changes to the myriad of stakeholders working with the transformed structure?
    • How do you benchmark the existing workloads and tune the design to achieve your new performance goals?

erwin Data Modeler by Quest helps you overcome all of these challenges.

By providing guided workflows and automated normalization and denormalization mechanisms, traditional design teams more easily adopt and more effectively deliver a query-optimized approach to design. The rich data visualizations and documentation capabilities combined with modeling standards and audit support ensure customers have a robust understanding of both the as-is and the to-be data models while providing the automated ability to compare and synchronize for governing the transformation process. It also ensures the right view for the right role to support stakeholder understanding and promote more effective collaboration.

These models form the structural guide to understanding the as-is nature of the workloads and make it possible to quickly adjust the resulting Couchbase structure in order to tune the performance on the new platform. Finally, it can be shared and integrated into downstream data management and governance technologies to increase the agility, efficiency and effectiveness of maintaining this new environment.

The Future of Legacy Database Migration

erwin Data Modeler by Quest has long been the producer of the most integrated and accessible data models in the industry.

erwin Data Modeler extends the design, documentation and engineering support for all the major RDBMSs as well as NoSQL databases, with the ability to share, reuse and convert models from one platform to another. In addition, erwin Data Intelligence and its integration with erwin Data Modeler enables harvesting of metadata from all major database platforms and integrates with enterprise ETL/ELT solutions like Talend, Informatica and many others.

While erwin Data Modeler enables the conversion and denormalization of data models, erwin Data Intelligence allows you to accelerate the design and development of ETL jobs required to move the data from legacy DBMSs to Couchbase by providing detailed specifications of the source and target structures and their related metadata early in the migration lifecycle. This provides organizations the ability to migrate both the structure and instance data more efficiently and accurately, resulting in faster time to success.

Conclusion

NoSQL database technologies are no longer only fulfilling the need of new use cases that relational databases aren’t able to address.

Couchbase delivers foundational capabilities to provide enterprises with a credible alternative to rigid RDBMS structures. Together with erwin Data Modeler and erwin Data Intelligence, you embark on your database migration journey with confidence by bridging the gap between the old and new.

For more information please attend the erwin by Quest breakout sessions at Couchbase Connect 2021:

    • Data Empowerment: A Comprehensive Approach to Assuring the Successful Modernization of Your Apps – 10:00 a.m. PDT on October 21
    • Leveraging Data Models to Accelerate and Assure the Success of Your Modern Data Platform – 11:30 a.m. PDT on October 21

Additional assets:

 

Author

Posted by Danny Sandwell & Yves Laurent

Danny Sandwell is an IT industry veteran, who has focused on delivering value from data for more than 30 years. As Senior Solutions Manager at Quest, he is responsible for communicating the technical capabilities, synergies and business value of the company’s data empowerment solutions. During Danny’s 20+ years with the erwin brand, he also has worked in pre-sales consulting, product management, business development and business strategy roles – all giving him opportunities to engage with customers across various industries as they plan, develop, govern, protect and deliver their enterprise data architectures and assets. His goal is to help enterprises unlock the value of their data assets while mitigating data-related risks. Yves has over 20 years of experience in building partner and channel go to market strategies for leading technology companies. He started his career at Cisco Systems where he held various sales and marketing leadership positions across EMEA, APAC and US. Before joining Couchbase he led technology partnerships at H2O.ai, a machine learning and AI company and held partner success roles at Hortonworks, Denodo and analytics start-up companies. His focus is on bringing customer value by delivering integrated technology solutions that align with business objectives. During his spare time, he enjoys the outdoors with his family and friends.

All Posts

Leave a reply