Cloud database technology can no longer be ignored in financial services
Recent developments in cloud based data technology points to the beginning of a wider step change in the delivery of data technology within financial services, including two recent announcements made at the 2017 Google Next event in London.
The first was from David Knott, Chief Architect at HSBC who announced HSBC’s ‘cloud first’ strategy for the bank. This is a significant moment and one that is of particular interest for banks trying to side step the legacy of technical debt that has been built up over the past 40 years. There are still significant hurdles to overcome that have thus far prevented wide scale adoption in financial services whilst other industries have ploughed ahead. The shoots of cloud migration for financial services are, however, beginning to emerge.
The second was the announcement by Google of their cloud relational database service, named Cloud Spanner, which was released in production form in May. Cloud Spanner is the first and only global relational database service that is both strongly consistent, horizontally scalable and delivers 99.999% availability. This adds another significant weapon to the Google Cloud Platform’s arsenal of data engineering tools which can no longer be ignored by banks wanting to reduce the cost and increase the speed at which they analyse data.
Cloud database technology
About ten years ago, Google realised that the future of cloud computing required a fundamentally different approach to the norm of colocation and virtualised data centres. Driven by the need for a technology solution to support their global business for Adwords and GooglePlay, the emerging technology has matured and is now offered as a service available to Google Cloud Platform customers. The underpinning technology is a combination of Google’s own global network backbone and globally synchronised ‘TrueTime’ atomic clock implementation.
As Google states, Cloud Spanner is a “horizontally scalable, globally consistent, relational database service”. It provides all the traditional benefits of a relational database: ACID transactions, relational schemas (and schema changes without downtime), SQL queries, high performance and high availability. In addition it offers automatic scaling, synchronous data replication and node redundancy. Cloud Spanner delivers high availability for mission-critical applications.
Unlike any other relational database service, Cloud Spanner scales horizontally, to hundreds or thousands of servers, so it can handle the highest of transactional workloads. This may be an attractive proposition for organisations who need to ramp-up their data storage and processing capability quickly. There are some trade-offs. The data model is similar to other relational databases but, as would be expected, there are some specific technical constraints for enforcing referential integrity.
Fully managed cloud service
Using a cloud service, monitored and maintained by Google engineers undertaking infrastructure activities, such as sharding, failover and disk management, means the expensive overhead of infrastructure maintenance goes away. Those of us who have been involved in projects that required a best of breed failover or sharding solution will recognise how much of a benefit a fully managed service provides in terms of effort, cost and time-to-market. Cloud Spanner is of particular interest to global organisations that need to be consistent across a wide geographical distribution, who require a fully managed infrastructure with no server or database provisioning and maintenance, high-availability, suitability for mission-critical applications and the ability to scale with new nodes at the push of a button.
Disruptive technology is beginning to change traditional banking technology architecture. However, the complexity of banks interwoven systems means there are plenty of transformation challenges.
Typically global banks have grown through merger and acquisition. This has resulted in a plethora of different systems, a lack of consistency in platforms and therefore a number of constraints governing the speed of change.
For new challenger organisations that have a ‘greenfield’ approach to technology infrastructure and require a fully managed, global database service, Cloud Spanner offers a powerful alternative from traditional infrastructure. For well-established organisations that have a mature and complex infrastructure, there may be some business functions that are more suited to cloud than others.
The first problem to overcome is to abstract the complex infrastructure from publishers and subscribers of data services, perhaps utilising an application programming interface (API) and/or a microservices strategy. Once this is achieved the migration business case for adopting the latest cloud architectures such as Cloud Spanner becomes much more compelling.
Opportunity to reduce costs
It is clear that the cloud paradigm can lead to significant cost reduction, with fully utilised hardware bringing economies of scale. Resilience without redundancy results in a more effective infrastructure.
The key benefits of Cloud Spanner include:
- Fully managed infrastructure, with no server or database provisioning and maintenance
- High-availability (99.999%), suitable for mission-critical applications
- Elastic and able to scale rapidly with minimum effort
Moving to cloud will inevitably require changes to operating models. New access and security measures may need to be implemented that adhere to existing policies, in order for the full benefits of a ‘cloud first’ strategy to be realised. Compliance to existing data jurisdiction boundaries and ring-fencing also needs to be factored into the design.
Typically, financial institutions approach global availability at a system and region level. Each team with global access requirements may have replicated or sharded databases, multi-region caching using open source NoSQL technologies, or full-blown multi-region data-grid solutions. The complexity of designing, maintaining and debugging these systems creates very high up-front and ongoing costs for each of these teams.
As a technology company, Google approaches these issues from the other direction, providing for all these needs with a single world-leading data platform. With Google now making this platform available to everyone, this adds a new dimension and opportunity for leveraging the benefits of a cloud infrastructure. The availability, scalability and consistency of Google Spanner running on Google’s own private network simply cannot be matched by existing distributed systems running on the public internet.
Regulatory, jurisdiction and data privacy considerations
The finance industry is a complicated one, and there are impediments to simply globalising current systems.
Jurisdictionally or regulatory constrained systems, such as trading or payments systems, may not be able to be re-designed for a single global-instance. However, for multiple regional databases with global access-patterns, guaranteed globally-consistent reads are a windfall.
Other critical but supportive banking activities such as compliance, financial crime, HR, CRM or ticketing and support, where there are fewer enforced constraints, could be much more straightforward to globalise with Cloud Spanner. Many of these systems are currently hosted in a single region and when accessed from offshore locations, the latency and throughput can significantly affect productivity.
Data migration considerations
Another key consideration for adopting cloud is the viability of the data migration path. Typical scenarios are:
- Simplest case of an isolated, low volume, non-regulatory constrained database
- Mid complexity with a small number of feeds, some data volatility, where latency is not an issue e.g. a reporting system with a daily feed of data
- Highly interconnected, multiple feeds, with golden source access, alerting / emails
When assessing systems for suitability to make this move, considerations include; volatility, latency, volume, security, relational database management system (RDBMS) or NoSQL and the degree of interconnectedness.
Migrating to cloud will also offer opportunities for reducing software licencing costs. In an environment of cost-reduction, the pay-as-you-go cloud model, where you pay for what you use, is an attractive proposition.
From a technology perspective, application and business logic is often tightly coupled to legacy relational database systems, often as PL/SQL in stored procedures. Cloud Spanner supports SQL syntax for reading data but there is no native support for such SQL commands as ‘insert’ or ‘delete’ or any of the more advanced functions. A migration to Cloud Spanner will therefore require a more loosely coupled solution based on APIs.
Whilst supporting these features will be less of a concern for ‘greenfield’ projects, most of the existing financial services implementations use relational databases; migrating from these will not be a trivial exercise and should not be underestimated.
Benefits to Total Cost of Ownership (TCO)
One interesting area of research is therefore to understand the total cost of ownership of a traditional relational database including licence, maintenance and fixed hardware costs, compared to the cost of migration to Cloud Spanner as well as the on-going pay-as-you-go costs. In other words, the benefits of any data migration effort should be weighed up against long term total cost of ownership.
Moreover from the perspectives of architecture, ongoing support and maintenance, moving to a loosely coupled API based solution has many advantages, which those of us that have had to upgrade or migrate such a system will appreciate. It could therefore be argued that the cost of migration could be discounted against the benefits of migrating to a services orientated architecture in the TCO calculation.
An opportunity to re-architect your business?
In the past, business architecture was somewhat constrained by technology. In order to provide the service required of business critical relational databases, a regional approach to systems architecture was required. Cloud Spanner is now challenging this approach as its inherent characteristics mean that banks are no longer constrained by traditional technology limitations. Banks looking at what Cloud Spanner means for their business, can do so with a new enterprise architecture lens and design a more global approach, not limited by traditional relational database constraints.
Google is currently the only cloud provider that offers a service like this and Cloud Spanner is certainly a step-change in what cloud can offer. It is a combination of hardware, software and operational support that, together provide a completely new paradigm in enterprise technology. We anticipate that there will be significant benefits to be leveraged for those firms willing to embrace the opportunity it provides.
This blog was co-authored by Nick Weisfeld and Rob Saundby (Principal Consultant, GFT)