You might have gone into an alcohol-induced hibernation over the holidays, but cloud databases did not.
More specifically, the cumbersomely named Microsoft Azure Cosmos DB did not, rocketing past Amazon Web Services (AWS) Redshift, as Begin founder Brian Leroux first noticed.
While everything was “as you were” for AWS database leader DynamoDB in 2017, according to DB Engines’ comprehensive ranking, Cosmos DB jumped 27 places, from 58 to 31.
What’s emerging is a very different approach to data across competing cloud vendors. AWS has introduced powerful options for familiar data needs: Amazon Redshift for data warehousing, Amazon Aurora/RDS for traditional relational workloads, and AWS DynamoDB for NoSQL.
Meanwhile, with Cosmos DB, Microsoft seems to be heading in the opposite direction, with a one-size-fits-all approach to data that seems to be catching fire.
Cloud databases are where the (new) action is
Most data remains firmly ensconced in traditional RDBMSs like Oracle, MySQL, and Microsoft SQL Server. While NoSQL has started to change this (MongoDB being the best example, thanks to its flexible schema document data store), databases remain the least likely enterprise infrastructure to change. There’s simply too much risk involved with changing databases.
Or there would be, if it weren’t for AWS, Microsoft, and, increasingly, Google. Risk-averse enterprises may shy away from doing business with small-fry NoSQL startups, but they simply can’t avoid doing business with AWS and Microsoft.
Indeed, judging from trends highlighted in DB-Engines January 2018 rankings, “avoidance” doesn’t remotely describe what’s happening with the top cloud database services.
Although DB-Engines isn’t a perfect representation of the database market, it aggregates and weights diverse measures of database popularity, including jobs postings, technical discussions on Stack Overflow, and search interest. As such, it’s a reasonable estimate of relative database adoption.
While Oracle, MySQL, and Microsoft SQL Server stand supreme at the top of the database heap, their cloud competitors have been gaining steam—and fast.
It’s probably not yet accurate to say that databases like DynamoDB and Azure Cosmos DB are gaining ground on the old guard, given that Oracle remains more than 100 times as popular as Cosmos, for example. But for new applications largely born in the cloud, these cloud-first databases dominate.
This matters because, as Gartner analyst Thomas Bittman has written, there’s a pronounced (and accelerating) shift from private data centres to public cloud environments: “New stuff tends to go to the public cloud, while doing old stuff in new ways tends to go to private clouds. And new stuff is simply growing faster.”
Not just a little bit faster, either: We’re talking about a 20X growth rate for the public cloud versus a 3X growth rate for private datacenters, by his analysis. Of course, legacy workloads dwarf these new cloud-friendly applications, but that won’t be true for long.
As such, it’s worth noting that among these cloudy upstarts, Cosmos DB has outpaced every other cloud database contender. If only symbolically, it recently surpassed Google BigQuery and Amazon Redshift, both of which were on the market before it was.
Although it still has a ways to go to beat out Amazon DynamoDB or even its sibling Azure SQL Database, at this rate of growth Cosmos DB may actually surpass both in 2018.
Where Cosmos DB gets its edge
The reason for Cosmos DB’s ascendance may stem from declining developer interest in “polyglot persistence.” Coined by Thoughtworks’ Martin Fowler back in 2011, polyglot persistence suggests that “any decent sized enterprise will have a variety of different data storage technologies for different kinds of data.”
Rather than forcing data to fit a relational data model, for example, an enterprise will more likely embrace wide-column data (Apache Cassandra) for some parts of an application, a graph database (Neo4j) for others, and relational (PostgreSQL) for still others.
The popularity of databases like MongoDB is a clear sign that, in fact, we do live in an increasingly polyglot world.
Microsoft’s genius with Cosmos DB is that developers may want to have their polyglot persistence cake and eat it too—all in one place.
As InfoWorld’s Serdar Yegulalp has written, “With Cosmos DB, Microsoft offers multiple consistency models in the same database, so the choice of model can be a function of the workload rather than the product.” That’s huge.
Equally huge is the multi-model approach that makes Azure Cosmos DB a bit of a jack-of-all-trades, as Yegulalp explains: “What Microsoft is offering here isn't one particular kind of database. It's a universal back end for different kinds of databases—likely including future styles of database that haven't been invented yet.”
This lets a developer dig deep into Cosmos DB for a wide array of application requirements, rather than having to learn an equally wide array of point solutions for different application needs.
There’s a risk that Cosmos DB’s general-purpose approach could make it a watered-down option for all applications, rather than the best tool for a particular job. Judging from its popularity, however, developers don’t see it this way.
All of this sets us up in 2018 to watch a classic battle between two companies at the top of their cloud games: Amazon and Microsoft. At the AWS Re:Invent conference in December, Amazon announced more new database technologies and feature upgrades than most companies could hope to create in a lifetime, each living in a separate product.
Microsoft, by contrast, seems to be doubling down on Cosmos DB, investing it with diverse powers that make it a great default database for any application a developer can think up.
Which will win? That’s for developers to decide. But if Cosmos DB does push past DynamoDB in 2018, expect it to have serious ripple effects in the broader public cloud market.