Welcome to the Cloud
It’s exciting! You’ve got a huge new toolbox with a cornucopia of choices. And forums with a bonanza of advice.
Whether you are creating, migrating, or something in-between, everybody has an opinion about building cloud apps:
- Development Model: Serverless? Micro-services? Containerized? Traditional?
- Platform: Cloud-Provider? PaaS? IaaS packages?
- Database Stores: MongoDB? PostgreSQL? DynamoDB? S3?
While you dig through the avalanche of advice, there is one topic conspicuous by its absence… what is the right way to handle persistent data?
Day 1: How Should I Manage My Application's Data?
There are even more ways to store data in cloud than there were on-premises!
You can use:
- Database as a Service: AWS RDS, MongoDB Atlas, etc.
- Standalone Databases: PostgreSQL, MongoDB, MySQL, etc.
- Files: local files or cloud NAS
- Objects: more tiers than an Instagram wedding cake
Most applications use a combination of tools to get the best results.
Once you decide how to store the data, you need to provision for capacity and performance. You might even consider how you’ll scale, deal with failures, secure the data, and meet compliance regulations (Hey, I’m an optimist).
It’s a stressful decision. Data is “the new oil”, “the lifeblood of business”, and “the new bacon”. You want to pick the right way to manage your data; your business depends on that information. Unfortunately, with so many variables, making the right decision seems impossible. Still, you’ll pick something reasonable, and then you’ll never have to worry about it again. Phew.
Except… data doesn’t take care of itself. In the cloud, there is no storage, backup, disaster recovery, or compliance team watching over your data. If you care about your application, you’ll be managing its data every day of the application’s life.
Day 2: A Day in the Life of a Cloud App Developer
Congratulations! You’ve released an application into your environment. You’ve created something that helps people. Take a moment to celebrate.
Now, take a deep breath. Your life will never be the same. Here’s a day in your life.
06:30: Make sure backups are happening and that they’re being replicated off site. Consider testing a restore to make sure the backups are good. Resolve to do it tomorrow… for the 100th straight day. Push down the guilt.
10:30: You get an escalation about application performance. After looking at processing and network, check out the data path. Figure out which storage resources are serving the application. Look at historical data to find changes in workload, resource performance, etc. Hope that application performance goes back to normal on its own.
12:10: Discuss with others how to scale the application. If you add more compute resources, how can they share the same data? Will that become a bottleneck?
14:07: Somebody hits a bug in the application. Create a clone environment, so you can reproduce the bug and test and fix. Wait hours to clone or create a data set.
16:20: Finance has asked you to reduce the cost of running your application. Explore ways to either use less expensive storage or use it more efficiently/dynamically.
18:30: As you walk out of the building, wonder when you will have time to ever build a new application.
23:30: Security has identified a flaw in the configuration of some AWS S3 buckets. They want to make sure your data is secure and encrypted. You need to send them details of what you’re doing and how. They also demand full audit logs of who has access to the data and what they’ve done.
06:30: Do it all over again…
The cloud offers infinite tools for building applications: UI, queuing, analytics, etc. Without data management, however, you won’t have time to create new applications. Instead, you’ll become a de-facto platform administrator: managing storage, backup, and compliance for your existing apps.
Day 2, Take 2: A Better Day
It may feel like you only have two options:
- Bad: Spend every day in data management hell.
- Worse: Use only one type of storage — e.g. object, DBaaS, etc.
There is a third answer: a data management solution that does the job of managing storage, data protection, and compliance for you.
In this world, today is not the worst day of your life…
06:30: Breakfast. [Backups have been happening automatically every 4 hours.]
10:30: Develop your new application. [Storage resources automatically scaled up to meet your existing application’s performance needs.]
12:10: Eat lunch and talk with co-workers about your new application.
14:07: Somebody hits a bug in the application. Instantly clone the environment to reproduce and test a fix; send the fix through the CI/CD pipeline.
16:20: [Storage resources scale down as the load reduces.]
18:30: Kick off a set of tests with real data for your new application; head home.
23:30: Sleep. [Your data is secure and the records are available to security, auditors, etc.]
The First Day of the Rest of Your Life
Cloud offers a tantalizing array of options to help build applications. Without data management, those tools will torment you because you won’t have time to use them. Plan ahead, though, and you can find a better way. Day 2 can be the best day of your life, not a nightmare.
Cloud should give you choice. Cloud should give you automation. Cloud should make you more productive. If that’s not happening, it’s time to look for a cloud-first data management solution.