Tag: cloud storage

We’ve come a long way over the last few decades. A ton of stuff about IT has changed, even in the last five years. And it is true in IT that the only constant is change.

We like to have control of our environment to provide the most stable platform for user satisfaction that we can and simplify troubleshooting when things do go wrong. But even such a simple desire has been stripped away in the flood of changes we face. To be certain, some areas of IT within most enterprises are very stable environments that only change after careful consideration, but much of the rest of IT–both within and outside of the enterprise–has become the wild west again.

And there is a lot of good in this cycle. Trying new things solves old problems and drives innovation, there can be no doubt. The changes we’re making address things like, “There was never enough testing,” or “We don’t have the security staff,” which means our apps are better because of the wave of changes we’re currently riding. The cloud is a standard option for new development even in the most stodgy enterprises, and some enterprises are moving older apps to the cloud as time permits.

That last bit is going to force some hard choices in the future. As many of you know, I spent several years writing exclusively about storage, so when I see something happening in the storage space, I get interested. A former coworker of mine recently wrote about Kubernetes storage options from an angle I hadn’t considered before, and it drove me to look deeper into what’s going on today.

In case you missed it, storage–specifically, data gravity–has tethered a large number of applications to enterprise data centers. The app needs access to the data; if the data is in the data center, the app has to be there and have secure, ready access to the data somehow. Cloud and containers offered some options/workarounds to deal with this issue, but they’ve lagged far behind the rest of the agile infrastructure.

That is changing. Higher-speed, distributed datasets are here and are being used. I have not followed the space closely, but my immediate thought is, “They have to be proven.” They’re going to have to show that they’re as stable and secure as the DBMS in the data center and that you can get your data out of them no matter what. In the end, data is a company resource–a very valuable company resource. In some cases, the data a company owns is worth more than the company’s products. So, total control and security of that data is imperative and must be proven on the ground.

But it will be. Because there is a need. And that will be the final step in making data centers far less critical for the average enterprise. Some enterprises already don’t need a data center, but most still do, largely because of data control and the resulting data gravity. Apps can be shifted from cloud to cloud or cloud to on-premises, or from cloud to hosting via Kubernetes, almost as an afterthought. But data is far less agile. In the end, a couple terabytes (or petabytes, depending on dataset) of data aren’t just flung over to a new cloud. Distributed, though; copies could be done in parallel, at high speed, making them far more mobile.

Vendors already recognize that subscription models are easier to sell, with far fewer user maintenance headaches. IT enjoys not having to go through huge checklists to make sure every bit of supporting software is up-to-date with the right version–in a hosted environment, it’s done for you. Enterprises already recognize that a fair number of applications can be cloud-native and are putting them out there. Slowly, over time, more and more applications–even data-heavy applications–will be viable in the cloud for your average organization. The bottleneck, as I see it, will be cloud storage costs. Cloud storage is both more cryptic than other cloud costs and more expensive. Ingress, egress, storage, instance … all these things add up, and the more instances required to achieve parallelism, the more they cost. Until that issue is resolved, don’t expect any mass migrations. But migrations will come, eventually. Kubernetes is preparing to do its part and make data easier to use and move; cloud vendors will just have to find a way to rationalize the bandwidth implications with the costs we, the users, are charged.

But over time, it will be less profitable to produce hardware and software targeted at individual enterprises. We’ve already seen some moves to hosted models that should open your eyes to the direction your vendors are headed–Atlassian and pretty much the entire source code scanning industry, for example, come to mind.

For most organizations that have been around awhile, the key to making the move will come from internal motivations. They’ll have to figure out how to manage containers with virtual storage internally, in their data center, before they’re comfortable throwing that architecture out onto the public cloud. It’s a good path, but for the short to mid-term, data gravity will slow or even halt the “To The Cloud!” charge for core applications.

For most large and established organizations, the ROI just isn’t there to move tons of data around. The core systems that work will be kept and may be improved, and only when they need to be replaced will things like a full-on container architecture be considered. But, again, this is likely to be an internal architecture, to start, (and often, will be forever) simply because that’s where the important data is.

For newer organizations, the opposite will be true for the short to mid-term. As the data lake grows in the cloud, the costs to keep it there grow in a non-linear fashion, eventually biting into profitability. This group will also want to understand how to run a fully agile container infrastructure internally, complete with virtual data access.

Whichever group you belong to, consider adding on-site Kubernetes or local cloud skills to your talent repertoire. Those are the skills that overlap; the juncture where all of these paths seem to cross, and the agility that full-on container automation with storage offers will not negatively impact the organization.

And keep rocking it. We’re enjoying your apps. Thank you for keeping them humming along!