staging-devopsy.kinsta.cloud

Tag: data gravity

  • NetApp Survey Shows Hybrid Cloud is Maturing

    NetApp Survey Shows Hybrid Cloud is Maturing

    A small survey of 79 midsized to large enterprises conducted by NetApp suggests enterprise IT organizations might finally be embracing hybrid cloud computing beyond use cases involving backup and recovery.

    The survey finds 20% of respondents are currently employing both on-premises and cloud resources to support the same workload in a production environment, with another 17% planning to do so within 24 months.

    Darnell Fatigati, senior product marketing manager for hybrid cloud at NetApp, noted that a hybrid approach to cloud computing will dominate with more than three-quarters of respondents reporting their organization plans to operate a hybrid cloud environment for the foreseeable future. Only 17% expect their organization to eventually migrate all their IT environments to the cloud.

    A major factor in the decision to not migrate all workloads to the cloud is the amount of data gravity that legacy applications running in on-premises IT environments exert, noted Fatigati. The amount of effort required to successfully migrate terabytes and even petabytes of data into the cloud is simply too great, he added.

    The primary benefits of hybrid cloud computing cited by survey respondents are faster innovation (26%), increased responsiveness (25%) and increased collaboration (22%). Survey respondents also noted they are looking to improve infrastructure flexibility and scale (27%), optimize costs (21%) and improve data accessibility (13%). Data protection involving disaster recovery, backup and archiving remains the most widely employed use case for hybrid cloud, with 29% either employing it today or planning to do so in the next 24 months.

    Most hybrid cloud computing environments today consist of multiple clouds and on-premises IT environments that are managed in isolation from one another. The fact that more organizations are now starting to deploy the same workload across both a cloud service and an on-premises IT environment has significant implications for DevOps teams that will be required to manage those workloads. Most DevOps workflows today do not span multiple platforms. However, as more organizations become impacted by the occasional cloud outage, the level of resiliency enabled by hybrid cloud computing is becoming a higher priority.

    In many cases, the shift to hybrid cloud computing was accelerated by the COVID-19 pandemic that drove many organizations to increase the number of workloads deployed in the cloud. Now, heading into 2022, those same organizations are looking to reduce IT costs by centralizing IT management as much as possible, noted Fatigati. Each time an IT organization adds a new platform, the tooling and personnel required to manage that platform increase the total cost of IT for any organization. Much of the shift to hybrid cloud is being driven by C-level executives looking to reduce those costs.

    It may be a while before most enterprise IT organizations are able to deploy workloads across multiple platforms. However, as both cloud and on-premises IT environments continue to evolve and as more platforms deployed in local data centers are increasingly managed using a cloud operating model, the way those platforms are accessed on demand will continue to converge. In fact, there may come a day when the term cloud computing itself disappears from the IT lexicon as every IT platform, by default, leverages one type of cloud or another.

  • How Dispersed Can We Be?

    How Dispersed Can We Be?

    It is interesting to consider DevOps from the storage perspective. DevOps in general—and specifically newer development and deployment models—tried to pretend that long-term storage was not a necessary component of software delivery. In short, it appears that because there wasn’t a good answer to long-term state in stateless development models, the storage and databases that used state were hand-waved away. Which, of course, failed. Container deployment models did the same thing and failed just as miserably. Oh, containers didn’t fail, but the attempt to ignore persistent storage did; from the very beginning, it was clear that was going to happen. We were doing early container implementations and working through how to ensure that containers ABC had access to the same shared storage. That need never went away and was never going to. Apps can run without persistence, but business doesn’t run without persistent data.

    Checking out the wave of cool new products like, for example, Ionir (disclosure—I know more about them simply because I’m friends with their CMO. Other vendors likely have good solutions, too) makes me realize the answers that we needed were right in front of us. And not just the answer to persistent storage, but to storage portability, which has been a problem from the beginning of the cloud. Data gravity is the idea that data holds apps in place. Tools like Ionir reduce the gravity of data. A lot. They can’t get rid of data gravity, because some platforms charge for data storage, data ingress and egress, etc. but that becomes a TCO operations problem rather than a near-insurmountable tech problem.

    But just as interesting is this: What if the slow but steady increase in core Internet speeds will allow us to access data from anywhere? We could do that now, but performant applications struggle if you force them to leave cloud A and go to your data center or to cloud B to get a dataset. Yet we use APIs that do just that on a regular basis. What are Google Maps and Google Places if not simply big old honking databases? And yet we call out to them from just about every platform.

    So, it’s worth trying out and seeing if the idea of data gravity is maybe an obsolete idea for you. For some applications that really do need super-fast response times, the concept is still relevant. But for many applications? Where your app sits in relation to data may not be as important as you think. Write or grab an API wrapper for a critical dataset and throw a simple test app up on a different platform. See what you get. If it performs well, consider putting apps where it makes sense and using a wrapper API with secrets to grant access to data from any validated app.

    Some of you are doing this already. Not many, but it’s out there. Excellent job; keep rolling!

    Everyone else is kicking rear and taking names, with the world still online every day because of your collective work. So keep rocking it, but consider if you really need the application close to the data these days and, if you do, how data mobility tools might facilitate moving data to the app—something most of us would not have considered for large datasets before the last year or so.

  • The Forces Massing Against the Data Center

    The Forces Massing Against the Data Center

    We’ve come a long way over the last few decades. A ton of stuff about IT has changed, even in the last five years. And it is true in IT that the only constant is change.

    We like to have control of our environment to provide the most stable platform for user satisfaction that we can and simplify troubleshooting when things do go wrong. But even such a simple desire has been stripped away in the flood of changes we face. To be certain, some areas of IT within most enterprises are very stable environments that only change after careful consideration, but much of the rest of IT–both within and outside of the enterprise–has become the wild west again.

    And there is a lot of good in this cycle. Trying new things solves old problems and drives innovation, there can be no doubt. The changes we’re making address things like, “There was never enough testing,” or “We don’t have the security staff,” which means our apps are better because of the wave of changes we’re currently riding. The cloud is a standard option for new development even in the most stodgy enterprises, and some enterprises are moving older apps to the cloud as time permits.

    That last bit is going to force some hard choices in the future. As many of you know, I spent several years writing exclusively about storage, so when I see something happening in the storage space, I get interested. A former coworker of mine recently wrote about Kubernetes storage options from an angle I hadn’t considered before, and it drove me to look deeper into what’s going on today.

    In case you missed it, storage–specifically, data gravity–has tethered a large number of applications to enterprise data centers. The app needs access to the data; if the data is in the data center, the app has to be there and have secure, ready access to the data somehow. Cloud and containers offered some options/workarounds to deal with this issue, but they’ve lagged far behind the rest of the agile infrastructure.

    That is changing. Higher-speed, distributed datasets are here and are being used. I have not followed the space closely, but my immediate thought is, “They have to be proven.” They’re going to have to show that they’re as stable and secure as the DBMS in the data center and that you can get your data out of them no matter what. In the end, data is a company resource–a very valuable company resource. In some cases, the data a company owns is worth more than the company’s products. So, total control and security of that data is imperative and must be proven on the ground.

    But it will be. Because there is a need. And that will be the final step in making data centers far less critical for the average enterprise. Some enterprises already don’t need a data center, but most still do, largely because of data control and the resulting data gravity. Apps can be shifted from cloud to cloud or cloud to on-premises, or from cloud to hosting via Kubernetes, almost as an afterthought. But data is far less agile. In the end, a couple terabytes (or petabytes, depending on dataset) of data aren’t just flung over to a new cloud. Distributed, though; copies could be done in parallel, at high speed, making them far more mobile.

    Vendors already recognize that subscription models are easier to sell, with far fewer user maintenance headaches. IT enjoys not having to go through huge checklists to make sure every bit of supporting software is up-to-date with the right version–in a hosted environment, it’s done for you. Enterprises already recognize that a fair number of applications can be cloud-native and are putting them out there. Slowly, over time, more and more applications–even data-heavy applications–will be viable in the cloud for your average organization. The bottleneck, as I see it, will be cloud storage costs. Cloud storage is both more cryptic than other cloud costs and more expensive. Ingress, egress, storage, instance … all these things add up, and the more instances required to achieve parallelism, the more they cost. Until that issue is resolved, don’t expect any mass migrations. But migrations will come, eventually. Kubernetes is preparing to do its part and make data easier to use and move; cloud vendors will just have to find a way to rationalize the bandwidth implications with the costs we, the users, are charged.

    But over time, it will be less profitable to produce hardware and software targeted at individual enterprises. We’ve already seen some moves to hosted models that should open your eyes to the direction your vendors are headed–Atlassian and pretty much the entire source code scanning industry, for example, come to mind.

    For most organizations that have been around awhile, the key to making the move will come from internal motivations. They’ll have to figure out how to manage containers with virtual storage internally, in their data center, before they’re comfortable throwing that architecture out onto the public cloud. It’s a good path, but for the short to mid-term, data gravity will slow or even halt the “To The Cloud!” charge for core applications.

    For most large and established organizations, the ROI just isn’t there to move tons of data around. The core systems that work will be kept and may be improved, and only when they need to be replaced will things like a full-on container architecture be considered. But, again, this is likely to be an internal architecture, to start, (and often, will be forever) simply because that’s where the important data is.

    For newer organizations, the opposite will be true for the short to mid-term. As the data lake grows in the cloud, the costs to keep it there grow in a non-linear fashion, eventually biting into profitability. This group will also want to understand how to run a fully agile container infrastructure internally, complete with virtual data access.

    Whichever group you belong to, consider adding on-site Kubernetes or local cloud skills to your talent repertoire. Those are the skills that overlap; the juncture where all of these paths seem to cross, and the agility that full-on container automation with storage offers will not negatively impact the organization.

    And keep rocking it. We’re enjoying your apps. Thank you for keeping them humming along!