Tag: devops

  • In DevOps, Chatbots are a People Problem

    In DevOps, Chatbots are a People Problem

    By now, you are undoubtedly aware that you can call up a chatbot or install an AI assistant into your IDE and generate entry-level code and deployment scripts.

    Indeed, if you just go to a site like ChatGPT and enter “generate Python code to do X” where X is something you need to be done–or an example like “read a file and print out the contents one line at a time,” it will do it. And if you keep it simple, it will do it well. If you say “again, in Java,” it will comply and do it well.

    Then say “new chat,” and tell it to “write a bash script to deploy an app named X written in Java and using Gradle to a cluster named Y,” it will do that also, informing you of the names of the deployment and service YAML files. Then you can say, “generate the deployment file mentioned,” and it will. And then you can say, “parameterize the Java version,” and it will do that, too. This iterative revision can continue until the deployment code, containers and deployment files are complete. If you have not yet, I urge you to spend the half hour going through these steps once required, so the potential is obvious.

    It is not yet to the point where I would (or you should) trust it with complex systems, but all of that busy work we’ve been trying to overcome with templates and preconfigured containers? We can actually get around the bulk of it now. The same is true for testing of all kinds. There are tools that will auto-generate test scripts for a given set of source code. They will also generate test execution code. That means that the only step left—and the one that has held testing back because of the volume of work—is reviewing results … But wait, there’s more! In security testing, AI can help eliminate false positives. In performance or QA testing, it can filter out all but the most egregious issues (as defined by the org or by a larger pool of apps).

    The problem we need to address today is the people issue. What we have traditionally used entry-level people for—and trained them in Dev/Ops/DevOps with—is no longer necessary. Because the AI can do it. That is stunning and really very cool, if you think about it.

    But it heralds a danger that every org needs to be aware of. What we do and how we do it is changing. For the better, but change always brings uncertainty, and the foreseeable bit we need to confront is, “Now that we don’t need entry-level people for these steps, how do we recruit and train them?” I do not yet have a definitive answer for this question, because I am a product of my time, so I see things in terms of learning the bits that make up the whole—advising people interested in DevOps to learn at least one Linux command shell, to understand how networking works in containers, to write the scripts to get containers (and/or public cloud instances) up and running, etc.

    I do think that every org should start using these tools at a pace that makes them comfortable because it allows us to continue our “more with less” growth without significant pain. But we need to make certain we are training the next generation of DevOps engineers in the process. For the foreseeable future, we’ll still be the ones bug fixing and troubleshooting, so we need to understand the environment and the applications. We are starting to get some sweet tools to help us at a higher level, though. Anyone who has inherited complex systems knows that the learning curve of tons of source files that make up many subsystems and all of the spaghetti code that these can be comprised of is painful. I don’t normally mention brand names here, so I’ll just say there is at least one tool out there that you can ask to “describe what this code does” and it will run with it and give you an answer in plain English. This might be the training tool of the future, with the ability to compare what you see in code with what the AI describes in English.

    But for now, even if you don’t hunt down app description AIs, having entry-level people review the files that the AI generates will serve two purposes: First, to validate what the growing AI space is capable of and second, to get entry-level engineers into the entry-level bits of the systems.

    And keep rocking it. Make sure you have a path for DevOps in the AI future, but remember that the creation of the calculator did not statistically change the number of mathematicians, just made the bulk of them more productive. That’s all we’re doing here, just a bit more flexibly … But in the end, like a calculator, the AI has an equation that it fills and reports back with.

  • Why Low-Code/No-Code Tech is Important in a Competitive Job Market

    Why Low-Code/No-Code Tech is Important in a Competitive Job Market

    The demand for tech professionals is increasing exponentially as digital skills become a requirement for businesses. However, the world is facing a serious shortage of tech professionals. According to a forecast by the World Economic Forum, 50% of the world’s labor force will need to upskill in order to keep pace with technological advancements and automation. Similarly, digital transformation will change over one billion jobs. Yet the majority of workers around the world lack the digital skills to meet the demands of their jobs. According to a recent study, three in four employers in the world (data from 40 countries) claim a work shortage. On the other hand, millions of young people find themselves unemployed as they lack the basic digital skills for in-demand jobs. Low-code/no-code software solutions can offer a solution to this labor shortage and close the gap for workers who quickly need to acquire digital skills. Proficiency in low-code/no-code applications will provide candidates with a competitive edge in the job market.

    Low-code/no-code software allows citizen developers and business experts to build business applications without requiring high technical expertise. With citizen development becoming an important part of all organizations’ business strategy, the demand for workers with proficiency in low-code/no-code app development is increasing. Expertise in low-code/no-code app development has become an expected skill set for potential employees.

    Benefits of Low-Code/No-Code App Development

    Citizen Development

    As discussed above, digital skills are becoming a part of almost all technical jobs. Employees should be proficient in using technology to streamline data collection, automate daily processes and leverage mobile technology. To achieve digital transformation, employees should be acquainted with advancements in digital technologies. This is where the skills gap seeps in. Learning code is not an easy task. It requires a lot of time, patience and effort. Add mobile development skills to the list, and there’s an even larger skills gap. Low-code/no-code app development can play a major role in addressing the skills gap. With pre-built apps and components, low-code/no-code development platforms offer an intuitive environment for designing software without the need to be proficient in coding. Such platforms can be easily used by people without programming skills to build flexible interfaces their companies need. However, low code can also be used by IT departments to deliver faster digital solutions while freeing up time to focus on more complex coding tasks.

    Additionally, low-code naturally aligns with the millennial proclivity to be digitally savvy. This generation is more comfortable than ever with using digital technologies in all aspects of their lives. Thus, they are more inclined and open to using low-code/no-code platforms and gaining some expertise in the domain.

    Empowering Users

    Low-code/no-code platforms empower citizen developers or business users to “skill up” in an increasingly digital society. By empowering business users to develop applications without the need for coding, low-code generates jobs directly or indirectly. The rate of technological advancement tells us that it is no longer favorable for companies to take months to develop a single app. Since the software has a life of three to five years, taking several months to develop apps is not conducive to the competitive market. As such, low-code/no-code app development can empower business users to develop applications quickly. However, citizen developers should work with IT management to ensure that the app adheres to all the company’s compliance requirements and governance guidelines. This is because citizen developers may not be proficient in scaling apps.

    Business Experts Understand Business Application Need

    Business needs are continuously evolving, especially since after the pandemic, when so many people are remote or using mobile apps to run businesses. At this rapid pace, it is not always easy for IT departments to continuously craft new apps and immediately update apps to match evolving circumstances and needs.

    Business users who are close to daily tasks, data collection requirements and ongoing processes, are in an excellent position to understand urgent needs for specific business applications. As a result, business users are in a much better position to collect department requirements and propose an application that caters to those specific needs. In such situations, low-code or no-code development solutions equip business users to develop applications directly based on their needs.

    How Can Low-Code/No-Code Help Employees Advance Their Careers?

    As discussed above, low-code has tremendous potential and benefits in the current digitally charged environment. Here are some benefits for candidates learning low-code skills:

    1. Bridge Business and IT

    Low-code/no-code is an easy entry point for someone in their early career phases interested in working in the IT department. By learning low-code solutions for streamlining business processes and automating workflows, employees can take a step towards interfacing with IT departments or understanding the basics of coding. With necessary low-code skills, employees can also become leaders in cross-functional groups or digital transformation teams responsible for introducing low-code/no-code solutions and rapidly introducing new business applications. There is a huge gap in this space, as only 29% of IT leaders feel confident that they have the right training mechanisms for non-developers to build applications.

    2. Increase Your Salary

    As employees grow their low-code skills, their salaries will correspondingly increase to reflect digital proficiency. Since low-code/no-code skills will increase in demand, proficient users will make significant gains in terms of the salary offered by major companies. Glassdoor suggests that the average base salary of a workflow automation specialist in the U.S. is $81,335 per year, while that of a data analyst is just $69,517 per year.

    3. Higher Job Satisfaction

    A full 71% of employees expect their work organizations to provide them with the same level of service using technology that customers’ products provide. Organizations can introduce low-code/no-code applications within the organization to increase the level of employee satisfaction.

    Low-code/no-code skills are becoming highly coveted in the current competitive job market. Offering benefits to both employees and organizations, low-code app development is the key to digital transformation.

  • Six Mainframe DevOps Predictions for 2023

    Six Mainframe DevOps Predictions for 2023

    DevOps, or the automation of application development and hand-off to operations, is more prevalent than ever on the mainframe. As we begin 2023, below are six predictions for what the coming year will bring. The composite themes will undoubtedly be speed, alignment and experience.

    1. Enterprise DevOps Will Break Down Silos Across Core Systems

    DevOps practices will continue to expand beyond cloud and distributed systems to include more core systems than ever before. Core platforms, including the mainframe, are the functional backbone of mission-critical applications that drive the world economy. In a recent IDC survey, 82% of organizations reported taking steps to prioritize using the same application development tools across distributed and mainframe teams. 2023 will continue to see a rise in enterprise DevOps that will bring down the silos between platforms. As core system workloads increase, organizations will see the benefit of aligning platforms such as mainframe and distributed to realize the full benefit of their DevOps initiatives.

    2. Growth in AI/ML as Key Components for Measuring DevOps Performance Metrics

    The value of analytics will increase as organizations emphasize the use of key performance indicator (KPI) data to measure the performance and maturity of DevOps pipelines. In this way, organizations will not only gain more insights into their pipelines, but will also be able to proactively manage the data in real-time. They will be able to adjust loads as needed and create smart pipelines with advanced error handling. Organizations will leverage benchmarks within the analytics to see how they compare with their peers.

    3. Cross-Functional Pipelines Will Create Hybrid Automated Everything

    This will enable the cloud, core systems and data to deliver a more holistic view of “platforms, processes and people.” Silos will finally disappear. Automation will become a key priority in this effort as organizations seek to free up time for developers to innovate and achieve greater velocity, quality and efficiency. A single integrated and connected pipeline will emerge that automates the creation and testing of applications in a single flow. The rationale of prioritizing this alignment between distributed and core systems includes the reuse of best practices, workflow simplification, cost-effectiveness, application integration and streamlined training.

    4. DevOps Will Embrace Platform Engineering and Site Reliability Engineering (SRE)

    Platform engineering is the next evolution of DevOps, taking a DevOps platform and turning it into a self-service product to improve the developer experience. It treats DevOps as a product and allows organizations to create an end-to-end offering that serves developers, operations and platform teams. SRE focuses on the “Ops” side of DevOps, specifically how to automate operations capabilities. This is vital to the DevOps mission since it provides the data that allows organizations to manage what they can measure.

    5. DevSecOps Will Play an Increasingly Central Role

    Security must become a central part of the toolchain and a “first-class citizen” in the pipeline. Static application security testing (SAST), also known as white box testing, along with dynamic application security testing (DAST), also known as black box testing, will be further integrated into processes to enable security teams to scan large loads at rest or in real-time quickly and efficiently.

    6. Git-Centric Enterprise Source Code Management (SCM) Strategy Will Grow

    Git has become the de facto standard for SCM for distributed application workloads. In a recent IDC survey of industry professionals across 11 countries and multiple industries, 87% said they will migrate all or some of their mainframe source code to Git over the next three years. This means Git and Git-native development will become important capabilities for mainframe DevOps companies to support. Organizations will increasingly prefer implementation approaches and tools that offer flexibility in both scope and pace.

    The techniques highlighted in these predictions are, naturally, related. They point to an ever-increasing capacity for efficiency and quality that will enable core and distributed technologies to work better together.

  • DevOps Use Cases for AI-Assisted Kubernetes

    DevOps Use Cases for AI-Assisted Kubernetes

    As indicated in my prior blogs Optimizing Cloud Costs for DevOps With AI-Assisted Kubernetes and Optimizing Cloud Costs for DevOps With AI-Assisted Orchestration, an AI-assisted Kubernetes orchestrator is needed to optimize cloud costs for DevOps, DevSecOps and SRE. This blog describes DevOps-specific use cases for an AI-assisted Kubernetes orchestrator. The blog also describes a roadmap for implementing an AI-assisted Kubernetes orchestrator and the benefits of a solution using an AI-assisted Kubernetes orchestrator for DevOps.

    Use Cases for AI-Assisted Kubernetes

    An AI-assisted Kubernetes orchestrator can offer many benefits to organizations running containerized applications on Kubernetes clusters. Here are some use cases for an AI-assisted Kubernetes orchestrator:

    1. Auto-scaling: An AI-assisted Kubernetes orchestrator can help automate the process of scaling up or down the number of pods based on the traffic or usage patterns of the application. The AI can analyze the performance metrics of the application and determine the optimal number of replicas needed for the application to function efficiently.
    2. Load Balancing: Load balancing is a critical component of any Kubernetes cluster. An AI-assisted Kubernetes orchestrator can optimize the load balancing by analyzing the network traffic and determining the best way to route traffic to the different pods in the cluster.
    3. Predictive Maintenance: An AI-assisted Kubernetes orchestrator can help identify and diagnose issues before they become critical. The AI can analyze the logs and performance metrics of the applications to identify patterns and anomalies. Based on this analysis, the AI can predict potential issues and notify the operations team.
    4. Optimization: An AI-assisted Kubernetes orchestrator can optimize the resource allocation of the Kubernetes cluster by analyzing the usage patterns of the application. The AI can identify the optimal amount of resources required for each pod and allocate them accordingly.
    5. Self-Healing: An AI-assisted Kubernetes orchestrator can automatically detect and recover from failures within the Kubernetes cluster. The AI can analyze the logs and performance metrics of the pods and take corrective actions to ensure that the applications continue to function properly.

    DevOps-Specific Use Cases for AI-Assisted Kubernetes

    An AI-assisted Kubernetes orchestrator can provide several use cases specifically for DevOps teams. Here are some examples:

    1. Automated deployment: DevOps teams can leverage an AI-assisted Kubernetes orchestrator to automate the deployment process of their applications. The AI can analyze the code and configuration changes, build container images and deploy them to the Kubernetes cluster without any human intervention.
    2. Continuous integration and continuous deployment (CI/CD): An AI-assisted Kubernetes orchestrator can facilitate the CI/CD process by automatically building, testing and deploying new code changes. The AI can monitor the code repository for new commits and trigger the necessary actions to build and deploy the changes.
    3. Security and compliance: DevOps teams can use an AI-assisted Kubernetes orchestrator to ensure that their applications comply with security and compliance standards. The AI can monitor the security posture of the cluster and identify vulnerabilities or non-compliant configurations. The AI can also implement remediation actions to address the issues.
    4. Disaster recovery: An AI-assisted Kubernetes orchestrator can help DevOps teams to recover from disasters quickly. The AI can analyze the backup data and restore the data to the Kubernetes cluster. The AI can also create a disaster recovery plan and test it periodically to ensure that the recovery process is reliable and efficient.
    5. Resource optimization: DevOps teams can use an AI-assisted Kubernetes orchestrator to optimize the resource usage of the Kubernetes cluster. The AI can analyze the performance metrics of the applications and identify resource-intensive workloads. The AI can then allocate resources more efficiently to ensure that the applications run smoothly and cost-effectively.

    Roadmap

    Implementing an AI-assisted Kubernetes orchestrator for DevOps involves a series of steps to ensure a successful deployment. Here’s a feasible implementation roadmap for an AI-assisted Kubernetes orchestrator for DevOps:

    1. Define your use cases: The first step is to define the use cases for the AI-assisted Kubernetes orchestrator that align with your DevOps team’s goals. Identify which areas of your DevOps process can benefit from automation and optimization using AI.
    2. Choose a platform: There are several AI-assisted Kubernetes orchestrator platforms available. Research the options and choose a platform that aligns with your use cases and technical requirements. For example, CAST AI provides an AI-Assist for Kubernetes solution that is specially targeted to reduce cloud costs by as much as 60%
    3. Plan your deployment: Plan your deployment by defining the architecture, infrastructure and resource requirements. You need to ensure that you have the necessary resources, such as compute and storage, to support your deployment.
    4. Prepare your data: The success of your AI-assisted Kubernetes orchestrator depends on the quality and accuracy of the data. Gather and prepare the data needed for your AI algorithms to train and make decisions.
    5. Train your AI models: Train your AI models based on the data you’ve collected. Ensure that you test and validate the accuracy of the models to avoid introducing errors in your deployment.
    6. Integrate with your DevOps toolchain: Integrate the AI-assisted Kubernetes orchestrator with your existing DevOps toolchain, such as CI/CD pipelines, monitoring and logging tools and configuration management tools.
    7. Test your deployment: Test your deployment in a staging environment to ensure that it meets your requirements and use cases. Run a series of tests to verify that the AI-assisted Kubernetes orchestrator is working as expected.
    8. Deploy to production: After testing, deploy the AI-assisted Kubernetes orchestrator to your production environment. Monitor the performance and health of the AI-assisted Kubernetes orchestrator and make any necessary adjustments.
    9. Continuous improvement: Once deployed, continue monitoring and optimizing the performance of the AI-assisted Kubernetes orchestrator. Collect feedback from your DevOps team and improve the AI models, infrastructure and deployment processes.

    Benefits

    There are many possible benefits of an AI-assisted orchestrator for DevOps, including:

    1. Increased efficiency: An AI-assisted orchestrator can automate many of the repetitive and time-consuming test environment and deployment orchestration tasks in the DevOps process, allowing teams to focus on more critical and complex tasks.
    2. Improved accuracy: AI algorithms can analyze data and make decisions more accurately than humans. By using an AI-assisted orchestrator, DevOps teams can reduce the risk of human error in their provisioning and re-provisioning processes.
    3. Faster innovation: With an AI-assisted orchestrator, DevOps teams can automate many of the deployment processes, resulting in faster and more reliable deployments. This can help organizations to deliver new features and updates to their customers more quickly.
    4. Better resource utilization: An AI-assisted orchestrator can analyze resource usage and optimize resource allocation, resulting in more efficient resource utilization. This can help to reduce costs and improve performance.
    5. Enhanced security and compliance: An AI-assisted orchestrator can monitor security and compliance issues in the DevOps infrastructure orchestration process, identify vulnerabilities and implement remediation actions. This can help to ensure that applications are secure and compliant with relevant regulations.
    6. Improved scalability: An AI-assisted orchestrator can help to manage and scale Kubernetes clusters more effectively, ensuring that applications are available and performant as demand increases.
    7. Predictive analytics: An AI-assisted orchestrator can use predictive analytics to identify potential infrastructure issues and recommend solutions before they become critical. This can help DevOps teams proactively address issues and avoid downtime or other service disruptions.

    What this means

    As indicated in my prior blogs Optimizing Cloud Costs for DevOps With AI-Assisted Kubernetes and Optimizing Cloud Costs for DevOps With AI-Assisted Orchestration, an AI-assisted Kubernetes orchestrator is needed to optimize cloud costs for DevOps, DevSecOps and SRE. This article explained DevOps use cases for an AI-assisted orchestrator like CAST AI. Such an AI-assisted Kubernetes orchestrator can help DevOps teams to streamline their operations, reduce costs and improve the quality and reliability of their infrastructures and applications for testing and deployment. By leveraging AI technology, DevOps teams can focus on innovation and delivering value to their customers.

  • How to Rethink Your Website Uptime

    How to Rethink Your Website Uptime

    COVID-19 rapidly accelerated e-commerce, almost shutting down in-person shopping during the lockdown. Ordering from Amazon became second nature. The online giant accounted for approximately 33% growth—50 million new Amazon Prime members.

    While there are concerns that consumer spending will fall short this year, according to Adobe’s 2022 holiday shopping predictions, online discounts are already playing a big part in creating record sales. Customers are concerned about their site’s availability. Are your stakeholders saying your service is running too slowly? Look at the latency metric and you may not see a change noted from the week before. With no real decrease in latency, your team is unaware of what “running too slow” means.

    Revisiting Technology Vendor Contracts

    Page latency is not new. Many retailers have increased their website conversion by improving page load time. The problem with some sites is that no one really knows what was defined as being fast enough and not too slow. And, of course, every stakeholder has different expectations of what the service should deliver. Internet traffic surged during COVID-19 and networks responded at different levels. While not unusual, the rapidity of growth was staggering. Globally, traffic in April 2020 grew 38% with upstream traffic skyrocketing up to 123.18% before leveling off.

    Service level objectives (SLOs) can create a common understanding of service capabilities to help define site requirements from the start. Unlike service level agreements (SLAs) which are engineering-driven, SLOs are a business decision that requires input from multiple stakeholders. SLOs allow retailers to match IT delivery models to business outcomes.

    Uptime is only one of an organization’s business objectives. First, service level indicators (SLIs) must be measured to identify users’ expectations of reliability. When you combine these individual data points into your SLO, then you’re set to meet your team’s business goals.

    SLOs for Existing and New Services

    Once your SLOs have been established, you should review all your existing and new services to make sure that your systems are running according to the objective. Moving from SLAs to SLOs isn’t simply a rebranding exercise.

    SLOs become the target for the availability of the system. Instead of looking at every individual data point, you can now look at the SLIs over a certain period — to establish an event and success criterion — and, for example, report that for 99% of them, the operation has less than five seconds to succeed, if that’s your goal. The SLO will indicate if a large portion of your SLI events were good. At 99%, there’s still a gap, but aiming for 100% reliability is unrealistic.

    No retailer can guarantee that their system will always be up and running 100% of the time. There are too many factors you can’t influence. And aiming for a higher number may not speak to your business case as it will contribute to unnecessary costs. Additionally, if your users have no expectations of the reliability of your service, it’s not prudent to make the investment. Your 1% gap, known as an error budget, is the number of negative events your system can withstand before violating your SLO—when negative events happen but are completely acceptable and fine.

    By using SLOs and error budgets, you will deliver less noise for engineers. Often engineers are paged too much which results in alert fatigue. You’ll have happier engineers who will not receive pages in the middle of the night, get fewer alerts and not have to deal with being alerted about small website spikes.

    Implementing SLOs

    While reliability is the single most important decision when making SLOs, teams are also faced with internal development challenges. Engineers would like to prioritize technical debt to rework existing services versus focusing on the business side demanding new features. When looking at SLOs, they should be based on your user expectations, not an arbitrary value.

    Undercutting the reliability of objectives also means that you’re not meeting user expectations. And not meeting user expectations will generate a direct business impact because unhappy users might move away from your service. Create this direct link between reserving tech debt and having a positive business impact by bringing reliability back up to the value which was defined. This approach will help prioritize between tech debt and new feature development.

    Getting started with SLOs for existing services is a good jumping-off point. Most likely, there will be information expectations as to what’s a service. Document the services and think about what users expect. You won’t have to deliver anything near 100% but be reassured that by making the informal formal, seeing the user perspective, and not trying to aim for perfection are areas to hone in on when delivering value for your SLO. Once defined, how often, what you want to achieve, and measuring the actual reliability of your service are the next logical steps to knowing how good you’re delivering in comparison to the objective defined.

  • Five Great DevOps Job Opportunities

    Five Great DevOps Job Opportunities

    staging-devopsy.kinsta.cloud is now providing a weekly DevOps jobs report through which opportunities for DevOps professionals will be highlighted as part of an effort to better serve our audience.

    Our goal in these challenging economic times is to make it easier for DevOps professionals to advance their careers. Of course, the pool of available DevOps talent is still relatively constrained, so when one DevOps professional takes on a new role, it creates an opportunity for others.

    The five job postings shared this week are selected based on the company looking to hire, the vertical industry segment and, naturally, the pay scale offered.

    We’re also committed to providing additional insights into the state of the DevOps job market. In the meantime, for your consideration:

    Dice.com

    Pyramid Systems
    Remote
    AWS Cloud DevOps Platform Engineer
    $160,000 to $200,000

    Monster.com

    Apidel Technologies
    Los Angeles, California
    Software engineer
    $140,000 to $160,000

    SimplyHired

    Cox Communications
    Atlanta, Georgia
    Senior DevSecOps Cybersecurity Architect
    $120,000 to $150,000

    LinkedIn

    EdgeLink
    Beaverton, Oregon
    DevOps Engineer
    $110,000 to $120,000

    Indeed.com

    CrowdStrike
    Remote
    DevOps engineer
    $105,000 to $195,000

  • How to Modernize a Data Warehouse in Months, Not Years

    How to Modernize a Data Warehouse in Months, Not Years

    Modernizing a data warehouse ranks among the worst imaginable tasks for IT. The end result is immensely desirable. The benefits of moving from an on-premises to a cloud data warehouse are well-documented. But database migrations require huge efforts and the stakes are high. What makes it so difficult? All applications, including ETL, BI, analytics and reporting, are designed to work with a specific legacy system. Hence, moving these applications to a different system requires extensive rewrites.

    Database migrations may well be the world’s most underestimated problem. In particular, data warehouse migrations are notorious for their horrendous cost overruns. Even when organizations plan for these projects to take several years, they tend to miss even the most generous deadlines. Worse, the vast majority fail to meet their objectives.

    In this article, we report on the exceptional success one major enterprise had migrating its systems. They managed to modernize their system in record time with little to no downtime at all. We’ll take a look at their journey and what sets this organization apart from their peers.

    The Big Challenge

    They don’t get much bigger. The system in question was long regarded as one of the largest, most complex data warehouses in EMEA. It is the backbone of a market-leading integrated container logistics and shipping company.

    The company’s 165TB of highly compressed data is used to track goods and shipping vessels around the world. Uninterrupted availability 24×7 is critical for this kind of operation, of course. As the central data hub, this system connects thousands of users that execute over one million queries every day.

    Migrating the workloads off an aging Teradata system to Microsoft Azure Synapse was key to modernizing the data estate. IT leadership found itself faced with the question of whether it would be possible to move without disrupting their business users.

    Database Virtualization is the Game Changer

    Database virtualization (DBV) is the extension of server virtualization to the database stack. Over the past two decades, we’ve seen a similar development in other spaces too. For example, networking and storage have been completely transformed by virtualization.

    The principle of DBV is as simple as it is powerful. At its core, it is software that sits between the applications and the database and translating between the two in real-time. Applications connect to the DBV component instead of the database and DBV provides the same language surface and APIs as the legacy system. DBV converts and optimizes all queries and data formats for the new cloud database. Applications can’t tell the difference between the original system and the virtualized one.

    Viewed in the context of migrations, DBV eliminates the most labor-intensive tasks. Instead of rewriting applications over the course of years, point them to DBV.

    Planning the Implementation—There is no ‘Easy’ Button

    If DBV eliminates the need for a rewrite, doesn’t that make database migrations easy? Indeed, it makes it simple. But easy it is not. Remember, DBV is only the runtime. As with other kinds of virtualization, the implementation requires extra effort.

    DBV, in this case, had over 99.75% coverage. In theory, there is no limit to what DBV can emulate. In practice, developers of DBV systems have to weigh each feature request. In our case, of the 0.25%, the customer decided to redesign the application that was outside of the current coverage.

    Another decision criterion is the choice of the destination system. For DBV to be successful, the new destination must be powerful enough to stomach the incoming workloads. This seems obvious, yet the failure of the Hadoop craze of the past decade showed it isn’t. In our case, the customer chose Azure Synapse, which was perfectly capable of processing these workloads.

    The biggest task during an implementation, though, is the data transfer itself. As a runtime, DBV relies on the data transfer to the new system being complete.

    People and Processes Are Your Biggest Obstacle

    Conventional migrations are the open heart surgery of the IT world. In contrast, DBV is a non-invasive procedure, but heart surgery, nonetheless. Implementing DBV is a team sport.

    To be successful, everybody must pull in the same direction. Executives must set the course and, once underway, relentlessly cheer on the team. Developers must refrain from wanting to rewrite code “just because.” Everybody else must execute in their clearly defined roles.

    Divide and conquer is the key principle. By structuring the implementation in tranches, the scope remains manageable at all times. Moving a limited number of applications at one time reduces distraction and keeps the team focused.

    Then, there are the naysayers. These are members of the organization who want to preserve the status quo. And finally, there is the legacy vendor. To them, DBV is a clear threat and they will do anything in their power to stop your project.

    Upfront alignment across the organization keeps the team on course even when there’s pushback.

    Move Fast to be Successful

    Because a data warehouse is constantly under development, it remains a moving target throughout the project. The longer the project takes, the more complex things tend to get.

    For each workload tranche, the team needs to execute two distinct steps. Check for functional correctness, then tune the performance of the workload. The latter concerns physical design choices such as indexes or table geometries.

    Rarely do queries need to be adjusted.

    At the end of the project, the team needs to catch up the data transfer so the new system is fully equivalent. Catching up on data transfer between tranches reduces the load in the home stretch.

    The time needed to implement the full virtualization of a system varies. For a mid-range system, this can be a calendar quarter; for a large system, it could take six to 12 months. In our case, the full implementation took 10 months.

    A New Way of Modernizing Data Estates Emerges

    What is most remarkable was the final cutover from the old to the new system. Our customer made the transition without telling their thousands of business users. Instead, they flipped the switch on a weekend, repointing all business users to the new stack.

    Business users coming back on Monday noticed their applications mysteriously running faster. But they had no idea of the magnitude of the change. That is unheard of. Even simple database upgrades cause more downtime.

    Looking back, how does this new way of modernizing a data estate compare to the conventional approach?

    Instead of 10 months, a conventional approach would have taken years. Instead of a small agile team of about 20, it would have required hundreds or even thousands of people to make changes. Coordination alone would have made the cutover a project for several calendar quarters.

    In the end, the customer saved an estimated nine figures on their five-year TCO over a conventional approach, assuming that a conventional approach would have succeeded at all. Given the high failure rate of conventional migrations, this is a very optimistic assumption.

    This project can serve as a blueprint for data warehouse migrations. Whenever time, cost and risk are of the essence, DBV is a strong contender.

  • FinOps & ITAM Must Team up to Cut Cloud Spend

    FinOps & ITAM Must Team up to Cut Cloud Spend

    Ahead of its Wisdom NA 2023 event taking place in Florida March 21-22, Rich Gibbons from the ITAM Review explores the relationship between FinOps and IT Asset Management when it comes to managing software costs. With on-premises revenues in decline, will the major software publishers respond by increasing policing and enforcement of their licensing rules in the cloud? Are cloud audits ahead in 2023? And will a marriage of convenience between FinOps and ITAM see us through?

    It’s already clear that, in 2023, corporate spending will be under the microscope more than ever. With software publisher price increases, rising energy bills and a global recession all contributing to the need for businesses to focus on cost, IT–and particularly software–is a prime place to focus. Concentrated efforts to monitor and manage software spending present a fantastic opportunity to materially improve an organization during these difficult times, enabling it to weather the storm and emerge from it lean, strong, and heading in the right direction.

    Annual spending on public cloud is predicted to reach $809 billion (IDC) by 2025, so ensuring this money is spent as efficiently and effectively as possible must become a key corporate-wide priority for every business.

    It’s Not All About Reduction

    One important aspect to realize is that, in the modern cloud age, cost management efforts shouldn’t be all about cost reduction. Rather, it is more appropriate to focus on spend efficiency. With cloud computing as a driver of digital transformation which enables organizations to offer new services to new audiences, it’s completely understandable that your cloud bill may increase; the important thing is whether the value to your business has risen commensurately.

    If your cloud bill rises by $1 million and your revenue rises by $1.5 million because you’re reaching more customers and/or have a new product line–happy days. If, however, you’re seeing your bill increase without that associated revenue increase, then that is a sign of spend inefficiency; you’re spending more for no return. This will likely be due to resources that are over-provisioned, under-used and/or incorrectly architected.

    Who’s Involved?

    The management of on-premises software such as Microsoft, Oracle, IBM, etc. has long been the domain of the software asset management (SAM) team or IT asset management (ITAM) team and their remit quickly grew to incorporate software-as-a-service (SaaS) products such as Adobe Creative Cloud, Microsoft 365 and Salesforce alongside SQL Server, Oracle DB and IBM Websphere.

    More recently, with the massive and continued growth of infrastructure-as-a-service (IaaS)/platform-as-a-service (PaaS) cloud environments, the discipline of FinOps has experienced considerable growth and become the de facto strategy for managing public cloud spending for many enterprises.

    While the two areas may seem separate, the growth of hybrid infrastructure means the two disciplines must learn from each other and work together to deliver the best overall results for a business. As John Donne said in 1624, “No man is an island,” and the same applies to teams managing corporate assets–what one team does (or doesn’t do) often impacts the other.

    Let’s take a look at some of the ways this collaboration can pay off in the real world.

    Azure Hybrid Benefit

    This is a perfect example of how regular, clear communication between ITAM and FinOps benefits everyone. Most enterprise organizations purchase their Microsoft volume licenses via an enterprise agreement (EA) and one of the characteristics of this agreement is the mandatory inclusion of software assurance (SA). This suite of extras includes a range of benefits such as new version rights, disaster recovery rights and, most pertinent here, the Azure Hybrid Benefit.

    Azure Hybrid Benefit is an SA benefit that allows you to allocate your existing on-premises licenses to cover portions of your cloud costs in Azure and on-premises hybrid infrastructure. It is currently applicable to:

    • Windows Server Azure virtual machines
    • SQL Server Azure: Virtual machines, managed instances, databases
    • Azure stack hyperconverged infrastructure (HCI)
    • Azure Kubernetes Service (AKS)
    • Red Hat Enterprise Linux (RHEL)
    • SUSE Linux Enterprise Server (SLES)

    Each of these scenarios can help an organization reduce their cloud bill, make better use of their overall Microsoft spend, and potentially use higher-level services–all increasing their return on investment (ROI) and spend efficiency. Microsoft claims savings of 50% for Windows Server and 85% for SQL Server are possible (see here), although your mileage will, almost undoubtedly, vary.

    However, while enabling the Azure Hybrid Benefit is, in many instances, as simple as ticking a box when deploying the cloud resource, that is far from the only requirement. First of all, there is a need to know how many applicable on-premises licenses you currently own and what your future plans are for renewals.

    Then you must be aware of the various licensing rules that pertain to the range of Azure Hybrid Benefits and understand their implications for your specific environment. Rules and usage allowances differ between editions, such as standard/enterprise/data center, while certain benefits may not be used concurrently. Also, given Microsoft’s penchant for regularly updating and changing licensing rules, this understanding needs to be frequently reviewed and any changes highlighted.

    Furthermore, given the direct link between on-premises licenses and cloud resources, changes at either side may have a knock-on effect. To keep hybrid spending as efficient as possible requires strong processes, capable discovery/monitoring tools and clean lines of communication between the ITAM and FinOps teams to keep things running smoothly:

    Example One:
    During the Microsoft Enterprise Agreement renewal process, a decision is made to reduce the number of SQL Server Enterprise licenses on which Software Assurance is renewed. This will require the Azure Hybrid Benefit to be removed from certain cloud workloads, resulting in increased cloud costs that need to be mitigated in other ways.

    Example Two:
    There is a blanket policy to activate Azure Hybrid Benefit for all applicable workloads in Azure. A new project is spun up consisting of dozens of Windows Server VMs and SQL Server instances which surpasses the number of on-premises licenses available; ITAM needs to be notified so they can purchase new licenses to ensure continued license compliance and, in most cases, keep spending at a minimum.

    Two Sides of the Same Coin

    In a hybrid environment, spend management and license compliance are often interlinked. So far, the software publishers have been relatively hands-off when it comes to policing and enforcement of their licensing rules in the cloud, but that could change at any point. Given that we’re already seeing the impact of the recession with layoffs across several major software vendors (Microsoft, Google) and Microsoft’s FY23 Q2 operating income dropping, traditionally on-premises software vendors such as Oracle, Microsoft, IBM, et al. may seek to realize untapped revenue in the cloud through license non-compliance audit penalties. While the examples above look at Microsoft licensing, the ability to use on-premises licenses in the public cloud exists with most software publishers and so, too, do the risks.

    A close relationship between ITAM and FinOps with regular and comprehensive data sharing and a clear understanding of each other’s priorities will be integral to the growth and development of any business using software and cloud services. Ensuring the best use of existing assets, enabling the most cost-effective cloud spending and reducing the risk of unexpected budget hits are all fundamental aspects of ITAM and FinOps collaboration.

  • Optimizing Cloud Costs for DevOps With AI-Assisted Kubernetes

    Optimizing Cloud Costs for DevOps With AI-Assisted Kubernetes

    As indicated in my prior blog, Optimizing Cloud Costs for DevOps With AI-Assisted Orchestration, an AI-assisted Kubernetes orchestrator is needed to optimize cloud costs for DevOps, DevSecOps and SRE. This blog describes the landscape for cloud-native optimization approaches and a features requirement blueprint for an AI-assisted Kubernetes orchestrator. The blog also describes a roadmap for implementing an AI-assisted Kubernetes orchestrator and the benefits of a solution using an AI-assisted Kubernetes orchestrator.

    Cloud-Native Optimization Approaches Landscape

    As indicated in the article Tricks for Cloud Cost Optimization, “Cloud cost optimization (CCO) is a technique of running cloud-native applications effectively, at minimal possible cost, without affecting the health of the services.” The most typical approaches are variants of the following:

    • Rightsizing pre-paid, fixed and reserved instances: Cloud service pricing models are intended to fit your application and business needs with the hundreds of choices available from cloud service providers. This approach can be cost-efficient when application and business needs are constant over time. However, this approach is not sufficiently cost-efficient for many applications and business needs that are constantly changing, such as use cases for DevOps, DevSecOps and SRE.

    • Spot instances can offer low cost-per-instance, but reliability is not guaranteed because the cloud provider can interrupt these services anytime. Unless the use of spot instances is limited to workloads that do not require a high level of reliability, this approach is limited. This approach does have the potential to have a broader application if a smart management layer is implemented to orchestrate the use of the spot instances.

    • Pay-as-you-go offerings by cloud service providers, used together with infrastructure-as-code (IaC) and auto-scaling, can help save on cloud costs. This approach facilitates the automatic spin-up of resources when needed and automatic shutdown when not needed. However, to realize the benefits of this approach requires constant supervision of cloud usage and manual adjustments to realize the benefits. Businesses may not even be aware of the resource scaling until the monthly bill arrives.

    • Cloud auto-stopping of entire applications and clusters can help to manage cloud resources automatically to ensure that they run only when needed, never when idle. This approach is not granular enough to sufficiently reduce costs for rapidly fluctuating resource needs for DevOps, DevSecOps and SRE use cases.

    • Manual cloud costs management, in which team members are taught to understand their own cloud spend and cloud cost management practices, can help reduce costs. However, this approach is subject to human involvement, is slow and generally does not provide granular controls needed to sufficiently deliver cloud cost savings 24/7.

    None of the traditional cloud cost optimization approaches described above have the requisite smarts, speed or granularity to sufficiently optimize the highly dynamic requirements for DevOps, DevSecOps and SRE use cases. An AI-assisted orchestration approach, when supported with the right features, can make up for the deficiencies of the typical optimization approaches.

    Features for AI-assisted Kubernetes Orchestrator

    As indicated by CAST AI an AI-Assisted Kubernetes orchestrator needs the following features:

    • Easy to start – It should take less than five minutes to fully onboard a cluster.
    • Easy to use – The orchestrator should abstract layers of complexity, so little knowledge is needed to use it other than a basic knowledge of running Kubernetes clusters in a public cloud. For example, a working knowledge of the kubectl command line utility for creating and managing Kubernetes clusters.
    • SaaS configuration – Avoids the need to host and manage the orchestrator itself.
    • Autoscaler policies – Define threshold values and upper bounds so instances are automatically added or removed as per the policy.
    • Cloud Support for AWS EKS, GCS GKS, Azure AKS and kOps clusters
    • Reports to monitor and track cost savings for each cluster.
    • Features to optimize each cluster.
    • High availability to ensure the orchestrator itself is not a failure point.
    • Collect and analyze each cluster configuration to provide the most optimal setup along with a savings estimation for the current cloud environment.
    • Security – Detect and monitor K8s vulnerabilities and configuration issues.

    Roadmap for Implementing AI-Assisted Kubernetes Orchestrator

    According to the article Roadmap To Cloud Cost Optimization, organizations need to understand that they need to deploy cloud computing the right way. This not only paves way for the actual digital transformation but is even instrumental to cloud cost management.”

    The article Top 3 Issues in Kubernetes Cost Management indicates, “Teams cannot manage and monitor Kubernetes’ costs without having the right tools in hand. No doubt, you can collect all the data yourself and create the right solutions. It wouldn’t be easy to modify these solutions if changes are required.” The speed of change with DevOps, DevSecOps and SRE is not compatible with the typical tools.

    A good example AI tool, CAST AI, indicates how easy it can be to onboard a Kubernetes cluster. First, connect your Kubernetes cluster to CAST AI and see how much you can save by optimizing cluster configuration. After exploring the available savings and cost reports, you can onboard the cluster into CAST AI and set the Autoscaler policies, which will manage the cluster for you. It takes less than five minutes to fully onboard a cluster.

    Benefits of Solutions Using AI-Assisted Kubernetes Orchestrator

    An AI-assisted cloud orchestrator can provide the following benefits.

    • Spot instance automation
    • Spot instance fallback
    • Real-time autoscaling
    • Instant rebalancing
    • Visibility into cloud usage and costs
    • Avoids over-provisioning
    • Cloud resources are not left idle
    • A consistent approach between teams and departments
    • Reduce monthly average cloud spend by up to 60%

    Summary

    The prior blog Optimizing Cloud Costs for DevOps With AI-Assisted Orchestration explained why an AI-assisted Kubernetes orchestrator is needed to optimize cloud costs for DevOps, DevSecOps and SRE. In this blog, specific requirements for such an AI-assisted Kubernetes orchestrator was explained using CAST AI as an example. The steps to onboard a cluster to the AI tool was explained.

  • Graduating From DevOps to MLOps? 5 Tools to Help

    Graduating From DevOps to MLOps? 5 Tools to Help

    DevOps is a set of practices that aims to bridge the gap between software development and operations. It aims to improve collaboration and communication between these two teams and to automate the process of software delivery so that changes can be made and deployed more quickly and easily. This can include continuous integration, continuous delivery and infrastructure-as-code. The goal is to enable faster delivery of new features and improvements to customers while also improving the reliability and stability of the software.

    What is MLOps?

    Machine learning operations (MLOps) is the practice of applying DevOps principles and practices to machine learning projects. It involves the collaboration of data scientists, machine learning engineers and operations teams to streamline the process of building, testing, deploying and maintaining machine learning models in a production environment. 

    The goal of MLOps is to increase the speed and reliability of deploying ML models while also improving the overall quality of the models. This can be done by using tools such as version control, continuous integration and automation of the model deployment process.

    Why DevOps is Not Enough: The Benefits of MLOps

    Machine learning models have unique requirements, such as data privacy, version control and model deployment that are not addressed by traditional DevOps practices. MLOps is specifically designed to address these unique requirements and to ensure that machine learning models are deployed and managed in a production environment in a way that is consistent with other software components. 

    There are several benefits to implementing MLOps in a machine learning project, above and beyond the benefits of DevOps:

    • Improved collaboration: MLOps promotes collaboration between data scientists, machine learning engineers and operations teams, which helps to ensure that models are deployed quickly and efficiently.
    • Faster deployment: By automating the model deployment process and using continuous integration and continuous delivery (CI/CD) practices, MLOps can help to speed up the time it takes to deploy a model to production.
    • Improved model quality: By using version control and automated testing, MLOps can help to ensure that models are thoroughly tested and of high quality before they are deployed to production.
    • Better monitoring and maintenance: MLOps can help to improve the monitoring and maintenance of deployed models by providing better visibility into the performance of models and enabling easy rollbacks if necessary.
    • Better model governance: MLOps can help to improve model governance by providing a clear process for the management of models, including versioning, testing, and deployment.

    5 MLOps Tools to Help Transition From DevOps to MLOps

    Making the move from DevOps to MLOps typically requires dedicated tools that can help manage the complexity of machine learning models and datasets. Here are a few tools, some open source and some commercial, that can help.

    MLflow

    MLflow is an open source platform for managing the machine learning life cycle. It aims to make it easier for data scientists and machine learning engineers to develop, deploy and track machine learning models. MLflow provides several capabilities that can help streamline the machine learning development process, such as:

    • Experiment tracking: MLflow allows data scientists to track and compare different versions of their models, including the code, parameters and metrics associated with each run.
    • Model packaging: MLflow allows data scientists to package their models in a format that can be easily deployed to a variety of environments, such as a local machine or a cloud platform.
    • Model deployment: MLflow provides tools for deploying models to a variety of environments, such as a REST API or a container.
    • Project management: MLflow allows data scientists to organize their work into projects, including multiple models and associated files.
    • Model registry: MLflow provides a centralized model registry that allows data scientists to store and manage models.

    Pachyderm

    Pachyderm is an open source platform that provides data versioning and pipeline management for machine learning and big data workloads. It allows data scientists and engineers to manage, version and collaborate on large datasets and create reproducible and reusable data pipelines.

    Pachyderm uses a version control system (VCS) like git to version data, which allows data scientists to track and collaborate on data sets and enables reproducibility and rollbacks. This allows tracking the changes in the data, seeing who made the changes and when they were made and rollback to previous versions if needed.

    It also provides a powerful pipeline management system, which allows engineers to create, test and deploy data processing pipelines. The pipeline management system is designed to work with containerized workloads, which makes it easy to deploy and manage pipelines.

    Pachyderm is built on top of Kubernetes, which allows for easy scalability and can be run on-premises or on any cloud provider. It also supports data processing frameworks like Apache Spark and Apache Flink, as well as machine learning frameworks like TensorFlow and scikit-learn.

    Amazon SageMaker

    Amazon SageMaker is a fully-managed cloud-based machine learning platform. It provides a suite of tools for building, deploying and managing machine learning models. It is designed to make it easy for data scientists and developers to quickly and efficiently build, train and deploy machine learning models in a production environment.

    Some of the key features of Amazon SageMaker include:

    • Model building: Amazon SageMaker provides a variety of tools for building machine learning models, including pre-built algorithms and frameworks for popular libraries such as TensorFlow, Apache MXNet and scikit-learn.
    • Model training: Amazon SageMaker allows data scientists and developers to train machine learning models on large-scale datasets, using distributed training and automatic scaling to improve performance and reduce training time.
    • Model deployment: Amazon SageMaker makes it easy to deploy machine learning models to production, with a variety of options for hosting models, including Amazon SageMaker hosted endpoints and Amazon Elastic Container Service (ECS) and Kubernetes.
    • Model management: Amazon SageMaker provides tools for monitoring, updating, and maintaining deployed machine learning models, including automatic A/B testing, canary deployments and automatic model rollbacks.
    • Experiment management: SageMaker experiment management is a feature that allows data scientists to keep track of the different versions of their models, the parameters and the metrics associated with each run.
    • AutoML: SageMaker AutoML is a feature that allows data scientists to automatically explore different algorithms and hyperparameters to find the best model for a given dataset.

    H2O MLOps

    H2O.ai’s MLOps is a platform that allows data scientists and engineers to collaborate and deploy machine learning models in production. It provides a set of tools for model development, management, and deployment, as well as monitoring and governance capabilities. 

    It aims to streamline the process of taking machine learning models from development to production while maintaining high standards of model quality and performance. Some of the main features include:

    • Model Development: It provides a set of tools for developing and testing machine learning models, including support for popular frameworks such as TensorFlow, PyTorch and H2O.
    • Model Management: It allows users to organize and manage models in a central repository, making it easy to track versions and collaborate with other team members.
    • Deployment: It provides an easy way to deploy machine learning models in a variety of environments, including on-premises, in the cloud or at the edge.
    • Monitoring: It offers monitoring capabilities that allow users to track the performance and quality of deployed models in real-time and automatically detect and diagnose issues.
    • Governance: It provides governance capabilities to help ensure compliance with regulatory requirements, such as data privacy, and to help manage the risks associated with deploying machine learning models in production.

    Neptune.AI

    Neptune.ai is an open source platform that provides a suite of tools for organizing, tracking and reproducing machine learning experiments. It is designed to help data scientists and machine learning engineers to keep track of their work, collaborate more effectively and improve reproducibility.

    Some of the key features of Neptune.AI include:

    • Experiment tracking: Neptune allows data scientists to track and compare different versions of their models, including the code, parameters and metrics associated with each run.
    • Experiment management: Neptune provides a user-friendly web interface for organizing and managing experiments, which makes it easy to keep track of progress and share results with others.
    • Reproducibility: Neptune provides a way to reproduce experiments by tracking the code, data, environment and dependencies used in each run.
    • Collaboration: Neptune allows data scientists to share and collaborate on experiments with others and provides a way to track who made changes and when they were made.
    • Model management: Neptune provides a centralized model management system that allows data scientists to store and manage their models, including versioning, testing and deployment.

    Conclusion

    In conclusion, MLOps is a practice that applies DevOps principles and practices to machine learning projects to streamline the process of building, testing, deploying and maintaining machine learning models in a production environment. The goal of MLOps is to increase the speed and reliability of deploying ML models while also improving the overall quality of the models. 

    There are several tools available that can help organizations to implement MLOps, including MLflow, Pachyderm, Amazon SageMaker, H2O MLOps and Neptune.AI. Each of these tools provides a different set of features and capabilities, but all aim to make it easier for data scientists and machine learning engineers to develop, deploy and track machine learning models.

    Image Source