staging-devopsy.kinsta.cloud

Tag: devops

  • What’s the Difference Between DevOps and Platform Engineering?

    What’s the Difference Between DevOps and Platform Engineering?

    The DevOps space is buzzing with a new trend—platform engineering. Platform engineering has been gaining steam over the last few years but has really peaked in interest lately. In fact, according to a study conducted by Puppet Labs, 51% of organizations have adopted platform engineering or plan to do so in the next year. But what exactly is platform engineering, and how is it different than DevOps?

    Below, we’ll explore the nuances between these two concepts. We’ll review what DevOps and platform engineering are and see how they’re similar and dissimilar. We’ll also outline the benefits of platform engineering and suggest some best practices to make the most of this emerging discipline.

    What is DevOps?

    DevOps is a broad topic, but it generally involves efforts to increase software release efficiency and frequency. Some of the hallmarks of the DevOps discipline include reducing siloes between teams and increasing collaboration. DevOps is often directly associated with various tools that automate the process of moving code from a local environment into production, also known as continuous integration and continuous delivery (CI/CD). Monitoring, incident management and configuration management tools also support the DevOps process.

    “DevOps reflects more of a philosophy and a way of working,” said David Sandilands, senior solutions architect for Puppet. DevOps enables more of a shift left mentality. Instead of having operations teams “guarding the gates,” developers work in partnership with Ops. In doing so, groups can decrease build time significantly, he said. This style helps engineers incrementally and rapidly release new features, replacing old waterfall development cycles.

    The ultimate goal of DevOps is to automate the software life cycle and shorten the feedback loop. Thus, DevOps takes a life cycle approach to shipping applications more efficiently, streamlining phases such as build, release, configure, monitor, plan, create and verify. Microservices architecture and containerization also often play a crucial role in enabling DevOps.

    What is Platform Engineering?

    Platform engineering is all about developing a self-service internal platform that serves the needs of developer consumers. Although DevOps opened things up and empowered individual developers with advanced automation, it left organizations with scattered tooling and processes, especially at large enterprises. For example, various infrastructure-as-code (IaC) configurations might exist for similar functions within the same organization but be unknown to engineers in different departments. Not to mention, overseeing a growing DevOps pipeline means added responsibilities around continual updates and security.

    Platform engineering places more control in the hands of a central platform team rather than individual developers. These teams typically treat the platform as a product. They do so by evaluating the needs of developers across an organization, building out internal self-service tools that suit the needs of most consumers and maintaining the platform.

    The 2023 State of DevOps Report found that this self-service approach typically handles areas like deployment, operation of infrastructure, monitoring production applications and security and compliance. Other capabilities, such as abstracting Kubernetes or overseeing secrets management, may also play a role in these efforts.

    Benefits of Platform Engineering

    • Improve developer experience: Platform engineering can enhance the experience a developer has with their software build and release systems. A better developer experience can do loads to improve morale and create a more productive engineering atmosphere.
    • Standardize DevOps practices: By bringing together common DevOps approaches into a shared platform, organizations can begin to standardize with reusable build processes and automated infrastructure. This can unite an organization on common runtimes, observability methods and deployment processes. This can streamline the introduction of DevOps practices to an organization.
    • Secure the DevOps pipeline: Various vulnerabilities exist within tooling that comprises modern CI/CD chains and cloud-native open source software. By outsourcing DevOps responsibility to a shared platform team, the organization can coalesce around more secure practices and avoid overlooking important updates or holes.
    • Supply additional guardrails: Platform teams might also assist in the acquisition of new tools as well as outlining and enforcing policies for software infrastructure. While this may come at the cost of customization, additional guardrails can help avoid shadow IT and governance around internal development. Letting platform teams handle it could make maintenance easier across an organization.

    Platform Engineering: Under the DevOps Umbrella

    I don’t see platform engineering as replacing DevOps. Instead, I view platform engineering as falling under the umbrella of DevOps since platform engineering takes DevOps processes and makes them more reusable and self-service. Both use a lot of automation and share many of the same end goals.

    However, there are nuances between the two approaches that reflect the recent trends in the IT market. With DevOps, the market was more interested in shifting responsibility left and empowering individual developers to oversee the entire software life cycle. Now, as sprawl and shadow IT emerge, the pendulum seems to be shifting back toward a bit more centralization and governance with platform engineering.

    Platform engineering is typically more of an in-house development initiative. So, what are some best practices IT divisions should remember to get the most out of platform engineering? Hope Lynch, senior director of platform at Cloudbees, shared some great strategies to consider that revolve around treating the platform as a product. Doing this involves accepting feedback from developers, not controlling everything and always maintaining a focus on developer experience.

    Hopefully, by following these tips, organizations should have a better grasp of the emerging discipline of platform engineering and have the confidence to begin introducing platform-based concepts within their own environments.

  • DevOps Security: Your Complete Checklist

    DevOps Security: Your Complete Checklist

    In a fast-paced environment like DevOps, your security strategy needs to be even more agile. While moving fast is great for delivering application features and functionality, speed shouldn’t come at the cost of security.

    DevOps security incorporates security strategies directly into your DevOps pipeline, creating a culture of pre-secured software deployments that work seamlessly for the end user. However, this requires a complete shift in your operational policies and security mindset. You need to start viewing security as an integral part of your DevOps lifecycle instead of a separate department.

    Here’s a checklist of DevOps security strategies for your development teams to deliver secure and reliable applications faster:

    Your Complete DevOps Security Checklist

    1. Analyze your development process for security hazards.
    The first step in implementing a DevOps security strategy is finding and fixing loopholes in your existing development process.

    For instance, continuous delivery and development are key in a DevOps environment. However, if each change doesn’t go through a code review, it has the potential to break your application.

    Deploying changes without passing them through a security checkpoint is like driving at high speed on a road with many curves. Each curve (change) is a potential hazard, and the danger increases when the driver does not know a car is coming from the other side (bug).

    Therefore, you must turn the spotlight on your development process and find those curves.

    2. Create a common goal between development and security teams.
    Traditional roles of developers and security teams might look like they are at crossroads — Developers write code, and security teams find fault with their code. However, in a collaborative environment such as DevOps, security and development must work hand-in-hand to deliver a secure and seamless experience to end users.

    Your organization should have a common goal of integrating security directly into the development process. Therefore, encourage open and regular communication between development and security teams. Further, establish joint milestones and review them to ensure there are no silos between both teams.

    Once these teams get used to this collaborative approach, your deployments will become faster because fewer things will break.

    3. Integrate security procedures in the development phase.
    Automation is the lifeblood of DevOps security. Integrating scanning tools into development workflows is critical to your entire process.

    For example, automated tools can scan the code, network and infrastructure to identify security vulnerabilities and provide recommendations for remediation. With this, development teams can fix a few security concerns themselves before passing them on to the security team for a final check.

    4. Add observability to your pipeline.
    One of the biggest challenges of the distributed nature of DevOps operations is visibility. How do you efficiently monitor a large number of operation nodes spread across several different locations?

    Observability helps you do this by using logs, metrics and traces—i.e., the outputs of your application to determine internal application health. With observability tools, you can see much deeper into the operations of your systems and identify the exact details of any security concerns. You can also easily identify potential bugs and fix them before they cause problems.

    5. Make changes in small, incremental units.
    It is infinitely easier to test code in small units and deploy them as minor but incremental changes than handling large, unwieldy blocks of monolithic code. Deploying gradual changes means you can resolve vulnerabilities faster.

    6. Scan and secure your entire DevOps pipeline.
    A large part of the security hazards in your DevOps process comes from unsecured containers and operations nodes. According to a study from ThreatStack, 94% of organizations say containers represent a security risk.

    However, the risk is not limited to containers.

    Other services like APIs, third-party tools like Docker and imported code pose a potential security risk to your processes. This makes it necessary to scan all your operations nodes periodically for vulnerabilities. And once you’ve cleared them, remember to lock the door behind you by requiring strict credentials for access to your administrative dashboards.

    Ticking Items Off Your DevOps Security Checklist

    If you implement most of the security guidelines on this checklist, your organization will have a solid security foundation. However, DevOps security is not a one-time process.

    You will need to continuously integrate security protocols into each part of your DevOps life cycle. Furthermore, you will need to re-train your employees and invest in new technology that makes your security pipeline smoother, such as observability.

    However, since this entire process makes your applications more secure and compliant while shipping them to market faster, investing in DevOps security is well worth it!

  • Boomi Brings Generative AI to Integration Platform

    Boomi Brings Generative AI to Integration Platform

    Boomi announced today it is applying generative artificial intelligence (AI) to enable organizations to automate integrations via a natural language interface.

    Boomi CEO Steve Lucas said Boomi AI makes it much simpler for professional and citizen developers alike to create integrations at scale using generative AI platforms to invoke the low-code engine that Boomi already provides to create integrations between thousands of applications.

    Those generative AI capabilities will complement a wide range of AI capabilities based on machine learning algorithms that are already embedded in the Boomi platform, he added.

    Generative AI’s Impact on Integration and Automation – Steve Lucas, Boomi from Techstrong.TV on Vimeo.

    Boomi has been widely used to drive business process integration via an integration platform-as-a-service (iPaaS) environment it delivers as a managed service. The low-code platform the company created significantly reduces the need to rely on professional developers to write procedural code using Java, for example, to create those integrations.

    Generative AI capabilities make it possible for almost anyone of any skill level to create those integrations across multiple types of application programming interfaces (APIs), noted Lucas.

    That capability also makes it possible for Boomi to notify end users when, for example, a data attribute in an application has changed and ask them if they would like to apply that change across their entire application environment.

    In addition, Boomi AI can also interpret the intent of a prompt to better orchestrate processes in addition to providing an audit history in natural language that explains how decisions were made and what outcomes occurred as a result. As part of that effort, Boomi algorithms are also trained to avoid biases and unfairness with established ethical guidelines.

    Boomi’s AI engine is trained using anonymized data collected from approximately 20,000 customers that have created more than 200 million integrations using the Boomi platform. Those integrations have been exposed to third-party large language models (LLMs) that the company is using to train Boomi AI, said Lucas. Boomi is not storing any customer data to train its AI models.
    DevOps teams, of course, spend hours creating integrations that should be much simpler to create, test and validate with the rise of generative AI. In fact, the number of integrations being created to drive a wide range of digital business transformation initiatives should substantially increase.

    In the meantime, DevOps teams would be well-advised to start evaluating which tasks they manually perform today that could instead be automated by a generative AI platform. Many of the routine integrations that DevOps teams create today using low-level APIs will be increasingly automated by end users. The ultimate goal should be to identify more complex tasks that previously DevOps teams would not have attempted simply because there wasn’t enough time.

    It’s still too early to determine what impact generative AI will have on the way IT is managed, but DevOps teams that are typically committed to automating as many processes as possible should be at the forefront of adoption. The challenge is going to be simply keeping track of the rate of change that might soon be enabled by these platforms.

  • Four Trends Defining the Observability Market in 2023

    Four Trends Defining the Observability Market in 2023

    Observability is a market that has grown exponentially over the past few years. It started as a buzzword in late 2015 and early 2016, and since then we’ve seen dozens of insurgent companies rush in, eager to stake their claim in a new (albeit loosely defined) market. Incumbent companies also rebranded their wares, hoping to capture some of the momentum that comes from the new term.

    Venture capital saw the market opportunity, too. By my conservative estimate, observability companies raised nearly $2 billion over roughly the last two years alone (I’m including some cybersecurity companies, which I’ll explain later).

    In the seven or so years since its inception, the observability market has fragmented and specialized into categories such as application, data, network and edge observability. On the artificial intelligence and machine learning side, there is model observability and AIOps. Connecting it all are observability pipelines.

    While more businesses than ever know about observability and have built it into their tech strategy, category awareness still has a long way to go. And though the dollars were flying fast and furious a couple of years ago, the economic cycle has turned against risk, and growth funding rounds are harder to come by. It’s time to step back and examine the key trends defining today’s observability market.

    Staffing Shortages: A Blessing and Curse to Observability

    Many companies in the observability space, regardless of their subcategory, represent net-new work for IT and security teams. A new application performance monitoring (APM) platform may offer a marginal improvement around APM’s golden signals, but that marginal improvement is a substantial lift for the end users tasked with implementing and deploying the new platform. While software may get sold, overworked ITOps and SecOps teams may decide the effort isn’t worth the perceived ROI, leaving the software on the shelf.

    On the other hand, observability products that reduce toil for operators will continue to outpace market growth rates. IT departments face acute shortages in skilled staff, leaving companies with little choice but to seek out software that automates high-effort, low-value tasks or otherwise offers a force multiplier to existing teams. Subcategories like AIOps and observability pipelines can address those needs.

    Consolidation is Needed to Simplify the Market

    Another looming hurdle for the nascent observability market is feature similarity between companies in a subcategory. Two subcategories where this is apparent are application and data observability. In the application observability space, several companies were founded in the wake of the availability of extended Berkeley Packet Filter (eBPF) technology. eBPF allows you to extend the capabilities of the Linux kernel at runtime, and a common use case is to trace applications running in user space. While this is a great technology with several uses, there are simply too many companies with the same idea, the same go-to-market strategy and same limited feature set.

    Data observability is another subcategory riddled with clones. Data observability started as a reaction to operational brittleness in the modern data stack, and there are only a few companies with marked differentiation here. I agree with Matt Turck’s assessment that data observability products will drift into the well-defined (but arguably less sexy) data quality category.

    CFOs are Looking to Reduce Tool Sprawl and Cut Costs

    The challenges with raising growth rounds have been well documented. VC funding represents the supply side of the finance puzzle and IT budgets are the demand side. The demand side is getting squeezed as CFOs clamp down on budget growth. Deals are taking longer to close and often involve more parties eager to reject a budget request from another team because it means more money for their team.

    A common refrain from CFOs today is that any new spending must be more than offset by corresponding savings somewhere else. This puts the products representing net-new work with little marginal utility at risk. Observability vendors today need a strong consolidation (e.g. can reduce tool sprawl, serve multiple teams, etc.) and cost savings story to survive in a difficult budget climate.

    Cybersecurity remains a budget bright spot. According to a recent survey from IDC, 27% of respondents stated cybersecurity budgets were immune from budget cuts. With changing SEC rules highlighting the need for board-level cybersecurity expertise and reporting of breaches, coupled with the new National Cybersecurity Strategy, the priority around preserving these budgets is clear.

    Cybersecurity is Pure Greenfield for Observability

    A few short years ago, SecOps teams, threat hunters and incident response teams didn’t have enough data to create an accurate picture of their environment. Endpoint data was sparse and incomplete, cloud-based application deployments were still uncommon, the network remained between the office and data center, and so on.

    Today’s SecOps teams, threat hunters and incident response teams have too much data to understand their environment. There are too many alerts and too little context, making finding the signal through the noise a difficult, if not impossible, proposition. From a compliance perspective, much of this data must be retained for longer periods, driving up storage costs.

    This is where the principles of observability benefit cybersecurity teams. Much like site reliability engineers need to trace an error well after it occurred, cybersecurity professionals need to trace a breach or find threats over huge volumes of data. They need robust exploration tools, rational data storage and routing options, and, most of all, the ability to do this while maintaining budgets. This is where observability and cybersecurity converge.

    Seven years young, observability remains a nascent market despite its rapid growth. Though the market will face growing pains, the opportunities and upside are bright.

  • Five Great DevOps Job Opportunities

    Five Great DevOps Job Opportunities

    staging-devopsy.kinsta.cloud is now providing a weekly DevOps jobs report through which opportunities for DevOps professionals will be highlighted to better serve our audience.

    Our goal in these challenging economic times is to make it easier for DevOps professionals to advance their careers.

    Of course, the pool of available DevOps talent is still relatively constrained, so when one DevOps professional takes on a new role it tends to create an opportunity for others.

    The five job postings shared this week are selected based on the company looking to hire, the vertical industry segment and, naturally, the pay scale being offered.

    We’re also committed to providing additional insights into the state of the DevOps job market. In the meantime, for your consideration:

    Dice.com

    ERP International
    Fort Meade, Maryland
    Senior DevOps Infrastructure Engineer
    $150,000 to $180,000

    LinkedIn

    Lark Health
    Remote
    Lead DevOps Engineer
    $127,700 to $183,900

    Simply Hired

    Shake Shack
    Remote
    Senior DevOps Engineer
    $123,000 to $155,000

    Indeed.com

    Peraton
    Arlington, Virginia
    DevSecOps Data Automation Engineer
    $112,000 to $179,000

    CareerBuilder.com

    Booz Allen Hamilton
    Fort Meade, Maryland
    DevOps Engineer and Release Manager
    $73,100 to $166,000

  • The Hidden Configuration Tax Affecting Uptime SLO

    The Hidden Configuration Tax Affecting Uptime SLO

    This article is a preview of a talk by Greg Arnette for SLOconf 2023 on May 15 – 18. To watch this talk and many more like it, register for free at sloconf.com.

    Tax season was just a few weeks ago, and that got me thinking about how frustrating it is to get hit with a surprise tax bill and how that can relate to uptime: Configuration sprawl. DevOps practitioners constantly strive to maintain uptime service level objectives (SLO) while working with complex and ever-evolving infrastructure and applications. Configuration sprawl is a significant challenge many teams face, negatively impacting system performance and reliability.

    My company’s recent survey of 900 DevOps and platform engineering leaders revealed a startling (but not too surprising) conclusion: Most teams struggle managing secrets and configs at scale for infrastructure and applications.

    Where Does Configuration Sprawl Come From?

    As organizations scale their operations, the number of configuration files and settings grows, making it increasingly difficult to manage and maintain consistency across the board. They may have multiple environments, such as development, staging and production environments, often requiring different configurations, leading to duplicated settings and increased complexity.

    Configuration sprawl may also lack standardization as teams adopt different naming conventions, file formats and storage locations without enforced configuration standards, resulting in fragmentation and confusion. Relying on manual processes for configuration management increases the likelihood of errors and inconsistencies.

    Grinding to a Halt

    So you’ve got a lot of configuration–why should you care? Sure, it means more files to manage, but does it really impact your environment? There are a few places where configuration sprawl will negatively impact you. You might see an increase in error rates because as complexity grows, the likelihood of errors and misconfigurations increases, leading to system instability and downtime.

    Increased configuration complexity also means slower troubleshooting. Identifying the root cause of issues becomes more challenging and time-consuming when configurations are disorganized and fragmented. Ultimately this means less agility as you are forced to slow the deployment of new features and updates, impeding the organization’s ability to respond to market demands.

    How to Overcome Configuration Sprawl

    Don’t worry–you can tackle configuration sprawl, keep your SLOs on track and reduce the efforts required to manage and maintain configurations. To mitigate the harmful effects of configuration sprawl, consider adopting these best practices:

    Don’t repeat yourself (DRY)
    DRY config means variable names are declared once. Then flexible variable values are injected across build, deploy and runtime phases as defaults, inheritances and overrides. DRY avoids errors by allowing single changes to propagate where they are needed.

    DRY solves the common problem of forgetting to update critical secrets or parameters and failing to synchronize related changes (e.g., client and server port numbers or rotating a database password.)

    Decouple and abstract
    Decoupling and abstracting config means externalizing config from source code, separating how code and configuration interact, and isolating the specific values (log level) from the configuration interface (how the log level value is fetched.)

    This approach allows the same code to run in different environments using parameterized config. Abstracting config supports a best practice of implementing independent “code and config change” life cycles. Teams that frequently need to change config independently from code should decouple and abstract.

    Centralized config
    Consolidate configuration complexity into a single location because it’s easier to manage complexity within a single location versus multiple locations. Syncing config to the edge (where it is consumed) mitigates downtime. For example, it’s easier to fetch the configuration for service X versus coding all the logic needed to generate the configuration for service X.

    Standardize your configuration management by enforcing consistent naming conventions, file formats and storage locations across teams to reduce confusion and streamline configuration management. Centralizing the configuration and generating that configuration with transformations allows all the places that consume the configuration to be simple. Simple equals easy to understand and troubleshoot.

    Version control & monitoring
    Use a version control system like Git to track config changes to configuration files, making identifying and resolving issues easier. And continue to monitor changes and identify potential problems before they impact system performance.

    Train your team
    Ensure all team members know your best practices and patterns for configuration management. The proliferation of microservices and teams operating in parallel, establishing their unique naming schemes and patterns, exacerbates this problem for DevOps teams.

    Cutting the Uptime Tax

    By understanding the causes of configuration sprawl and implementing best practices to manage and reduce its impact, teams can improve their uptime SLOs and maintain a more reliable, high-performing infrastructure. Remember, investing in the proper tools and processes for configuration management is not only a best practice but also a critical factor in ensuring the long-term success of your DevOps initiatives. Those are tax cuts I can get behind.

  • Nixpkgs: How to Build Better, More Open Software

    Nixpkgs: How to Build Better, More Open Software

    The world we know today depends in fundamental ways on open source software. As providers of this software, package managers serve their respective ecosystems from their centralized, opaque hubs in ways that are sometimes difficult to reconcile with the open source nature of the software they’re serving. I want to talk about the extent to which that’s true and discuss Nixpkgs—the largest open source package repo in existence today—in this context.

    People sometimes focus on Nix the language or Nix the package manager, but it’s the Nixpkgs package collection (pronounced ‘nix packages’) that is core to the success of Nix. But it’s more than that: We can also understand Nixpkgs as a powerful tool to ensure the continued growth, transparency and resilience of open source software, first and foremost because it democratizes the process of curating this software and its build instructions. It is the living compendium of the efforts of thousands of contributors who both stand on the shoulders of giants (the software they’re packaging) and provide the ladder for others to climb further.

    A Purely Functional Grimoire

    The Nix package manager is different from other package managers in several ways, but one key aspect is that Nixpkgs contains only recipes to build packages, not the packages themselves. This is a fundamental idea to Nix: If we build a package reproducibly, we can separate the specification of a build from its built output. When we build a “recipe” for a package, we determine the output hash of that build–without actually building it. This means you don’t need a cache of build artifacts: Recipes are enough to rebuild your software world.

    Nixpkgs is just that: The big fat recipe book of open source. Anybody can view it, fork it, patch it and use Nix to inspect the largest set of free software recipes in the world. Anybody can create a cache that uses those build recipes (or your own!) to publish packages: The default cache is the official one run by the NixOS Foundation at cache.nixos.org.

    Derivation and Substitution

    So what happens, in practice, when I download a package from the Nix cache? When you query for a package, the CLI creates the specification (we call it a derivation) of the package to be built. This derivation includes a hash: The summary identifier of all source code, patches, build flags and hashes of the dependencies themselves.

    With that output hash in hand, Nix the package manager just asks whether that path exists in its cache. If it does, Nix fetches the contents of the package you asked for; if it doesn’t, Nix will build the package locally. We call this move substitution: Creating a derivation from a Nixpkgs expression gives you all the information you need about the output of the build, so the cache can substitute a binary with that output hash if we have it. Of course, if you end up having to build the package locally, it will result in the expected output hash anyway.

    Democratizing Package Curation

    The term ‘lingua franca’ refers to the dialect spoken by sailors across the Mediterranean in previous centuries, and it was the shared language with which one could communicate and do business from Morocco to Greece. Today we package software for independent islands, each with its own rules, languages and cultures and, until Nix, we had not found a way to “speak” across the many ports-of-call of our software archipelago. What this allowed for is packages being available on more platforms, which drew more developers in, which resulted in Nixpkgs becoming more complete, and so on in a positive feedback loop.

    Nix is a cross-platform, cross-architecture package manager. If we had to design the next-generation package manager, this would probably be at the top of the requirements list, yet few package managers enable this today. Each of the teams at RedHat or Debian maintains their own packaging format and package cache and has a fixed team of vetted contributors to keep packages secure and up-to-date. This results in few who effectively contribute to packaging open source projects and do so for one of many platforms.

    An open book of package recipes written in this lingua franca results in another surprising effect: Nixpkgs maintains more packages than anybody with an order of magnitude fewer maintainers [1]. It makes sense: Since it’s a common language, people can review the packaging code for packages they don’t necessarily know intimately, which in turn leads to contributors and reviewers cycling through updates faster and providing more up-to-date packages, which reinforces the previous feedback loop from earlier.

    Even if the rest of the Nix ecosystem stalled out, we’d still have this incredible resource in Nixpkgs to curate and share open source software; flox sources its packages exclusively from Nixpkgs for this (and many other) reasons. As of February 2023, there were around 6,000 contributors on the Nixpkgs repository and growing.

    (via (repology.org)[https://repology.org/repositories/graphs])

    Stay Unstable, my Friends

    Nixpkgs embodies a new model for package management and one that opens it up to more people to contribute. Providing cross-platform and cross-language packages is obviously attractive, but it’s the fact that it provides a lingua franca for software packaging that makes it a pillar of open source moving forward. Because Nixpkgs unifies a broader swath of software and makes it easy for anybody to contribute new packages and fix existing ones, it continues to grow and serve the open source community: A positive feedback loop propelling the existence of a more free and open world of software.

    References:

    Nixos.org

    https://reproducible-builds.org/

    https://wiki.debian.org/SecureApt

    https://rpm-packaging-guide.github.io/

  • Why We Need to Rethink Data Pipelines

    Why We Need to Rethink Data Pipelines

    Data is the fuel that powers modern business. But as demand for data surges, so does the pressure on data leaders and practitioners to deliver it. Businesses need resilient data pipelines that deliver critical insight for real-time decision-making to users on demand. However, against the backdrop of today’s chaotic modern data ecosystems, this is much easier said than done.

    DevOps teams are struggling with a complex combination of legacy and diverse technology systems across multiple environments. The resulting siloed systems and an ever-shifting data supply chain cause significant data integration friction. This friction keeps businesses from creating seamless and resilient pipelines to drive digital transformation and other business outcomes. Businesses need a better way to take the pressure off data leaders and DataOps practitioners and enable line-of-business teams to do more of the “last mile” data collection and analysis themselves. To achieve this, they need a unified, end-to-end platform designed to build resilient pipelines.

    Data Demand Sparks Friction

    The acceleration of digital transformation has created a demand for data that far exceeds supply. This has become especially true as competitive advantage gets harder to carve out and worrying macroeconomic headwinds gather. In today’s climate, all areas of the business are demanding data, but it’s hard to meet every request all the time.

    Research that polled data leaders and practitioners shows nearly half (48%) of admin and operations and customer service departments request data at least weekly, followed closely by accounting and finance (44%), IT and digital (43%) and sales and marketing teams (40%). And the strain is showing. Over half (59%) of data leaders said changing priorities have created significant data supply chain challenges.

    The problem is one not just of capacity but complexity. Building pipelines from source to destination requires rules to integrate, transform and process data. But when data is siloed across cloud, legacy and mainframe systems and stored in inconsistent formats, creating bespoke data pipelines to fulfill departmental requests is a huge challenge.

    Legacy systems also bring another challenge. Over half of data leaders say it’s so difficult to unlock data from legacy systems like mainframes that they simply don’t bother. That will ultimately undermine the value of the cloud analytics tools that many business leaders see as a single source of truth for decision-making–especially as legacy systems often hold decades of vital business insights.

    This patchwork approach creates extra toil for DevOps, DataOps and technical teams and means that over two-thirds (68%) of data leaders said they are being prevented from delivering data at the speed the business needs.

    The Scourge of Broken Pipelines

    For many organizations, building pipelines is also a labor-intensive job requiring a high degree of manual effort to produce hand-coded, one-off solutions. The resulting pipelines are brittle and vulnerable to being disrupted whenever there is a shift in the environment–such as adding new data sources. In fact, two-fifths (39%) of data leaders admit their pipelines crack at the first sign of trouble, and 87% have experienced a break at least once a year. More than one in 10 say it happens at least once a day.

    The most significant cause of breakages is due to bugs and errors being introduced during a change (44%) while infrastructure changes such as moving to a new cloud (33%) and credentials altering or expiring (31%) are also creating disruption. These broken pipelines immediately impact the corporate bottom line. They also force technical teams into a vicious cycle of firefighting. Data engineers spend, on average, almost a third (31%) of their time troubleshooting and recoding broken pipelines–time that could be better spent on value-adding tasks.

    But the negative impacts aren’t only felt operationally. The breakage of any pipeline can also lead to bad decision-making. For example, a supply chain director working with old data may over- or under-order goods or a financial trader making stock picks on out-of-date intel may lose money for a client.

    Unleash the Power of Data

    For organizations looking to overcome these challenges, the first goal is to reduce the workload on DevOps, DataOps and technical data teams by empowering business units and end users to do more. Research shows that 70% of technical data leaders are responsible for the last mile of data collection and analysis while 86% of them would prefer lines of business teams, such as marketing or finance, to be empowered to do this independently. However, this can only become a reality if organizations invest in the right data integration platform. It must be able to build, run, monitor and manage smart data pipelines at scale from a single console–across all cloud and on-premises environments. And it must be able to deliver resilient pipelines built to withstand continuous change.

    Only by eliminating data integration friction and enabling self-service analytics for lines of business can organizations accelerate digital transformation. Doing so will unleash the power of data across the enterprise and reduce the burden of overworked data leaders and practitioners.

  • Black Box SLIs

    Black Box SLIs

    This article is a preview of a talk by Stephan Lips for SLOconf 2023, on May 15 – 18. To watch this talk and many more like it, register for free at sloconf.com.

    SLOs are fast becoming the industry standard to measure reliability and help teams decide when to prioritize it. The first step in adopting a service level objective (SLO) culture is to identify the metrics that matter without drowning in noise and alert fatigue. This article explores how to apply the black box concept to aggregate granular metrics into service level indicators (SLIs) that focus on the user experience as an indicator of system reliability.

    To SLI or Not to SLI

    In general terms, SLOs define targets for the proper level of reliability of a given product, such as a service or a website. SLOs are applied to or informed by SLIs. An SLI is a measurement determined over a metric, or a piece of data, representing some property of a service. And this is where we, as engineers, can get lost in the details, since the perpetual proximity to the systems we build and support often leads us to think of system reliability in technical terms or metrics (e.g., response time, error rate, throughput). While these are certainly valuable metrics, the user experience may be compromised even if the error rate is zero and the duration is well within SLOs. Consider, for example, the response data. Even if well-formed, it may not be current, or flat-out wrong. An error-free and quick response is of no value to a user that expects current and correct data. Error rate and response time remain valuable metrics and SLIs, but focusing exclusively on them would leave higher-level issues undetected.

    We could add freshness and correctness SLIs, but by doing so, we increase the number of signals we monitor. And with each signal—or SLI and associated SLO and error budget—we increase alert frequency and make reliability reports unnecessarily complex. In other words, adding SLIs may address a particular aspect of system reliability, but it also introduces additional complexities.

    Tales of Black and White Boxes

    So, let’s take a step back and borrow a concept from a related discipline: Quality engineering—in particular, software testing. Tests commonly fall into one of two categories: Black box tests or white box tests.

    In systems theory, the black box is an abstraction representing a class of concrete open systems that can be viewed solely in terms of its stimuli inputs and output reactions, without any knowledge of its internal workings. A given input is expected to result in a particular output, without any consideration for the processing steps. Common examples include end-to-end tests.

    White box tests, on the contrary, are designed with knowledge of, and to test, internal structures and workings of an application. Common examples include unit and integration tests.

    User Journey as Black Box

    Now that we understand the concept of black box versus white box tests, let’s apply it to our SLIs. As mentioned above, a good SLI considers the entire user journey. Conceptually, a user journey aligns with the black box paradigm: For a given input, a particular output is expected. For example, requests to our API (the “input”) result in responses that provide fresh data to clients within a given time frame (the “output,” including success criteria). There are several aspects worth mentioning with this SLI:

    ● The SLI is applied at a system level
    ● The SLI aggregates lower-level metrics implicitly and explicitly
    ● The SLI is binary; it is either true or false.

    These aspects combine to inform an SLI that represents the user experience (system level), via measuring many indicators by measuring only a few and supporting pass/fail attribution to an SLO target (by being binary). In other words, the user journey is measured as a black box SLI.

    White Box to Black Box: An Example

    Let’s consider a concrete example. A user requests a new account for a website. After the request is processed successfully, the user receives a confirmation email with an activation link. The user follows the link to activate the new account and log in. This workflow is visualized in the following sequence diagram.

    Of particular interest are the account creation and user notification via email steps. Both steps occur asynchronously. In particular, the event processing engine where the request for account creation is queued offers several opportunities for insightful SLIs: Queue length, average processing time, etc. Those SLIs, however, fall into the white box category: They contribute to the user experience, yet are opaque to the user (black box). The user journey begins with the initial request for account creation (input) and ends with the email containing the activation link (output). Rephrasing the example from earlier—a user-focused (black box) SLI could be a request for a new account (the ‘input’) that results in sending an email with a valid activation link within 1 minute (the ‘output’, incl. success criteria). This single high-level SLI aggregates several lower-level metrics; it measures many things by measuring only a few.

    Let’s switch to the engineering mindset mentioned in the introduction and assume the processing queue is stuck. The high-level black box SLI does not capture queue-specific metrics, suggesting a more granular SLI specific to queue size may be needed. However, white box metrics like this will affect the error budget burn of the aggregate SLO associated with the high-level black box SLI. Monitoring and observability tools will allow engineers to diagnose and troubleshoot particular issues, such as a stuck queue, while understanding the impact on system reliability (via the higher level, black box SLO’s error budget and burn rate). The solution to the stuck processing queue used in this example is not an SLI dedicated to the queue, but reliability-focused work to diagnose and correct the root cause of the queue getting stuck.

    Summary

    This article introduces an SLI thought model that uses a common paradigm from quality engineering. This thought model offers a different way to think about SLIs. It supports implementing the fundamental objective of SLIs and the associated reliability stack they inform: Ensure a positive and reliable user experience by measuring reliability and providing quantitative support for decisions on prioritizing development efforts. Only a happy user is a continuous user.

  • Open Source Communities Need More Than Funding From Big Tech

    Open Source Communities Need More Than Funding From Big Tech

    In recent years, major players like Microsoft, Amazon and Google have increased their involvement in open source projects, providing both financial and technical support. More consistent contributions from big tech—whether in the form of people or resources—is exactly what the open source community needs. However, underdeveloped collaboration guidelines can lead to friction between large corporations and open source developers, which ultimately harms both parties. Friction jeopardizes tech companies’ ability to rely on open source for operations while leaving the open source community without the support it needs. To make their relationship more sustainable, both parties need to cultivate a greater level of trust, develop clear guidelines for collaboration, and prioritize community.

    Big Tech Contributions: Moving From Cash to Collaboration

    For years, major tech companies have used open source solutions in their tech stacks. But many open source developers—who receive little to no money for their work—have voiced concerns about companies that take more from the open source community than they give.

    In many ways, big tech has answered the call. For example, Amazon recently passed IBM as the fifth-largest contributor to open source projects. Companies like IBM and Amazon have a vested interest in supporting these projects—if open source projects are understaffed and underfunded, the quality of the solutions suffers.

    But many tech companies are grappling with an unstable financial environment, which makes hefty donations more difficult to justify. As a result, support from big tech often involves donations of their developers’ time. However, this is also challenging as tech companies like Twitter and Google continue to lay off the developers who were keeping many open source projects afloat.

    Yet, even before recent layoffs, collaborations between big tech and open source teams were strained. There’s a natural power imbalance between a corporation like Amazon and the open source community, which includes members who contribute to these projects in their free time, sometimes for little or no pay. Big tech developers must juggle the interests of the open source team they’re collaborating with and their own employer—with little guidance on how to do so.

    A lack of transparency and communication from big tech also puts open source developers in a tricky situation. Many developers need financial and personnel support but may not want to relinquish control of their solutions to an outside corporation.

    To alleviate this tension, big tech needs to establish clear, mutually beneficial guidelines for open source collaboration that prioritizes sustainability.

    Reconciling the Fraught History Between Big Tech and Open Source

    Past tensions between big tech and the open source community can affect collaboration if left unaddressed. For example, Microsoft has become one of the biggest contributors to the Linux kernel, but many open source teams still remember former Microsoft CEO Steve Ballmer calling Linux a “cancer” in 2001.

    To overcome this tension, big tech companies need to clarify their intentions from the beginning of a project. They should disclose their reasons for selecting the project, the developers who will work on that project and how their contributions will look. Open source maintainers also have a role to play here. They must properly vet these companies and inquire about their prior open source experience and strategies before welcoming them into the fold.

    One way to reduce the risk of power imbalances or other tension is by collaborating through a project like the Linux kernel. Google, IBM and many other corporations contribute toward Linux kernel, and this distribution of contributors creates a much more balanced playing field. Rather than one company assuming operational control, multiple companies play a small part in keeping the Linux kernel up and running.

    The Benefits of an Open Source Program Office

    While these guardrails certainly help, big tech companies also need to formalize their open source contribution processes. At Aiven, we’ve done this through our open source program office (OSPO), which helps us manage our relationships with the open source communities we rely on. OSPOs ensure organizations remain compliant with open source licensing and provide a foundation to expand their involvement with the community over time.

    When developing an OSPO, tech companies should be intentional about how they measure success. The most obvious metrics relate to the work developers do—for example, the number of features delivered, the number of fixes and the number of projects they contributed to. But OSPOs should also measure how developers contribute to the community itself, not just the projects within it. For example, do developers review work from other developers in the community? Do they attend conferences, write blog posts and share knowledge with other developers?

    Big tech leaders must place as much emphasis on community-building as they do the number of contributions developers make. If they don’t evaluate developers based on their engagement with the community, the developer is less likely to focus their efforts there, injecting toxicity into the communities they’re trying to support.

    Open Source Thrives on Collaboration. Let’s Keep it That Way

    The challenge of reducing the stress and burnout of open source developers can’t be solved solely through financial means. No amount of money will reduce the amount of work it takes to maintain an open source project.

    To make collaborations between open source teams and big tech sustainable, both sides need to prioritize supporting the community above all else and develop guidelines that support this mission. More sustainable collaborations will ensure that the open source community gets the support it needs while big tech continues to reap the rewards of the community’s innovations.