staging-devopsy.kinsta.cloud

Tag: devops

  • There’s No Such Thing as DevSecOps

    There’s No Such Thing as DevSecOps

    Should DevSecOps be categorized separate from DevOps?

    After years of debate, few doubts remain about the viability of the public cloud for enterprise workloads. Enterprises are shifting to cloud-first and cloud-only strategies, looking to take advantage of all the good the public cloud has to offer. For applications built and deployed in the public cloud, it is all about automation. The cloud enables you to be elastic, reduce costs and deliver products faster. To achieve these goals, you need to automate your processes—building, testing and deployment.

    Security Challenges of DevOps

    This need for faster innovation led to the rise of DevOps, a set of practices and tools that result in tighter collaboration between development and operations. In the past, controlling the deployment processes was the responsibility of dedicated teams that wrote scripts. However, with DevOps, development and deployment merge. Software engineers must do more than write their infrastructure and business logic code; they also need to write the automation that surrounds it and take responsible for deploying it through the different stages until production.

    However, even as development and operations come closer together, security has remained a function apart. While DevOps workflows are heavily automated, security checks for application and infrastructure code often are manual. DevOps and security teams are measured differently—delivery versus protection—and often are at odds with each other. The siloed approaches to security hardening that worked in the past are incompatible with the holistic, iterative software development and deployment model of DevOps.

    For a developer, adding the responsibilities associated with DevOps is not complicated, but securing the environment is a different level of challenge that requires mindset shifts in multiple areas:

    1. Knowledge: Software engineers are great at writing business logic. They write efficient code and develop highly available environments that can handle millions of concurrent users. However, the vast majority of developers don’t know about the latest CVEs or the proper configurations of SSL/TLS in the load balancer.
    2. Pace: Successful development organizations move fast, frequently adopting new development trends and frameworks. Public cloud vendors release new features every week, and it can be difficult to follow and keep up, as engineering is usually measured by the ability to deliver and the quality of output. DevOps enables rapid release cycles, and the security must not hold them back.
    3. Scale: Modern applications make the transition to work with microservices. When each development team owns its cloud environment (and sometimes multiple environments), the security teams suddenly find themselves dealing with huge distributed deployments. They can’t manually configure security when new features are released a few times a day in different environments.
    4. Heterogeneity: Multi-cloud deployments are another emerging trend. New product teams are often given a free hand choosing their tools and infrastructure, and they may pick a new cloud vendor. While security teams have a voice in this decision, they do not have the last word. It may have taken the security team years to learn to deal with the existing cloud solution, and suddenly the organizational policy needs to be applied across different infrastructures.
    5. Consistency: Enforcing organizational policy across different development teams is challenging. Each team has its own style, environment and development frameworks, meaning it’s never easy to ensure the security is applied uniformly.

    However you choose to address all these challenges, one thing must be clear: Your DevOps cycle is not complete without handling the security.

    Choosing a Security Solution

    Clearly, you can’t deploy services without properly securing them—you don’t want anyone to hack into your systems and steal your (and your customers’) information. Since cloud environments are dynamic and deployments tend to span across dozens of accounts, manual steps are not an option. Automation is necessary. Whether you acquire an enterprise security solution, adopt an open source security project or write your own tools, you need to make sure that they are:

    1. CI/CD ready: The solution can easily integrate into the continuous integration/continuous compliance pipeline. User-friendly APIs are required.
    2. Scalable: Assessing hundreds of cloud environments requires the solution to be scalable and robust. When covering more cloud vendor capabilities, performance becomes critical and increasingly challenging.
    3. Flexible: Each organization is different, and security needs can vary with each deployment. The baseline configuration of your security solution will never meet your requirements by 100 percent, so it’s important that you be able to adapt it to your needs.
    4. Continuously updated: Cloud vendors add new features at a rapid pace, which also means new potential misconfigurations are generated all the time. Whatever solution you choose, it needs to match the cloud vendors’ speed and agility. Security solutions that are not being updated soon become useless.
    5. Conveniently Reportable: Aggregated reporting is critical. Consuming the security findings in a convenient manner can assure that issues are properly mitigated.
    6. Automatic remediation-capable: Many of the findings are easy to remediate automatically. Applying automatic remediation would enable the security team to focus on complicated tasks and reduce alert fatigue.

    DevOps and DevSecOps Are (or Should Be) the Same Thing

    Security is clearly an integral part of DevOps. DevSecOps is emerging to bring security and compliance management practices in line with the rest of DevOps. DevSecOps serves to call out the temporary chasm that exists between security and DevOps.

    As security checks and controls get automated and integrated, DevOps and DevSecOps are converging to the same thing. Just like adjusting the mirrors and putting on the seat belt are part of standard procedure before pulling your car out of the driveway, assessing your security posture before releasing a new version of your product must become habit. To remain relevant, your security team must integrate into the CI/CD and make sure the newly created environments meet organizational policies.

    — Offir Zigelman

  • Living in a DevOps World (Part 1)

    Living in a DevOps World (Part 1)

    The concept of DevOps has grown and evolved into a conceptual and working model for more effective and efficient software development and implementation. That said, there are some differences of opinion on the real-world value of any DevOps approach to date, and on the best way to create and implement a real-world DevOps environment. This two-part article will focus on what an agile DevOps approach is meant to address, and what it is not meant to address.

    DevOps sits at the nexus of three essential business technology functions: software development, quality assurance and operations. A short and concise definition of DevOps proposed in 2015 seems as appropriate as any:

    DevOps is a set of practices intended to reduce the time between committing a change to a system and the change being placed into regular production while ensuring high quality.

    The definition was suggested in the book, “DevOps: A Software Architect’s Perspective,” and the authors have hit upon the essence of the practice. The key, of course, is how to put that concept into practice.

    Perhaps the first step on the journey to effective DevOps is the recognition that the concept is the result of the rise of the Lean and Agile software development methodologies. Those methodologies, among other things, emphasize the following:

    • A focus on customer value.
    • The elimination of waste.
    • Reduced cycle time (accomplishing work faster, releasing faster).
    • Shared learning.
    • Avoiding batching (don’t do things until required).
    • Theory of constraints (break things up, focus on individual issues).
    • Continuous integration, testing and delivery.
    • Faster time to market.

    DevOps in Practice

    Adherence to the principles above meant that something had to be invented to accomplish them and that something was DevOps. Over time, an effective DevOps practice should address any number of business technology pain points. The following short list of those pain points and the DevOps response should prove instructive.

    System Downtime

    The developers’ curse since systems were developed, system outages are inevitable as long as systems are designed, tested and implemented—even with increased automation—by imperfect beings. DevOps acknowledges that by changing the focus from trying to create applications that never fail to designing systems that can recover quickly, thus decreasing aggregate systems outage time over the life cycle of any application or system.

    Stagnation

    This was a staple of traditional systems development and is most closely associated with the waterfall methodology for systems development. After requirements were created, the development team would be locked away for weeks, months or, in some cases, years before emerging with “fully” working software that inevitably no longer satisfied rapidly evolving business requirements. DevOps is designed to fit hand-in-glove with the Agile practice of short windows of incremental changes instead of long release cycles, putting working software in the hands of customers as quickly as possible.

    Team Conflict

    Having been borne from the cultural combination of Agile and Lean, DevOps has taken on the problem of functional silos that are often erected between development, operations and the business customers. It follows the methodological approaches of collaboration and teamwork first to understand what others know and to leverage the best of it to solve business problems more rapidly. There is also a cultural bent toward experimentation, continual learning and constant improvement. This leads to blameless post-mortems, where instead of finger pointing when something goes wrong there is collaborative discussion and learning to correct and prevent the problem from occurring again.

    Knowledge Silos

    Functional silos have led to compartmentalized knowledge. If the old game was that knowledge is power, the new game in the DevOps world is that knowledge is freely exchanged as an enabler to solving business problems. DevOps addresses the problem of information being lost in translation between the development and operations functions by eliminating the functional barricades and making knowledge sharing the highest form of collaboration.

    Inefficiency

    Waiting for things to happen used to be a standard operating procedure in the pre-DevOps world. Project plans were created and managed to account for the time it might take for new code to be moved into a testing, quality or even production environment. This was a momentum killer for projects and at times a morale killer for developers waiting to see what changes they might need to make to their code set. The combined Agile and DevOps approach has rewritten the traditional approach to code migration, smoothing and eliminating wait times so that projects flow more seamlessly from start to finish. It also has the benefit of keeping business resources—testers, approvers, etc.—more engaged as a result of a constant flow of new functions and features to test and use. And lest this be viewed as simply a way to keep the technical stuff moving along, it’s important to remember that there is a financial aspect to this as well. Reducing speed to market with new functionality, reducing or eliminating idle hands—be they technical or business—and delighting customers with a steady stream of enhancements and features all go directly to an organization’s top and bottom lines.

    That, after all, is in many ways what the DevOps approach is all about. All of these critical areas become the means to accomplish it. Part two of this article will focus on some more of the benefits of a DevOps approach, and how to achieve them.

    — Dave Farinelli

  • Using CALMS to Assess an Organization’s DevOps

    Using CALMS to Assess an Organization’s DevOps

    Of the tried-and-tested frameworks that allow enterprises to assess DevOps in their organization and how it can be improved, the CALMS model remains particularly useful.

    CALMS, which stands for Collaboration, Automation, Lean, Measurement and Sharing, is particularly helpful for analyzing an organization’s DevOps structure, and ultimately, its utility in any organization. The CALMS framework covers all stakeholders within DevOps, including business, IT operations, QA, InfoSec and development teams, and how they collectively deliver, deploy and integrate automated processes that make business sense.

    “DevOps as a technique is slowly but steadily maturing, so it’s good to see general assessment methods like the CALMS model,” Holger Mueller, an analyst for Constellation Research, said. “The CALMS model gives a good frame of reference to compare the maturity of a DevOps team, and as such,  is invaluable for assessing the state of teams for the transformational change that goes with it.”

    Applying CALMS

    Here are a few ways CALMS can be applied to DevOps for enterprises either thinking about shifting to a DevOps structure or seek to make improvements to an existing deployment.

    Culture

    It is long-established that technology adoption must be to serve a business need rather than investing in technology for its own sake. This shift in mindset applies to how the “culture” component of CALMS supports how the forecasted ROI associated with automating a process must be championed across the DevOps team. This mindset is especially important for getting the non-technical business teams onboard, Mueller said.

    “DevOps must have the backing from the business executives. That’s a crucial factor in the culture part,” he said. “One thing software teams can do is use successful software deployments to help convince the business executives in DevOps to make the investments. Those are the kinds of tribal events that every DevOps organization needs.“

    Automation

    Netflix is well-known as an exemplary case study for DevOps automation. Among other things, it can deploy code and software updates in a matter of minutes across its entire massive infrastructure, many times throughout the day. But Netflix’s example of automation does not take into account what can go wrong at firms that want to follow Netflix’s example when automating their software deployments and updates, Mueller said.

    “The risk is always deploying too fast and you end up deploying bad code,” Mueller said. “Automation, of course, needs to take this into account and compensate. But not everybody can do that.”

    Lean

    Once completed, it is relatively easy to document cost savings by eliminating resources. Within the CALMS framework, the corresponding “lean” component in DevOps is thus relatively easy to assess, Mueller said.

    The good thing about DevOps, Mueller said, is companies can come to a consensus ahead of time regarding what they consider to be a lean operation.

    However, he added, “There is always the risk of being so lean you begin to err on the QA side.”

    Measurement

    The success of software deployments is relatively easy to measure once completed. However, attempting to measure future deployments as part of a forecast model is, of course, much more challenging.

    “It can be surprisingly tricky to measure how good DevOps is ahead of time,” Mueller said. “Measurements should thus allow errors to be pinpointed  quickly during the post-deployment stage to fix the code if needed very quickly.”

    Sharing

    Sharing, in many ways, overlaps with culture. This is where assessing whether all of the different teams within DevOps are working together and communicating comes into play.

    “Thanks to  DevOps, the days of the lone developer are long gone. But sharing is not always easy,” Mueller said. “Even within separate teams, it is easy to fall into the trap of wanting to get your own work done without necessarily thinking about the team as much as you should.”

    The End Result

    While applying CALMS to assess DevOps never hurts, the framework is still not as important as one of the main objectives of DevOps—namely, using technology to make organizations’ operations more agile and reactive to customer needs, Mueller said.

    “At the end of the day, assessment methods such as CALMS are great, but they are never as important as focusing on implementing next-generation applications,” he said.

    — B. Cameron Gain

  • Microservices Momentum Accelerates

    Microservices Momentum Accelerates

    A global survey of 353 application development professionals conducted by Dimensional Research on behalf of LightStep, a provider of application performance management (APM) tools, finds a full 86 percent of respondents expect microservices to be the default architecture within five years.

    A total of 60 percent of respondents said they already have microservices in production or deployed as part of a pilot, and 92 percent said they expect to grow their use in the coming year.

    Top reasons cited for employing microservices are agility (82 percent) and scalability (78 percent).

    The downside of increased reliance of microservices is the need to manage them all. The survey finds 99 percent report challenges in using them. More than half (56 percent) reported that each additional microservice increases operational challenges, and 87 percent of those that have deployed microservices in a production environment reported they are generating more application data.

    To make matters more challenging, a full 98 percent of respondents said they have trouble identifying the root cause of performance issues associated with microservices. More than three-quarters (76 percent) said it also takes longer to resolve issues. In fact, 73 percent said it is more difficult to troubleshoot application performance problems compared to a traditional monolithic application.

    Not surprisingly, just less than three-quarters (74 percent) said they plan to increase investments in microservice performance management tools in the next year.

    LightStep CEO Ben Sigelman said that while microservices come in many forms, advances in how software is developed is clearly coming at a cost. Not only is software becoming more challenging to manage, APM tools are now a prerequisite. Historically, IT organizations only monitored their most mission-critical applications. But the level of dependency each microservice has on others requires a more holistic approach to APM, Sigelman said, noting that IT teams now need to be able to see well beyond the border of any one microservice.

    Challenges associated with managing microservices are not likely to deter organizations from eventually adopting them. The level of agility and resilience inherently built into in a microservices-based application are too high to ignore. But the rate at which they are employed might be impacted. There’s usually a direct relationship between the availability of tooling and expertise and the rate at which any emerging technology gets deployed in a production environment. The good news is advances in machine learning algorithms and other forms of artificial intelligence (AI) should make it easier to manage microservices in the years ahead.

    In the meantime, IT organizations can take some comfort in the fact that developers are being held more accountable for the microservices they build. That level of accountability tends to vary widely by organization. But the end result is generally higher levels of software quality because developers increasingly realize they can’t throw code over the proverbial IT operations wall without it coming back to them. The challenge now is to bridge the historic divide between developers and IT operations teams in a way that works best for everyone concerned.

    — Mike Vizard

  • How DevOps Can Create Smarter, More Agile 5G Telecom Networks

    How DevOps Can Create Smarter, More Agile 5G Telecom Networks

    There is no doubt DevOps revolutionized software development with its introduction of lean and agile techniques. By applying continuous deployment and continuous integration methodologies into software development, enterprises can build, test and deploy applications and release new services more rapidly. In this article, we will see how the DevOps approach is being utilized in 5G telecom networks with the rise of software-defined systems.

    Emergence of Software in Legacy Telecom Networks

    In last few decades software has changed our lives in myriad ways—mostly for the better. Now, software is starting to disrupt the telecommunication industry with the eagerly anticipated 5th generation of telecom network, also known as 5G. It is expected to provide low latency and high bandwidth network connections to serve use cases including artificial intelligence, smart city, 3D video, industry automation and self-driving cars, and to provide end-to-end real-time experiences such as virtual reality and augmented reality. A variety of these use cases have different network requirements and dependencies, and 5G is expected to provide such flexibility along with agility.

    Software-defined networking (SDN) and network functions virtualization (NFV) form the backbone of the upcoming 5G network, enabling service providers to deliver lucrative services to their customers and realize a competitive advantage. SDN makes the network programmable and centrally controlled, while NFV replaces dedicated network appliances with software packaged in virtual machines on top of high-performance servers. SDN and NFV enables:

    • Better control over overall network.
    • Reduction of manual network configuration processes.
    • Lower CAPEX.
    • Network resiliency.
    • Orchestrate network dynamically.
    • Faster time to market, i.e. faster introduction of new services and features to customers.

    Application of SDN and NFV technologies in 5G telecom networks indicates the use of software in the legacy telecom network, wherein many network functions including firewall, content delivery network (CDN), network address translation (NAT), routers, deep packet inspection (DPI) and broadband remote access server (BRAS) can be virtualized and serve as virtual network functions (VNFs) in NFV architecture. This stands as the core feature of upcoming 5G networks, as most of hardware-dependent operations will be performed by software, providing higher performance with tons of benefits to end users.

    Cloud-Native Enables DevOps

    Virtual network functions (VNFs) are the software representation of network function appliances in NFV environment and a core element in 5G networks. Services can be provided by chaining different VNFs developed by different vendors. There are complexities associated with having multivendor VNFs, as each VNF has its own performance and operational issues including manual integration in NFV infrastructure, utilization of NFV hardware resources, lack of interoperability and multitenancy. For a 5G network to deploy successfully, each VNF must be configured and managed in automated way.

    As software becomes part of telecom network, the term “cloud native” becomes part of the vernacular for VNFs.

    By harnessing a cloud-native approach—including containerization, microservices-based architecture and dynamic orchestration—VNFs with their own life cycle enable:

    • Decomposition of sub processes into microservices—resources required will be automatically scaled and communicated with other services using APIs.
    • Management and simplified orchestration of VNFs.
    • Deployment within containers and connected by APIs.
    • Orchestration by container scheduler—manages provision into service and lifecycle management.
    • Portability to different NFV environments.

    More importantly, a cloud-native approach allows VNF vendors and service providers to impose DevOps (continuous deployment and integration) methods to automate process of building, validating and deploying VNFs into NFV environments.

    A DevOps approach in 5G networks brings benefits for overall network operations, enabling deployment automation and management of service chains and 5G network slices. Complete automation is required, especially for 5G network operations, which may increase operational efficiency and decrease human error, saving OPEX and decreasing expensive downtime.

    For 5G networks, requirement of DevOps methods in VNF can be for:

    • Automated onboarding and deployment of VNFs in NFV environment.
    • Automated performance testing and benchmarking for VNFs.
    • Enabling self-scaling of VNFs—automated resource provisioning for VNFs.
    • Rnabling self-healing after an error in VNFs residing in VMs or containers.
    • Real-time monitoring of VNF activities for analysis.

    More ‘Intelligent’ Network

    Cloud-native capabilities and a DevOps approach opens the possibility of incorporating artificial intelligence in 5G telecom network operations analysis and real-time monitoring. Cisco Systems has already introduced its solution based on machine learning, which it calls Intent Based Networking. We can imagine a network operation having “zero touch” feature by just providing an intent to complete network for any configuration or specific network-based instruction. The collected analysis data can be used in the management and organization of NFV for corrections of network errors, adjustments in VNF and NFV elements configuration and self-healing/scaling of network functions.

    Summary

    Development of 5G network is still in process. By taking into account a revolution in SDN, NFV, cloud native and DevOps, it is expected that we will have 5G networks by some of leading players Verizon, AT&T, Telefonica and Vodafone reasonably soon.

    — Sagar Nangare

  • Monitoring Your Data Center Like a Google SRE

    Monitoring Your Data Center Like a Google SRE

    Back in the early ’00s, when Google was beginning to expand its portfolio of services beyond search, it encountered a combination of challenges. Some of these emerged from familiar, classic disconnects between developers and operations folks or between IT services and line-of-business owners. Others were brand new, never-before-seen failure modes that arose from providing services on novel cloud platforms—and doing so at planetary scales. 

    To confront these challenges, Google began evolving a discipline called Site Reliability Engineering, about which the company published a very useful and fascinating book in 2016. SRE and DevOps (at least the contemporary version of DevOps that’s expanded into a vision for how IT operations should work in the era of cloud) share a lot of conceptual and an increasing amount of practical DNA; particularly true since cloud software and tooling have now evolved to enable ambitious folks to begin emulating parts of Google’s infrastructure using open source software such as Kubernetes. Google has used the statement, “Class SRE implements DevOps” to title a new (and growing) video playlist by Liz Fong-Jones and Seth Vargo of Google Cloud Platform, showing how and where these disciplines connect and nudging DevOps to consider some key SRE insights.   

    Understanding Google SRE Principles

    First, some basic principles: 

    Failure is normal – Achieving 100 percent uptime for a service is impossible, expensive or pointless (e.g., given the existence of masking error rates among your service’s dependencies). 

    Agree on SLIs and SLOs across your organization – Since failure is normal, you need to agree across your entire organization what availability means, what specific metrics are relevant in determining availability (called SLIs, or service level indicators) and what acceptable availability looks like, numerically, in terms of these metrics (called the SLO, or service level objective). 

    Use agreed-upon SLOs to calculate an “error budget” – SLO is used to define what SREs call the “error budget,” which is a numeric line in the sand (e.g., minutes of service downtime acceptable per month). The error budget is used to encourage collective ownership of service availability and blamelessly resolve disputes about balancing risk and stability. For example, if programmers are releasing risky new features too frequently and compromising availability, this will deplete the error budget. SREs can point to the at-risk error budget, and argue for halting releases and refocusing coders on efforts to improve system resilience. 

    This approach lets the organization as a whole balance speed and risk with stability effectively. Paying attention to this economy encourages investment in strategies that accelerate the business while minimizing risk: writing error- and chaos-tolerant apps, automating away pointless toil, advancing by means of small changes and evaluating “canary” deployments before proceeding with full releases.     

    Monitoring systems are key to making this whole, elegant tranche of DevOps/SRE discipline work. It’s important to note (because, remember: Google isn’t running your data center) that this has nothing to do with what kind of technologies you’re monitoring, the processes you’re wrangling or the specific techniques you might apply to stay above your SLOs. In short, it makes just as much sense to apply SRE metrics discipline to conventional enterprise systems as it does to 12-factor apps running on container orchestration. 

    Applying Google SRE Principles

    So here are a few things Google SRE can tell you about monitoring, specifically: 

    Alert only on failure, or on incipient failure – Alert exhaustion is a real thing, and “paging a human is an expensive use of an employee’s time.” 

    Monitoring is a significant engineering endeavor – Google SRE teams with a dozen or so members typically employ one or two monitoring specialists. But they don’t busy these experts by having them stare at realtimecharts and graphs to spot problems: that’s a kind of work SREs call “toil”—they think it’s ineffective and they know it doesn’t scale. 

    Post-hoc analysis, no magic – Google SREs like simple, fast monitoring systems that help them quickly figure out why problems occurred, after they occurred. They don’t trust magic solutions that try to automate root-cause analysis, and they try to keep alerting rules in general as simple as possible, without complex dependency hierarchies, except for (rare) parts of their systems that are in very stable, unambiguous states (their example of “stable”: when they’ve redirected end-user traffic away from a downed data center, systems can stop reporting on that data center’s latency). Elsewhere, their systems are in constant flux, which causes complex rule-sets to produce excessive alerts. One exception to this general rule about simplicity: Google SREs do build alerts that react to anomalous patterns in end-user request rates, since these affect usability and/or reflect external dependency failures (e.g., carrier failures). 

    Heavy use of “white box” monitoring – Google likes to perform deeply introspective monitoring of target systems grouped by application (called business service monitoring in Opsview Monitor). Viewing related metrics from all systems (e.g., databases, web servers) supporting an application lets them identify root causes with less ambiguity (e.g., is the database really slow, or is there a problem on the network link between the DB and the web host?) 

    Four golden signals – Because part of the point of monitoring is communication, Google SREs strongly favor building SLOs (and SLAs) on small groups of related, easily understood SLI metrics. As has been widely discussed, they believe that measuring “four golden signals”—latency, traffic/demand, errors and saturation—can pinpoint most problems, even in complex systems such as carrier orchestrators with limited workload visibility. It’s important to note, however, that this austere schematic doesn’t automatically confer simplicity, as some monitoring makers have suggested. Google notes that “errors” are intrinsically hugely diverse and range from easy to almost impossible to trap, and “saturation” often depends on monitoring constrained resources (e.g., CPU capacity, RAM, etc.) and carefully testing hypotheses about the levels at which utilization becomes problematic. 

    Conclusion

    The bottom line is that good DevOps monitoring systems need to be more than do-it-yourself toolkits. Though flexibility and configurability are important, more so is the ability of a mature monitoring solution to offer distilled operational intelligence about specific systems and services under observation, along with the ability to group and visualize these systems collectively, as business services. 

    — John Jainschigg

  • Barracuda Networks Survey: DevSecOps Is Hard

    Barracuda Networks Survey: DevSecOps Is Hard

    Organizations that have adopted DevSecOps processes may be making progress when it comes to improving cybersecurity, but it’s far from easy. A survey of 618 IT decision-makers conducted by Dimensional Research on behalf of Barracuda Networks, a provider of IT security software and appliances, finds half have adopted either DevOps, DevSecOps or a continuous integration/continuous development (CI/CD) platform. A full 93 percent of those IT decision-makers said they faced challenges integrating security into those practices.

    Top challenges cited include: Existing security technologies don’t integrate with application development environments (63 percent), application developers are not comfortable with security (48 percent), and security processes have not evolved (48 percent).

    Much of the frustration can be tied to reliance on firewalls that don’t feature application programming interfaces (APIs) or are natively integrated with a public cloud, said Tim Jefferson, vice president for public cloud at Barracuda Networks. Each cloud service provider has a unique environment that requires providers of IT security software to write code that natively integrates with that platform. IT organizations can’t simply cut and paste security controls created for on-premises IT environments mainly because the cloud provider has already implemented its own controls at the infrastructure level. The IT organizations need to develop their own controls for the application layer, each of which then must be layered on top of the controls put in place by the cloud service provider.

    In the absence of those capabilities, developers get frustrated because a firewall running on a virtual machine on a public cloud presents them with a graphical user interface (GUI) rather than an API. Developers in the age of the cloud are not going to engage with any technology that they can’t programmatically invoke via an API, Jefferson said.

    Not surprisingly then, the survey finds the most requested firewall capabilities requested are cloud-native integration (74 percent), ease of deployment and configuration by cloud developers (59 percent), ability to regulate traffic flows between on-premises IT environments and public clouds (56 percent) and distributed policy enforcement (53 percent).

    Because many IT professionals don’t have these capabilities many of them conclude on-premises IT environment are more secure than public clouds. More than half the respondents (56 percent) say on-premises cybersecurity is still superior to a public cloud. Clearly, there a lot of technical and emotional factors that go into those assessments, but regardless of the truth, perception remains reality.

    Regardless of the approach taken, however, organizations that deploy firewalls in the cloud are expecting to pay for those services on a consumption basis, which doesn’t always align with taking a firewall running on virtual machine and then deploying it on a public cloud.

    Jefferson said that as cybersecurity in the age of the cloud continues to mature, expectations are rising regarding the capabilities a firewall should be able to provide—especially when it comes to controlling the flow of traffic coming into and out of a public cloud.

    Obviously, anything to do with DevSecOps these days is still evolving. But whatever the cybersecurity future holds the relationship between on-premises and cloud security is far more nuanced than many appreciate.

    — Mike Vizard

  • Can Agile and DevOps Get Along?

    Can Agile and DevOps Get Along?

    There is a common misconception that Agile and DevOps cannot work together, which does a disservice to everyone in the software development and delivery field. In fact, both approaches can significantly add to the Scrum pillars of transparency, inspection and adaption to better enable product teams to develop, deliver and sustain complex products. Both Agile and DevOps embody great ideas and enthusiasm that can add value to how teams work, ensuring the success of their organization in a world where business is both driven and disrupted by software.

    For businesses to take full advantage of the opportunities presented by Agile and DevOps, both communities—the Agilists and DevOps professionals—need to rectify the misunderstandings between the two approaches that amplify the differences between them. They must understand why these myths permeate and learn how to overcome these barriers so Agile and DevOps can finally play nice in the sandbox.

    Agile and DevOps: The Misunderstandings

    Both Agile and DevOps pursue the desire to deliver value to customers in a more effective way, but from different starting points—Agile with the developer, and DevOps from operations. Their emphasis also varies, leading people to believe that they are different things: Agile emphasizes team interactions, culture and values, while DevOps emphasizes delivery pipelines and flow.

    These apparent differences are further amplified by misunderstandings that do nothing but to pit the two approaches against each other:

    • Some Agilists think all process is bad, while DevOps is steeped in a history of stability through process.
    • DevOps uses tools and automation to increase speed, while some Agilists view tools as unavoidable at best and a distraction at worst.
    • DevOps uses holistic systems to solve problems, while Agile uses people to drive change and discover it through work.

    It is disheartening that these misunderstandings exist because Agile and DevOps share more in common than most realize. For instance, both include faster delivery cycles, smaller increments or releases and use feedback to improve and remove waste and impediments. Both approaches can also influence a cultural shift in an organization that promotes small changes that are easily manageable so they lead to bigger and better business outcomes over time.

    If they are more alike than dissimilar, then what exactly is holding these two methodologies back from getting along?

    Politics, Organizational Silos Widen the Gap

    Organizational changes, such as new hires and managerial moves, etc., attract strong personalities who get recognized for driving change and who get rewarded when those changes produce results. When their success is tied to the success of either their Agile or DevOps program, competing political goals are deemed a threat and lead to confusion:

    • Different tool strategies: How many times do you hear, “The Agile teams use JIRA, but the DevOps teams use Service Now”? These tools and the vendors selling them focus on a change initiative to drive purchases. This makes it even more difficult to integrate Agile and DevOps, as their data and processes are hidden within different tools.
    • Politics rather than value to make decisions: The saying, “It is not what you know, it is who you know,” is not just true in national capitals, but in any organization. Politics is the reality of most organizations and that means that the associated initiatives not only come with baggage of the work, but also with the politics of who is driving them.
    • Increased division and organizational conflict: Ironically, the DevOps and Agile movements call for unification and collaboration between different groups. However, when practiced, they can sometimes lead to even deeper gaps within traditional groups due to their different terminology and ideas.

    Different Cultures

    Agilists consider process to be “emergent” in response to the needs of the situation. This means teams are empowered to make decisions about how they work via a light framework (such as Scrum) and then add any other process they think are needed to achieve more value and faster feedback.

    DevOps traditionally takes a different tack, with a focus on systems thinking with flow models and value streams.

    Despite the culture differences, Agile and DevOps need consistent language and concepts to work together, or else communication breaks down and collaboration suffers. This leads to alienated stakeholders, increased fear and defensive behavior, poor product and service quality, lack of transparency and visibility and, more painstakingly, never getting anything done.

    Scrum Breaks Down the Barriers to Agile and DevOps

    Delivering value to customers through software requires elements from both Agile and DevOps coming together in a comprehensive process that is fluid and enables the teams to work in a dynamic manner. Through Scrum that empowers cross-functional, business-oriented teams to deliver software continuously, work flows seamlessly between all the right stakeholders, in real time.

    However, simply having cross-functional teams in your organization isn’t enough to make sure all involved are working in a holistic way to deliver customer value. Organizations need to make the right decisions about who should be in or out of a team, since different technologies require specialist skills and infrastructure to support it.

    A cross-functional team must have all the competencies needed to accomplish the work without depending on others who are not part of the team, as described in the Scrum Guide, to deliver a working product.

    As technology, market and environmental complexities and their interactions have rapidly increased, Scrum’s utility in dealing with complexity is proven daily—Scrum teams are highly flexible and adaptive so that they can deliver products of the highest possible value. When misunderstandings are put aside, and people learn to communicate with each other, cross-functional teams comprised of Agilists and DevOps with all the right skills can deliver done software, faster for your organization.

    — Dave West

  • What Can Conway’s Law Teach Us About DevOps?

    What Can Conway’s Law Teach Us About DevOps?

    I first heard about Conway’s Law from reading the “DevOps Handbook” (an excellent read, by the way, if you want to dive head-first into DevOps). I was intrigued by how this idea came from slightly left field, yet seemed to have big implications for businesses on their journey to increased efficiency.

    In this blog, I will share my understanding of what Conway’s Law is and how it is relevant for your business, whether you are in the middle of a DevOps transformation or just taking your first steps.

    What is Conway’s Law?

    In 1967, Dr Melvin Conway submitted a paper to Harvard Business Review. It stated the following observation: “Organizations which design systems … are constrained to produce designs which are copies of the communication structures of these organizations.”

    This statement was later dubbed “Conway’s Law“.

    What Conway’s Law Means for the DevOps Journey

    Conway’s Law explicitly mentions the system design and communication structure. However, this idea can also be extended to include team structure, which is usually aligned either to systems structure or communication channels.

    When it comes to delivering streamlined and efficient change, Conway’s Law suggests that we should align all these areas to the overall business model. This is a concept that also sits at the heart of DevOps, and is a cornerstone for improving efficiency and overcoming delivery bottlenecks.

    The Importance of Alignment

    Alignment between these four areas is critical for becoming truly agile and thriving in today’s online and digital world. Without it we will hampered by heavyweight handoffs, conflicting team goals, and resistance to change.

    The larger the organization and the more siloed the teams, the more pronounced the constraint described by Conway’s Law is. However, organizations with more nimble, collaborative teams are free to design their systems in the most efficient way, not restricted by inefficient wasteful processes.

    Dealing with Misaligned Team Structures

    In IT, with its constant flow of new projects and requirements, we are always improving our systems in response to changing strategies and directions from the business. So why shouldn’t we have the same mindset when thinking about our team structures?

    If our team structures aren’t evolving to meet the current needs, then we are inadvertently constraining ourselves to only deliver solutions that fit in with our existing team structures. This makes innovation difficult. Similarly, we need to mold our communication and collaboration structures into the right shape for our systems, based on business goals. This will, in turn, define how we organize our teams.

    With knowledge and understanding of Conway’s Law, we can actively design systems, team structures and communication channels to avoid the pitfalls of misalignment.

    Overcoming DevOps ROI Barriers

    DevOps is an incredibly powerful approach to IT. But for DevOps to provide value to your organization, you need to address team silos and collaboration gaps. If you cannot align collaboration channels with your business needs, then no amount of automation or DevOps change will help your business to achieve agility.

    For some businesses, team structures may need to change dramatically. For others, it may just be a case of increasing collaboration between specific teams, or creating ways to improve the efficiency of handoffs and reduce wastage in processes.

    Aligning Teams to Business Goals

    Overall business success isn’t siloed, and team/department success shouldn’t be either. Sharing KPIs and dashboards is one important way of ensuring that teams stay aligned to business goals by improving staff mindset and reducing distraction.

    So what do fully aligned teams look like? In businesses delivering IT change, teams will be aligned to products or market sectors. This means each team will be as autonomous as possible, without cumbersome handoffs.

    Often aligned teams will incorporate all speciality skills (such as information security, infrastructure and DBA) within their own team. However, it is becoming more common to solve the alignment problem by improving the efficiency of interactions between teams through self-service. For example, self-service environment provisioning enables delivery teams to function independently throughout the delivery life cycle without any environment requests or delays.

    The Art of Collaboration

    We are currently seeing a big shift in culture across many organizations, powered by the fast uptake of new collaboration systems. Tools such as MS Teams, Slack and HipChat remove barriers between teams and improve communication, making it easier to align collaboration channels with your business goals.

    However, just using a shared collaboration tool does not guarantee streamlined business delivery. While it may be quick and easy to create a new channel or chat group, creating a culture where collaboration evolves alongside business needs is much more difficult.

    The key to a collaborative culture is trust. Everyone needs to know that the entire team is working toward the same end goal, with shared responsibility for the results.

    Closing Thoughts

    When staff are in heavily siloed teams, they can be easily diverted away from the business goals. Staff might also resist change as they haven’t got the business view. But when teams, systems, and collaboration channels are fully aligned to the business goals, all employees will understand their impact on the business—and measure their success against the business success.

    I hope this insight into DevOps from the perspective of Conway’s Law was useful, and you have some ideas to foster greater collaboration and efficiency in your business.

    — Bronwyn Davies

  • DevOps in the Cloud: Security’s Groundhog Day

    DevOps in the Cloud: Security’s Groundhog Day

    Throughout the internet age, enterprise IT has given security a lower priority than qualities such as speed-to-deployment, scalability, availability and usability. We first saw this in the late 1990s and early 2000s, when IT teams rushed applications onto the web. Amid the glee surrounding the potential of fancy new internet architectures, few thought about the possibility that these applications could become portals into the enterprise for criminals. So, the credo among development organizations was “move fast, deploy fast and seize the future.” There was no concept of risk management, beyond considerations such as availability, scalability and usability. This laid fertile soil for what would become the breach epidemic.

    As we progressed through the years, breaches became so common and massive as a result of the “web evolution” and there became a dire need to manage information security risk. Thus, enterprise security was born. As it evolved, these departments worked closely with IT management, development, infrastructure and business teams to create frameworks that aligned business goals, risk, security and agility. Security and risk management became important parts of development processes and business operations.

    Finally, there seemed to be a path forward to a more secure future … until the future changed.

    Related Content:

    How Developers Can Take a More Proactive Approach to Security

    How Blockchain Tech Can Make DevOps Better

    The future became the cloud revolution, and it plunged us back into the same festering pool of security negligence that we saw two decades ago. Indeed, when it comes to deploying applications in the cloud, security professionals likely can relate to Bill Murray’s character in “Groundhog Day.” Only in the real-life case, they repeatedly relive the same scenario regardless of what they do.

    As DevOps Teams Gain Power, Things Become Less Secure

    Since the emergence of cloud computing, enterprises once again have rushed to take advantage of a new deployment paradigm, and they are prioritizing factors such as scalability, availability and cost-containment, leaving security as an afterthought. As DevOps teams lead today’s much-ballyhooed business transformation initiatives, they gain elevated status within companies.

    Perhaps the most critical development has been DevOps teams gaining control over infrastructure, not to mention development and even corporate purse strings. While these teams follow their own best practices, their processes and protocols often don’t align with their organization’s enterprise security model. Additionally, many cloud providers increasingly are working directly with DevOps personnel, circumventing IT, security and business teams. Not only has this eradicated the interdepartmental collaboration that emerged to manage risk following the first generation of internet applications, but traditional IT and IT security processes have been disintermediated.

    The net result is that, even if a company understands the importance of security, no one team or individual is directly responsible for it because those personnel have been disintermediated. This leaves a gaping hole in enterprise cloud strategies, which is due in large part to confusion related to the shared responsibility model in cloud computing. Many organizations incorrectly assume that cloud platform security equates to application security. But, just because a cloud provider is HIPAA-compliant, for example, doesn’t mean an application deployed within that environment is as well.

    We have critical business applications exposed to a brand-new environment—and security is nowhere in sight. Companies are moving forward with cloud migration without fully understanding what their requirements should be or for what the business, IT and security teams are responsible, paving the way for a second iteration of the breach epidemic. Welcome to “Security’s Groundhog Day.” 

    Securing Cloud Migration and DevOps Processes

    We don’t have to make the same mistakes with the cloud that companies made when they first deployed web-enabled applications in-house. We can break the repetitive cycle by making security and risk management a fundamental part of cloud migration and DevOps processes. Here are a few ways to accomplish this:

    • Know your data. Everyone within your organization has a responsibility to understand what types of data are moving to the cloud on a per app basis, as well as exactly what types of controls are required to mitigate enterprise risk associated with that data.
    • Use a matrix of reference architecture to design secure-by-default solutions to meet varying workload requirements. This will help DevOps teams know when to apply more arduous requirements and when lower risk data needs fewer controls.
    • Verify your plans regularly. Agility requires compromises, and the cloud moves quickly. Without continuous monitoring and rigorous testing, you can’t be sure that what you wanted to build actually made it into production.

    Ending ‘Groundhog Day’

    Many companies are forging ahead with DevOps in the cloud without giving security a second thought, and the resulting consequences are giving other organizations pause. In the movie “Groundhog Day,” Bill Murray’s character had to re-examine his life to break the endless time loop. DevOps would be well-served to do the same: to rethink security and make it a focal point in cloud migration and processes. This is how DevOps can avoid repeating the mistakes of the past. Most importantly, there needs to be a change in the DevOps mindset.

    Security should not be the reason organizations are saying “no” to cloud adoption. When engaged as part of the process, it actually becomes the key to unlock the cloud. Making security part of the process is how you avoid “Groundhog Day”!

    — John Turner