staging-devopsy.kinsta.cloud

Category: Features

  • Busting The 4 Biggest DevOps Security Myths

    Busting The 4 Biggest DevOps Security Myths

    Last month when the IT security world convened in San Francisco for the annual RSA confab, a number of security experts tackled DevOps in talks that covered the crossover between continuous improvement and risk management. Among them was a lively panel mythbusting some of the most prevalent misconceptions about DevOps held by the security community.

    Moderated by Dwayne Melancon of Tripwire, the panel was made up of Josh Corman, CTO of Sonatype, Nick Galbreath, vice president of engineering at IPONWEB, Gene Kim, author of The Phoenix Project, and David Mortman chief security architect and distinguished engineer for Dell Enstratius, all of them security professionals and very active DevOps evangelists. The banter backed-and-forthed between the participants, with four myths bubbling to the surface as particular highlights from the discussion.

    Myth #1: DevOps Pace Will Leave Security In The Dust

    One of the most prevalent myths about DevOps within the security community is that the rapid deploys of the approach will throw all risk management precautions to the wind and leave security practices behind as a result. Quite the contrary, panelists argued. They believe that DevOps patterns make it more possible than ever to bake security practices into IT processes more than before, when it is frequently bolted on “at the end of the caboose,” as Galbreath puts it.

    “You’re not throwing out your current security system,” he said. “What you’re doing is removing a lot of the junk work that eats up so much time and keeps people underwater. You’re really enabling different teams to all improve security together.”

    As Corman explained, security people say all the time that complexity is the enemy of security.

    “It also happens to be the enemy of stability,” he said. “That is the reason DevOps people come willingly with open arms to the security community.”

    Not to mention, Kim added, “because you can’t be highly available and have stuff coming continuously through the pipeline if it is not secure.

    Myth #2: DevOps Destroys Compliance And Control Frameworks

    This myth came up during the audience Q&A portion of the panel, where one audience member from the financial industry wondered at how possible it would be to institute DevOps in a highly regulated industry. According to Mortman, as seemingly chaotic and fast-and-furious as the DevOps pace can seem to the uninitiated, it is actually far more process-oriented than people realize.

    “People say ‘Oh, there’s no process to this.’ But it is actually very process oriented,” he said. “It’s just not very bureaucracy-oriented. There’s not a lot of paperwork. In its very guts, it is very service-oriented and very delivery-oriented.”

    As he puts it, every compliance framework in the planet says you need to log, audit and approve changes. But none of them stipulate a workflow where each change has to be approved one at a time.

    “The workflow can be ‘Please approve the following list of actions that I can do without notifying you first,’” he explained, “and as long as you have an audit trail showing that that has been done properly, I have yet to meet an auditor who will not buy into that.”

    Myth #3: DevOps Gives Developers Too Much Power

    Many dyed-in-the-wool security pros fear DevOps because they believe it is a cowboy environment that gives developers too much power.

    “I’ll be honest, when i first started observing DevOps patterns, developers doing their own deploys seemed immoral to me; irresponsible, reckless, just shouldn’t be done,” Kim said. “No sane person would want to be associated with that, right?”

    The thing was, after Kim dove in and also did a benchmark study with Tripwire several years ago across 4,000 organizations, he found that when comparing groups where ops did the deploys to those where developers shepherded code through deploys, change success rates and mean time to repair were not adversely affected, and in fact, “it is often considered indispensible that developers take code all the way through the deployment pipeline and you know they are getting great outcomes.”

    And from a security controls perspective, the truth is that in a DevOps situation developers “are not logging individually into each server and SSH’ing the code up, untarring it and installing it,” Mortman said. Instead these actions are being done by automated systems.

    “They’re actually going to give you much better timestamps, much more accuracy, and more consistent results and visibility than a manual process being done by someone in ops,” Mortman said. “So, actually, you’re getting better change management out of that system than you are in other situations.”

    Myth #4: DevOps Cuts Important Testing

    One discussion point also brought up during audience Q&A was the possibility that DevOps would somehow excise important testing functionality from the IT process. The panel quickly put that one to bed.

    “You can do bad testing in any model,” Corman said. “If DevOps starts eliminating any kind of testing, its a bad DevOps program. I think that’s orthogonal to some of the benefits or opportunities of DevOps.”

    Kim pointed to Google as an example of how testing fits into the DevOps methodology.

    “So, Google does about 5,500 code commits per day, they have about 15,000 software engineers, and 75 million hours of testing is run each day,” he said. “You can’t get that massive amount of deployment frequencies without a tremendous amount of testing.”

  • Partnership Between Tufin, Puppet Labs Indicates Need for Security Baked Into DevOps

    Partnership Between Tufin, Puppet Labs Indicates Need for Security Baked Into DevOps

    A recent partnership between the IT automation developers at Puppet Labs and the security policy orchestration gurus at Tufin Technologies could start setting the pace for how orchestration of security policies can more easily be baked into DevOps practices. Announced at last month’s RSA Conference, the partnership yielded an integration between Puppet Enterprise and Tufin Orchestration Suite. This will make it possible for joint customers to streamline the configuration and provisioning of security policy changes to iptables, a host-based firewall commonly found on physical and virtual Linux servers.

    “We expect Security Policy Orchestration to become a core requirement for our customers,” says Nigel Kersten, CIO at Puppet Labs. “Integrating with Tufin enables our customers to quickly and effectively address connectivity requirements across the enterprise without compromising security.”

    The integration is an indication of the market’s need for better tools to help DevOps shops speed up the pace of application and network changes while maintaining a strong security and compliance posture. Today’s firewall and security policy changes are often handled in a manual, error-prone process—even at some of the largest enterprises.  A recent survey commissioned by Tufin found that the vast majority of IT professionals reported having to correct 20 percent to 60 percent of security policy changes in their organizations after the fact.

    “This is incredibly useful to our customers and ensures that security is woven into the server’s connectivity requirements,” says Reuven Harrison, CTO of Tufin. “As the convergence of security and network operations continues, this module enables our customers to manage change with automation that delivers tangible business value.”

    According to Harrison, firewalls play a dual role in network operations, both providing security through network segmentation and connecting applications to the network across multiple touch points.

    “It’s the latter that makes them relevant to DevOps folks,” he says. “Firewalls see the both application and the network, which makes them an ideal source of information for modeling applications.  Load-balancers are also good for this.”

    This is why the industry is starting see a convergence between these two technologies, and also why Tufin has pushed for an evolution from firewall management into the sphere of security policy orchestration.

    “Security policy orchestration aids with use cases for firewalls and other network infrastructure as they relate to applications,” Harrison says. “It ensures security policy changes are automated and account for security and compliance considerations not just for network access but also for application connectivity.”

    Harrison believes that while security may not necessarily be the chief selling point for evolving to DevOps practices at some firms, the movement could present opportunities to more closely fold security into a more rapid application release cycle. He believes it will be an imperative in complex cloud environments.

    “As cloud infrastructure matures and gains acceptance from larger and more traditional companies, security will have to be baked into the application release process,” he says. “That means securing not just the application code but also the infrastructure on which it resides.”

    As things stand, Harrison believes that security is typically split into two distinct layers. There’s the application security layer, typically dominated by code security and penetration testing and then network security that is preoccupied by zone segmentation, encryption, authentication and the like.

    “Eventually these two layers, along with some emerging technologies such as virtual networks, will converge,” he says, “and DevOps will handle it end-to-end.”

  • Perforce: The biggest little secret in DevOps

    Perforce: The biggest little secret in DevOps

    Quick can you name a company that is 15 years old, never took a dime of venture money, has over 250 employees, over 10,000 customers and was widely profitable from day one? Until last week me neither. That was when I  was briefed by Dhruv Gupta, Director, Product Marketing and Bob Dever, Director, Corporate Marketing for Perforce.

    perforce customersPerforce is a DevOps powerhouse. Take a look at the customer cloud to the right to get an idea of who some of the 10,000+ customers they have are. Perforce is all about version control. They do that and do it well. Whether you are talking about version control of your code, documents or analytics, Perforce has a solution for you.

    When you think about it, you can see why Perforce is such a DevOps play.  DevOps is all about continuous delivery or at least rapid release.  How can you accelerate your releases without keeping track of the different versions? Without version control you quickly have a mess. Perforce has in some ways enabled continuous delivery and rapid iteration. They truly are the biggest, little secret in DevOps.

    For many years Perforce was really the only choice in town for version control (there were others, but it became a leader in the space pretty quickly) but the rise of GIT, a distributed version control system (DVCS) gave some an alternative. But GIT lacked the enterprise features (read security and team management) that larger organizations needed.  Its popularity among lone wolf and smaller dev teams though was unmistakable. It is easy.

    Perforce has recognized this and has just released a new hybrid version of their solution. This hybrid version allows developers and dev team managers to use GIT and Perforce’s P4 in one hybrid version control platform. A best of both world solutions give DevOps teams the ability to move fast with GIT and have the security and enterprise functionality that P4 allows.

    When you start thinking about DevOps Unicorns like Netflix for instance or Salesforce, the sheer magnitude of what Perforce is managing comes into clarity. In Salesforce for instance you have 3,000 Engineers making 4,000 – 9,000 Code Changes a day with 550,000 Tests / Hour. Keeping versions straight is not the kind of thing you do in a spread sheet.

    It in a Continuous Delivery environment where Perforce really shines. They are making a big bet on enterprises adopting the Continuous Delivery model. However there are some fundamental questions to answer first. According to Perforce though you need to define what does Continuous Delivery mean to you? They conducted a survey:

     perforce cd

     

    According to Perforce they think of Continuous Delivery as this:

    • Deliver working product to users as quickly as possible
    • Every change (check-in) leads to a potential release
    • Give business the option to release – what, when, to whom
    • A change in process and culture

     perforce cd pipeline

    Most importantly though, Continuous Delivery is not just something Perforce thinks is crucial. Waterfall release models are obsolete and even Agile is just a way stop towards Continuous.

     perforce cd arrow

     

    According to Perforce’s own survey they see Continous Delivery attaining critical mass:

     perforce cd adoption

     Is this a case of when you are a hammer everything is a nail or are we at the dawn of the age of Continuous Delivery? Perforce is making a bet that it is. They are already very successful, any more and they may just lose their status as the biggest, little secret in DevOps.

  • DevOps: An Opportunity to Build a Better Resiliency

    DevOps: An Opportunity to Build a Better Resiliency

    Security professionals like to check code for security-related defects before the code is delivered. When they start hearing about multiple builds a day, or even a few builds a week, and a radical decrease in development time along with operational changes, they are bound to give pause. After all, most are having a hard time keeping up even now.

    The industry is moving in this direction. An Evans Research survey of more than 600 software development professionals in the UK and US found that 51% of respondents have already started the practice of continuous delivery across a number of teams. The same survey found that both culture and technology are the primary stumbling blocks, with 53% believing that they have the technology in place for continuous delivery and 59% believing that their staff is ready.

    However, those that do successfully make the move to continuous delivery and embrace DevOps find that the real impact on security will be much brighter than the naysayers predict, according to those  interviewed. In fact, it is automation that underpins continuous delivery and much of the DevOps movement, when done properly, goes a long way to enhance security and system resiliency.

    Security often gets blamed for slowing down the development process, explains Andrew Storms, senior director of DevOps at CloudPassage. But the reality is that it slows it down no more than QA and other development checks. “Those who are at the end of the chain are typically the ones who get the fingers pointed at them. They are the ones who are blamed for slowing down time to market, or delays in getting product in front of the customers,” says Storms “DevOps is an opportunity to integrate all of that together.”

    “One of the interesting things about moving to DevOps is that it requires a very high level of automation,” says David Mortman, chief security architect at Dell Enstratius and a contributing analyst at security research firm Securosis. “Part of the whole continuous deployment model is the move to continuous building and continuous integration. So you need to start writing security-oriented tests that can be automated against that code; when you are running your regression, integration, and other tests, you are testing for security states as well,” Mortman says.

    Jeff Sussna, founder of the IT service innovation consultancy Ingineering.IT, agrees “In continuous delivery and DevOps you start thinking, from a nonfunctional requirements perspective, about continuous integration—and that’s about finding software problems sooner, when they’re easier to fix. That’s what security professionals have been fighting for, for years,” Sussna says.

    That certainly makes DevOps an easier sell to information security teams. “When you suggest to security teams that security scans be completed with every build so that problems are found sooner, my experience is that security folks, who don’t seem to smile a lot, suddenly start to smile,” Sussna says..

    It’s not just talk, “it works,” says Sussna. “When we run those kinds of scans, we find problems earlier, and it’s easier to fix them. When it comes to running through the traditional gates that security puts up, it’s a lot less painful because they tend to run clean within the processes.”

    Another, and perhaps much more subtle, yet deep, benefit, is how continuous security testing keeps security at the forefront of developers’ minds. “Even when a build comes back clean, and the notification says, ‘Security violations, zero,’ it puts in their minds—daily—that security is something they need to think about it. And it’s not confrontational; DevOps puts it within the development process because the developers are seeing the feedback from their reporting tools, and so they are getting instant feedback that they need to fix security issues. Therefore, the feedback isn’t a nag from QA or information security; it’s coming from their systems,” .

    The benefits of DevOps can run still deeper. “Forget the deployment. Forget the automation. Just having this culture of sharing and working together that DevOps fosters, as opposed to being combative, helps to drive excellence,” says Mortman.

  • The DevOps Journey

    The DevOps Journey

    Maintaining a constant thirst for learning is essential for anyone dedicating their career to the IT, computer and networking industries. It’s more than just learning about the latest technologies, rather, it’s imperative we embrace new methodologies and paradigms for doing work, thinking about problems and creating solutions. Enter DevOps, which is a healthy mix of all the above.

    You might find things a bit confusing if you’re new to DevOps. There are many perspectives on what it means. For myself, there were two significant milestones on my journey to understand and embrace DevOps;  1) rethinking how we design large, dynamic applications and manage the underlying infrastructure in cloud environments such as Amazon AWS, and 2) reading the book The Phoenix Project by Gene Kim, Kevin Behr, and George Spafford.

    At first blush, I thought DevOps suffered the “definition bloat” of so many other industry buzzwords (for example, “cloud” comes to mind.) Depending on who you talked to or what you read, DevOps definitions frequently encompassed one or more of the following:

    1. embedding development resources within business units for better alignment and reduced cycle time,
    2. high frequency software releases into production (multiple per day, hour, etc.)
    3. automating infrastructure and system administration tasks vs manual provisioning and sysadmin
    4. embedding operations functions within development organizations
    5. operations performed by developer, even doing away with traditional operations in some cases

    If these seem like a somewhat confusing, loosely assembled list is ideas, don’t fret — they struck me the same way at first too. These ideas actually fit very well together around a few core ideas essential to DevOps.

    Quicker Time To Value – Being Fast and Agile

    DevOps is about changing traditional paradigms, enabling us to more rapidly deliver production updates, new capabilities and improved operations. Outcomes include reducing software releases from months to multiple, micro releases per day, and even hours. To do this requires much more than can be achieved through traditional operational efficiencies. It requires tight alignment, coordination and execution by product, development and operations functions. Embedding multiple disciplines together helps gain this alignment, shorten communication paths, setting priorities quicker, enabling lightning fast coordination. DevOps promotes new practices like embedding development and operations functions within other parts of the business, breaking down long standing barriers between IT / Ops and development / business groups.

    Automated Provisioning, Operations and SysAdmin

    From app development to infrastructure operations, DevOps is about automation. DevOps is about faster software testing and integration, quality improvement, and working in much more dynamic and rapidly changing operating environments. For example, rather than building system images by hand (stick built), automation is used to rapidly generate and spin up new virtual environments on a small or large scale. Systems wake up, check in to learn their personality, install, configure or update code (OS, database, web server, application code revs, etc) and join the network dynamically. This can occur with a single instance or with hundreds simultaneously. Initial upgrades and feature updates can occur on smaller or isolated sections of the network, then are rapidly rolled out across the network, and rolled back quickly if necessary. Sysadmins, be they developers or ops people, use scripts, code and automation tools to manage much larger environments than would ever be possible using manual methods. Automation is an essential differentiator in any DevOps environment.

    Culture of Open Collaboration, Communication and Failing Fast

    DevOps breaks down silos and places a premium on collaboration and communications amongst all players involved. It’s no longer us vs. them or this department vs. that other department. The blame game is gone and the focus is on making work happen, and fast. Communication lines are drastically shortened and conflicts in chains of command are collapsed. It’s about how “we” get things done, no longer about what “they” didn’t do. Failures are viewed much differently. Failures happen and from them we learn and develop newer, better, more innovative solutions to solving problems and prevention problems in the future. Because of the speed DevOps can bring, recovery from failure is much quicker. If a new release into production is causing issues, we quickly flip back to a previous known-good version running on other virtual instances. Updates can also be rolled out much more quickly.

    I hope these and coming views on DevOps are valuable to you as a reader. I know our conversation will help me and the teams I work with, as we learn and grow our expertise in DevOps. Please share your ideas, disagreements and learnings as we move along this journey together into the fast moving and quickly changing world of DevOps.

  • A Q & A with Gene Kim: DevOps and What Separates High Performing Enterprises

    A Q & A with Gene Kim: DevOps and What Separates High Performing Enterprises

    We recently caught up with Gene Kim (@realgenekim) as he was wrapping up the final touches of a draft of the upcoming DevOps Cookbook. We wanted to get his thoughts on what causes IT deployment failure, the DevOp movement, and finally what separates high performing IT organizations from low performing ones. If you’re following DevOps, no doubt you are familiar with Gene. In case you are not, Gene is an award winning CTO, researcher and author. He founded the security firm Tripwire, and was the CTO there for 13 years. To date, he’s written three books, including The Visible Ops Handbook and The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win. Gene is a huge fan of IT operations, and how it can enable developers to maximize throughput of features from “code complete” to “in production,” without causing chaos and disruption to the IT environment. Now, let’s jump into our discussion.

    staging-devopsy.kinsta.cloud: When organizations decide they want to move to DevOps, or continuous delivery, what is a barrier that they immediately run into? And is DevOps something all types of companies can embrace?

    Gene Kim: In my mind it’s the unicorns (leading web app companies) that are sort of showing the way out. Just to show why I’m so interested in this, you know I’ve been studying high performing IT organizations since 1999. What I’ve noticed is that there is this downward spiral that happens in almost every IT organization. It led to more fragile application in production, longer deployment times, building up of technical debts, and the business goes slower and slower. I think one of the reasons that I cared so much about is that it led to preordained failure. Where people felt, especially downstream (operations, test, security) trapped in a system where we were powerless to change the outcomes. No matter what we did, failure has been preordained. I think the way out of the downward spiral is what organizations like Google, Amazon, Twitter, LinkedIn, GitHub, are doing. These organizations are doing tens, hundreds, or even thousands of deploys a day. In a world where most of us are stuck with one deploy every nine months with catastrophic outcomes. I think that the first objection is that we’re not an “Amazon,” we’re not a “Google,’ we’re a bank, we’re telco, we’re a retailer. That it doesn’t apply to them. However, it turns out if you look at who’s adopting DevOps it’s Nordstrom, Walmart, General Motors, SAP. One of the statistics that I saw was, that Google has 15,000 engineers, or developers. HSBC, the European bank, has 15,000 developers. I think every company is an IT company regardless of what business we think we are in. I think it just shows that anything we try to get down relies on IT. So for DevOps not to be a core competency is absurd.

    staging-devopsy.kinsta.cloud: And even increasingly software has to become a core competency, as more devices become API-enabled? All companies are software companies now.

    Gene Kim:  I love that Jeff Immelt, CEO of GE, quote: “Our biggest fear is being disintermediated by software. So I’m going back to school on software and big data.” I imagine that they run into that same objection internally. “Hey, we’re not Netflix, we’re not Amazon.” Right? They are trying to sell the idea of moving to DevOps or becoming more agile. Anyone can do it and benefit from it. Even in general conversations that was the first objection that always came up. Well that’s good for so and so, but we’re not so and so. I think that makes it a belief issue. So that’s what we’re talking about here, right? It can’t work here. I think it a serious of problems and it’s very predictable what the bottlenecks or obstacles are. The first one is that it takes too long to do the deployments, actually you can go a step before that. One of the biggest problems is that operations is always in the way. Whenever a dev or test wants a test environment it’s not uncommon in typical enterprises, that it takes 40 weeks to get one.  Back in my Tripwire days, I use to think that developers didn’t like testing the code, because that’s what I saw. The truth is even worse than that; the developers really want to test their code. Their problem is they can never get an environment when they want one or they need one.  So the survival mechanism kicks in, they’ll always go to the guy they know that’s really good at getting an environment, Their Corporal Klinger or Nash. They don’t want to ask any questions about where it came from, because you know someone is missing a server somewhere. This person probably just stole an environment from another project. With virtualization and infrastructure code there’s no almost no excuse not to be able to create environments available on demand. One of the things that we did, with Puppet Labs, is we reached out to me to help design their DevOps benchmark. This was awesome, when I found out about it I jumped out of my chair because, I saw this as a logical continuation of ten years of benchmarking we had done.

    staging-devopsy.kinsta.cloud: What have you learned that separates high-performers and makes DevOps possible?

    Gene Kim: There were three findings about it that blew me away. The first one is the high performers were massively outperforming their non-high performing peers. Thirty times more frequent code deployments. They were able to complete the deployments from code committed to running a production 8,000 times faster. In other words they could do a deployment measured in minutes or hours. Whereas low performers it took them weeks, months, or quarters. Furthermore, they actually had better outcomes. Twice the change success rate and when they caused outages they could fix it 12 times faster. There were 4,200 that completely responded and it just showed that you could be more agile and reliable at the same time. I think if we just connect some dots, they’re also going to be more secure, they’re probably going to have a faster find fix cycle time. We didn’t actually test that but, if that were true it would mirror what we found in the previous benchmarking work I had done. So that’s surprise number one the extent that they were out performing their non-high performing peers. Second surprise, we found two behaviors that were present in every high performer but they were universally absent in every non-high performer. The first one was that they are they checking in all their production artifacts and version control. Meaning, they could reproduce a production environment based off of what was in version control. Versus only it being an indescribable asset that only one person understood, if they knew about it at all. The second one was is there an automated code deployment process? Which includes being able to create environments available on demand. The next one is the code deployment process. I think it’s very typical in most enterprises that whenever you want to do a new release or deploy it takes days or weeks to complete. Involving tons of hand-offs from the developers, DBA, release team, back to the developers to the security people, and the firewall change and so forth. All that creates delays and is error prone and so forth. In the high performer there usually a one click to deploy or an automatic deploy. If you look at Google, Amazon, Twitter, Facebook, in general they have a continuous deployment process so that when a developer checks in code it automatically runs all the tests. Then it puts it into an integration test environment and runs all the tests there. There might be some manual work, but then in general it automatically migrates to production. Clearly, that’s what’s required to do multiple hundred of deploys a day.

    staging-devopsy.kinsta.cloud: That is incredible to learn, Gene. A lot of fascinating findings. Thanks so much for your time.

    Gene Kim: Thank you, George. Always a pleasure speaking with you.

  • Is DevOps Ready to Cross the Chasm to Mainstream?

    Is DevOps Ready to Cross the Chasm to Mainstream?

    There are just a handful of business books that I have read that have had a profound influence on me.  One was Gene Kim’s “The Phoenix Project”, another was Eliyahu Goldratt’s “The Goal”. A third one was Geoffrey Moore’s, “Crossing the Chasm: Marketing and Selling Disruptive Products to Mainstream Customers”.  Even though written in the early 90’s, it is as relevant today as it was then.  Though our technology has changed (dare I say advanced), the adoption curve of who uses it has remained static.

    When I look at the world of DevOps today, to me it is on a classic chasm model.  The real question is will it or rather how quickly will it make the hop over the chasm into mainstream. The picture above is a great representation of the chasm. Most technologies go through the enthusiast period, moving to the visionary.  I would say that DevOps has been in this visionary period for a few years now.

    In order for any technology to jump the chasm it needs to build up enough momentum and critical mass to successfully make the jump.  Now there are some technologies that never build up enough critical mass to successfully jump over to the pragmatist’s cliff of mainstream adoption.

    Back in my StillSecure day’s NAC had a tough time making the jump. It was not until mobile technology and BYOD served as ignitions that NAC made the cross-over to mainstream.  But enough about NAC lets return to DevOps.  For DevOps the Cloud and virtualization have been ignition points.  Using software to replace hardware has allowed for rapid deployment, automation and continuous delivery models that characterize DevOps.

    With so many pundits and analysts predicting that this is the year Cloud goes mainstream (I thought it did already), DevOps will not be far behind. As disruptive as Cloud is, could it be as disruptive if not for DevOps? There is a symbiotic relationship between the two.

    But it is not just Cloud alone. DevOps has other boosters that will help it over the chasm. SDN is another big boost.  Software will eat the world according to Marc Andresseen. It may very well start with networking hardware.  But software is eating more than hardware.

    One of the hottest trends in security (and one you will see featured here in the SecOps neighborhood of blogs on staging-devopsy.kinsta.cloud) is Software Defined Security.  It may sound like marketing speak to you, but it involves putting security right into the virtual, software stack.  It is DevOps for security.  In fact DevOps and security is something you are going to hear more about from the likes of Josh Corman, Dave Mortman and Dwayne Melançon and others.  Their “Rugged DevOps” discussion is ripping through the security industry for the last year or more.

    All of the above aside the real key for DevOps to making the leap is that instead of just enthusiasts and visionaries being involved, what Moore calls “the Pragmatists” get involved.  These pragmatists are not hung up on labels or new versus old.  They just want to get the job done. If DevOps will help them get the job the done, that is good enough for them.

    The pragmatists make up a good chunk of the mainstream market.  The silent majority if you will.  To win these pragmatists over DevOps just has to show that its benefits are real, quantifiable and worthwhile. The good news is that the Pragmatist are the first group on the other side of the chasm.

    Having the Pragmatist as the first part of the mainstream is a good thing, because after that it gets harder to gain traction.  According to the Chasm model after the Pragmatists come the Conservatives.  Though not quite reactionary, this group doesn’t like change. They won’t change until they are nearly forced to by business pressures. They will resist adopting DevOps just because it is different than what they do now.  Generally technology has to be pretty mature, its benefits well established before this group will consider doing anything more than a small trial.  With DevOps representing as much a change in philosophy and culture as it does new tools, this is a really bitter pill to swallow for the Conservatives.

    I would think we are years away from DevOps making real traction into this group. Especially when there are still some folks who claim that DevOps is DOA, DevOps is much ado about nothing, that it is doomed to failure. But hey there are people who still say that the science doesn’t point to climate change and that we should teach creationism in schools.

    Call me the eternal optimist though. I do think that as Cloud and the other catalysts behind DevOps continue to expand, DevOps will be swept up even into the conservative base.

    Between Conservative and Pragmatist, a large slice of the market is spoken for. Added to the pre-chasm market and DevOps is well beyond critical mass.  Whether the last segment of the market, the skeptics ever adopt is not even relevant.  Many of these skeptics have a bone to pick. As I have grown older I am convinced that they are just wired to be skeptics.  They literally look at everything through a half empty glass.

    So there is little doubt in my mind that DevOps will not be a stillborn technological movement. It has the catalysts, the benefits and most importantly timing and circumstances are lined up to help push this from the early adopter crowd and right over the chasm into the mainstream. What’s more is I think this is already under way and will be much quicker than most people think.

    This is all a big reason why we launched staging-devopsy.kinsta.cloud.  There is a big enough critical mass now to support a site dedicated to DevOps like this.  But as we cross the chasm to mainstream, we want to ride the crest of the wave, helping to bring DevOps to pragmatist, conservative and anyone else who would benefit from it.

  • Let the Code do the Heavy Lifting

    Let the Code do the Heavy Lifting

    One of the first jobs I had when I started in IT was as the IT administrator for a dot.com startup. IT admin at a smaller company like that was typically an umbrella term used to describe a one-man IT department, which is essentially what I was.

    The company was growing exponentially—both in terms of customers and revenue, as well as traffic to our website. What that meant for me was a steady stream of UPS shipments from Dell, and a virtually full-time job of mounting servers on racks in the data center, getting them properly configured, and deploying them to the network.

    Dealing with 50 or 100 hundred servers in 2000 was a daunting task, but dealing with hundreds or thousands of virtual servers in 2014 is—for all practical purposes—impossible. That is, it would be all but impossible doing things the old-fashioned way. However, the DevOps revolution changes the rules of the game because it uses code to do the heavy lifting, and automate those tasks which can be automated so the IT admins of today are free to engage in more productive tasks.

    If we were still dealing in physical servers, the benefits of automation that you get in a DevOps environment would still not be possible. What allows so much of the traditional “racking and stacking” today to be automated is that we are dealing primarily with virtual servers. Although a virtual server may act like a server, and fulfill the same duties as a physical server, it is essentially just a software program. The fact that a virtual server is comprised of nothing more than code means that virtual servers can be created, configured, and deployed automatically through code as well.

    It’s a good thing, too. The fast pace of a DevOps culture would be challenging to maintain without the efficiency of automation. The ability to create, provision, destroy, or move virtual servers around on a whim is part of what facilitates agile development and accelerated execution by DevOps teams.

    There are also some adverse consequences of automation. People have lost their jobs in manufacturing plants as machines have come along to automate tasks they once performed. There is a risk that automating many of the routine tasks normally delegated to IT personnel could result in fewer people being necessary, but that is not a reason to not embrace automation. Automation enables people to develop more advanced skills and focus on more important tasks—those functions that can’t be easily automated.

    One of the primary aspects of functioning in a DevOps environment is a faster development cycle, and more frequent deployments. Traditional organizations might deploy updates once a quarter, or once a month, but DevOps teams often deploy updates at least daily—sometimes multiple times per day. Automation reduces the possibility of error—it is a simple, repeatable process that works reliably and predictably.

    A survey of DevOps Conference attendees published by JumpCloud and SoftLayer found that about 75 percent of the respondents use a configuration automation tool such as Puppet or Chef. More than half use scripts of some sort to execute tasks, and about one-third reported using a patching tool of some sort. Many of those surveyed admitted that they have the tools they need, but there simply isn’t enough time to get things done.

    Perhaps that explains why 82 percent of survey respondents plan to automate more. Breaking the data down a little farther, nearly a third are looking to increase automation of configuration and deployment, and about 10 percent want to automate user management, and a variety of tasks like log management, scripting, testing, patching, and monitoring all came in around five percent or below.

    It’s still possible to create virtual servers, and configure them manually one at a time. That may work if you’re bored, or nostalgic for the “good ole days” of IT, but it’s impractical, and frankly just dumb. It’s also possible to cut your lawn with a pair of scissors, or paint your walls with a Q-tip, but it is highly inefficient and ill-advised.

    Don’t waste time and resources performing tasks that can be automated. Identify the functions you routinely perform that are consistent and repeatable, and embrace automation to take care of those things for you so you can focus on more important things.

  • Ops is dead, long live DevOps

    Ops is dead, long live DevOps

    Yes, that’s a bold statement, but if the vibe at the recent AWS re:Invent conference was any indication there are a number of methodologies, technologies, and companies working to make that a reality. This article isn’t focused on whether this is a good or bad thing (my personal belief is that ops will always be a critical part of the IT delivery process). Regardless, it is critical for IT operations personnel to understand the implications of this tectonic shift. As Marc Andreessen, founder of the venture capital firm Andreessen Horowitz, has said, “software is eating the world.” As a result, ops professionals must have the ability to automate tasks that used to be managed manually. Whether or not you think that is the death of ops as we know it I will leave to you, but it does mean we will live in a world where DevOps will offer distinct advantages.

    Before we dive into what this change means to ops personnel, let’s review the underlying assumptions. The fundamental driver at the a high level has been the DevOps movement. Yes, DevOps is quickly becoming the buzzword dejour, but the underlying principles are lasting and changing IT to its core. DevOps is about integrating the various functions that it takes to release software or applications: software development, quality assurance, operations, and security among many others and rapid iteration. The process of integrating these tasks is accomplished through new processes and automation. Different functional areas such as development and operations are brought into the process earlier, and their processes are automated to the extent that they can be. This results in fast cycle times, increased quality, and hopefully increased innovation for the business. All of this is accomplished through software – at each functional level.

    Some technologies that have enabled this massive shift include Infrastructure as a Service (IaaS), Platform as a Service (PaaS), automated configuration management, server management tools, virtualization, and software defined networking. IaaS and PaaS offerings are eliminating the need for organizations to procure, provision, and deploy servers. Further, this class of technology is also dramatically reducing the need to deal with the network layer. This allows organizations to concentrate on their applications and development, and focus less on their infrastructure. The amount of time and effort spent on server and network configuration is minimized. Configuration management solutions are enabling the automation of server setup. An organization’s technology stack can be pushed, pulled, cloned and copied versus manual configuration thereby saving time and adding consistency. Top performing organizations employing DevOps go a step further and automate the scaling of capacity based on utilization. To reduce day-to-day operational activities and costs, organizations are also automating the management of servers, including users access, patches, security, and other areas through both traditional written code and SaaS solutions. Another area changing the game for ops personnel is software defined networking (SDN), which is effectively morphing data center and hardware functions into software programs.

    In short, DevOps and associated innovative new technologies are driving a paradigm shift in the way operations is done. No longer are ops personnel required to manually provision, configure, and manage the network, servers, and application stack. Combining technologies from organizations such as AWS, Rackspace, and SoftLayer along with configuration tools such as Chef and Puppet drives the provisioning and configuration of applications through software automation rather than manual effort. Once applications are deployed, the underlying infrastructure can be managed with tools such as my own company, New Relic, this author’s own JumpCloud, and others. Ops personnel have more companies than ever focusing on helping them automate the tasks that they need to deliver applications for their organizations.

    The result from all of this effort is that ops as we know it won’t exist in the future. The network operations wizard that could manually troubleshoot routing and switching issues, provision servers quickly, and tweak configurations will be tasked with automating those tasks through software. Some of that automation will be through code that ops person writes herself, some delivered by third party providers, and other portions delivered by open source or SaaS-based solutions. But, don’t shed tears for the forced obsolescence of the IT ops people. They will still exist and perhaps be more important and powerful than ever. In fact, because of DevOps ops personnel will in essence be set free from mundane, repetitive “busy work” to perform higher value tasks as well as managing over the software and systems at work. The black magic of enabling different layers of the stack to function together will be less about art and more about science, or rather good software. Solutions will be engineered to work together and be tied together through code. Caveat: yes, service providers and some large enterprises will  still run their own data centers and networks, but the same reasons that DevOps works in IaaS or PaaS will apply to private data centers or private clouds as well.

    If all of this trends toward being true as we see it happening today, what are the opportunities for ops personnel to be at the forefront of this shift? Beyond embracing DevOps – which many operations personnel are doing already – learning to code is the most critical step that ops professionals can take. Many ops professionals already know how to script which is absolutely a minimum, but many will need to go further. Embrace the DevOps movement and push further towards the development side by learning how to code. There are any number of amazing services to teach us how to code or code better. Take the step. It may be the difference between a vestige of the old school versus driving the new world in your organization. The second significant step you can take is to automate your job. Leverage the very best automation tools to code your network, server, and application infrastructure. Not only will you spend less time on these tasks manually, but you will increase availability, reliability, and performance. In most organizations, that’s more than a gold star. Finally, make the mental shift that operations in the future will be less art and more science. DevOps is driving action through data, and the technology stack is responding to this demand with more telemetry than ever. Problems are now solved through understanding performance, availability, and other metrics. And code.

    DevOps is the future. Automation is key. Code is the answer. Developers and ops are in it together for the good of their organizations. Ops pros that embrace this will win! Ops is Dead, Long Live DevOps!

  • DevOps’ Chicken Or Egg Origins

    DevOps’ Chicken Or Egg Origins

    When acolytes explain the DevOps movement to the uninitiated, they usually run into a chicken and egg line of questioning. What is DevOps, really? Is it a culture? Is it a pattern of processes? Or is it a technology? What comes first?

    The ratios of the perceived importance of each of those elements may shift depending on who you talk to in the DevOps community. But the inevitable answer to these questions is usually that DevOps depends on at least a little of all of them, but probably on culture above all else.

    “I think it’s first and foremost cultural. The development team is a tribe. So is operations.  They have norms.  They have things they value.  They have things that they reward and admonish,” says Josh Corman, chief technology officer for Sonatype, a component lifecycle management firm. “Developers are incentivized to cause change.  And ops people are incentivized to prevent change, to promote stability.  So they were natural enemies.  And when they work it out and align to each other’s tools and incentives, they’re able to go faster.”

    Therein lie the historical roots of DevOps, agrees Andrew Storms, senior director of DevOps at cloud security firm CloudPassage, who explains that it is an answer to the blame game that inevitably occurs in the old model.

    “I think that existing corporations or companies or groups are probably doing DevOps in response to existing pain,” he says, explaining that in previous non-DevOps shops he worked at, it was inevitable that development would write a piece of code, throw it over the fence and let ops catch the heat when trouble brewed. “The question is, why is the person at the end of the train always feeling the pain for something that originated at the beginning of the train? To me, DevOps says, ‘There is no train.’ There’s no more division of who’s in trouble for anything. Everybody is on the same darned team.”

    It’s for this very reason that Jez Humble, a principal for the consultancy ThoughtWorks and co-author of Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation, says that the fundamental ingredient to successful DevOps is trust.

    “You need to trust each other. Developers can’t just assume the ops people are all dumb and don’t know anything about code or are just command line monkeys. And the ops people can’t assume the developers don’t know anything about how systems work in real life,” he says. “These assumptions make it all a big finger-pointing exercise.”

    A lack of trust is the underlying roadblock to better teamwork between dev and ops, but it can be difficult to achieve within enterprises.

    “In small organizations people trust each other because they can see each other and work with each other and must learn to trust to get anything done,” he says. “But as organizations scale you lose that human contact.”

    It’s replaced instead with brittle and bureaucratic processes that only feed into those distinct walls between each tribe. However, as organizations feel the pain that’s inflicted by those bureaucracies that have evolved so far in IT and by the walls thrown up as a result, they face a choice. Engender that trust and a culture that allows  dev and ops to simplify processes and tear down the walls, or face further roadblocks that will hurt the business.

    From there, it is a matter of establishing behavioral and process patterns that are a hallmark of DevOps cooperation.

    “I think ‘patterns’ is the right word for it. People get hung up on this textbook or technical definition of DevOps.  And while there are certain common processes and things like Kanban boards and Scrums, and sometimes it’s a hybrid of Agile, sometimes not that ,” Corman says, “what matters the most is an attitude to fail fast and iterate, to continuously look for ways to increase flow, to automate, instrument and find and root out weakness. And to be responsive to new ideas of the business, or new ideas of your install base, or new market pressure.”

    According to Corman, the beauty of DevOps is that it’s a ‘jump ball’ for IT, so there are no set processes that make or break whether a shop is DevOps or not.

    “It’s in a state of flux, and the cement hasn’t dried.  It’s a singular opportunity to redefine the way we engage with the business and drive value, the likes of which we will not see again for 30 years,” he says. “It’s very, very rare.”

    And while few within the DevOps community would say it is defined by technology, it certainly is enabled by it.

    “To me it is more a cultural shift than a tool shift, but that tools shift presents a huge change in the way IT runs,” Storms says, explaining that instrumentation and orchestration tools like Puppet and Chef makes it possible to achieve continuous improvement and fast iterations. “So does it come about through the cultural change or the technical change? I think, actually, neither. I think it happens at the same time. That’s the force multiplier, that two plus two equals five kind of thing.”

    But like tectonic shifts, changes in IT happen sometimes all of a sudden, but mostly in increments.

    “One part of it is focusing on what we can do every day to make things a little better for everybody. You have to proceed incrementally and accept that it’s going to take time,” Humble says. “You can’t say, ‘Well, we’re going to adopt Agile now across the whole organization—go!’ There has to be enthusiasm from the people on the ground to do this stuff and then you have to work together though a process of experimentation and trial and error to work out how to make it work in practice, probably with a small group of people initially.”

    Even with that force multiplier, Storms says, you’ll still see positive steps forward by establishing an element of DevOps one at a time.

    “Even if you’re in a difficult situation where you can’t have the force multiplier of culture and technology changes at the same time, just one or the other is still going to be a big win,” he says. “Let’s say you just took the one piece of ‘Let’s move our dev and ops teams together and make them be friends,’ you don’t necessarily have to orchestrate and instrument like crazy to win. But you can also do the opposite and just picked the orchestration piece and say, ‘Hey, ops people, stop doing things by hand.’ You send them to Chef conferences and make them learn Ruby and guess what? Their skill sets just went through the roof.”