Category: Features

  • The 2016 DevOps Dozen Winners Announced

    The 2016 DevOps Dozen Winners Announced

    In what turned out to be a much bigger job than we anticipated, we have gone through the numbers and checked them twice (actually more like 20 times). All of us here at staging-devopsy.kinsta.cloud are happy and proud to announce the winners of the 2016 DevOps Dozen!

    With thousands of votes cast in just the finals alone, it was a daunting task to verify votes and select the winners in each of the 12 categories for awards this year. Also several of the categories had incredibly close races with just a few votes separating the winners from the other finalists. But there can be only one, winner in each category that is.

    So without further adieu, here are the winners of the 2016 DevOps Dozen awards:

    Best DevOps Open Source Project: Jenkins

    The DevOps world is chock full of great open source projects. Many of them have huge communities who are  very passionate and involved. The Jenkins community holds its own with any of them. More importantly, Jenkins seems to be at the center of the DevOps/CD  world.

    Also congrats are in order for CloudBees who while not managing the Jenkins project do a great job of shepherding and nurturing it.

     

    Best DevOps Commercial Solution: Automic

    Another extremely competitive category with so many great finalists. Automic was the choice of the voters here and I guess to CA Technologies as well, who just closed on the acquisition of Automic last week.

    We can’t move on without acknowledging and congratulating the other finalists in this category as well as the bevy of DevOps tools and tool vendors in the market though.

    Best DevOps Company Executive: Barry Crist, CEO, Chef

    Well deserved recognition for one of the leaders of our industry. I still remember when Barry was first announced as the CEO of Chef, as they embarked on a major rebranding effort, shedding OpsCode and firmly moving to an enterprise solution path.

    Along with CTO Adam Jacob, Chef continues to lead with innovation in both product and go to market. In many ways Chef represents the best of the DevOps community and Barry has a lot to do with that.

    Most Innovative DevOps Solution Provider: Ranger4

    OK I have to admit it, I was pulling for Ranger4 in this category. Helen Beal and Steve Green are great people who were out there evangelizing and leading in DevOps consulting way before the current Cambrian explosion of DevOps consultants we see today. Their DevOps Liftoff Workshop and their latest, DevOps for CEOs, are both high water marks in the industry. Congrats to the entire Ranger4 team.

    Make no mistake though that DevOps consulting is a rapidly growing market with many different solution providers out there. Ranger4 on top of their a first mover advantage continues to blaze a path that others follow.

    Most Innovative DevOps Solution: IBM Bluemix Garage Method

    There is no shortage of innovation in the DevOps space, but what IBM has done with the Bluemix Garage Method is really ingenious. It is not software or hardware, it is not SaaS, it is truly a methodology.

    The method takes IBM’s lessons learned during their own DevOps transformation, combined with tools and processes, to form a complete blueprint that allows organizations to “innovate like a startup, while scaling to the enterprise.”

    Top DevOps Evangelist: Jason Hand, VictorOps

    Again a very close margin of victory here, but a well deserved accolade for evangelist/author Jason Hand of VictorOps.  Luckily, for VictorOps they don’t pay Jason by the mile as it seems he is constantly on the road. I don’t remember the DevOps Days or other DevOps event where I didn’t see Jason there. Somehow in between all of this travel, Jason also found time to write a book, ChatOps.

    We are blessed in the DevOps field to have many worthy recipients for this award, but kudos for Jason for being the people’s choice here.

    Successful DevOps Transformation: Nationwide

    Many of our stories here on staging-devopsy.kinsta.cloud, as well as just about every presentation at Gene Kim’s DevOps Enterprise Summits are about successful DevOps transformations. Name the vertical and there is probably a DevOps transformation story you have heard from one or more organizations in that vertical.

    Nationwide is a DevOps transformation story that we here at staging-devopsy.kinsta.cloud are very familiar with. Primarily through the efforts of Carmen DeArdo who is one of the leaders of that transformation. We have interviewed Carmen a number of times and for me the way Nationwide has rolled out DevOps in a deliberate manner across the organization and still does so today is truly a model to be emulated. Congrats to the Nationwide DevOps team. When it comes to DevOps, “Nationwide is on your side (sorry couldn’t resist).”

    Best New DevOps Solution Company: DBMaestro

    Winning the “rookie of the year” in an industry that is itself still immature is no easy task. There seems to be a new DevOps company coming out of the woodwork everyday. But DBMaestro has managed to stand out from the crowd. Pioneering in the “DevOps for the database” market, DBMaestro has gained attention not just with a great product/service, but also by producing high quality content to help evangelize and educate the DBA market about DevOps.

    Of course the thing about winning best new company is now the pressure is on. We expect big things from DBMaestro going forward. Can’t wait to see what they do for an encore.

    DevOps Transformation Executive of the Year: Angel Diaz, IBM

    Dr. Angel Diaz is a charismatic, dynamic leader at IBM. He is VP of Cloud Architecture and Technology. In that role Angel’s team pioneered IBM’s BlueMix platform. Hearing the story how Bluemix came to be is a great story of innovation and entrepreneurial inspiration.

    Angel and his colleagues were able to move “Big Blue” and teach the elephant to dance by adopting Bluemix. Not just the Bluemix platform, but also such innovations as the Bluemix Garage and the Bluemix Garage Method. Dr. Diaz is not afraid to take chances and innovate to keep IBM at the forefront of technology. This award is well deserved for leading a massive DevOps transformation that has not only made IBM a leader in offering DevOps solutions to market, but also having IBM eat its own DevOps dogfood.

    Best DevOps Presentation of the Year: Better, Faster, and Cheaper. How? by Damon Edwards & John Willis

    Again, how blessed are we in the DevOps space to have such a wealth of great presentations to pick from? Whether they be slides from webinars or video from the many, many conferences around the world, there are some really great presentations that inform and educate us all.

    One presentation stood out among them this year. It happens to be one that I was lucky enough to catch while in London for the 2016 DevOps Enterprise Summit. Damon Edwards and John Willis (who literally had just stepped off a redeye from the US and rushed over from the airport) delivered what I have come to call “2 out of 3 ain’t bad, but why settle”. To me it is the best way to show what the advantages of DevOps. Why you can have both better, faster and cheaper (and might I add more secure).

    Of course Damon and John have been delivering killer DevOps presentations for years now and this is just the latest in the line. Both John and Damon are very deserving of this award and congratulations to them.

    Best DevOps Book/eBook of the Year: The DevOps Handbook, Gene Kim, John Willis, Jez Humble and Patrick Debois

    While there are several great DevOps books that came out last year, one really was in a league of its own. The highly and long anticipated companion/sequel to the Phoenix Project, The DevOps Handbook has lived up to the expectations of the industry. Gene Kim assembled a DevOps superband of rockstars to help him as co-authors and they delivered a great book.

    The DevOps Handbook is a collection of over 40 case studies broken into several categories. It claims to show, “How To Create World-Class Agility, Reliability, & Security in Technology Organizations” and it succeeds on every level. The DevOps Handbook will be a required component of DevOps collections for years to come.

     

    Best staging-devopsy.kinsta.cloud Article of the Year:  5 Things DevOps is NOT by Shelbee Smith-Eigenbrode

    While not the story with the most views, 5 Things DevOps is NOT struck a chord with our readers. It was one of the most widely commented articles of the hundreds of articles we published in 2016. Additionally, it has a long tail in social media with comments on Linked In, Facebook and Twitter as well.

    Shelbee Smith-Eigenbrode hasn’t written a lot of articles for us here at staging-devopsy.kinsta.cloud (hint, hint), but she brings a real life view of DevOps in the field that she has learned in her role at IBM. It is this hands on view that I think allowed her to capture the sentiments in this article that so many identified with.

    Congrats to Shelbee and hope this encourages her and you to consider contributing your own thoughts on DevOps to us here on staging-devopsy.kinsta.cloud. We are always looking for quality content. You can write us at editor@devops.com with your thoughts on contributing your own article of the year.

    Well there you have it. A dozen winners in the 2016 DevOps Dozen awards. Congrats to all of our winners and also congrats to all of our finalists. This was such a strong class of finalists.

    The trophies are already on their way to the winners and we hope to see pics of them on the desks of the winners soon!

    The DevOps Dozen for 2017 nominations will start in August so stay tuned for details if you are interested in throwing your hat in the ring for next year.

    — Alan Shimel

  • 5 Ways DevOps Can Help Improve IT Security

    5 Ways DevOps Can Help Improve IT Security

    You already know DevOps can make software delivery faster and more efficient. But did you know it can help to keep software more secure, too? Here’s how.

    DevOps refers to a set of practices and cultural priorities related to the way software is designed, produced and deployed. It emphasizes collaboration between developers (the people who write code), IT Ops (the folks who manage software in production) and everyone in between—such as software testers and designers.

    When done properly, DevOps greatly reduces the time it takes to bring software from idea to implementation to end user delivery. It also adds efficiency to the software delivery process in many ways. It allows different team members to work in parallel, for example. It also ensures that coding problems are found early in the delivery pipeline, when fixing them requires much less time and effort than it does once a bug has been pushed into production.

    DevOps and Security

    But making software delivery faster and less costly isn’t the only thing DevOps does. DevOps also helps make software more secure.

    It does this in several ways. Consider the following:

    • Visibility. DevOps encourages constant communication between all members of the IT organization. That maximizes each team member’s visibility into the software being developed. In so doing, it places as many eyeballs as possible on the code, at all stages of delivery—which maximizes opportunities for your staff to find security flaws before code is released into production.
    • Automation. In the DevOps world, software delivery tasks are automated as much as possible. With automation comes consistency and predictability—and a much lower chance that human error during a manual process could introduce a security vulnerability.
    • Fast updates. DevOps enables a much faster pace of software delivery. That means that security bugs (and bugs of all kinds, for that matter) can be squashed very quickly by rolling out a fix.
    • Tool-agnosticism. With DevOps, you can use whichever tools or programming frameworks you like. You’re not bound to particular platforms or vendors. That’s important from a security perspective because it empowers you to adopt the tools that are most secure for your needs—not the ones you’re locked into because you have an outdated, inflexible software release process.
    • Agile technology. While DevOps doesn’t require you to use any specific type of tool, DevOps teams tend to favor next-generation architectures and technologies, like microservices and containers. These help to make apps more secure by reducing attack surfaces and enabling quicker reaction. If you deploy your app using containerized microservices, it becomes harder for attackers to compromise your entire app, because an attack against one microservice doesn’t give them control of the other ones.

    DevOps isn’t a silver bullet when it comes to cybersecurity, of course. Keeping your systems and data secure requires careful planning on a number of fronts and implementing a variety of tools.

    But DevOps—especially a highly automated, DevOps-based release process—should be one important weapon in your cybersecurity arsenal.

    — Chris Tozzi

  • What Do We Do When Everything is Automated?

    What Do We Do When Everything is Automated?

    Recently, I came across a study the caused me to think beyond the day-to-day world of code-slinging in which I live. According to the study conducted by Center for Business and Economic Research at Ball State University in Indiana:

    “Almost 88 percent of job losses in manufacturing in recent years can be attributable to productivity growth, and the long-term changes to manufacturing employment are mostly linked to the productivity of American factories.”

    Just translate the term “productivity of American factories” into the word “automation” and you get the picture. Other workers are not taking jobs away from the gainfully employed, machines are.

    This is not a new trend. It’s been going on since before Eli Whitney invented the cotton gin. Industry creates machines that do the work of humans faster, cheaper, with more accuracy and with less failure. That’s the nature of industry; nothing new here. However, what is new is rate by which the displacement of human beings from the workforce in happening.

    The Bureau of Labor statistics projects that by 2024 the U.S. economy will need 50 percent less car mechanics, 42 percent less telephone operators, 26 percent less postal clerks and 20 percent less parking enforcement officers. Or, to put it another way, cars will be better at self-repair, call routing technology will get better at conducting verbal conversations with humans, self-service postal machines will get better at taking care of human transactions in the post office and parking meters will know how to write tickets.

    Here’s another another point to ponder. As the quality and reliability of self-driving vehicles improve, it’s not far-fetched to imagine self-driving trucks becoming the norm on the nation’s highways. How many truck drivers will be displaced once self-driving trucks are everywhere?

    Also, think about this: As Uber and Lyft mature in the business landscape and self-driving cars become commonplace, the ride-share companies will replace human driven vehicles with self-driving vehicles. Then, Uber or Lyft will start to offer a pricing model that is flat fee for 24/7 service with unlimited mileage. So, to keep it personal, imagine Uber/Lyft offering me a deal in which I pay $150 a month to have all my driving needs taken care of. Or, I can continue to pay the $400 it costs me for my car lease and insurance coverage (yes, I lease a modest vehicle). Which way do you think I will go? Spend $400 for the pain of driving in a car in LA traffic or spend $150 to get where I need to go while I read a book in the back seat? Me? I’d fork the car over in a New York minute.

    As if this isn’t startling enough, try this one out: Walk down your street. See all those cars parked on the side of the road. In the near future, using the Uber/Lyft automation scenario, they won’t be there any longer. Any car made will be used 24/7. All this translates into less cars doing more work. And less demand for cars means less cars being made. Less cars being made means less people working making cars. You get the picture.

    What about software? Let’s take a scenario that is part of my day-to-day coding world, logging. In the old days, you’d log to a file on disk. Over time, that log file got so large that a human would need to move the file manually onto an archive disk so as to not eat production space. Somebody comes along and says, “Hey, I can write a piece of code that watches the size of the log file and moves it automatically when the file gets too big.” So much for the person doing the manual file move. But, still, you had to keep adding capacity to the archive machine, which typically meant a human walking down to the server room and twiddling with some hardware. Then a someone has a great idea, “Let’s use this thing called cloud storage as a back-end for a log collection service. We’ll leverage the elastic capacity features of cloud storage and write out logs to that service. Hell, why make a service? We’ll subscribe to an existing one and save the development costs.” Thus, no more humans needed to monitor capacity, move files, write incidental scripts—any of it. In terms of logging, a bunch of automation is doing the work a lot of humans used to do. Is this progress? Might be. But the trend is unmistakable, so much so that I have no problem imagining a future in which 90 percent of work that used to be done by humans is done by low-cost, automated, robotic, labor.

    We’ve always been on our way to automating everything. The only difference between then and now is that now we are really, really close to full automation. Asking the question, “What will we do when everything is automated?” is no longer fantastic, it’s essential.

    So what will we do?

    “Simple,” you say. “We’ll retrain the displaced workers.” Really? Many of us in the software game know that notion of retraining is a joke. If you are not constantly learning the new stuff, you are out of work. For a knowledge worker, continuous learning is an ongoing requirement for commercial viability. And, the baseline from which you are learning is very high. In other words, you need to know a lot to get the basics of how to write a deployment script in Ansible that runs against AWS. And, even as you are learning that basic technique, somebody is already creating a technology that will make the technique you are learning obsolete.

    So back to the question, what do we do?

    First, let’s do the obvious. When near total automation has the result that there is no longer any work for most humans to do, we need to come up with other means of survival. Survival in the modern world means having money to live on. Charles Murray suggests in the Wall Street Journal that we plan to implement Universal Basic Income. Finland is starting a pilot project based on the idea. Other countries are experimenting too. Universal Basic Income is a program in which citizens of a state are provided with a base income on which to live. Some programs vary basic income according to income produced otherwise. The common thread is an understanding that jobs are not plentiful and that depending on one for survival is not realistic.

    The notion of Universal Basic Income is not that new. It’s been around since the 1960’s, proposed by folks such as Milton Friedman as the negative income tax.

    Second, we need to address the elephant in the living room: the birth rate. In the old days when agriculture and small, private farming was a major industry in the United States, having a large family made sense. You needed your family members to work the land. As the Industrial Revolution created demand for factory workers regardless of age, large families still made economic sense. But now the trend is that we need less people to do more. Thus, having large families becomes counterproductive to the economy overall. Yes, there are certain sensibilities out there that encourage large families and adherents to such thinking have large families as a matter of choice. Still, many families end up with large families for reasons other than intention. It just sorta happens.

    Thus, to ensure that humans have a chance at a viable existence in a fully automated world, implementing effective family planning policy and practice becomes as useful to the society of the near future as the anti-smoking efforts that started in the 1960s does today. I am not advocating draconian, government mandated family planning. We cannot legislate family size anymore than we could smoking behavior. Rather, we need to create a social condition in which the wisdom of the idea prevails.

    Finally, we need to address the need for human dignity in a fully automated world. For better or worse, we need to accept that in the near future the traditional relationships between demand, productivity and income no longer may be applicable. The old days of human workers making stuff and getting paid for their product or labor are coming to an end. Yet, one of the intangible benefits of the old days was the dignity and satisfaction that went with pulling your weight and earning your keep. Being able to provide for yourself and your family defined your worth in your own eyes as well as those around you. This need for dignity will remain no matter how the technical and economic landscapes evolve.

    When I started to doing the research for this piece, I asked others in the know to respond to the prospect of a world that is fully automated. One the people I talked to was Phillipe Kahn, founder of Borland, Starfish Software, LightSurf Technologies and most recently, Fullpower Technologies. Here is his response:

    “I remember being in India and watching a construction site: Lots of people, elephants, activity. I asked, ‘Why don’t you all use bulldozers and tractors and prefabricated concrete parts?’ The answer was enlightening: ‘How would this community make a living with dignity?’ That’s the whole paradox that we are dealing with: Productivity versus dignity for communities that could be unemployed for generations.”

    There is a difference between work and labor. Work is an activity a human does to give one’s life meaning. Labor is what a human does to survive. In the old days work was your avocation, labor was your vocation. Some humans were fortunate enough to have their activities be both avocation and vocation. Others went to work in factories, farms and offices to earn a buck. They pursued other interests at other times. But whether you sang in the New York City Opera full time or in the choir on Sunday morning, the dignity of your work, bringing music to the ears of other humans, was in full force.

    So, the final part of answering the question, “What do we do when we’ve automated everything?” is this: Preserve the dignity of the person and the community. A fully automated world will be a wealthy world unless we blow it up. Productivity produces wealth. That’s the way it is. Regardless of who is or what is the owner of the wealth—government, corporation or private citizen—sharing will become more important than ever. Concretely, this means allocating a significant amount of money to people and organizations that do work that has little or no commercial value, but extraordinary amounts of human value. When it comes to commercial value, the machines will have that end of the economy covered and then some. However, stuff that has little commercial value—curing low occurrence diseases, space exploration, academic research, expanding the arts—this is where human activity can excel.

    In the old days we had a notion called the commonwealth, that shared space where our individual assets grew. We shared some to get some. In the automated future it might have another name. But no matter the name, it will be a place where humans work with dignity in communities and most labor is the left to automated robotics. If not, the common thread of our individual survival might indeed be nothing more than alms for the poor while automation creates wealth for a few.

    — Bob Reselman

  • Mesos: Security Awareness and Considerations

    Mesos: Security Awareness and Considerations

    binaryAs regular readers know, I’ve been digging into Mesos quite a bit these days and, for the most part, enjoying the entire ecosystem that has grown up around it. There are some very cool possibilities when organizing servers into one massive platform for application (or micro-service) delivery. While there is a bit of a learning curve, I wouldn’t say it is as bad as the application provisioning tools out there, and there is the ability to scale and manage apps while maintaining but a single reference architecture.

    But every technology has its issues that we need to be honest and open about if they are to be successful in the enterprise. Mesos is no exception.

    Most open-source projects start out with, “If we just did X, Y and Z, we could do this cool thing!” and only afterward consider the security needs of the target audience. This is kind of the nature of the beast, and Mesos is no exception. The Mesos team is currently on the path to enterprise-grade security from little (or optional) security, but for right now, here are things to consider to help make your Mesos installation secure.

    According to Ben Hindman, Mesosphere co-founder and co-creator of Apache Mesos: “For folks that were running without authentication, we didn’t want to break their clusters when they upgraded. Authentication will be on by default after sufficient time for frameworks to get upgraded. We enable a mixed mode where some frameworks can authenticate and others don’t exactly to support this upgrade path.”

    The thing is, the default upon installation of Mesos at this time is to have no—I repeat no—authentication for frameworks, and no limitation on what those frameworks can see. If you use command line switches, you can modify this default behavior, but the default is still to be wide open where frameworks are involved. The point of this blog is to urge you to use those command line switches if at all possible in your environment.

    It is industry standard to do the exact opposite of what Mesos does—to offer an option to turn off security—but to have it turned on by default. And while it makes initial pilot deployments more difficult, that is precisely what Mesos needs (and, judging from Ben’s comment above, plans) to do.

    The following items, taken together, create the core problem:

    1. By default, Mesos allows frameworks to register without authentication (see link labeled “command line switches” ,above). Normally, some form of authentication is required. Increasingly, OAuth access control is being utilized. See Stormpath or DigitalOcean for examples of API Authentication design.
    2. Mesos uses “roles” to determine which frameworks have access to what resources. By default, a framework can register for any role.
    3. Persistent disks (such as your database or your contracts directory) can be resources assigned in Mesos. Access is controlled by role only (last bullet point before version history), to make them available to multiple apps on multiple frameworks. This also makes access available to any framework that assumes the same role as those with access.
    4. Mesos can distribute agents to each node in the cluster as needed via the mesos fetcher. So distribution of agents, which would be a limiting factor to this weakness, is actually easy—a framework can request that its agents be downloaded to any server in the cluster, on demand.
    5. There is a public framework development guide.

    So with a default Mesos installation, and any machine that can hit the REST API for frameworks, I can register a framework of my own construction. The Mesos system has automated approval of registration, so there is no human decision-making involved; registration (called scheduler subscription) implies approval if authorization is not enabled, and the response includes registration information. Considering that the frequency of adding frameworks should be minimal, this is a good place to have a human involved in the process. Note that older framework development using libmesos.so could also abuse this default, but that route is harder, and thus less likely to be utilized. My new framework can supply roles that exist in other frameworks, and I can then gain access to these static resources simply by waiting for Mesos to hand me the correct resources. These resources could include any persistent disk assigned to that role, with read/write access (because the current incarnation of Mesos only supports read/write). This came to my attention while working on Mesos with Marathon, but is possible simply with Mesos.

    Again, this is a known issue and the Mesos team is slowly moving toward resolution. But for now, users need to lock down their own implementation if their environment will allow it.

    The workaround is easy enough, there are command line options to turn on authentication for frameworks, and to limit which frameworks can use what roles. But you have to turn both on—they are off by default—and looking through the logs on my Mesos cluster, it does not appear that framework registration is logged with default logging settings. Note that role management has overhead, but when applied to frameworks is not a large security burden unless operations is constantly adding/removing frameworks from the system (which is highly unlikely). Also remember that the reason the system is currently designed to work both ways is that some frameworks do not yet support authentication, so testing with this feature turned on is a must before rolling it out in production.

    Do I rate the risk from this issue to be huge? Honestly, it depends upon your implementation. If the Mesos cluster in question is putting sensitive information on a persistent disk, the risk is potentially high. If the cluster in question is not, there is still risk from the architecture design and lack of authentication, but the data available—and how it is protected outside of Mesos—determines how hazardous an attack would be. The ability to modify the configurations of running applications is a further concern; HTTP redirections alone could be troublesome.

    In short, the existence of this transition period is going to make me more aware of the security implications of everything else in Mesos, simply because it shows a failure to follow common practices and might indicate that other places in the project suffer similar security design decisions, though the move toward fixing authentication will make me less hypercritical.

    Note that I wanted to get this info out there, thus have not yet taken the time to explore exploiting this issue on my cluster. Implementation almost always provides different information/issues than researching offers, as it is a different stage of the process. I will update when I have time to make a pair of frameworks, one “normal” and one “attacker,” to test with. In security, it is always best to test out issues first, but these are well-enough documented, I opted for notifying the community before embarking on a potentially lengthy proof.

    It is not the point of this post to imply that Mesos should be avoided. The point is to advise you that the “Command Line Switches” link (above) should be followed, read and the guidelines for turning on authentication should, at a minimum, be implemented. I would recommend limiting roles also, but at a minimum, turn on authentication for frameworks and test if it works in your environment.

    I’d like to thank Ben Tomhave, Security Architect at New Context, for helping me out on some of these points. If there is any error in my analysis, it is mine alone, but the blog is far better for his assistance.

    — Don Macvittie

  • Continuous Integration vs. Delivery vs. Deployment: What’s the Difference?

    Continuous Integration vs. Delivery vs. Deployment: What’s the Difference?

    “Continuous” is one of the most popular words in the DevOps lexicon. What does it really mean, and what is the difference between continuous delivery, continuous deployment and continuous integration? Keep reading for some perspective.

    Like many technological concepts, continuous integration, continuous delivery and continuous deployment are terms that DevOps teams tend to use without defining them precisely. For that reason, it’s worth taking a closer look at what each of these terms actually means.

    Continuous Integration

    Continuous integration can really mean one of two things, depending on the context.

    In the narrower sense, continuous integration refers to the work done by a continuous integration server, such as Jenkins or Bamboo. A continuous integration server automatically tests code written by individual developers to make sure it can be merged into the main code base. By automating this process, continuous integration servers help developers to write and test small chunks of code on an ongoing basis, which minimizes the risk that a code change could cause serious problems and force developers to revert their application to an earlier state.

    In a broader sense, continuous integration means the constant integration of changes to an application at all stages of the delivery chain. This includes but is not limited to the automated integration testing and code merging performed by an integration server. In this broader definition, continuous integration can include the integration of new design concepts to the delivery pipeline, for example. It also can involve other types of software testing beyond just automated integration tests.

    Continuous Delivery

    In most definitions, continuous delivery refers to the process of delivering software updates to users on a nearly constant basis. Continuous delivery is made possible by continuous integration and other optimizations at earlier stages of the development process. But the definition of continuous delivery gets a little cloudy when you start comparing it to continuous deployment.

    Continuous Deployment

    What’s the difference between continuous delivery and continuous deployment? For some DevOps teams, nothing. People tend to use these terms interchangeably. In fact, they are usually simply abbreviated as “CD,” leaving it up to the listener to decide what the speaker is actually talking about when he hears the term during a conversation about DevOps.

    Some developers draw a distinction between continuous delivery and continuous deployment, however. For example, Mirco Hering writes that continuous delivery requires that the DevOps team manually release updates to users. In contrast, the continuous deployment pipeline is fully automated; users get updates as soon as they are written and tested, with no manual intervention by developers.

    This distinction is a good way to think about the difference between the closely related concepts of continuous delivery and continuous deployment. It’s a distinction that will become increasingly important as technologies such as Docker make it easier to automate application deployment by allowing DevOps teams to drop new container images into production.

    What Does Continuous Mean, Anyway?

    It’s worth mentioning, too, that the word “continuous” is usually something of an exaggeration when used by DevOps folks. It implies that software changes roll down the pipeline in a totally constant, never-stopping fashion.

    That is an exaggeration. Integration, delivery and deployment are almost never completely continuous. In practice, a “continuously” integrated application is likely to be rebuilt and delivered something like every 24 hours, not every single time a code change reaches the end of the pipe.

    That rhythm is still a lot faster, and closer to continuous, than the “waterfall” delivery pipelines that were common before the introduction of DevOps. But it’s still important to understand that, even with all of the automation tools available today, software development is rarely completely continuous.

    — Chris Tozzi

  • Don’t Leave Software Testers Out of DevOps

    Don’t Leave Software Testers Out of DevOps

    The term DevOps implies that software delivery is all about integrating the development and IT operations teams. But the software testing team is an important, if oft-overlooked, part of the picture. Here’s why.

    Could you deliver software continuously without involving the QA team in the continuous delivery pipeline? You could try, but it would probably not go well. Software testers play a vital role in continuous delivery, for the following reasons:

    They are the glue connecting developers to IT Ops. If there is no one to test the code that developers write, IT Ops cannot deliver a stable product.

    They help find problems before they become costly. If you skimp on integration, unit and performance tests, you won’t find problems in your software until after it reaches users. By that point, fixing those problems will be much more time-intensive than it would if they had been caught early on. Plus, your organization’s reputation will have taken a hit, because problems that could have been nipped in the bud prior to delivery ended up affecting users.

    They know things that developers and IT Ops don’t. If DevOps is all about continuous visibility across the delivery chain, then it only works well when the delivery team’s collective expertise covers all parts of the delivery process. Since developers and IT Ops people are not experts in quality assurance or testing, you need to involve testing professionals in the continuous delivery chain so that they can help everyone else understand the parts of the process that QA professionals know in detail.

    All of the above is to say that if the QA team is not plugged into every stage of the delivery process, your software either won’t be tested efficiently, or it won’t be tested at all. Not testing efficiently means that the QA team is not in sync with the rest of the organization. In this case, the QA team would delay software delivery because it would have to wait on developers and admins to complete its work, and vice versa.

    Not testing at all, of course, is even worse. No one should deliver unvetted software to users.

    Don’t Forget about QA

    The takeaway for organizations seeking to adopt DevOps practices is this: Even though the DevOps conversation on the surface may seem to be about only developers and IT Ops professionals, QA teams play an equally crucial role in continuous delivery. They need to be as involved in the continuous delivery pipeline as everyone else if an organization wishes to reap the full benefits of DevOps.

    — Chris Tozzi

  • Q&A: CNCF on Instilling a DevOps Culture, Part 1

    Q&A: CNCF on Instilling a DevOps Culture, Part 1

    The Cloud Native Computing Foundation (CNCF) is a non-profit comprised of some 49 industry participants that have come together to drive development and adoption of cloud-native applications and services. The CNCF is uniting such giants as Google, IBM, Intel and Cisco Systems to distill a collection of container technologies that end-user companies deem praiseworthy to reach its goal.

    The CNCF intends its membership to represent end users (the CNCF website lists one end user, Goldman Sachs), cloud and container technologies, online and IT services, and academic institutions.

    On behalf of the CNCF, Craig McLuckie, its Governing Board chair, and Brian Grant, its Technical Oversight Committee representative, weigh in on leading the DevOps transformation and instilling a DevOps culture in this two-part Q&A.

    staging-devopsy.kinsta.cloud: How is the CNCF specifically involved in leading the DevOps transformation by instilling the DevOps culture, in the industry or in member companies, for example?

    McLuckie: We believe that at the heart of the DevOps culture is the need to reduce the friction and, indeed, the barriers between teams that build an application and the teams that run an application. Developers learn to think about an application as a living service that evolves continuously and rapidly to meet the evolving business needs. Operations is there to deliver the tools and environment needed for them to successfully drive their application.

    staging-devopsy.kinsta.cloud: Why do developers need an environment that is safe to change? What kind of change? Safe how?

    McLuckie: Safe change has many potential meanings: The first kind of “safe” is about creating deterministic behavior in operations. Does the system handle a change correctly every time? If I make a change to two systems, will they have the same state? How about to 1,000 systems? It’s important to focus on creating strong determinism by removing the moving parts in a production deployment.

    The second kind of “safe” is about being able to validate that a change is good—not just in pre-production environments, but in the actual production environment—with a simple and deterministic rollback. It is important to make it trivial to replicate environments, and offer a way to run a simple experiment in production (e.g. route 1 percent of my traffic to this new version) with the option to deterministically roll forward or backward without effort.

    A third kind of “safe” is about being able to ensure that every change is deliberate. One of the problems we often see in production systems is “configuration drift” that isn’t driven through any kind of formal process. This should be addressed by creating a strongly declarative toolset that makes every change deliberate and trackable.

    staging-devopsy.kinsta.cloud: How do you recommend providing these things that developers need? How do you help provide them?

    McLuckie: There are multiple issues like data, security governance and applications that need to be addressed. Additionally, companies want to deliver apps with high velocity. How do you solve all of this? A vendor-neutral organization that can harmonize emerging technologies is important for facilitating cloud-native. Given the large number of players in the marketplace with competing approaches, having an open source way to solve this is also important.

    Grant: Cloud-native infrastructure enables self-service and “hands-off” operation, including provisioning, deployment, scaling, healing, discovery, monitoring and more, with relatively little marginal effort for specific applications. By supporting open source projects such as Kubernetes and Prometheus that provide such functionality, the CNCF hopes to democratize high-velocity application delivery, which was previously achievable mainly by web-scale companies.

    staging-devopsy.kinsta.cloud: How does this align with instilling a DevOps culture?

    Grant: In a DevOps culture, operators are both partners and customers of the developers. Developers take responsibility for meeting operational requirements: security, reliability, availability, scalability, efficiency, actionable monitoring and logging output, necessary configurability, compatibility with the production environment, etc. Cloud-native infrastructure provides mechanisms developers and operators need to facilitate and capitalize on those capabilities.

    Stay tuned for part two of the CNCF interview.

  • Scaling Collaboration in DevOps

    Scaling Collaboration in DevOps

    Those familiar with DevOps generally agree that it is equally as much about culture as it is about technology. There are certainly tools and practices involved in the effective implementation of DevOps, but the foundation of DevOps success is how well teams and individuals collaborate across the enterprise to get things done more rapidly, efficiently and effectively.

    Most DevOps platforms and tools are designed with scalability in mind. DevOps environments often run in the cloud and tend to be volatile. It’s important for the software that supports DevOps to be able to scale in real time to address spikes and lulls in demand. The same thing is true for the human element as well, but scaling collaboration is a whole different story.

    Collaboration across the enterprise is critical for DevOps success. Great code and development needs to make it over the finish line to production to benefit customers. The challenge organizations face is how to do that seamlessly and with as much speed and automation as possible without sacrificing quality or performance. How can businesses streamline code development and deployment, while maintaining visibility, governance and compliance?

    Emerging Trends

    First, I want to provide some background and share some data gathered by 451 Research on DevOps and DevOps adoption in general. Cloud, agile and DevOps capabilities are important for organizations today—both in perception and reality. 451 sees enterprise adoption of these things, as well as container technologies, growing—including increased usage in production environments.

    There are a number of advantages to embracing these technologies and methodologies, such as increased flexibility and speed, reduction of costs, improvements in resilience and reliability, and fitness for new or emerging applications. According to 451 Research, organizations also face some barriers including a lack of familiarity and required skills internally, the immaturity of these emerging technologies, and cost and security concerns.

    In the “Voice of the Enterprise: SDI Q4 2015 survey,” 451 Research found that more than half of the respondents (51.7 percent) consider themselves to be late adopters, or even the last adopters of new technology. The flip side of that is that almost half (48.3 percent) label themselves as first or early adopters.

    Those general sentiments are reflected in the survey responses to other questions. When asked about implementation of containers, 50.3 percent stated it is not in their plans at all, while the remaining 49.7 percent are in some state of planning, pilot or active use of container technologies. Nearly two-thirds (65.1 percent) indicated that they use agile development methodologies for application development, but only 39.6 percent responded that they’ve embraced DevOps approaches. Nevertheless, while agile software development has been in the industry for years, 451 notes the impressive adoption of containers and DevOps, given they are emergent trends.

    When asked what the top three IT pain points are, the leading responses were cost or budget, insufficient staff and legacy software issues. As organizations move to cloud, DevOps, and containers issues such as these will need to be addressed, along with how to scale both technologies and collaboration effectively.

    The Current State

    The industry—driven in large part by the DevOps revolution—is in the midst of a sea change, where software development is becoming more highly integrated across the entire business. The creation of software is less segregated and is more and more a function of collaboration and socialization.

    Concepts and methodologies that were novel or niche just a few years ago have matured quickly to become the mainstream technologies and frameworks that are driving value today. Businesses rely on concepts such as agile, lean, virtualization, cloud, automation and microservices to streamline development and enable them to work more effectively and efficiently at the same time.

    To adapt and evolve, enterprises need to accomplish a number of key tasks. The challenge companies face today is how to accelerate development while reducing costs. Organizations need to eliminate the barriers that exist between IT and the rest of the business, and work cooperatively toward a strategy that provides more effectiveness in a technology-driven, competitive environment.

    Agile, cloud, DevOps and containers all play a role in that process, but the one thing that binds it all is effective collaboration. Each of these technologies and methodologies provides unique benefits, but the real value comes from the organization as a whole—and the tools and platforms used by the organization—being able to collaborate at scale. Successful DevOps implementations also require participation from other stakeholders beyond development and IT operations teams, including security, database, storage and line-of-business teams.

    Collaboration-as-a-Platform

    There are services and platforms online—such as GitHub—that facilitate and streamline collaboration. The online platform functions as a code repository, but the value extends beyond just providing a place to store code.

    Such a collaboration platform helps developers and teams collaborate more effectively because it provides a community where the code and process can be shared and discussed. Managers can monitor progress and track what code is shipping next. Developers can experiment with new ideas in a safe environment before taking those experiments to a live production environment, and new ideas and experiments can be effectively communicated to the appropriate teams.

    One of the keys to more agile development and DevOps is to allow developers to test things and gather relevant feedback quickly. The goal is to produce quality code and features faster, not to waste time setting up and managing infrastructure or scheduling more meetings to talk about it. The GitHub platform, for example, enables more effective and scalable collaboration because code review can occur when it is most convenient for the participants. There is no need to try and coordinate and schedule code review meetings, so the developers can continue to work uninterrupted, resulting in greater productivity and job satisfaction.

    Steven Anderson of Sendachi noted that GitHub is a collaboration platform, but it’s also a place for your tools to work with you, too. This means it can help not only with collaboration and continuous integration, but also with code quality.

    One of the benefits of a collaboration platform is that large teams of developers can be broken down into smaller teams that can focus more efficiently on specific components. It also allows things such as document sharing alongside code development to blur the lines between technical and non-technical contributions and enable increased collaboration and visibility.

    Collaboration is Key

    The importance of collaboration can’t be stressed enough. It is a key tenet of DevOps culture, and it’s vital to agile development and maintaining a competitive edge in today’s world. Executive or management support and internal evangelism are important. Organizations also need to embrace the culture shift—blending skills across functional areas toward a common goal.

    With that culture established, though, effective collaboration is crucial. A collaboration platform is an essential element of collaborating at scale because it streamlines productivity and reduces redundancy and effort, and yields higher quality results at the same time.

  • Walmart OneOps Won’t Be One-Upped

    Walmart OneOps Won’t Be One-Upped

    One of the biggest names in retail also is kind of a big deal in DevOps. Walmart has developed its own platform, OneOps, designed to streamline operations and keep its application development schedule in check.

    “OneOps is an open source cloud and application lifecycle management technology platform with benefits including cloud portability so developers can seamlessly move applications, databases or entire cloud environments from one cloud provider to another,” says Brian Johnson, VP of Engineering, Platform, Walmart Labs. The latter feature allows developers to leverage the broad selection of cloud environments and pricing schemes to their advantage.

    The OneOps platform deploys Walmart website adds/changes, launches new mobile features such as Walmart Pay, and creates APIs so third parties such as Hiku can tap into Walmart’s online grocery offerings, at speed and with a lower investment in the cloud, says Johnson. The OneOps platform once only managed the Walmart.com e-commerce site, but now it also serves individual Walmart stores and supports other initiatives.

    According to Johnson, OneOps undergirds and empowers the work of developers as follows:

    • OneOps continuous lifecycle management continuously auto-pilots the process of automating application scaling and repair whenever any unforeseen changes take place.
    • OneOps promotes accelerated innovation so software engineers can spin up new environments for app hosting in minutes, since the DevOps platform skirts the need to specify the intricate details of any one cloud environment by modeling that for the developer.
    • This abstraction of the cloud environment gives developers control over the tools and technologies they select regardless of the cloud provider they use.

    Life Before OneOps

    “Before OneOps, the process of updating Walmart’s website was so tedious that we rolled out updates only every two months. The process used hundreds of manual steps and required significant resources,” Johnson says. “This was time-consuming and created room for manual errors.”

    Before OneOps, A/B testing was expensive and time-consuming, taking months to set up multiple cloud environments and conduct tests. By the time testing was done, the rationale for the new feature had waned and whatever opportunity it was created to take advantage of had passed, he says.

    At that time, creating enough virtual machines (VMs) in time to handle peak shopping events and traffic would have taken teams of people months to complete, at the risk of exorbitant operating costs, Johnson adds.

    The OneOps Afterlife

    OneOps now fully manages the Walmart e-commerce platform such that in a typical day the site sees more than 1,000 new software deployments, which developers execute on demand in a span of minutes on average, says Johnson.

    OneOps serves a variety of customers such as consumers of Walmart.com, who on black Friday found their shopping experience supported by the tens of thousands of VMs OneOps was able to spin up across multiple clouds to cradle the peak traffic, Johnson says. OneOps is the DevOps and cloud platform for SamsClub.com as well.

    Solving the Single Cloud Provider Dilemma

    Walmart targeted OneOps to use multiple cloud providers easily, because of the pitfalls of being locked in to a single provider. “The fear of vendor lock-in is the major challenge to cloud service adoption. Cloud vendors like AWS don’t make it easy for customers to move from one cloud to another. The complexities of cloud service migration mean that many customers stay with a provider that doesn’t meet their needs simply to avoid the cumbersome and expensive process of migration,” says Johnson.

    Further vendor lock-in concerns, he notes, include:

    • Contracts of lengthy duration geared toward ensuring higher server utilization, maximum ROI and profitability for the cloud provider but providing little benefit for the cloud customer.
    • Lack of freedom of choice where storage space, processing power, bandwidth and tools and technologies are concerned.
    • Because it is typically difficult to find any two cloud providers that use the same tools, compatibility is an issue. “You can’t simply port your site to a new provider without rebuilding it using the new provider’s toolsets,” says Johnson.
    • “While you build and test your site on the new cloud platform, you are still paying to maintain it on the old provider, doubling your costs,” says Johnson.

    This is why OneOps abstracts the use of differing cloud providers, making porting easy.

    The Big Question Is …

    OneOps solves a unique set of DevOps challenges related to the cloud. Can you name another platform that does something similar?

  • Zero Trust Is a Key to DevOps Security

    Zero Trust Is a Key to DevOps Security

    Once upon a time, network and endpoint security was a simple matter of us vs. them—and all you had to do is keep unauthorized stuff outside your perimeter from gaining access to devices or data inside your perimeter. It’s not like that anymore, though. BYOD, mobile devices, cloud service, virtual systems and changes in attack techniques combine to create a scenario where it’s virtually impossible to assure security with any degree of confidence. At this point, it’s almost better to assume you’re compromised, and work from there to minimize the potential impact—especially when it comes to DevOps security.

    Consider the fact that threats such as Stuxnet and Duqu were found to have been circulating in the wild for years before they were discovered. According to one recent study, it takes organizations an average of 256 days to discover a malicious compromise—in large part because they’re functioning from the semi-delusional mindset that they’re effectively protecting against such attacks. When you believe that your network is secure, you have no reason to dig deeper and scrutinize activity on the network to detect and identify suspicious activity.

    Mitchell Hashimoto, co-founder and CEO of HashiCorp, is an evangelist for the idea of a zero-trust data center. Borrowing a page from the Ronald Reagan national security policy, Hashimoto stresses a strategy of trust but verify. He believes in a security mentality in which all resources are assumed to be compromised—particularly in organizations that have embraced DevOps.

    DevOps brings much greater flexibility and agility, but with the more rapid pace of development and deployment also comes increased volatility and potential uncertainty when it comes to maintaining accurate visibility of the assets and data in the environment, and the vulnerabilities you’re exposed to. With the unprecedented scale and automation of modern data centers, the zero-trust model keeps infrastructure secure while still enabling automation.

    “As the scale of modern computing continues to increase, our security mindset must change. Automation and DevOps help manage this scale, but introduce more change. With change comes vulnerability,” cautions Hashimoto. “The best way to approach this problem is to assume a zero-trust model. With this mindset, all resources are assumed to be compromised and require specific permissions to connect to other resources in the environment. If the permission policy does not match, the resource gets denied.”

    The idea behind this zero-trust approach is to greatly reduce the number of opportunities an attacker might have to gain access, and reduce the scope of potential damage in the event of a successful breach of your network. Applying this concept at scale—in a dynamic DevOps environment—can be more of a challenge. So much changes so quickly that you need to have automated tools capable of managing this process. Attempting to do this with any sort of manual system would be lunacy.

    If you still think there is an inside and outside to your network, or that it’s possible to win in a battle of us vs. them, you’ve probably already lost. There is no such thing as impenetrable security, and there is simply too much volatility in any network environment—but especially in a dynamic DevOps environment. Trust but verify. Embrace the idea of a zero-trust data center to improve your security and minimize the potential for attackers to access and compromise your network.