staging-devopsy.kinsta.cloud

Category: Features

  • The benefits of micro-services

    The benefits of micro-services

    The adoption of a micro-services architecture has two main advantages, according to ThoughtWorks technologist Sam Newman.

    One is that micro-services increase the autonomy of individual development teams within an organization, as ideas can be implemented and deployed without having to coordinate with a wider IT delivery function.

    This allows new or changed features to be delivered more quickly. Organizations that understand they make money through technology – Netflix is one example Newman cited – know they need to get new features out the door as quickly as possible.

    The other main advantage is that micro-services aids the adoption of new technologies, although this is arguably a subset of the autonomy argument.

    Since a micro-service is by definition a small, self-contained function, it provides a low-risk opportunity to try a new technology. That is definitely not the case where monolithic systems are concerned, Newman said, as greater care is required when all the eggs are in one basket. But where a less-critical element is concerned, there are fewer barriers to experimenting with a new programming language or database, for example.

    So which organizations are setting the pace? “Netflix is a very good example,” said Newman, observing that the driver for its adoption of micro-services was to ship new features quickly, rather than handling scaling issues.

    Locally, REA (the company behind realestate.com.au and related real-estate sites) “is probably a couple of years ahead” in the use of micro-services, he said, noting that REA has been a ThoughtWorks client for many years, and has also embraced DevOps and automation. Its technology architecture is aligned with its organizational architecture, so REA can readily absorb companies that it acquires and therefore grow more easily. Organizational issues are key to understanding the significance of micro-services, he said.

    REA approached micro-services progressively. In the first three months it made sure the foundations were in place and put two micro-services into production. But in the second trimester it was able to deploy around a dozen more, and in 18 months more than 60 were live.

    Other micro-services adopters mentioned by Newman included MYOB (see DevOps in Action: MYOB) and PwC (which is progressively migrating its system for handling expatriates’ tax returns and related issues from a traditional enterprise system to a finer-grained, more decomposed architecture using micro-services, to reflect the way some parts of the system are common to all geographies while others are country-specific).

    There is a very real risk that building micro-services will lead to duplication, but that is not necessarily a bad thing, Newman said. Many SOA (service-oriented architecture) implementations have been hamstrung by efforts to optimize services for reuse slowing the pace of change, so organizations may choose to accept a degree of duplication in order to speed delivery.

    An Australian bank is currently exploring this issue, realizing that it previously made a mistake by emphasizing reuse over time to market, he said. “It has to be a balance… [but] reuse in itself shouldn’t be the goal.”

    That said, it can be beneficial to eliminate at least some of this duplication over time in order to improve the maintainability of the systems. Otherwise the result can be “quite a chaotic environment.”

    “It’s a governance thing,” Newman explained – organizations need to detect any duplicate micro-services, and then make a deliberate decision to either continue as they are or to converge them. This is healthier than simply forbidding divergence and duplication as that can negatively affect time to market.

    Allowing duplication – even if only in the first instance – also mitigates questions of ownership, allowing the team developing a micro-service to proceed without the need for ‘sign off’ from other parties that may be potentially affected.

    In any case, developing a micro-service typically takes around two weeks, so there is no great loss if duplicates are retired. It is a different situation when monolithic systems are involved, as they are very hard to retire in an SOA environment – an enterprise application may be grossly underused, but it can’t be removed as long as other programs are consuming its services.

    The idea behind SOA was good, he said, but it became dominated by software vendors rather than by consideration of what works well for their customers.

    The micro-service architecture can be thought of as ‘SOA done well,’ Newman suggested, especially as it provides a mechanism for surfacing capabilities in multiple different ways, such as (in the case of a retailer) via web sites, native and web apps for mobile devices, and in-store software. When different capabilities are to be combined in different ways, working with smaller pieces is easier than larger ones.

    Newman’s book Building Micro-services is published by O’Reilly Media. It is now available as an ebook and will appear in print shortly.

  • Five Great Books on DevOps

    Five Great Books on DevOps

    While the DevOps movement began in earnest in 2009, there are still a lot of questions about what DevOps means, how it’s done, and what the purported benefits of DevOps are. Big questions, for sure. And there are no quick answers. In fact, to thoroughly answer each would require numerous books.

    Which brings us to the question, what are the best books that define DevOps, describe how to operationalize it in the real world, and show what beneficial outcomes one could expect?

    Here are my choices of five great books on DevOps, in no particular order:

    Book: The Phoenix Project

    Authors: Gene Kim, Kevin Behr, George Spafford

    This one is a fictional story, written in engaging novel style, that tells the story of how to apply DevOps to unleash IT’s potential. The story is about Bill, the IT manager at the fictional company Parts Unlimited. Not an exciting business name, but the adventure starts in earnest when on a drive to work, Bill gets the call from the CEO that sets the story in motion.

    The book’s cover description tells it best:

    The company’s new IT initiative, code named Phoenix Project, is critical to the future of Parts Unlimited, but the project is massively over budget and very late. The CEO wants Bill to report directly to him and fix the mess in ninety days or else Bill’s entire department will be outsourced.

    With the help of a prospective board member and his mysterious philosophy of The Three Ways, Bill starts to see that IT work has more in common with manufacturing plant work than he ever imagined. With the clock ticking, Bill must organize work flow streamline interdepartmental communications, and effectively serve the other business functions at Parts Unlimited.

    In a fast-paced and entertaining style, three luminaries of the DevOps movement deliver a story that anyone who works in IT will recognize. Readers will not only learn how to improve their own IT organizations, they’ll never view IT the same way again. 

    As one Amazon reviewer put it: “This “parable” of how applying DevOps to pull out of an IT death spiral of delays and defects is an enjoyable read backed with great references and support materials for understanding the principles underlying DevOps. It’s not enough to get you all the way there, but it will get you inspired to start down the path.”

    Book: Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation

    Authors: Jez Humble and David Farley

    This seminal read is all about putting into practice rapidly and consistently delivering new highly-functional software to your users. From the back cover: “Through automation of the build, deployment, and testing process, and improved collaboration between developers, testers, and operations, delivery teams can get changes released in a matter of hours—sometimes even minutes-no matter what the size of a project or the complexity of its code base.”

    The book covers automated infrastructure management and data migration and the use of virtualization, and automating creating, integrating, testing, and deploying software, as well as improvement collaboration within the teams.

    As one Amazon reviewer put it, “This is an absolute must read for anyone in software or application development. It is amazing how well the authors capture the real pain points and provide all the information you need to getting to a more effective IT practice.”

    Book: The Visible Ops Handbook

    Authors: Kevin Behr, Gene Kim and George Spafford, IT Process Institute

    This is the first book I ever read on IT process improvement, and it was an eye-opener for me. This book came out about 4 years before the term DevOps was coined, but it nailed the essence of what high-performance IT is all about. And it’s an essential read today:

    Visible Ops codifies how these [High performance enterprise] organizations achieved their transformation from good to great, showing how interested organizations can replicate the key processes of these high-performing organizations in just four steps:

    1. Stabilize Patient, Modify First Response – Almost 80% of outages are self-inflicted. The first step is to control risky changes and reduce MTTR by addressing how changes are managed and how problems are resolved.
    1. Catch and Release, Find Fragile Artifacts – Often, infrastructure exists that cannot be repeatedly replicated. In this step, we inventory assets, configurations and services, to identify those with the lowest change success rates, highest MTTR and highest business downtime costs.
    1. Establish Repeatable Build Library – The highest return on investment is implementing effective release management processes. This step creates repeatable builds for the most critical assets and services, to make it “cheaper to rebuild than to repair.”
    1. Enable Continuous Improvement – The previous steps have progressively built a closed-loop between the Release, Control and Resolution processes. This step implements metrics to allow continuous improvement of all of these process areas, to best ensure that business objectives are met.

    As this Amazon reviewer this had to say, this fall, on the book: “This is the first and foremost required reading on my team! If you are an IT professional, you must read this. It is a concise, powerful expression of how to reinvigorate your operation and become a top performing IT organization.”

    Book: DevOps for Developers

    Author: Michael Hüttermann

    This book is for software developers and engineers that aim to help improve the delivery of software by using lean practices to get apps and software into production.

    From its description:

    DevOps for Developers is for motivated software engineers, particularly programmers, testers, QA, system admins, database admins, both beginners and experts, who want to improve their software delivery process. It’s the perfect choice for engineers who want to go the next step by integrating their approaches for development and delivery of software. This book is for engineers who want to shape their processes and decide on and integrate open source tools and seek for guidance how to integrate standard tools in advanced real world use cases.

    And as an Amazon reviewer put it: “Developers tend to produce updates in short cycles whilst admins prefer a stable system. And if something bad happens, they often start finger-pointing to blame the other side. One solution is to tear down the silos of development and operation. In other words, set up integrated teams.

    This is the core message which Michael Hüttermann would like to stress out. Agile methods and tools might be used for development as well as for deployment and operation. In his book, Michael explains how to do and which (free) tools to use.”

    To be fair, the book has taken dings about not being a programming book, and repeating some of its concepts. But the value in this book is its real-life examples, and explanations as to how different roles on teams in Dev and Ops can work together more collaboratively. It also dives into common pitfalls organizations hit in their journeys of operationalizing DevOps.

    Book: The Goal: A Process of Ongoing Improvement

    Authors: Eliyahu M. Goldratt, Jeff Cox Author, David Whitford (Other Contributor)

    The Goal is the second fictional book in this list. Written more than 30 years ago, it could have been written yesterday. When discussing concepts and systems, many times story is the way to go because the motivations and thought processes of different roles can be discussed in depth and elegantly so. The Goal succeeds in this, and is a must read for anyone who seeks to help be a catalyst of improvement in their organization.

    From the book description:

    One of Eli Goldratt’s convictions was that the goal of an individual or an organization should not be defined in absolute terms. A good definition of a goal is one that sets us on a path of ongoing improvement.

    Pursuing such a goal necessitates more than one breakthrough. In fact it requires many. To be in a position to identify these breakthroughs we should have a deep understanding of the underlying rules of our environment. Twenty-five years after writing The Goal, Dr. Goldratt wrote Standing on the Shoulders of Giants. In this article he provided the underlying rules of operations.

    This article appears at the end of this book.

    Here’s what an Amazon reviewer had to say: “I have often heard myself complaining that companies over complicate the simplest of activities. Measurements do not truly measure the end result and are often held in a high regard by executives. These measurements cause further loss and mask the true issues. This book was fascinating, very “simple” in a sense which lends to making it even more fascinating. I learned a lot about the production aspect of my “department” which would often not be looked as a production department in the literal sense of the word. Thought provoking and inspirational, every leader can benefit from the story told. A must read.”

    I would love to hear from you, what are the most important books on DevOps that you’ve read?

  • 11 DevOps start-ups funded for a busy 2015

    11 DevOps start-ups funded for a busy 2015

     

     Slides

    [seoslides embed_id=”4b8ee43e173e” script_src=”https://staging-devopsy.kinsta.cloud/embed-script/11-devops-start-ups-funded-busy-2015/title/” overview_src=”https://staging-devopsy.kinsta.cloud/slides/11-devops-start-ups-funded-busy-2015/” site_src=”https://staging-devopsy.kinsta.cloud” site_title=”staging-devopsy.kinsta.cloud” title=”11 DevOps start-ups funded for a busy 2015″ /]
    Advance slides with arrow buttons or touchscreen

  • Henry Ford Discovered The DevOps Secret Sauce (and you can too).

    Henry Ford Discovered The DevOps Secret Sauce (and you can too).

    DevOps Celebrates it’s 100th Birthday!

    Yes, you read the title right – DevOps is officially 100 years old. Way back before Dr. Deming, Agile, Jez Humble or anything labeled “continuous”, the principles of DevOps were being created and refined. But like a rebellious teen, our current iteration of DevOps may be ignoring some of the guiding principles that have proven to be so successful in previous generations.

    Think about the automobile industry circa 1905. These high-tech shops were pure DevOps as cross-functional teams collectively engineered, fabricated, assembled and tested every single automobile that was “released”. These DevOps forefathers strived for the same important ideals our current generation has set out to achieve – to increase release speed, improve release quality and gain a competitive advantage.

    Back then, DevOps teams were super agile and could adapt quickly to the changing needs of their end-users. Quickly, teams at Buick, Oldsmobile and Cadillac began to discover the power of interchangeable components and automation (i.e., open source tools). Productivity and efficiency skyrocketed as a result.

    But starting around 1912, Henry Ford began thinking beyond the prevailing brand of DevOps. Ford came up with is own DevOps secret sauce. Unlike the other DevOps teams of the era, Ford placed relentless focus on two key concepts: repeatable processes and standardized inputs/outputs. He recognized the lack of standardization resulted in low process repeatability which in turn raised cost and lowered quality. In 1913 he began to standardize processes and the inputs they require, thereby launching the very first Continuous Delivery pipeline for the auto industry. Once implemented, Ford’s new CD pipeline was assembling, testing and releasing new automobiles at a rate far exceeding anything previously thought possible.

    What was true about delivering cars in 1913 is also true about delivering software today. Without highly standardized inputs, DevOps teams can not design repeatable processes. Without repeatable processes, automation is not possible. This means DevOps craftsmen will always be required to manually build, test and deploy software change.

    Today a handful of select companies like Netflix, Facebook, Ancestry.com and Flickr have largely removed this type of craftsmanship from the software delivery process, replacing it with highly-repeatable processes and relentless automation. Some call these companies “Unicorns” because they seem to have magical powers. A few months ago I talked to the DevOps team at Ancestry.com. They have hundreds of fully automated delivery pipelines and production deployments happen on average about every six minutes with the same level of fanfare as the next new Ford rolling off a production line. These organizations build, test and deploy software change effortlessly by relying on Henry Ford’s two core ideals: standardized inputs and repeatable processes.

    Here’s a quick story… John DiMarco, Chief Operating Officer at CEDAR Document Technologies, is very happy with his agile Development process. New development output is very strong, so strong in fact, that Delivery is quickly becoming the agile bottleneck. To combat this problem, he’s well on his way toward a complete transformation of CEDAR’s current Delivery methodology. When we first met, he equated the process of delivering software with the effort required to deliver “finely handcrafted Italian marble”. At that time, his Delivery process was able to produce a high-quality output but it was overly reliant on individual effort and creativity. Now, one of DiMarco’s strategic objectives for 2015 is to evolve his software Delivery process toward the kind of high-tech assembly lines found at Ancestry.com.

    Assembly lines require highly standardized inputs in order to function efficiently. Therefore, a clear line between Development and Delivery must exist to define the discrete point at which Delivery inputs (i.e. software change) must tightly conform to a set of very strict standards agreed upon by all stakeholders. Without this formal entry point and standardized inputs, you’ll be handcrafting marble all the way to production.

    Devops needs to take a cue from our fellow microservice advocates. Imagine a Service Oriented Process (SoP) where Development and Delivery are both first-class citizens in the larger agile machine. Both services have inputs and outputs, are bound by contracts and interface through a standard “API”. With this type of mindset, Development and Delivery services can evolve as needed without affecting each other, as long as all contracts are honored.

    Payoff – The Unicorn Secret Sauce

    Over the past six months, I’ve talked to over 50 CIO’s and VP’s of Dev/Engineering. A few are unicorns, others are well on their way, while most are just now plotting their course. Based on these conversations, and so many others with customers and DevOps thought leaders, I have identified the Unicorn Secret Sauce.

    Standardization – There is one simple reason you must break your agile Develop/Deliver stream into two (or more) discrete services – Standardized Inputs/Outputs. Whether you define two big services (develop and deliver) or twenty micro services, each service along the pipeline should have documented inputs and outputs as well as detailed contracts between any two interacting services – in writing. While this step won’t be easy, it’s absolutely critical for two reasons:
    Assembly lines require standardized inputs. With no standard inputs you are handcrafting Italian Marble.
    When the definition of “done” can be objectively proven via satisfied contracts, pipeline throughput is no longer dependent on cooperation and the DevOps group hug (while that’s still awesome). Now, statistics can be generated, quality can be measured and performance can be objectively quantified and compared between and across services and teams. Welcome to the DevOps Party Dr. Deming!

    Test-Driven-Design – The Development service must be able to hand off a standardized package that can be easily ingested by a downstream service without the need for creative thought or subjective analysis. This has always been pretty easy when it comes to the change package itself (code), however, the question of quality has always been a much tougher nut to crack. The best way around this is by adopting some form of TDD. It doesn’t matter which teams create and define quality tests – what matters is that success and quality have been implicitly defined and that proof can be easily achieved (manually or automatically) downstream. Without tests as a primary input to delivery, there can be no contract between services and we’re back to Italian Marble.

    The Journey – It’s a Marathon not a Sprint

    Clearly the metamorphosis from marble craftsman to unicorn will not happen overnight or this quarter – It took Ancestry.com two years to get there. But here is a proven roadmap that will get you from here to there:

    Define your version 1.0 Service Oriented Process design. Pay careful attention to ensure inputs/outputs have been standardized and clear contracts are in place between “service API’s”.
    Next implement basic delivery infrastructure (the assembly line operating system). This platform will orchestrate flow across/between services, gather data, and provide real-time status, reporting and analytics. This will get all teams and service providers on the same page and rowing in the same direction leveraging a unified world view.
    Next, start eating the elephant one bite at a time by continuously replacing manual services with automated replacement versions still remembering to focus on inputs/outputs, API’s and contracts.

    Over time, your body of automated quality tests will grow at a speed equal to your rate of change. Soon coverage will be broad enough to accurately predict release quality allowing manual release testing to quickly decline as automated test coverage (read confidence) increases.
    Twelve to twenty-four months later…congratulations, you’re a unicorn!

    No doubt, many will disagree with my perspective. I value every opinion and will learn from your shared ideas. Please express yourself and add your voice to the discourse with a comment below!

    About the Author/Dennis Ehle

    dennis-150x150Dennis Ehle, is a pioneer and thought leader in continuous delivery automation and agile delivery methodologies.  Dennis is passionate about helping agile teams dramatically reduce the transaction cost associated with delivering incremental change. His company, ClearCode Labs, does just that by helping organizations continuously deliver high-quality software releases more frequently merging proven methodologies with empowering tools and technology. Twitter: @DennisEhle

     

  • DevOps Video Rodeo: DevOps 101

    DevOps Video Rodeo: DevOps 101

    It can sometimes be difficult as a DevOps newbie to get the straight dope on what DevOps is all about without ending up in the weeds with descriptions and tutorials full of specialized details. As you ramp up, that kind of material is invaluable. But when you’re new to the game, it gets in the way of clarifying the basics.

    If you or one of your colleagues is looking into the basics of DevOps, we’ve got a good place for you to start. Today we’re launching a new video round-up feature we’re calling DevOps Video Rodeo and we thought there would be no better way to kick things off than to focus on DevOps 101. We’ve collected some of our videos meant for DevOps beginners. They explain what DevOps is all about, how continuous delivery can help IT deliver better business value and how organizations can begin their DevOps journeys. Some of these are new, some are oldies but goodies, and some are considered classic talks that helped drive the way the industry first started talking about DevOps.
    So let’s get started.

    DevOps Demystified – An introduction to the ideas that are driving DevOps
    Now director of IT and operations at Chef, Ben Joyent has long been spreading the gospel of DevOps. When he was at Joyent a couple of years ago, he gave one of the most thorough talks breaking down DevOps for newcomers that still exists today. This is ideal for practitioners on either side of the aisle who want to dive deeper into the fundamental principles behind DevOps. It’s a long, but thorough primer.

    What is DevOps? – In Simple English
    If you’re looking for a short summary to pass along to the upper-level, line-of-business managers who you’ll need to help support a DevOps transformation, this may be it. Don’t let the slick, vendor-produced nature of this video turn you off. Rackspace kept to the basics without trying to hide a sales pitch in there. It does an especially good job explaining the business value of DevOps.

    What is DevOps
    Like the previous video, this is another DevOps description that clocks in under 10 minutes, but this one is particularly good at offering explanations of how developer and sys admin roles change under the DevOps model. This is particularly good one for new DevOps acolytes to send over to colleagues they’d like to convince to join forces with them. It also offers a quick sketch of how the infrastructure-as-code model works. Plus, its quick-and-dirty, back-of-the napkin style is kind of endearing.

    The (Short) History of DevOps
    If you’ve ever wondered where the DevOps movement came from and who helped refine the ethos, this is about as concise of a history as you’re going to get. Nicely done by Damon Edwards of DevOps Café podcast fame.

    10+ Deploys Per Day: Dev and Ops Cooperation at Flickr
    This is a classic talk, one that many DevOps early adopters point to as a pivotal demonstration that started the snowballing momentum going with a very real case study on how DevOps can improve speed and flexibility. DevOps practitioners may have moved far beyond 10 deploys a day, and John Allspaw and Paul Hammond have since moved on to other organizations, but this stands as an excellent introduction to DevOps.

     

  • A DevOps Metamorphosis Echoes Through Reverb.com, Part Two

    A DevOps Metamorphosis Echoes Through Reverb.com, Part Two

    Last time, Dear Reader, we discovered how the Reverb.com e-commerce platform, which connects musicians who want to buy and sell gear sought to automate its workflow software development process, speeding deployment and bug fixes while satisfying customers who want to trade in an online environment that is as efficient and error-free as possible. Now, the conclusion of how Reverb.com adopted DevOps to achieve its goals.

    A Round Trip Through The Reverb.com CD Process

    Today, Reverb.com focuses the DevOps mindset on its development process using Chef [https://www.chef.io/chef/] for configuration management and Jenkins [https://www.cloudbees.com/jenkins/about] for continuous delivery and infrastructure automation tasks. Developers use Capistrano [http://capistranorb.com/] for rolling out new code changes and running database migrations, CircleCI [https://circleci.com/] for unit testing code, and DataDog [https://www.datadoghq.com/] for metrics and monitoring. “We also use Logstash [http://logstash.net/], ElasticSearch [http://www.elasticsearch.com/about/], and Kibana in unison for log aggregation and visualization,” says Adam Enger, Senior Infrastructure Engineer, Reverb.com.

    In a typical deployment for Reverb.com, an engineer clicks the build button in Jenkins, which triggers a Chef converge on all the web heads. This rolls out any configuration changes that the developers have scheduled to go out, such as NGINX configuration updates or app config updates.

    Once this is finished, the system signals the application to perform a Zero Downtime Deploy in which the application reloads without dropping any requests. “If this process finishes successfully, Jenkins kicks off a code deploy using Capistrano and simultaneously deploys new code to our web application servers. If a database migration is scheduled to be run, that will also happen on one of our worker boxes,” says Enger.

    This may sound like a standard deployment so far. But that’s where DataDog and the ELK stack come into play. “That’s where we really gain the benefits of CD/CI and DevOps,” says Enger. Every time the development team converges Chef, they use a Chef Handler to submit an event to DataDog. And once Capistrano starts and finishes, another event goes to DataDog. The team has configured Logstash to watch the application logs on the web heads and to send an event to DataDog once the system renders an error. “The single point of information now becomes DataDog because we can see into our config and code changes and immediately know when a bad change goes out the door,” says Enger.

    The last thing the development team looks at is the performance logs in Kibana, to see how the application is performing based on the logs it is generating. “With the ELK stack in place, the team can answer questions like: Are there any 500 errors? If so, how many and how often are they occurring? Did development increase or decrease application response times? Do people like the new feature rolled out at /my/feature?,” says Enger.

    Hurdles

    The biggest hurdle in moving to DevOps is the mental shift you have to make from that of a traditional systems administrator or developer to a more DevOps-based mindset. This means the operations side learns coding and development learns about load balancers.

    “If we propagate continuous change in our computer systems, we should also expect to be continuously changing our mental models to adapt to new ways of doing things,” says Enger.

    Benefits You Can See Through

    Transparency is the number one thing the Reverb.com developers gained from this new CD approach. “Capistrano tells HipChat [https://www.hipchat.com/], “Hey, code is going out!” and I know to pull up my dashboards in Kibana and DataDog to watch for any anomalies,” says Enger. Humans can’t always be expected to communicate to each other efficiently. Computers help the Reverb.com team communicate across the organization through programmed notifications as to when changes go out.

    Code deploys are now predictable and automated. No more worrying about a failed deploy due to a bad connection. “When failure does occur, we are able to open up the Jenkins Build output and find the error pretty quickly,” says Enger.

    A Nice Surprise

    One really positive result that surprised the development team after implementing CD/DevOps is that they learned a lot about their user behavior and traffic patterns. “Now we know that 4.30am is the slowest period on the site,” says Enger.

    The team knows exactly when is the best time to roll out big configuration or code changes. They know immediately how deployments affect users and especially whether they ever break critical paths in the site, such as the checkout or signup functions. And the automated approach that DevOps provides enables them to fix bad changes within minutes of their original deployment.

    The Affirming Roar of Fan Feedback

    Reverb.com recently introduced more automation and monitoring to prevent site downtime and abuse by bots. The feedback: Customers don’t care about DevOps; they care about sites being operational and efficient. But it’s DevOps that makes that happen!

  • DevOps: Where are we with that?

    DevOps: Where are we with that?

    There’s a great scene in the Kids in the Hall movie Brain Candy, where the CEO of a major pharmaceutical company informs his executive team that the board has recommended a massive restructuring plan that will cut their research outlay by 60%. Only seconds after first mentioning this undertaking, the CEO turns to his yes man, Marv, and asks:

    “Where are we with that?”

    Marv responds: “The thing you just mentioned just now?”

    And after getting an affirmative response, Marv foolishly, yet not surprisingly answers:

    “Oh, we’re on top of that, Don.”

    It’s not hard to imagine this same scene playing out in real life where the restructuring being recommended is that found in DevOps. Marv and his fellow executives have agreed to a restructuring plan that’s going to dramatically affect the flow of work and the lives of many people at their company, but without having any real idea of why this plan is needed, how much it will cost, or even where to begin. They’ve just been asked to reduce the cost of their product by 60%, and to begin, well, five seconds ago.

    Sound familiar?

    There was a time when DevOps was possibly an even bigger buzzword than it is today, but that doesn’t mean it has gone anywhere, or lost popularity. In fact, its adoption has only increased since the term was coined, and it can be found everywhere from the smallest shops—all the way to massive enterprises.

    So why is it still so hard to successfully pitch DevOps to executive leaders?

    We’ve written before that the ambiguity around DevOps’ definition is one of the primary causes of the confusion surrounding the term, and the debate between whether it’s a “mindset” “practice” or “cultural shift” will likely never die (just don’t let anyone tell you it’s a “tool”).

    Rather than argue the buzzword’s definition, many at last year’s DevOps Enterprise Summit argued that the best way to promote a DevOps initiative at your organization, especially to your executive leaders, is to not call it DevOps at all.

    Since I’ve always liked the term DevOps, it’s taken me a while to come around to this concept, but I think those esteemed speakers are on to something.

    Vendors, analysts, the media, and other stakeholders have all had enough time to develop their own personal assumptions around DevOps. Maybe it was first proposed to them by an overblown consultant, or it was part of the failed vision of a former executive who is now on the outs. All it takes is one negative DevOps word association within your leadership and your noble pursuit of its buy-in could be quickly derailed.

    So if you don’t call it DevOps, what do you call it? “Lean IT?” “Agile Innovation Processes?” “Continuous Flow Delivery?” “ Industrialized Software Development?”

    It’ll take you more words, but I’d recommend using DevOps’ results, and not any perceived buzzworthiness of the term alone, as its primary selling point. While “more words” means “more time”—something your e-staff likely has very little of—you’ll both get and hold their attention much quicker than just saying “DevOps” and expecting the applause and champagne to start flowing.

    Gene Kim’s definition of DevOps supports this concept, and in a recent interview he remarked:

    I would define DevOps by the outcomes. In my mind, DevOps is those set of cultural norms and technology practices that enable the fast flow of planned work from, among others, development, through tests into operations while preserving world class reliability, operation and security. DevOps is not about what you do, but what your outcomes are.

    For those who are somehow just learning about DevOps, or for those who have known of its existence for years now, but who have been hesitant to pitch it to their own leadership, the good news is that the outcomes of DevOps are being proven not just at big names like Disney, GE, and Netflix—they’re even being felt by those who’ve implemented it at non-Dev or Ops fields like Security, and even Design.

    Your VPs of App Dev and QA will likely be impressed by your mention of accelerated feedback loops and reduced defects. Your IT Operations leadership, no matter what distrust they harbor for app dev, would have a hard time arguing against increased visibility into the delivery pipeline, and better quality when the software lands in production. Then when the CEO hears about faster service delivery and improved time to market, you may as well start sketching out your plans for that corner office.

    (Editors note: Author’s promise of a corner office is his own, and is not guaranteed by Skytap)

    About the Author/Noel Wurst

    noel wurst

    Noel Wurst, Managing Editor at Skytap, Noel is responsible for setting and executing Skytap’s content marketing strategy. That includes overseeing the creation and distribution of high-quality, targeted content, and managing the team and external resources associated with those efforts. By leveraging deep online marketing, project management, and editorial experience, Noel plays a crucial role in helping generate and nurture leads with compelling content. You can follow Noel and Skytap at @NoelWurst and LinkedIn and @Skytap, Skytap’s LinkedIn 

  • Five fruitful DevOps business drivers gaining speed

    Five fruitful DevOps business drivers gaining speed

    In talking with those in the enterprise trenches working to make DevOps a reality – admins, developers, CIOs, and business leaders who get it and are trying to help transform their enterprise culture and workflow to be more DevOps oriented – a number of key drivers surfaced from these discussions.

    What drivers are they? Below are the Five Fruitful DevOps Business Drivers Gaining Speed right now that many say are currently acting as a catalyst for pushing DevOps forward in their organization.

    These are in no particular order, nor is this list meant to be all inclusive, but these benefits are what was top of mind when speaking with enterprise IT over the past couple of months:

    1. Clearing application backlog. Enterprises are busier than ever before, and more employees, partners, and customers are requiring more apps to complete their day-to-day work. These demands, plus the pressure in recent years to build more apps for more form factors is pushing many development teams further and further behind schedule. They find themselves buried in application backlogs.

    Increasingly, seeing the speed and reduction in errors, more enterprises will clear through their app backlog with DevOps.

    2. Improving mobile development workflow. The needs of mobile users are constantly changing, as are mobile devices and mobile apps. Mobile devices and operating systems are a never-ending river of new capabilities, updates, and integrations. The evolution is so rapid that traditional mobile development and operations teams have trouble keeping up.

    It’s clear that creating dedicated mobile development teams isn’t working for all organizations that seek to maximize their productivity. But DevOps and continuous delivery and integration strategies make it more likely that development teams will be able to build and deliver these apps as fast (well, almost) as their enterprises demand.

    3. DevOps is mainstream. The debates over whether DevOps is real, or not, have come to a close – except for a few who may have kept themselves buried too deeply in an underground data center for too long. The question about whether DevOps is having a positive impact on enterprise outcomes is over.

    The discussion now switches, most everywhere, how to move rapidly to a DevOps organization, or how to improve the effectiveness of existing DevOps efforts.

    There will be more organizations working to collaborate more effectively, increase automation, and integrate teams to increase effectiveness. DevOps and collaborative toolsets and movements such as ChatOps will also mature and help accelerate the transition.

    4. Design teams, work more closely with DevOps teams.

    It’s surprising that DevOps isn’t more entrenched in enterprise mobile design. One of the key benefits of DevOps is the ability it provides for enterprises to design and build applications more quickly, and be able to react more swiftly to market demands. It’s why DevOps can be so powerful for mobile teams. More effective mobile app development, testing, and operations means more effective business integration and focus on designing apps that the business needs. “all DevOps does is make sure your app works well. It has nothing to do with it being popular,” says Katz.

    Some have called this focus on building better UX/UI functionality DesignOps, and more organizations are working to bring the same levels of collaboration and team integration with design and development as they’ve down with operations and development.

    5. DevOps and Better Feedback from Business Leaders

    With the enterprise embracing DevOps, certainly that is a culture change but the toolsets and the application service delivery platform is also changing, and with new toolsets and new workflows and tighter levels of collaboration and workflows means new assumptions of how business management can or will work with IT. Savvy businesses recognize the opportunity and are already engaging with their IT departments.

    It’s the manifestation of what Gene Kim called The Third Way: Culture of Continual Experimentation and Learning:

    The Third Way is about creating a culture that fosters two things: continual experimentation, taking risks and learning from failure; and understanding that repetition and practice is the prerequisite to mastery.

    We need both of these equally. Experimentation and taking risks are what ensures that we keep pushing to improve, even if it means going deeper into the danger zone than we’ve ever gone. And we need mastery of the skills that can help us retreat out of the danger zone when we’ve gone too far.

    As part of the culture of “The Third Way” business leaders need to help guide IT on some of that risk taking – where it makes sense, were it doesn’t, when those risks could affect direct business outcomes.

    This will help DevOps keep IT more aligned with business goals than was largely possible in large organizations previously.

  • A DevOps Metamorphosis Echoes Through Reverb.com, Part One

    A DevOps Metamorphosis Echoes Through Reverb.com, Part One

    “Reverb.com is an e-commerce platform that connects musicians, enabling them to buy and sell used gear and instruments,” says Adam Enger, Senior Infrastructure Engineer, Reverb.com. As an ecommerce site, it is critical to Reverb.com that its workflows such as checkout, offers to purchase, and messaging work in tandem with its selling workflows, which include creating new listings and managing orders.

    The Reverb.com team also develops a free, comprehensive online price guide and a complex in-house content and curation system that allows them to hand pick and recommend the best gear for their customers. In the Reverb.com marketplace, all software components must run smoothly for customers to have the optimum research, buying, and selling experience when conducting trade on musical equipment.

    To create those components, Reverb.com developers code primarily in Ruby on Rails on Macs. The etailer runs Linux in its production environment. “Our backend stack includes PostgreSQL  for our database, ElasticSearch powering our search engine, and Redis for queueing and key value storage,” says Enger.

    Not So Good, Old-Time Deployment

    Prior to DevOps approaches, Reverb.com deployed code from individual user workstations. “For configuration changes to our infrastructure we had to use SSH to connect into production. All while our fingers were crossed, of course,” says Enger. This loose-knit deployment process presented Reverb.com with some obvious challenges.

    “If a deploy failed due to an engineer being on a bad internet connection, we could potentially have had two updated hosts while three were still running an older version of code,” says Enger. This was especially problematic because these deployments often included database migrations.

    There was also no real way to know when a deployment failed or who started the deployment. “The process was devoid of any deployment log records, so errors and anomalies were harder to spot when the deployment was swallowed in the developer’s terminal,” says Enger.

    These risks made it possible for Reverb.com to face an even bigger risk: a lapse in faith on the part of its customers. Reverb.com’s reputation as a marketplace hinges on the perception of trust among its customers. Every aspect of the organization generates trust. From how Reverb.com interacts with customers as people, to how its systems present to users, the firm wants to exude confidence. “If we experience outages and slow performance, we may erode our trust with our audience,” says Enger.

    Goals For DevOps

    Reverb.com had many goals for DevOps in the hope that it would meet their challenges to fast, scalable, reliable delivery of code and configuration changes. Reverb.com needed to know when a change was going out. The firm needed to be able to deploy at a moment’s notice.

    “All processes must be behind the click of a button or an SCM post/pre trigger,” says Enger. All Reverb.com deploys must be logged and graphed. Scaling the team and the business should become easier if the teams have quick, repeatable processes that allow them to push new features and bug fixes quickly onto the new DevOps platform. This should allow for a scalable and dynamic workflow for the Reverb.com marketplace so it can be nimble, adapting to the users’ needs as they arise.

    Reverb.com’s Reasoning: An Automation Proclamation

    Reverb.com made its move to DevOps on the basis that the more the automation, the less the likelihood of human error. From that, Reverb.com development staff should create fewer issues for its customer base as consumers browse the website, make purchases, and sell hardware.

    “It’s extremely important to look at root causes here: if we only treat the symptoms every time the site slows or goes down, we will be playing Whac-a-Mole forever,” says Enger. By using the DevOps mindset, initiating repeatable and automated deploys, and measuring performance and application metrics in rea-time, Reverb.com developers should be able to trace when something was released, how it affected users, and whether customers are having a poor experience as a result.

    In the hopes that this would equip them to quickly pinpoint issues and fix root causes, the band of coders at Reverb.com set out on the road to DevOps, fired off its first continuous delivery power chord, and awaited feedback.

    Conclusion: Cliffhanger

    Tune in next time when staging-devopsy.kinsta.cloud reveals how Reverb.com adopted DevOps and used it to meet these development challenges and goals in the exciting conclusion of “A DevOps Metamorphosis Echoes Through Reverb.com”.

  • Docker tinkers with Docker Hub for performance improvements

    Docker tinkers with Docker Hub for performance improvements

    Docker’s strengthening its infrastructure components to accommodate the volume of use and wide range of use cases that inevitably crop up from such hockey-stick growth. The latest improvements come by way of refinements made to Docker Hub over the last month, according to an announcement made by the company yesterday.

    The Docker Hub team reported that many of the changes were precipitated by performance issues experienced at the end of 2014, following the addition of security enhancements to prevent brute force login attempts. They reported that the enhancements added “overhead to the login process.”

    “Under normal load there wasn’t much difference, but as the number of login attempts increased the site performance started to degrade,” they wrote. “The decreased performance affected not just the Docker Engine logins, push, and pulls, but also the Docker Hub website as a whole.”

    In the process of identifying immediate issues causing the problem, Docker’s engineers audited the entire system and is in the middle of rearchitecting the authorization, push and pull system to improve performance and isolate systems for a bulkhead effect, ensuring that there aren’t cascading problems.

    In the immediate term, some of the fixes already made include bulking out infrastructure capacity, including a tripling of servers in the Docker cluster for four-times the compute power and a production database upgrade for twenty times the RAM. It also improved login infrastructure to include added database indexes and more caching. The team also tuned queries for speed and fixed an issue that was causing database locks. Docker also switch its background worker queue broker from Redis to RabbitMQ and moved background workers to their own server cluster. Additionally, improvements were made to monitoring and metrics gathering, as well as to search functions.

    Meanwhile, down the line they expect to make further improvements that include API throttling,  breaking up larger services to improve scalability, replacing the brute force prevention library with a better performing library and adding API metrics and monitoring to head off performance issues earlier.

    “We have made some good progress, but we are not finished yet, and we still have a lot more planned,” the Docker Hub team wrote. “We will continue to make improvements with every release, so that Docker Hub only gets faster and more reliable as time goes on.”

    Such hiccups as the ones in December are understandable given the meteoric growth of the Docker community. Even just six months ago, the total number of downloads of Docker were under 3 million. By December that number had risen to 100 million. In spite of some questions about the security of Docker containers, Docker’s trajectory seems promising. Just in September the company closed on $40 million in series C investment from Sequoia and over the past few months it has been announcing a number of high-powered partnerships, including those with VMware, Microsoft and AWS.

    Docker’s most recent partnership was announced last week when it teamed up with Google so that the online giant could offer a private registry service for Docker on its cloud platform. Currently in beta, the Google Container Registry will provide Google cloud customers with secure hosting, sharing and management of private container repositories that will include strong access control and server-side encryption.