staging-devopsy.kinsta.cloud

Category: Features

  • staging-devopsy.kinsta.cloud sponsors FlowCon

    staging-devopsy.kinsta.cloud sponsors FlowCon

    staging-devopsy.kinsta.cloud is happy to be a media sponsor of FlowCon this year. FlowCon is a popular event held in San Francisco that focuses on lean product development, continuous delivery, lean UX, and (our favorite) DevOps. From the show producers:

    FlowCon brings together technologists and industry leaders passionate about innovation through lean product development, continuous delivery, lean ux, and devops. We’ll be exploring the role of culture, technology and design in growing organizations that thrive in an environment of continual change. We will provide inspiring and actionable information for key decision makers responsible for products and services that depend on software.

    Our full program includes speakers from Google, Netflix, Heroku, Nordstrom, Soundcloud, Macy’s, HP, Joyent, ThoughtWorks, and IBM. The second day features an open space unconference, and workshops from Don Reinertsen, Mary and Tom Poppendieck, Whitney Johnson, and Sarah B Nelson.

    This conference is especially relevant for our audience because of the intense focus on environment with “continual change.” DevOps is a necessary ingredient for this environment because when the team is focusing on innovation and improvement, it becomes vital to have tedious tasks automated – not just to save time, but to ensure accuracy and efficiency. The most valuable resource at many organizations is time, and one mistake in a simple process can be critical.

    Our mission is to help build the DevOps community; to help the DevOps movement cross the chasm into the mainstream. We want to unite entire organizations behind a DevOps culture, taking lean and agile from being an exclusive club where only the dev team is invited, to being integrated into the overarching company strategy. Everyone is invited.

    We’re bringing this message to FlowCon. Come join us. If you do use this code:  Devops50,  for $50 off your registration.

    FlowCon Banner 2

     

    This is from last years event:

  • Mobile App Development Will Soon Grow More Enterprise-Focused

    Mobile App Development Will Soon Grow More Enterprise-Focused

    As the global mobile app market begins to swell, many consumer-focused developers inexperienced in the ways of enterprise development will soon follow the money to become better initiated, a new report out this month explained. Until then, the more mature types of development methods and tools that enterprise developers depend upon are still relatively scarce in the mobile app development community.

    According to analysts with VisionMobile, which recently released its Developer Economics State of Nation Q3 report,  mobile developers accounted for 12.6 percent of the global developer population in 2013, but the number of developers in these ranks is expected to grow to close to 5 million in the next three years.  As things stand at the moment, more than two out of three mobile app developers currently target consumers. And even among those 27 percent who do focus on business users, 11 target individual business users—the independent BYOD early adopter type that many an IT department  has struggled to keep up with.

    That leaves just 16 percent of the mobile app developer population focused on the enterprise, with about half of them working in the enterprise environment and the other half in third-party or freelance situations.

    Given those demographics, it is no surprise how few tools the entire class of mobile app coders uses during development. Currently, just 9 percent of these developers use any kind of automated app testing tools, only 16 percent use crash and performance monitoring tools, and just 21 percent use beta testing tools. Even the use of cloud computing tools lags, with just under 30 percent using services such as AWS or Heroku to aid their development.

    It’s a perplexing problem, according to the report’s authors.

    ” Developer tools add value at every step of the developer journey from prototyping through to customer support,” the report stated. “It makes sense that hobbyists with no intention of making money from their apps would avoid tools that might incur costs at some point. However, 26% of developers that are interested in making money don’t use any third party tools.”

    This could potentially change as the economic incentives for more mobile app developers to shift to the enterprise and business users push them to create the more dependable apps features that these users demand.

    According to VisionMobile, overall the app economy stacked up to $53 billion in 2012, and its analysts predict 28 percent growth annually through 2016, when the market will reach $143 billion. Currently, the enterprise market leads the pack in terms of revenue generation opportunity, with developers focused on enterprise audiences far and away leading the pack in revenue from their apps, followed by those focused on individual professionals.

    “Smartphones and tablets were introduced as consumer-first technologies, which led to developers initially building consumer apps. Now a growing number of enterprises are adopting mobile technology; this is all businesses, governmental and non-profit organizations, not just large corporations with IT departments,” the report said. “The platform owners are providing clear signs that there is much greater enterprise adoption and significant opportunities for developers to come.”

    In an earlier report this year, VisionMobile predicted that the business and productivity market will double in size by 2016, reaching $58 billion by then.

  • In Support of DevOps: Kanban vs. Scrum

    In Support of DevOps: Kanban vs. Scrum

    The Kanban software development methodology is getting a lot of attention as of late, particularly for its ability to enable DevOps. Some organizations are even going so far as to move from Scrum to Kanban to improve efficiencies. We use Scrum at OutSystems, since we’re doing feature development that requires stakeholder feedback. The sprint lock and end-of-sprint demos to stakeholders are invaluable to us. But I was curious to see how the two methodologies differ and how Kanban may further enable an organization’s move to DevOps. Here’s what I found.

    There are two key differences between Scrum and Kanban: the rules and the workflow.

    The Rules

    Both software development methodologies provide rules for how work is performed. If your organization uses Scrum, then you know that it is quite prescriptive. Agile Advice lists 23 mandatory and 12 optional rules for implementing Scrum. These include, for example:

    • A product owner creates and manages a product backlog
    • Teams are cross functional
    • Interruptions are prohibited during sprints
    • Work is time boxed
    • Scrum meetings are held daily
    • A burn-down chart is used to measure progress

    Taken together, these rules create a fairly rigid box in which teams must operate in order to be successful with Scrum. As I see it, this presents two big challenges. First, few organizations actually follow all of the rules, leading to what is called ScrumBut – We practice Scrum, but… ScrumBut means a company ignores specific Scrum rules for internal reasons, leading to a suboptimal use of the framework.

    Second, at the core of Scrum is the time box. Time boxes are good for providing distraction-free time for developers to deliver the product, and define regular milestones for stakeholders to evaluate and steer the project. But, from a DevOps point of view, these artificial delivery checkpoints break the flow. Coordinating dependencies between sprints and ensuring their successful move to production becomes a big challenge.

    Kanban, on the other hand, is less restrictive. Its rules are as follows:

    • Visualize the workflow
    • Limit work in progress

     With only two rules, Kanban is an open and flexible methodology that can be easily adapted to any environment. (Some companies are using it across the entire organization – from manufacturing to marketing.) You can add rules as needed, borrowing from Scrum, if you so wish.

    The focus on the flow instead of time boxes makes Kanban an attractive choice to use with DevOps. Deploying work items becomes just another step in the process, and with its emphasis on optimizing the whole delivery rather than just the development process, Kanban and DevOps seem like a match made in heaven. With the contrast in rules understood, let’s look at the workflow.

    The Workflow

    In addition to rules, the workflow differs in Kanban vs. Scrum. This is a direct consequence of the rule sets. In Scrum, you decide beforehand what features will be completed in the next sprint. The sprint is then locked, the work is done over the course of a couple weeks (the usual sprint duration), and the queue is empty at the end of the sprint. Locking the sprint ensures the team has the necessary time to work on a problem without being interrupted with other apparently “urgent” requirements, while the feedback sessions at the end of the sprints ensure stakeholders approve the delivered work and steer the project as the business changes.

    In Kanban, you do not have the time constraints of a sprint, and the focus is on making sure the work keeps flowing with no known defects moving downstream. However, limits are placed on the size of the queues. This means that a maximum number of features or issues can be worked on at a given time, allowing the team to focus on delivering those work items with high quality. Making the flow visible creates a sense of urgency to keep things moving along. Note that Kanban was born in manufacturing, and its big focus is on productivity and efficiency. Kanban has to be extended in order to incorporate fundamental aspects of software success, such as stakeholder participation.

    Enter DevOps

    Organizations that adopt DevOps are looking to increase efficiencies, deploy features more frequently and be more responsive to business demands. You can see, then, how each methodology addresses various aspects of DevOps to greater and lesser degrees. Choosing one over the other isn’t cut and dried. Here’s what I advise:

    If your team is responsible for feature development, which requires stakeholder feedback, and developers need to focus, Scrum is probably a better fit. The sprint lock and end-of-sprint demos to stakeholders can be invaluable.

    If your team is responsible for maintenance and tends to be more reactive, then consider Kanban. There is no need to lock your backlog and you have more flexibility to respond to customer input.

    Ultimately, every team is different. You know where your team excels and where it needs extra guidance. I suggest you borrow from both methodologies to give your team just what it needs to achieve DevOps’ goals.

    About the Author

    Rodrigo CoutinhoRodrigo Coutinho is Principal Software Engineer, OutSystems. A member of the founder’s team at OutSystems, Rodrigo has a passion for web development, great products, and geeky stuff. He spends his time designing future versions of OutSystems Platform and dreaming about the cool future of the web.

  • DevOps by any other name still gets things done

    DevOps by any other name still gets things done

    My version of “A rose by any other name would smell as sweet” is not nearly as eloquent as Shakespeare’s, but the sentiment is the same. Juliet argues in Romeo and Juliet that the name of a thing such as a rose is irrelevant—it’s still a rose no matter what you call it. The same thing is true when it comes to DevOps.

    The term DevOps has achieved industry buzzword status. Most people—at least most people in IT—are familiar with the term even if they don’t have any idea what DevOps is. Even among those who think they know what DevOps is, there may not be consensus on how to define or implement the concept. It is understandable that some people don’t have a clue what DevOps really is.

    The Wikipedia entry for DevOps is a great illustration of the issue. Wikipedia is a crowd-sourced encyclopedia, and as such it is comprised of a variety of contributions. The first few paragraphs of the DevOps Wikipedia entry actually offer up different attempts at a definition. The various explanations don’t necessarily conflict directly with one another, but they also don’t agree completely and reading the Wikipedia entry will leave most people just as confused about what DevOps is or isn’t as they were before they started.

    It’s a “software development method that stresses communication, collaboration and integration between software developers and information technology (IT) operations professionals.”

    The goal of DevOps is “to maximize the predictability, efficiency, security and maintainability of operational processes. This objective is very often supported by automation.”

    It “targets product delivery, quality testing, feature development and maintenance releases in order to improve reliability and security and faster development and deployment cycles.”

    It “aids in software application release management for a company by standardizing development environments.”

    All of these are true, and yet none of them is necessarily THE definition of DevOps. The simple reality is that it doesn’t really matter.

    DevOps is not a single thing per se. It is a culture, and how that culture is embraced or implemented van vary from one organization to the next without altering the fact that both are employing DevOps. DevOps is not a product or tool–it’s a culture that has evolved organically to meet the needs of a more rapid pace of IT. Many organizations are utilizing DevOps processes or tools, and have no idea what DevOps is.

    If your organization has developers who manage operations, or IT ops staff who do development, or developers and IT operations staff who work closely together to get things done, you might be doing DevOps.

    If your organization uses scripts or other tools to automate the provisioning and/or configuration of servers, virtual servers, or entire virtualized environments, you might be doing DevOps.

    If your organization rapidly develops new iterations of an application, or continuously rolls out new features and tools, you might be doing DevOps.

    If your organization continuously monitors for suspicious activity or potential malicious attacks, you might be doing DevOps.

    If your organization makes extensive use of the cloud in order to be more agile, and be able to scale quickly to meet demand, you might be doing DevOps.

    If your organization lacks rigid structure between departments, and functions more like a startup where everybody does everything, and just does what it takes to get things done, you might be doing DevOps.

    If your organization automates simple tasks and functions with technology rather than wasting valuable man-hours doing everything manually, you might be doing DevOps.

    It doesn’t matter if you understand what DevOps is. It doesn’t matter if you call it DevOps within your organization. All that matters is that you take the elements that make sense for you, and adopt them in order to function more effectively and efficiently, and remain competitive.

  • Q&A with Aater Suleman: Successfully moving to DevOps

    Q&A with Aater Suleman: Successfully moving to DevOps

    For this Q&A on DevOps implementations, we caught up with Aater Suleman, co-founder and CEO of Flux7, an Austin-based IT consultancy that is focused on cloud technology, DevOps, and advanced development. Suleman began his career in hardware architecture and performance optimization, but has spent nearly a decade now working with cloud technologies and is a frequent speaker at DevOps and Cloud development events. Suleman also serves as a member of the faculty of the University of Texas, Austin.

    We wanted to ask Suleman a number of questions regarding what he’s learned from the DevOps implementations he’s been a part of since founding Flux7. He’s that interview.

    DevOps: When you see teams moving to DevOps, what do you think they need to do in order to move forward with good workflow?
    Suleman: A very strong focus on automation and continuous improvement is required. And to succeed there, you need to monitor developer workflows very cautiously.
    Simply looking at architectural diagrams and making plans is not enough. You don’t learn much about an organization from diagrams. You want to look at the IT infrastructure, sure – but you need to break into conversations with developers and IT team members about what they are actually doing day to day. You will learn much more from those interviews. When we interview developers and IT team members, it’s astonishing how much emerges out of these discussions. Companies learn that they really don’t know as much as they thought they did about their employee’s day-to-day activities and workflow.

    DevOps: What are some of the common lessons learned?
    Suleman: One of our clients recently requested that we help them set up Continuous Integration and Continuous Delivery (CI/CD). I insisted that make sure that was what they actually needed. We held conversations with business executives to try to understand their business requirements, and then moved along the rest of the enterprise and IT stack.

    And through that process, it became very clear that while CI/CD would be great, they did not have anyone with the skills to craft automated tests. So even if we put in place a framework for automated testing, there’s no QA person who could write those tests.

    Another thing that became very obvious was that their developers were experiencing a lot of pain in their development environments. They were trying to perform basic testing, but they did not have an efficient way of testing from their laptops.

    The developers were, for instance, logging into a remote server for testing, but the Internet connection to the server was slow enough that the connection would break. It was common for them to press a key and not see a result for a long period of time.

    It was a very frustrating environment for them to be able to get any kind of development done. That’s when the priority actually became to build a robust developer environment for this firm.

    Fortunately, and this is where interviewing team members really benefits, we discovered that one of the developers had actually created a local developer environment. He hadn’t told anybody, and was using it on his own laptop, but had never standardized it for use across the enterprise.

    The interesting story here is that they, despite wanting to move to CI/CD, they were not yet ready. Rven more interesting, they already had a solution to their biggest problem in the company that just nobody knew about.

    DevOps: What do you see as some of the ways enterprises often fail?
    Suleman: DevOps implementations tend to be very buzzword-driven. People have heard of CI/CD, and they want to implement integration testing right away, because everyone’s saying that it’s a good thing.

    But you have to look at your specific needs and where your top issues are right now from a business requirement and delivery standpoint. CI/CD is an excellent thing, and I think it needs to be there for every single team. But what’s also important is to see exactly where your current challenges are and address those first.

    Also, typically, if it’s a mid-size startup that is just beginning to hit the rapid “hockey stick” phase of growth, those tend to make for very interesting assessments. That’s when there’s enough flexibility in the company to make quick changes with very big impact. And that’s why it is also a very important time for companies to do something like this is.

    One aspect of assessments at this stage of the business is whether this infrastructure team really understands the need for the enterprise to have the ability to scale. I’ve seen infrastructure teams make mistakes on both ends of the equation here. At times, the IT teams are overestimating, overvaluing the scaling they need. And they learn from the business team that, while they are scaling, there’s a more important challenge somewhere else.

    The other extreme, at times, is the business teams are foreseeing a huge hockey-stick style increase in demand, but IT is more focused on putting out more features, and deploying more, instead of scaling the infrastructure. We see these kinds of disconnects all of the time.

    DevOps: What are some easy wins enterprises can achieve when it comes to being able to innovate more quickly?
    Suleman: Maintaining uptime and reliability in a DevOps environment also means that your QA team has more responsibility to production-ready and developers also now need to be move out code that’s relatively clean and correct.

    Another aspect is that developers still need an IT playground so that they can innovate. You want to provide them with environments that are local to them, so they’re not afraid to try to build creative stuff. And these environments need to be fast, and they need to be production-like. And they should be completely isolated and easy to recreate so that developers can innovate fast. And if they break something, the cost of a mistake is very low, because that’s the only way you innovate, is by lowering the cost of failure.

    That’s exactly how we did it, actually. So killing and spinning up a new dev environment was done in one minute and twenty seconds. And developers leveraged that capability quite a bit. Many developers are now actually rebuilding the environment multiple times a day. Previous to having an automated development test environment, the developers didn’t innovate as much because they didn’t have the ability to fail fast and cheaply.

  • Recent Stats Show PaaS Positioned At Tipping Point

    Recent Stats Show PaaS Positioned At Tipping Point

    The numbers are in and it seems that growing demand for greater speed and more line-of-business control over application development has pushed platform as a service (PaaS) to a tipping point of ubiquity.  Several recent surveys show the state of PaaS is in rapid growth mode as a number of application deployment trends have raised the bar for continuous delivery.

    One of the surveys, conducted among 1,358 IT and business pros by GigaOm Research and North Bridge Venture partners, clearly illustrated the dramatic rise in PaaS usage with just 7 percent of respondents stating they used PaaS in 2011 and 41 percent saying they now use the technology. Within two years, the survey shows that number will rise to 62 percent. And that’s slightly more conservative than a different survey conducted by Vanson Bourne on behalf of Progress, which shows that 70 percent of 700 respondents are either already using PaaS solutions for application development or are planning to use them in the future.

    The second survey points to increased pressure for even faster application delivery as a major driver of this push, with 85 percent of IT leaders reporting demand within their organizations to further reduce the time it takes to develop and deploy applications. That demand is coming from the top reaches of organizations and from the most important outside influencers—approximately 43 percent of respondents said rapid development was demanded by the C-suite, while 47 percent said the pressure was coming from existing customers or partners.

    The push for faster development from business stakeholders is understandable given overall cloud trends. The GigaOm survey reports that 65 to 70 percent of respondents will move some or significant business application processing to the cloud in the next one to two years. This trend is led by front-office business functions, with 52 percent of sales and marketing moving significant portions of application load to the cloud 44 percent of customer service and 44 percent of business analytics.

    This movement is led by a vanguard of those championing what executives at Progress are dubbing the DYOA movement—develop your own application—as many departments are increasingly trying to get what they need out of applications by developing their own.

    “Departments are keen to develop their own bespoke applications for specific business functions rather than relying on developers to translate their business needs,” says Mark Troester, a product management leader for Progress, who explains this is in turn driving further PaaS uptick

    Rob Walters, CTO of Engine Yard agrees that PaaS will not only be used to help speed up development in IT but to allow line of business some flexibility while metering the chaos that would come from a complete circumvention of the central development staff.

    “Business managers commonly circumvent lengthy procurement processes and use the cloud to deploy new apps or launch time-sensitive campaigns faster. This can cause headaches for it departments that have to maintain security and regulatory compliance,” Walters says. “The most successful companies are finding ways to reap the benefits of the cloud-agility, efficiency and cost savings—while integrating security and governance best practices to ensure they’re driving business growth without undue risk.”

    The process of how that happens on the development side is still up in the air and all over the map. Some organizations have developers distributed across department,  others maintain a tighter centralized leash around projects.

    However, what’s not in doubt is that it will take more than PaaS to speed up development. How organizations use these solutions within DevOps and continuous delivery models may prove the make-or-break element of how well the organization can successful deploy with speed and agility.

  • How to Find Your Continuous Delivery Rhythm

    How to Find Your Continuous Delivery Rhythm

    This article is a part of a series of pieces on digital innovation, accelerating change and the rise of the coded business from innovators in the Chef community. In this article we present an interview with Jez Humble, Principal Consultant, ThoughtWorks, on digital innovation.

    Focusing on speed and scale allows you to respond more rapidly to weak signals in the market, and to arbitrage those opportunities quickly. This is different from the way it used to be. In the past, we’d only respond to the strong signals in the marketplace. But now, with the lower cost of investment in product development, and the resulting optionality in that process, we can service subtle market signals effectively.

    Here’s an example of this. A company noticed that when people searched for bird-cages online, very few entries came up. So, to fill the void, the company began making bird-cages. And now this business has created a new revenue stream selling bird-cages. We’re also learning that the way we service customer demand – in all its varieties – is part of the product development process. This comes with the emergence of user-experience design. And it shows that marketing, as well as technology, is an essential part of product development today. Zappos is a great case study that illustrates my point. Many companies try to minimize the time spent on customer support calls. Zappos spent a maximum amount of time on customer calls every day. And, by deeply engaging with the customer, they learn about the customer.

    Code is really the key to exploring and exploiting new opportunities in an inexpensive and scalable way. And here, we have to give credit to Toyota, which has been using Lean to show us how to re-tool on the fly in response to customer demand. As a result, software is now following automobile manufacturing.

    A major part of all this is continuous delivery of code, which, from my perspective, is fundamentally about reducing the transaction cost of putting out incremental changes. And I want to emphasize the importance of incremental changes and small batches; continuous delivery of code really can’t be effective if it’s one-piece flow.

    Think about it this way. You don’t buy eggs at the supermarket once a year. And you don’t buy an egg as a replacement every single time you use one. No, you buy eggs on an ongoing basis in small batches. This is the kind of rhythm that powers our ability to move rapidly when there’s a weak signal in the marketplace. It’s the kind of rhythm that powers our ability to change code based on customer demand. And it’s the kind of rhythm that powers our ability to compete effectively – and with a decided advantage.

    Netflix, Etsy, Amazon, Google and Facebook are five of the best at this. And I also like the way the British government’s digital service has been re-tooled and re-built with Lean, customer experience, and continuous delivery principles in mind. The government site is, without question, all about customer demand and customer service, and it has saved citizens a lot of money as well. The take-away here is that software companies shouldn’t be thinking narrowly about software; instead, they should be thinking about how their software can better serve customers. To do that, the entire business has to be run differently and more effectively. And, again, technology people – as well as product marketing people, financial people and governance people – all have to join in and help make it happen.

    There are three practices of continuous delivery that can help you create the most efficient path from hypothesis to continuous feedback:

    • Comprehensive configuration management
    • Continuous integration
    • Comprehensive automation of build, test, deploy, and infrastructure management

    A new kind of “MVP”

    One of the key ideas that the Lean startup movement has popularized is the minimum viable product (MVP), which is defined as the least possible amount of work you can do to test your business hypothesis.

    Of course, people in manufacturing have been producing minimum viable products for decades – they’re called prototypes. I like the MVP because you’re not betting the company, or spending big wads of cash.

    Another key Lean concept is validated learning; here, you gather actionable metrics based on real usage of your product, without asking people. To do this, however, you’ll need to be able to answer questions like:

    • Did the changes we made to the product improve signup, retention, revenue? Or is it time to pivot?
    • Which version of our new feature came out better when we did A/B testing?
    • All our system metrics look okay, but a user reports that our site isn’t working. Are we down?
    • Which features of our product are generating the most revenue?

    These questions should be answerable by looking at a dashboard, and the information should be completely auditable. But this kind of learning is only possible if metrics are built into the “hypotheses” that are being tested. Once you have an idea for a minimum viable product, you need to start delivering software. The first step is analysis. My view is that you need to do just enough analysis up-front, and this approach needs to continue throughout the lifecycle of the project, with the emphasis on creating very small, incremental stories.

    Doing less and learning more

    Let’s say you’re working on a hotel booking site, and you want to add a feature to allow people to choose whether they want breakfast. Don’t create this feature for all hotels, or for all partner sites. Instead, start with a “hypothesis” that allows you to add that feature for a single hotel, with no configuration options, and get feedback for that possibility before you proceed further.

    Do less, so that you can focus your efforts on learning from your users.

    If you want to increase your release frequency, it’s essential to decouple your deployments from the act of taking a feature live. Feature toggles (also known as bits or switches) help you control who can see which features. This setup also allows you to release new versions of revised software that include partially complete features, containing perhaps a story or two of work, but nothing you’d want the general public using.

    In addition to enabling incremental delivery of functionality, feature toggles let you turn off a problematic feature when a release goes wrong, as an alternative to rolling back your deployment.

    The bottom line is that continuous delivery like this liberates us by reducing the cost and risk of releasing software, and by powering much-needed innovation at the same time.

     

    jez-humbleJez Humble is a principal consultant with ThoughtWorks and co-author of “Continuous Delivery” and “Lean Enterprise.” He began his IT career in 2000. Over the past 14 years, he has been a developer, system administrator, trainer, consultant, manager and speaker. He has worked with a variety of platforms and technologies, consulting for non-profits, telecoms, financial services and on-line retail companies. Since 2004, he has worked for ThoughtWorks and ThoughtWorks Studios in Beijing, Bangalore, London and San Francisco. He holds a BA in Physics and Philosophy from Oxford University and an MMus in Ethnomusicology from the School of Oriental and African Studies, University of London.

  • The evolution of built-in DevOps features in cloud platforms

    The evolution of built-in DevOps features in cloud platforms

    These days DevOps is being widely adopted by many IT organizations. Such IT organizations become more agile and reliable, deploying code more frequently with fewer bugs and failures. DevOps helps to get better IT performance and build additional competitive advantages, and companies with high-performing IT organizations are twice as likely to exceed their profitability, market share and productivity goals.

    The adoption of DevOps is being driven by factors such as agile development methodology, significantly increased rate of production releases, wide choice of public and private cloud platforms, and a wide range of tools for data center automation and configuration management.

    Today there are many different automation tools on the market that are used by DevOps teams for provisioning, orchestration and configuration management (Puppet, Chef, CFengine, bcfg2, Vagrant, Fai, Kickstart, Preseed, Cobbler), for build and deployment automation (Jenkins, Maven, Ant, Cruisecontrol, Capistrano), for infrastructure monitoring (Zabbix, Nagios) and application performance measurement (NewRelic, AppDynamics, DripStat).

    However IT organizations still struggle with assisting both developers and administrators that prioritize two completely different goals – speed of application development and feature completion versus production stability and uptime. Automation tools vendors promote their products to solve the challenge of DevOps teams, but still very often fail to face the complexity of workflows for operations engineers, or application releases and deployment speeds desired by developers.

    Another issue is that IT organizations with application release and deployment issues usually use some kind of automation, but they want to have more flexibility to manage their automation without the need to invoke everything manually at the command-line. Ideally, this automation should be available for developers in their development environments to give developers more control of the environment and understanding of the infrastructure usage.

    These issues have driven the evolution of DevOps tools. Today, cloud platform vendors consolidate and integrate automation and management tools directly into their cloud platforms. As a result, IT organizations can get better automation features in the same turnkey cloud platform. We can claim that PaaS solutions will become the next generation of DevOps tools. Cloud platforms have already removed the need to worry about many aspects of provisioning and configuration management of virtual machines/containers. These cloud platforms solve it in a very efficient way, because they have more control of resources and events, they manage the whole cluster of resources and take into account different metrics and status indicators at the same time. As a result, these platforms provide significantly better density of resource, better utilization, smart distribution of the load across the cluster, automation of new release deployments, automation of scalability and maintenance, out-of-the-box high availability and fault tolerance features.

    In fact, many IT organizations use automation tools such as Puppet and Chef to solve the challenges of template management, updating middleware stacks, application configuration management and zero-downtime deployment of new releases. However, the latest cloud solutions include more “native cloud” features designed to solve the same tasks. There are many examples of this trend for cloud platforms to evolve in this new, DevOps, direction, providing  native automation features in popular cloud platforms.

    Let’s take one of the visionaries in the cloud computing industry – Amazon Web Services. One of their cloud services, CloudFormation (http://aws.amazon.com/cloudformation/) allows developers and administrators to create and manage resources, middleware stacks and application templates. It helps to manage configurations and updates of already provisioned resources and applications. Literally, it‘s designed to solve the same tasks as Puppet and Chef. You can find many threaded discussions related to the comparison of CloudFormation vs Puppet/Chef. OpsWorks (http://aws.amazon.com/opsworks/) is another service for application management that simplifies application deployment and lifecycle management. It helps to define application topology, specification of package installations and software and resource configurations. In addition, this service includes automation for scaling based on time or load and allows scripting of these tasks.

    Another good example of DevOps evolution is the new cloud services released by Google Cloud Platform. Recently this public cloud platform released several different features that help to solve similar tasks to those mentioned above – Replica Pool (https://developers.google.com/compute/docs/replica-pool/), Startup Scripts https://developers.google.com/compute/docs/howtos/startupscript  and Cloud Deployment Manager https://developers.google.com/deployment-manager/. These natively integrated cloud tools provide the ability to create and manage sets of virtual machines (VMs) from common templates, resize, update, or delete them, define different configuration attributes such as base image, attached disk details, attached networking objects and scripts to be executed after the boot, install new software and updates, define environment/OS variables persistently available to VMs whenever it starts. In addition it provides the ability to easily declare, deploy, and maintain complex applications using a simple template mechanism. It provisions, scales, and monitors VMs and applications. Moreover, it can perform regular health checking and can restart or re-build sets of VMs that fail to respond appropriately. Recently announced Cloud Debugger and Cloud Trace (http://googlecloudplatform.blogspot.com/2014/06/enabling-developers-to-tame-production-systems-in-the-cloud.html?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+ClPlBl+%28Cloud+Platform+Blog%29) allow developers simply setting a watchpoint on a line of code and to get a snapshot of all the local variables, parameters, instance variables and a full stack trace. It helps quickly identify and fix performance bottlenecks and enables engineers to compare performance from release to release with a detailed report. It works with zero setup uptime and no complex configuration, and is suitable for use in production. The tools described above cover a large breadth of DevOps tasks.

    Docker (http://www.docker.com/) is another good example of a tightly integrated cloud automation mechanism and a further example of DevOps evolution. The promise and focus of this automation solution are to remove complexity of configuration and deployment into different environments. Many cloud vendors have announced integration with Docker. Many observers believe that Docker is still at an early stage and not yet production ready, but the product has generated lots of industry buzz and the technology seems to be significant.

    In addition, there is something even bigger in the DevOps evolution – the open initiative for unification of cloud automation as Topology and Orchestration Specification for Cloud Applications (TOSCA) https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=tosca. The goal of TOSCA is to enhance the portability of cloud applications and services, enabling the interoperable description of cloud services and relationships between parts of cloud services and operational behaviors. Examples are template configuration and provisioning, application deployment, software patches and updates, application scaling and maintenance, etc. In other words, it drives a unified standard for independent cloud platform vendors or cloud hosting service providers. As a result, it should simplify the work and tasks of DevOps teams by creating a vendor-neutral ecosystem enabling portable deployment to any compliant cloud platforms, smoother migration of existing applications to the cloud and the ability to build and run multi-cloud provider applications.

    In conclusion, the future will bring more cloud native absorption of different automation features for DevOps needs.  The evolution of cloud platforms in the DevOps direction is extremely beneficial for IT organizations because they do not need to manage and learn different tools and approaches for automation. This unneeded complexity will be replaced by seamless integration of different automation features inside the same cloud platform. It means in the near future very popular DevOps tools like Puppet and Chef will compete with built-in functionalities of cloud platforms.

    However, during the evolution period, choosing the right cloud platform is critical. It should allow IT organizations to use multiple approaches – the old legacy automation methods and the new native features to give the maximum flexibility from built-in automation and an already collected knowledge base. Flexibility of choice is the key point in the DevOps evolution for IT organizations – the ability to use both worlds at the same time: the old automation tools and the next generation features of cloud platforms.

    In addition, it makes sense to mention the importance of seamless integration between PaaS and IaaS for the evolution of DevOps. The public cloud platforms discussed above (AWS and Google Cloud) provide merged solutions today. The integration of both layers is very important for enterprises that heavily use private and hybrid clouds, as it replaces the unneeded maintenance complexity of two different solutions and removes extra overhead on integration of products from different vendors.  In other words, merging PaaS and IaaS provides extreme flexibility of automation and ease of use in the same solution. Seamless cloud platforms provide the best combination of the automation features in the same turnkey cloud solution that help to drive innovation and maintain legacy apps.

    The next generation of cloud platforms will establish a win-win relationship between the dev and ops departments, and when the outcome of dev and ops interactions is a win-win, IT organization performance wins as well.

    ruslan4

    About the Author

    Ruslan Synyntsky, is founder and CTO of Jelastic. With over 15 years in the IT industry, Ruslan is an expert in large-scale distributed Java applications and enterprise platforms. Before founding Jelastic, he led engineering teams a and at the Space Agency, Ukrainet iQueLab, SolovatSoft, Datamesh.

  • Clone your entire IT infrastructure in the cloud at the push of a button

    Clone your entire IT infrastructure in the cloud at the push of a button

    Rapid development and deployment is a hallmark of DevOps. The faster pace doesn’t change the need for testing applications before they’re pushed to production, or QA testing the code and infrastructure, though. The tricky part for many organizations is that different groups need access to the same environment, and creating duplicate environments can be tedious.

    There are plenty of tools out there that enable an IT admin to quickly create and provision virtual servers. The virtual servers can even be created from base images that include all of the necessary applications and configuration to match the servers in production. That part is fairly simple.

    Where it starts to become challenging is the rest of the infrastructure. The virtual servers might be identical to the production servers, but unless the switches, routers, connections to other servers and applications, IP addresses, and every other aspect of the production infrastructure match identically, the testing or development done in the secondary environment might not be completely valid.

    In 2011, the core team that introduced the KVM hypervisor—now the standard virtualization technology in Linux—founded Ravello Systems to address and solve this challenge. The Ravello Systems team understood that it’s possible to migrate virtual servers to a new environment, but that doing so introduces other issues, and the resulting environment is not an exact duplicate of the original. They set out to find a way for an entire infrastructure to keep up with the pace of agile development.

    Ravello provides organizations with a way to implement repeatable deployments—complete clones—of complex multi-tier production environments at the push of a button. It’s a concept they call nested virtualization. Ravello saves a blueprint of the environment, and enables IT admins to spin up instances of the complete IT infrastructure, including networking and storage configurations, with a single click or API call.

    One organization that has adopted Ravello for managing rapid development is Deutsche Telekom. Deutsche Telekom (DT)—the fourth largest wireless service provider in the world—maintains a cloud telephony service that is a complex multi-tier application with multiple virtual machines, and network appliances from F5 Networks, Brocade, and Palo Alto Networks. DT uses Chef and Jenkins to automate many aspects of continuous integration and continuous development, but it needed a solution to keep up with capacity needs without the expense and effort involved in expanding the physical datacenter environment.

    “Creating a copy of our production-like environment in Ravello was easy. We simply uploaded our base image, and virtual appliance form factors of our physical appliances, and then used the Chef bootstrapping support that Ravello offers to bootstrap into our existing Chef infrastructure,” said Ram Akuka, director of DevOps at Deutsche Telekom HBS in a statement. “What we are building is a truly large scale system with millions of concurrent users and transactions, so by continuously developing with Ravello we can easily scale to meet our customer’s critical business communication needs twenty-four by seven.”

    The first wave of virtualization made server platforms hardware agnostic. A virtual machine of any server operating system could be deployed and run from virtually any x86 hardware through virtualization. Ravello takes that a step farther by taking the entire virtualized environment, and making it cloud agnostic. The Ravello virtualized infrastructure can be deployed on AWS, Azure, or any other cloud platform.

    Ravello also keeps the pricing and invoicing for the service simple. Ravello charges customers on a per application basis rather than per VM. The Ravello pricing also includes the underlying cloud platform. You get one bill from Ravello, and you don’t need to pay the cloud provider separately.

    If you’re struggling to scale the scope and capacity of your infrastructure to keep up with demand, or you’re having challenges with development or testing in environments that aren’t exact clones of the production environment, you should take a closer look at Ravello and see if it will help you resolve those issues.

  • ChatOps: Communicating at the speed of DevOps

    ChatOps: Communicating at the speed of DevOps

    DevOps is about knocking down barriers between developers and operations teams, and in doing so reducing organizational frictions wherever possible. And that’s certainly the spirit behind ChatOps – the term for the hyper-collaborative way of running DevOps, including operating aspects of systems and infrastructure, through online chat.

    Proponents of ChatOps say it streamlines communication in a centralized and powerful way. Born within Github, ChatOps evolved from its open source chat bot, Hubot, which has changed the way the web-based Git repository runs operations through the automation of deployment, graphing, monitoring, provisioning, and even mitigating security events.

    Think of Hubot, a Node.js app that is commonly run on Heroku, as a robot that helps keep everyone informed in a centralized place and in real-time, or in a near real-time, way. There are many benefits to ChatOps, says David Mortman, David Mortman, contributing analyst at Securosis, including the integration of ops activities with notifications into one place, making it easier for everyone to know what’s going on across the enterprise, as well as everyone throughout the organization having the ability to see error messages on events as they occur, he says.

    Github, as a highly distributed organization, with its several hundred employees spread across 103 cities, can use all of the communication efficiencies it can muster. As Mark Imbriaco, Ops engineer at Github, in his presentation ChatOps, Augmented Reality for Ops, uses Github, email, internal apps, other web systems that they’ve built to help teams communicate asynchronously – and yet they needed an option for the real-time or “semi”-real time communication. “I say semi-real time because, when we talk about chat, the social contract at GitHub around chat doesn’t require you to be immediately responsive to everything you see. Chat is still asynchronous, but it’s more synchronous than GitHub or email or other sorts of notification mechanisms,” said Imbriaco.

    Without enhancement, Hubot doesn’t do a whole lot right away. But when added to chat with one of the dozen or so connectors that include access to Google Talk, IRC, Skype, and Hubot monitors for events that come from within chat or from one’s systems and applications. “The magic happens when you build additional functionality into him to automate, well, everything,” Imbriaco said.

    Hubot will do certain things, such as notify people not present in a chatroom that they were mentioned, phrases can be registered so that notifications to those interested in certain events will be notified. Github has also plugged Hubot into the GitHub service itself. Today, when anyone creates an issue or pull request in GitHub, according to Imbriaco, a link and definition of the event is sent to chat. While Github already has ways to send notifications, by entering events real-time into the chat stream, others can jump in and comment, help, or otherwise advise on events and interact as they’re occurring.

    As for the adoption of ChatOps, Github isn’t the only one: but it’s not yet widespread among DevOps practicing organizations, either, based on our reaching out to numerous organizations. Nor, is the concept especially well known.

    That’s not slowing Owen Zanzal a developer at systems management provider VividCortex down from adopting ChatOps. Zanzal says in his blog post, ChatOps at VividCortex, that, despite ChatOps being off to a slow rise, “ChatOps has really revolutionized how we operate,” he wrote.

    The first Hubot mission at VividCortex was to help improve how code is deployed. The problem Zanzal highlights is that encryption keys and new developers apprehension around code deployment, was slowing down production.

    To streamline deployment, Zanzal moved the execution of Capistrano commands to a Jenkins (a continuous integration server), and created Hubot commands so that jobs could be created using a Jenkins exposed REST API. When done, developers needed only run a chatroom command, such as “hubot stage webapp” or to deploy “hubot release webapp,” Zanzal explained. “Now, no one is blocked by the gatekeeper from pushing code into staging or production, and every developer can push code when they are ready. The unfamiliar Capistrano commands were simplified into much easier to understand Hubot commands,” he wrote.

    VividCortex is also using ChatOps to fetch logs and more quickly remediate issues as they arise, Zanal wrote.

    Chat-as-Ops
    GitHub has also pushed ChatOps much further than simply as a place to stage and deploy code. It’s using ChatOps as a way to help actually run ops. For instance, Imbriaco spoke to how Hubot and ChatOps is used for efforts such as rebalancing shards across its file servers. “Honestly, it took me a little while to get used to some of these. We have some [Hubot actions] that do things like rebalance shards across our file server. And that’s horrifying to me, because the consequences of failure are so bad. But the reality is, I’m either going to run it from a chat room, or I’m going to run a shell script from a terminal. If I ran it here, at least everyone saw it. So at the end of the day, this is repeatable, it’s tested, it’s commonly used,” Imbriaco said. “And if it stopped working we would notice,” he said.

    And one time when Github was fending off a distributed denial of service attack, Imbriaco needed only enter the name of their service provider to direct traffic and the attack was successfully mitigated.

    Moving to ChatOps isn’t without risk, however, Imbriaco explained. A lessoned learned when it became a little too easy to deploy to Amazon Web Services. “We went from zero to way too much spend on Amazon in about two weeks, when I turned up the ability for us to provision EC2 hosts from Chat,” he said. But that issue was resolved by giving users information at the time of provisioning, such as including metadata into the instance that details who created it, as well as informing users how much it’s likely to cost,” Imbriaco said.

    Despite the efficiencies of ChatOps, don’t expect a widespread embracing of ChatOps, but more likely a slow adoption. “GitHub is a flying unicorn. They’re like Netflix,” says Mortman. “People will pick up some of what they are doing but it’s too advanced for most,” he says.