staging-devopsy.kinsta.cloud

Tag: build or buy devops

  • DevOps Stack on a Shoestring Budget

    DevOps Stack on a Shoestring Budget

    There are competing products, both open source and licensed, in any DevOps area that you can think of. The Operations groups in companies tend to accumulate such applications over time as determined by ongoing projects. Team members’ familiarity with a certain application could be a factor in adding it to the tools stack.

    Companies, therefore, end up with a mixed bag of freeware, open source or community versions of applications and software licensed from the get-go. In a startup environment where the Operations group might be on a shoestring budget, it might make sense to build the DevOps infrastructure on the cheap; using open source software (OSS) products would be one way to achieve that goal.

    The Shoestring Budget

    Below is a list of some of the applications and software for an effective DevOps environment. The list here includes only core applications that are needed to build the DevOps infrastructure. The basic communication and productivity tools that would be used by everyone in the company—and usually managed by IT—are not in the scope. Neither are the compilers and debugging and testing tools used by product development teams. However, in a small company setting, managing all kinds of tools might drop into the laps of the same people driving the DevOps initiatives.

    The objective here is to make a list of software that could be used license-free. That means some of the obvious applications in certain areas might not make it to the list. That doesn’t mean that the open source alternative is an inferior choice; availability of reliable product support is a major factor in most of the software purchase decision-making processes, and licensed software scores over OSS on that front.

    Git for Source Code Management (SCM)

    A code repository might be the first DevOps infrastructure component that would be set up even before a product idea is formalized into a company. Also, both developers and DevOps engineers need it for collaboration, within their own teams and across various engineering teams. Typically in a startup, developers get started with the DevOps work such as build and deployment automation before a dedicated DevOps team would even be part of the engineering staff.   

    In this category, It is hard toimagine choosing anything other than Git. Even though GitHub is not free for maintaining private code repositories, Git server can be setup on-premise for free. AWS has CodeCommit, another SaaS version of Git, as one of the services, and there is no extra cost for using it if you are already on AWS.

    Bugzilla for Bug Tracking

    The popular Atlassian suite of products, JIRA in this category and Confluence for collaborative documentation, seems to be default choices in respective categories. But, both on-premise and SaaS offerings of  JIRA and Confluence are licensed. Bugzilla is a powerful OSS alternative to JIRA that can support a large user base and it has been around for a while.

    If you are open to customizing Bugzilla for specific requirements, it could work better than some of its licensed competitions. Though its primary use is bug tracking, Bugzilla is feature-rich enough to be used for managing all phases of software development and even as an IT ticketing system.

    MediaWiki for Collaborative Documentation

    Though there are multiple, open source wiki applications available, MediaWiki might be the best supported when it comes to the availability of third-party extensions. Out-of-the-box features of MediaWiki are limited, but it can be extended using the free extensions available. Such customizations are very straightforward.

    Jenkins as Build & Continuous Integration Platform

    Though there is an enterprise offering of Jenkins from CloudBees, its open source version combined with freely downloadable plugins would make up a powerful integration platform for build, release and deployment automation—and any other areas in between. Describing Jenkins any more here would be simply preaching to the choir.

    Artifactory for Artifacts Management

    This is one of the applications a lot of companies might not have in their DevOps stack in the initial stage. However, the need to store built artifacts and to cache third-party tools is real, and the packages and binaries would normally end up in S3 buckets or shared filer volumes. There is no reason for not using the OSS version of Artifactory right from the beginning, as it covers the storage requirements already mentioned and much more:

    • Automated uploading of artifacts as part of the build process, typically done from Jenkins
    • Caching third-party tools locally for builds and deployments
    • Storing software artifacts for meeting build dependencies
    • Distributing packages and third-party tools for deployments

    Ansible for Configuration Management and Orchestration

    There are two great products in this space that could be used for free: the OSS versions of Puppet and Ansible. If you are on AWS, Chef is also available, at no extra cost, in the form of OpsWorks. But I would recommend Ansible for those getting started for these reasons:

    • It is an agentless tool and beginners would feel safer to roll out automation using it. A configuration management agent can play havoc with production systems if something gets misconfigured inadvertently.
    • Using the playbooks, deployments can be orchestrated very well using Ansible. In many companies, Puppet or Chef primarily covers system management aspects and there would be additional tools for deployments. Ansible simplifies that scenario.
    • Ansible’s support of Python turned out to be more useful, in my experience, than the support of Ruby in Puppet and Chef. The chances of a DevOps engineer knowing Python is much more than Ruby.
    • Ansible’s handling of host inventory is advanced, especially the dynamic generation of inventory lists. The latter feature is compatible with the provisioning of cloud infrastructure where its elasticity demands the use of dynamic inventory lists in configuration management systems.

    Nagios for Monitoring

    A lot of us in the DevOps community despise Nagios for its blandness and lack of features out of the box. But if you are willing to customize it by developing application-specific plugins, it could be converted into a very effective monitoring platform that can be used to consolidate outputs from the rest of the monitoring and various admin applications. I recommend it for these reasons:

    • Writing plugins for Nagios is very simple. Also, there are tons of free plugins available that would meet most of the system-level and third-party component level monitoring.
    • Developing plugins to add application-level monitoring and to integrate with third-party admin applications is easy.
    • When you want to move on to a next generation cloud-based monitoring application such as Datadog, existing Nagios plugins are normally migrated natively or as an integration.

    In a sysadmin environment where coding skills would be in short supply, Nagios could be of limited use as a systems monitoring application. But DevOps teams should treat it as an easily extendable tool that can be used to implement specific and fine-grained monitoring goals of the company.

    ELK for Log Management

    Splunk was instrumental in changing the way we dealt with log files, and, now there are several good products including Loggly, Logentries and Sumo Logic in that space. Though some of these SaaS products have a free tier and that might be enough for a small operation, it is better to start with ELK (Elasticsearch, Logstash and Kibana stack), mainly because the size of the logs can grow very fast and so would the usage of the log management application in the company. ELK requires substantial DevOps resources to set it up and maintain, as there are three applications involved.

    Closing Remarks

    It is important to have these essential applications, or similar ones with comparable features, implemented or made available to empower the engineering  staff. They facilitate the application development, lay the foundation for automation initiatives, and provide the toolkit to maintain production environments 24×7.

    Though it is normal, the organic growth of the DevOps organization and infrastructure can bring in OSS products with overlapping features, mainly because they could be used license-free. Carefully selecting the tools would help avoid the otherwise-inevitable clutter and the confusion later.

    One aspect not discussed is the total cost of use of a software. If SaaS offerings such as GitHub, Datadog and Loggly, and hosted versions of JIRA, Confluence and Artifactory are used, the cost would only be the subscription charges. OSS applications will need computing resources and some amount of customization and administrative overheads, which could add up to the overall cost.  

    If the objective is to minimize total cost, the upfront decision-making can become much more complicated than it looks. However, for a startup or a small company, getting started with the OSS stack for setting up the DevOps infrastructure can be advantageous if the applications are selected carefully.

  • What’s the cost of build versus buy DevOps?

    What’s the cost of build versus buy DevOps?

    Nearly every business today relies on faster, innovative technology to succeed in the marketplace. How about the coffee you drink, or the movie you’re watching, or the phone on which you’re watching it? Name every business or walk of life. You can argue that technology serves to make it better. If DevOps helps deliver the best services and experiences to you, then should companies making the coffee, movies, and phones also make their own DevOps solutions?

    I completely empathize with the challenges the developers and DevOps teams face in transforming a team or company from traditional development and delivery to an agile lifecycle. I experienced this firsthand at Trend. I led a business unit that acquired a SaaS company for online backup and storage; this young company updated and deployed code several times a day. To integrate their processes into ours was painful. I wish ElasticBox had been an option then. I cringe when I consider the time and costs of lost productivity, wasted development time, and delays in getting to market.

    Faster deployment nirvana

    Every day, I speak to several technology companies. Speed is at the top of their mind. They all experience the pain of not deploying and delivering applications to the cloud fast enough. Most have some number of cloud applications, some customer-facing and many internal on both private and public clouds. To give you an idea of how widespread this pain is, see the 2014 451 Research report. The 451 Research team reported that more than 60% of businesses wanted to deploy more frequently.

    But moving from agile development to agile delivery across an application lifecycle comes at a cost. Companies calculate the cost implicitly and explicitly. I’ve heard feedback from prospects who say explicitly: “I love your product, but I can build the same thing with my developers in xx country.” They usually have many smart development resources and tend to be born-in-the-cloud companies. Others tell me, “This is not our core competency. We don’t want to maintain a cloud application manager. We want yours.”

    Build versus buy DevOps

    At what cost do companies decide to build versus buy a cloud application manager from ElasticBox or one of our competitors?

    • First, you have to answer this question: What is your company’s core competency? Do you really want to divert resources to build your own cloud application manager versus invest in your core offering?
    • Second question: What is the urgency and need? If you have time on your side, taking a year to build your own solution may make sense. If you need a solution tomorrow, then what is the cost, in terms of market impact, competitiveness, customer satisfaction, of delaying a year.

    Once you answer that, you calculate the total cost of owning (TCO) the complete lifecycle of build versus buy. These values must encompass all the phases of building and ownership: acquisition, operation, change management, training, ongoing maintenance and internal support. Time and time again, enterprises only look at the initial build or acquisition cost and fail to fully calculate the ongoing implicit costs.

    Recently, I met with a VP of Engineering who had built a barebones deployment solution a few years ago. Now she bemoans the fact that the costs of maintaining a less-than-perfect solution is too much. Her TCO calculation included the developers, ongoing maintenance, workflow delays, meetings, support tickets, delayed product launches. Her decision to now buy is a no-brainer.

    I’m not arguing that it’s always better to buy than build. But consider the factors—core competency, time-to-market delays, and TCO. We want enterprises to keep building great software that complements their core value proposition. When it comes to agile delivery, consider an experienced partner who handles the same problem for dozens of enterprises. Simplicity, flexibility and speed—those are the barometers of agile delivery success. So developers feel productive, applications deploy faster, and your customers relish the delicious cup of coffee as they watch Netflix on their iPhone.