staging-devopsy.kinsta.cloud

Category: DevOps Toolbox

  • The Risks and Potential Impacts Associated with Open Source

    The Risks and Potential Impacts Associated with Open Source

    Open source software (OSS) is built by communities of developers who contribute their knowledge and time to OSS projects they find appealing. That code can then be used by individuals, communities and organizations in their software products—the only obligation they have is to play under the rules of the license with which the OSS project was published. 

    This type of knowledge sharing brings many benefits to OSS users as it speeds up software development time and can help companies become more competitive in the market. Unfortunately, there is also a catch. Those benefits also come with certain risks which every OSS user needs to be aware of and take necessary actions to mitigate.

    The OSS License

    One specific risk to consider involves the OSS license. Not knowing what your obligations are under the license (or not abiding by those obligations) can cause an OSS user to, for example, lose intellectual property or experience a monetary loss.

    Operational Impact on the Product

    Another risk is the operational impact that the OSS in use will have on the product. It’s important to understand whether the community is consistently managing the OSS project and its issues in a timely manner, and that your organization has implemented the latest version and deployed patches have been applied. Using out-of-date, unpatched or unmaintained OSS could impact your release capabilities, product quality and customer trust.

    Security Vulnerabilities

    The third risk to consider when using OSS is that of security risk. As open source is software—and in many cases very complex software—it brings with it the possibility of introducing software security vulnerabilities into your source code. Some vulnerabilities can be introduced into your projects through dependencies that the OSS your organization is using has on other OSS projects. Additionally, vulnerabilities could be unintentionally introduced by developers as they code a functionality using open source. The reason for this is that software developers, whether professionals or hobbyists, oftentimes do not consider security while coding. This has a substantial impact on their work and their contributions to open source projects. 

    It’s not necessarily that developers intentionally skip the implementation of security procedures. Secure coding is a specific discipline in software development that guards against the accidental introduction of vulnerabilities—a discipline that must be learned. 

    Understand that many developers are self-taught or have been trained in schools and universities (where we are only recently seeing courses in software and application security emerging). Many use their software development skills as a hobby, to contribute to OSS projects or to pursue their own personal projects. Unfortunately, training programs, programming courses in schools and online tutorials often neglect to convey this knowledge or to stress the importance of building secure software. 

    It’s also important to note that many organizations find it difficult to introduce a secure coding discipline into their software development process. In other words, this struggle persists beyond open source projects. If you’re with an organization experiencing this pain point and unsure where to begin, you’ve come to the right place. There are several strategies that can be used to begin your secure coding journey. 

    Training

    By giving developers the education required for them to become security aware, you’re setting them up for consistent, secure development success. There are a variety of in-person and computer-based training options on the market.

    Software Composition Analysis (SCA) Tooling

    Implementing an SCA tool will uncover incorrectly coded strings or vulnerabilities that have been introduced in code or OSS components. Not only can SCA tools assist developers in identifying potential software risks, some SCA solutions can help to train developers by offering remediation guidance or short training videos on how to avoid such vulnerabilities in the future.

    It’s essential for software issues to be discovered early because the deeper into the process they’re identified, the more time-consuming and costly they are to remediate. As such, in recent years there has been a push to shift left, introducing security into the product development life cycle from the very beginning. 

    Companies and individuals need to also understand that this security mindset doesn’t start and end in the coding phase of the development process. A security mindset needs to be a part of every phase: from design to deployment to maintenance, all the way through to the product’s end of life. 

    Emphasizing a security mindset also has a long-term, broad reaching positive effects on the whole software development ecosystem, including the OSS world. As developers are trained to code securely (through university courses and other training institutions, coding portals or professional development through their employer) they pass this knowledge on to the projects with which they’re involved. This then leads to the production of higher quality proprietary and open source software, which in turn minimizes risk across the board.

    — Boris Cipot

  • GitLab Updates Core CI/CD Platform

    GitLab Updates Core CI/CD Platform

    GitLab this week delivered an update to its namesake continuous integration/continuous delivery platform that promises to make pipelines more efficient and easier to manage, in addition to making it easier to set up pipelines in Windows environments.

    Version 12.7 of the core GitLab platform adds support for Parent-Child Pipelines and Pipeline Resource Groups. Parent-Child Pipelines make it possible to run separate child pipelines concurrently, while Pipeline Resource Groups enable DevOps teams to limit pipeline concurrency to better manage jobs and resources.

    A Code Review Analytics module is designed to make it easier to find Merge Requests in review that require some level of intervention, while enhanced Merge Request widgets show when changes have made it to a specific environment.

    Other new capabilities include being able to automatically stage all changes with the GitLab integrated development environment (IDE) and the ability to share group access with another group.

    Finally, GitLab has added a beta release of Windows Shared Runner, which brings Windows environments to the GitLab platform via a managed service. Darren Eastman, senior product manager for Runner at GitLab, said instead of having to separately download agent software through which jobs on a GitLab pipeline are executed, Windows environments are becoming full-fledged citizens of the GitLab platform.

    Eastman said IT organizations running Windows compared to their Linux brethren are still playing a little catch-up in terms of DevOps maturity. However, with the arrival of microservices based on Docker containers on Windows, the need to embrace best DevOps practices to manage application development and deployment is becoming more apparent, he said.

    Of course, Microsoft is pushing its own rival CI/CD platform in the form of Azure DevOps, formerly known as Visual Team Studios. GitLab is betting that most organizations will prefer to consume a fully managed CI/CD service that spans both Windows and Linux platforms, especially as hybrid cloud computing gains traction in the months and years ahead.

    Regardless of the CI/CD platform employed, the overall size of the DevOps community should expand considerably in 2020 as more organizations that rely on Windows to build and deploy applications embrace DevOps’ core principles. Naturally, competition between Microsoft and providers of rival CI/CD platforms will be fierce. Microsoft in many ways is a behemoth that was constructed mainly by a loyal base of developers that the company can be expected to guard jealously. However, as Microsoft has embraced Linux alongside Windows in the cloud, the number of Windows developers who have been exposed to alternative platforms has increased.

    In the meantime, organizations that have teams that build and deploy applications on both Windows and Linux will have some cultural challenges to overcome should they decide to standardize on a common set of DevOps processes. Rather than managing each platform in isolation, there’s a clear opportunity to reduce costs and increase application development efficiency. The issue now is determining which DevOps platform might best enable them to begin a journey that is likely to take a few years to fully complete.

    — Mike Vizard

  • Launchable Aims To Improve Delivery Velocity, With Kohsuke Kawaguchi and Harpreet Singh

    Launchable Aims To Improve Delivery Velocity, With Kohsuke Kawaguchi and Harpreet Singh

    A new company, Launchable, aims to improve your delivery velocity by helping you be much smarter about which tests you run and when. By applying machine learning (ML) and better use of the data output by your software development work, the two co-founders think they can make a meaningful impact on development efforts.

    Sure, you say, lots of developers say they can change the world. But in this case, the co-founders have some very impressive résumés to give you confidence they just may do it. The dynamic duo is none other than Harpreet Singh, former GM of Atlassian BitBucket and VP of Product and Design at CloudBees, and the one and only Kohsuke Kawaguchi, founder of Jenkins and CloudBees Chief Scientist, most recently.

    Wait a second, you may be asking, you’re saying KK from Jenkins/CloudBees has started a new company? Yes, that’s what I am saying. In a podcast that will be available very soon, KK and Harpreet talk with us about Launchable’s why—and why now. Stay tuned for that.

    It’s worth noting that KK has been at CloudBees for 10 years and working on Jenkins for 15 years. In the tech business, that is nearly a lifetime. During our podcast, KK said he felt that with Jenkins being passed to the Continuous Delivery Foundation (CDF), coupled with a healthy group of leaders within the Jenkins community, now was a good time to move on to the next adventure.

    Beyond Jenkins, CloudBees has grown into its own next phase, with lots of exciting developments announced and more coming. The scientist in KK has led him to explore solutions to problems he sees in the way so many companies are doing development. His longtime friendship with Harpreet, dating back to their days at Sun, has energized them both to once again work together and tackle the real problems that development teams are facing.

    What will come out of Launchable? I don’t know. But the company has already attracted some blue-chip VC investment, as well as an impressive list of angels including Sacha Labourey, CEO and founder of CloudBees.

    They (like everyone else, it seems) are hiring development engineers and data scientists. So if working with KK is something that appeals to you, head over to the Launchable site and check out the Careers section.

    I am sure we will be hearing more about Launchable in the days, weeks and months ahead. In the meantime, best of luck to Harpreet and KK, and many thanks for a job well done with Jenkins.

    — Alan Shimel

  • CircleCI Extends Orbs Ecosystem for CI/CD

    CircleCI Extends Orbs Ecosystem for CI/CD

    CircleCI announced today it has added integrations in collaboration with 20 partners to the family of connectors, known as Orbs, that it makes available for its continuous integration/continuous delivery (CI/CD) platform.

    Thomas Trahan, vice president of business development for CircleCI, said integrations are now available with Heroku, AWS, Cloud Foundry, Google Cloud, Red Hat, Helm, Kublr, Alcide, Azure, DeployHub, Fairwinds, VMware, Kubernetes, Rafay Systems, Pulumi, Convox, Spinnaker, realMethods, Quali, Pantheon, LaunchDarkly and Salesforce.

    Since launching Orbs a year ago, CircleCI claims there are now more than 13,000 organizations using Orbs across nearly 40,000 repositories and 9 million CI/CD pipelines. There are also more than 1,200 Orbs provided by more than 116 partners listed in the CircleCI registry. In the future, Trahan said the company plans to begin surfacing recommendations concerning which Orbs to employ based on the nature of the workflow being constructed.

    Trahan said Orbs are gaining traction because they reduce the amount of custom code that needs to be written to set up workflows across a CI/CD pipeline. Without the existence of Orbs, DevOps teams are forced to rely on application programming interfaces (APIs) that are more liable to break, he said.

    Competition between CI/CD platform providers is expected to become fiercer as adoption of CI/CD platforms continues to accelerate. A report published by Data Bridge Market Research this week forecasts demand for continuous integration (CI) tools alone will grow to $18.9 billion by 2025, representing a 19% compound annual growth rate. While the current CI/CD market is dominated by open source platforms managed by internal DevOps teams, CircleCI is betting interest in accessing a managed CI/CD service delivered via the cloud will rise as organizations opt to devote more resources to building applications versus managing the platforms they are built on.

    A recent CircleCI study of more than 30 million workflows spanning 1.6 million jobs runs per day on its platform found 80% of all workflows finish in less than 10 minutes. Those jobs spanned 150,000 projects launched by more than 40,000 organizations.

    The report noted 50% of those organizations start six workflows per day across their projects and that 50% of CircleCI projects never had a failure during pipeline changes made over the 90-day period when the study was conducted. When a failure did occur, 50% recovered in less than an hour, with 25% of organizations recovering in 15 minutes or less, according to the report.
    As organizations embrace best DevOps practices more broadly, CircleCI is betting these kinds of metrics will become more important as a platform differentiator.

    Naturally, it’s too early to say how the CI/CD platform wars will play out in the months and years ahead. But the amount of code organizations will need to deliver at increasingly faster rates is only increasing. The challenge organizations are facing now is finding a way to deliver all that code at an unprecedented level of scale.

    — Mike Vizard

  • Red Hat Extends Runtimes Middleware Portfolio

    Red Hat Extends Runtimes Middleware Portfolio

    Red Hat has made available the latest instance of Red Hat Runtimes, a suite of lightweight open source components and frameworks that makes it easier to discover the middleware most appropriate for building a specific type of application.

    James Falkner, product marketing director for Runtimes at Red Hat, said as organizations embrace cloud-native application architectures based on microservices it’s become increasingly challenging to determine what middleware to deploy optimally and where. Red Hat Runtimes not only makes it easier to navigate all those options, Falkner said, but all the components and frameworks are certified to be pre-integrated.

    The latest version of Red Hat Runtimes adds support for Open Liberty, a lightweight runtime for building Java applications developed by IBM. Red Hat was acquired last year by IBM.

    The latest installment of Red Hat Runtimes also provides access to what has become a standard distribution of Node.js based on Red Hat Software Collections that will be implemented across all Red Hat offerings.

    Finally, Red Hat has also added support for Red Hat Single Sign-On, an access control framework based on the Keycloak project.

    Falkner said Red Hat Runtimes also serves to reduce the total cost by providing a single subscription through which organizations can access everything from container runtimes to the 3Scale application programming interface (API) management middleware, which Red Hat acquired in 2016.

    Very few IT organizations these days are standardized on a single class of middleware, especially as they embrace microservices-based applications built using containers. Most IT teams for the foreseeable future will need a flexible way to access multiple runtimes in support of everything from monolithic applications based on Java to applications built using Node.js code deployed using Docker containers.

    Rather than a piecemeal approach to the middleware infrastructure required to build, deploy and integrate all those applications, Falkner said Red Hat Runtimes provides a more holistic approach.

    It’s not clear how much or even whether IT organizations are centrally managing middleware in the age of DevOps and the cloud. Historically, many IT organizations preferred to standardize on a single application server platform to deploy various classes of middleware. However, many developers today are deciding what middleware to include within their application stacks without much input from a central IT function.

    Via Red Hat Runtimes, Red Hat is trying to make a case for providing developers with the flexibility they cherish in the form of a subscription service that provides IT teams with a modicum of control over what middleware gets employed.

    Tensions between DevOps teams and central IT organizations have arguably never been higher. In fact, many DevOps teams are operating within lines of business that are outside the direct control of centralized IT. As such, the level of standardization that can be imposed by central IT organizations has become more constrained. Many centralized IT organizations are now trying to influence middleware platform decisions rather than outright make decrees that might get ignored anyway.

    Regardless of the middleware platform choice that ultimately gets made, however, the one thing that most centralized IT teams will have to come to terms with is finding a way to support it.

    — Mike Vizard

  • JFrog ConanCenter Extends DevOps Reach to C++

    JFrog ConanCenter Extends DevOps Reach to C++

    JFrog this week announced it has updated ConanCenter, a repository for storing and discovering software packages that can be downloaded for free, to include support for C++ artifacts.

    Jagan Subramanian, vice president of community and partner engineering at JFrog, said ConanCenter provides a repository for the open source Conan package manager that JFrog includes as part of its continuous integration/continuous delivery (CI/CD) platform. By making ConanCenter available free of charge, JFrog is trying to encourage organizations that rely on C++ to build applications to embrace best DevOps practices, Subramanian said.

    JFrog is also making available a free Community Edition of its Artifactory CI/CD platform for C / C++ available for download and later this quarter plans to integrate ConanCenter with other JFrog tools to make it even easier to securely access packages.

    As a general-purpose programming language, C++ has been employed for decades to build applications in IT environments where resources are constrained, such as embedded systems. Subramanian said many organizations that rely on C++ to build those types of applications are experiencing the same pressures to accelerate application development as the rest of their software development colleagues.

    To make it possible to achieve that goal, he said, JFrog is now extending its DevOps portfolio to include support for applications developed using C++ tools. ConanCenter not only provides a means to discover artifacts based on the metadata they generate, but it also provides a list of configuration options and a free binary building service that can be applied across multiple operating systems, platforms and compilers.

    Despite its longevity and the rise of alternative programming languages, C++ continues to fulfill a unique role across a range of vertical industries. There are still plenty of C++ developers are that building some of the most challenging applications to develop. In fact, given the rise of robotics and internet of things (IoT) applications involving autonomous vehicles, interest in using C++ to build new classes of applications is on the rise. The ConanCenter repository promises to make it easier for C++ developers to share artifacts across those diverse projects.

    In terms of adoption of best DevOps practices among organizations that rely on C++, Subramanian said it’s still very early, as most are still at the proof-of-concept stage. However, he said most regularly experience all the pain points that best DevOps practices address, which should drive more organizations to embrace DevOps in the months and years ahead.

    It’s too early to hazard a guess as to how long it might take organizations that rely on C++ to embrace DevOps. Many of them have been relying on waterfall-based processes to build and deploy applications for decades. Nevertheless, just like in other programming environments, the need to build and deploy applications faster requires development teams to embrace change. As always, however, the biggest challenge when it comes to DevOps has little to do with the core technology platforms. Rather, the existing culture of the IT organization always proves to be the biggest hurdle.

    — Mike Vizard

  • Che: A Revolutionary IDE for the Mainframe

    Che: A Revolutionary IDE for the Mainframe

    Tools such as the green screens of ISPF and the Eclipse desktop IDE, enhanced with proprietary plug-ins, have served mainframe application developers well over the years and, for those comfortable with them, will continue to do so. However, changes in the broader world of development are creating the conditions for a revolution in mainframe tooling.

    First, mainframe developers are aging out of the workforce, leaving behind extensive code libraries and a workforce skills gap. Those likely to back-fill these roles know and love modern IDEs, especially Visual Studio Code, while the popularity of Eclipse is waning. For example, the annual Stack Overflow Developer Survey found that, over the past year, the popularity of Visual Studio Code grew from 35% to 51%, once again making it the most popular IDE, while Eclipse fell from 19% to 14%. According to Business Insider, 8.5 million developers worldwide use VS Code, creating a huge talent pool.

    Second, application development has made great strides in productivity since the current mainframe dev tools were created, especially in DevOps automation with enablers like task runners, scripting and testing frameworks. The Stack Overflow survey cites scripting language Python as the most sought-after skill among developers, by far.

    Finally, as the velocity of overall software delivery increases, mainframe is a critical component of digital transformation initiatives. According to 451 Research, 24% of companies are releasing application software daily or hourly while, similarly, DORA’s 2019 State of DevOps survey shows 20% of teams are deploying multiple times per day. Software delivery expectations have changed with continuous deployment becoming the new normal and, to remain a vital computing platform for the long term, mainframe app development needs to support this paradigm.

    An Open Source and Cloud Development Technology 

    Rather than an evolution of the existing Eclipse IDE, Eclipse Che is a radical departure. According to the Eclipse Foundation, “Che 7 is the world’s first Kubernetes-native IDE that’s built from the ground up for cloud native application development. It simplifies and accelerates cloud development by allowing developers who are not Kubernetes experts to immediately contribute to cloud native application development efforts. In turn, Che 7 relies upon Eclipse Theia, which provides a highly modular and extensible IDE platform built on modern web technologies that runs on both your desktop or in your browser.”

    For large enterprises, a good analogy for key value proposition of a hosted IDE is Git: usability and collaborative power have made Git the de facto standard for version control and Che’s workspace server offers comparable benefits.  

    Developers working on mainframe applications can use Che via the Che4z subproject. Che4z, whose primary goal is to bring a modern developer experience to mainframe software development, offers an extension pack providing access to the z/OS file system, remote debug and remote editing of COBOL programs with all the expected modern editor capabilities including syntax highlighting, code assist and real time syntax validation. After a beta period, Che4z is now GA.

    Beyond the foundational benefits mentioned above, Che is revolutionary for the mainframe for many reasons.

    Visual Studio Code and More

    Che was architected around a VS Code-like experience from the outset which, with the growth of VS Code, means Che extends its value.  Che is based on Eclipse Theia which provides an in-browser VS Code experience complete with the latest tooling protocols: language server, debug adapter and compatibility with VS Code extensions. 

    The Che4z sub-project, contributed by Broadcom, provides language support for both COBOL and High Level Assembler (HLASM).  The language server protocol (LSP) enables developers to use these languages with Che, VS Code and any other LSP clients (e.g., IntelliJ, VIM, Emacs).

    Bottom line: developers can use VS Code for mainframe development independent of Che adoption (the Code4z extension pack is available on the VS Code marketplace).

    Cross-Platform Applications

    For many Fortune 500 companies, the mainframe hosts critical system-of-record applications and data while the cloud and mobile applications have evolved from user-oriented systems-of-experience. With tech-first startups disrupting entire industries, large companies are deploying more cross-platform applications to maintain, if not win, market share by harnessing the best of both worlds. Cross-platform apps combine modern user experiences with the transactional power and data assets of mainframes.

    Because cloud and mobile developers need to integrate mainframe applications, allowing them to use the tools they know and love is critical. This includes the popular code editors and IDEs they love today as well as the new innovations, such as browser-based, container deployed tools that make them more productive.

    With its workspace server, Che also facilitates pair programming, which is a powerful approach when mainframe and distributed developers need to collaborate on cross-platform applications.

    Bottom line: Che makes the mainframe less of a silo and more like other platforms.

    Open Source-Enabled Vendor Independence

    While open source is not new to the mainframe–zLinux has been around for years–access to off-platform toolchains became possible with the introduction of the Zowe open source framework (described in more detail below). Unlike the incumbent Eclipse vendors, and consistent with the open source vision, the Che4z extensions are open sourced and free to use. There is now an alternative to the proprietary, vendor-controlled model: true open source tooling for mainframe developers.

    Furthermore, Eclipse locked mainframe developers into using a specific set of editors, runtimes and user interfaces for required capability and functionality–thus imposing a steep learning curve. Che, by comparison, offers flexibility via modular design, allowing individual developers to choose the components they need.

    Whether developing for mainframe or other platforms, the Che community welcomes contributions from those passionate about software development. Want to incorporate a lesser-used language? No need to plead with a vendor. Anyone can seek community support for an idea or simply do it themselves. The Che/Che4z communities will ensure the proper procedures are followed before any code is ever released.

    Bottom line: Che provides a more frictionless, fast-moving toolset allowing coding innovations to extend to the mainframe.

    Onboarding and Cost Reduction

    Consider the compatibility and maintenance headaches created by the heavyweight plugins required for Eclipse. For mainframe leaders, the need to reduce costs is an ongoing challenge as is the need to quickly onboard the next-generation of developers. As a container-based IDE with no client footprint, Che eliminates the need for local installation, maintenance and configuration. The potential of moving entire teams currently using Eclipse with customizations to Che provides an attractive opportunity to materially reduce operating costs while simultaneously improving developer productivity.

    Because Che workspaces are containerized, new team members can onboard instantly–open the container, begin work. This means developers, who are high cost resources, can more easily move across projects, teams, departments, etc.

    Bottom line: For mainframe leaders focused on staffing and costs, Che increases the talent pool while eliminating costs.

    Built on Zowe

    The extensions that open the mainframe to the Che IDE are powered by the Open Mainframe Project’s Zowe open source framework, which maintains mainframe-native security standards. Zowe, which was founded by Broadcom, IBM and Rocket Software and is the first open source project based on z/OS, enables developers to manage, control, script and develop on the mainframe just like cloud and other platforms. 

    Command line interfaces (CLIs), which come with platforms such as AWS and Azure, are popular with developers wanting the productivity lift of task runners, scripting and higher levels of automation (i.e., CLIs facilitate DevOps adoption). The Zowe CLI (the mainframe equivalent) magnifies the value of the Che4z stack as any set of commands that run through the CLI can be automated.

    Bottom line: Using Zowe as the foundation for the Che4z mainframe extensions ensures the modern experience extends far beyond the IDE.

    Path Forward

    For leaders wanting to empower developers integrating mainframe code and resources with the most popular and powerful app development tools, Che provides the best of both worlds–the immediacy of VS Code use with the path to full-on, in-the-cloud team collaboration.

    Expanded tool choice, greater alignment with off-platform peers and DevOps-fueled software delivery are all reasons why Che is not only the first Kubernetes-native IDE, but a real revolution for mainframe development.

    — Jean-Louis Vignaud

  • Is COBOL Here to Stay? Five Signs IT is Ready to Modernize

    Is COBOL Here to Stay? Five Signs IT is Ready to Modernize

    With increased demands and challenges around compliance and regulations, plus growing expectations for IT to provide more value-added services, companies are looking for more flexible and collaborative options. Interestingly though, while some IT departments have updated legacy systems, COBOL code has continued to be a mainstay. In fact, COBOL just turned 60 and many are saying it’s here for the long haul.

    There are a lot of reasons that make it challenging—or for some, unnecessary—to move away from the language, but there are also several clear signs companies and IT departments should know that indicate it’s time to make a change.

    Here are five signs IT leaders should pull the trigger on modernizing their legacy environments.

    You’re Constantly Worried About the Risk of Losing COBOL Expertise from Your Team

    COBOL has more production lines of code than any other language in existence, with over 220 billion lines of COBOL today—or around 80% of the world’s actively used code. One of the problems is that many that are well-versed in COBOL language are also nearing retirement, and the amount of COBOL programmers entering the market is much lower than it was 10 years ago.

    Author Glenn Fleishman put it best in his article on the subject stating “the language never died, though its early practitioners have faded away, and the generation of programmers who built systems towards the end of the predominant mainframe era in the 1970s and 1980s are largely near or past retirement age. It’s a slow-moving crisis with no crackling deadline, like Y2K, to focus the minds of Chief Information Officers.”

    Pressure from Business Leaders to Take Advantage of More Agile IT Methods

    Whether it’s cloud, DevOps or microservices, businesses want faster and more frequent IT delivery of new services to customers. In order to drive these results, embracing more agile methods of IT and developing a strategy that can support rapid application delivery is key. Moving to the cloud, integrating with Java or shifting to Linux requires open-based models that can handle legacy COBOL applications.

    You’re Quickly Losing Your Competitive Edge Because of the Reliance on Dated Technologies

    Direct and even indirect competitors are encroaching more and more in the industry and in new deals, and you feel as if your company’s position in the market is slowly declining. This may be a product of competitors’ willingness to open source systems that promote the business agility needed to react far more quickly to changing market dynamics and customer demands.

    If they are modernizing their IT and investing in new technologies quicker than you are, it’s likely they’ll catch up to or surpass you quickly.

    Your Confidence in Your Current IT Environment Is Extremely Low

    No matter the modernization strategy you pursue, you should ultimately feel confident and peace of mind when it comes to your environment today and as you look to the future. Without flexibility, collaboration and a tailored approach however, the business will likely suffer and may be the reason for the growing insecurities.

    For instance, if you’re interested in programming COBOL in some areas, but currently operate in a significantly pure native Java environment, a flexible and manageable solution is key. If you want to generate Java as the compiled object, the result will be inflexible static Java objects, rather than something flexible and extensible — and will require access to the Java source. That source is not nearly as maintainable as the source generated in the COBOL to Java conversion, but can be leveraged in Java-operating environments, and is much more flexible than other alternatives.

    You’re Experiencing Vendor Lock-In

    You’re wedded to an established tech company or consultants on-hand that seem like safe bets, but instead have caused you to miss the mark in being flexible and future-proof–not to mention are likely far over your budget to maintain. While transitioning to a new partner may seem daunting, conducting an assessment of your operational, infrastructure and application environment can provide a more comprehensive picture of the strategy that is right for the business, help manage the process more efficiently and ultimately cut unnecessary costs well before an exact migration or conversion path is decided on.

    Although many organizations have run COBOL without substantial updates and issues for decades, the ability to solely rely on the dated technology is quickly giving way to a growing need for additional languages and more agile systems. If IT leaders are experiencing one or more of these issues now, waiting any longer to address them can result in much bigger problems in the future and considering a modernization strategy may be the better path to success.

    — Cameron Jenkins

  • Building a Real-Time App: 13 Open Source Projects You Should Be Tracking Right Now

    Building a Real-Time App: 13 Open Source Projects You Should Be Tracking Right Now

    Open source is the dominant model of software consumption in the modern era. Cutting-edge startups and entrenched incumbents alike find open source software development and community building a significant part of their overall business strategies.

    For many open source projects, independent steering committees ensure that projects remain stable and are not beholden to a single company or profit motive. Alternatively, for open core projects where a single company financially supports and contributes the vast majority of open source code, open sourcing makes a project publicly available and accessible for potential new users. Opening a project to public contribution widens the pool of potential users and contributors, therefore ensuring more rapid innovation than commercial software offerings.

    Choosing the Right Open Source Projects for Your Real-Time App

    There are seemingly limitless open source projects to evaluate. Further complicating matters, each project has its own domain competence and cultural nuances. Considering a single distributed application will have tens or (many) more open source dependencies, development teams must continuously test new open source offerings and hone new skills to use these solutions, while simultaneously architecting their applications.

    In order to help sort through the noise, I’ve looked at six categories of open source projects for this article: data storage, message systems, service meshes, REST frameworks and streaming frameworks. For each category, I’ll identify any dominant players as well as some other projects of note.

    Here are the top 13 open source projects by category.

    Data Storage: Cassandra, Hazelcast

    Apache Cassandra, originally developed at Facebook, is one of the most widely used open source projects for real-time applications. Functionally, it’s a distributed NoSQL database management system. But practically, it’s become the linchpin of most at-scale, real-time applications. This is because Cassandra was designed to handle large amounts of real-time data across many servers, and has robust clustering support.

    For applications that require extremely low latency or have exceptionally high data rates, Hazelcast may provide a useful alternative. Unlike Cassandra, Hazelcast is an in-memory data grid database, which enables development teams to benefit from the latency advantages of in-memory computing. 

    Message Systems: Kafka, Pulsar, NATS

    Apache Kafka is the dominant open source messaging system used by application development teams today. Originating at LinkedIn, Kafka has grown into the de facto standard for high performance open source messaging systems. For streaming use cases, Kafka users can also turn to Kafka Streams.

    Apache Pulsar, developed at Yahoo!, is another alternative messaging system which has been optimized for streaming data sources. Pulsar further differentiates itself with TTL message support and queuing via shared subscription.

    For applications where latency is a critical consideration, NATS.io (a CNCF project) is one of the fastest pub-sub messaging systems available.

    Service Meshes: Istio, Linkerd

    Trends such as microservices, serverless and hybrid cloud deployment have contributed to the increasing architectural complexity of distributed applications. With hundreds or thousands of distributed application services, service meshes provide a mechanism for pulling all the disparate services together into a manageable and cohesive whole.

    The most notable open source service mesh offering is Istio. One of the most popular open source service mesh projects, Istio is primarily used for applications built on top of Kubernetes. Another alternative is Linkerd, a flexible service mesh solution for Kubernetes or other frameworks.

    REST Frameworks: Node.js, Spring

    Node.js is a popular JavaScript framework for building scalable web applications. When comparing web development frameworks, Node.js typically ranks among the top three most (along with frontend frameworks React.js and Angular.JS). While Node.js can support large scale applications, it does not support real-time use cases well at scale.

    Spring is another application framework, optimized for distributed microservices-style architectures. Like Node.js, Spring “focuses on the ‘plumbing’ of applications so that teams can focus on application-level business logic, without unnecessary ties to specific deployment environments.”

    Streaming Frameworks: swimOS

    SwimOS is a software framework for distributed streaming applications. Unlike REST-based frameworks, swimOS utilizes a unique and lightweight Web Agent architecture, optimized for composing and transforming many data streams in real-time. Whereas DevOps oriented projects, such as Spinnaker, seek to provide an abstraction layer to converge distributed cloud services, swimOS provides an abstraction across distributed data streams, so teams can build applications by composing many distributed data streams in a single mesh environment.

    In order to ensure real-time consistency of distributed services, swimOS automatically ensures eventual consistency between all Web Agents. Complete with a UI toolkit and it’s own stateful messaging protocol called WARP, swimOS is the fastest way to build stateful streaming applications.

    DevOps: Kubernetes, Spinnaker, RancherOS

    Kubernetes is one of the most talked about open source projects in the world. Kubernetes describes itself as “an open-source system for automating deployment, scaling and management of containerized applications.”

    While it may simply be a platform for container orchestration, rightly or wrongly, Kubernetes has become a stand-in for DevOps on the whole. In spite of its dominance, there are alternatives to Kubernetes, for example there’s RancherOS to manage Docker containers.

    Furthermore, as application architectures continue to become more distributed and complex there is a need for platforms which can converge multiple distributed edge and cloud environments. Spinnaker provides an abstraction layer to connect the various environments and provides a simple mechanism for continuous integration/continuous delivery (CI/CD). 

    Final Thoughts

    Your own particular use case should determine the appropriate open source technologies for your team’s real-time application. Whether you’re building an ultra-high performance low latency application or simply trying to tame large volumes of real-time data, knowing your options and choosing the right framework or platform can save you significant time and effort.

    The open source software landscape is changing every day. If you have something to add about one of the technologies above or think I skipped over the next OSS breakthrough, I’d love to hear about it. Let me know your thoughts in the comments section below.

    — Chris Sachs

  • PagerDuty Focuses Incident Response Efforts on the Cloud

    PagerDuty Focuses Incident Response Efforts on the Cloud

    After helping to pioneer the adoption of incident management software in on-premises IT environments, PagerDuty is now focusing its attention on the cloud with the launch of PagerDuty for Cloud Operations.

    Jukka Alanen, senior vice president of business development and corporate strategy for PagerDuty, said that as cloud computing evolves to include containers and serverless computing frameworks, it’s becoming apparent that managing IT incidents on public clouds is fundamentally different than on-premises IT environments. As such, demand for incident management platforms optimized for specific cloud platforms is increasing, he said.

    PagerDuty initially will focus its efforts on the Amazon Web Services (AWS) platform because that’s where most of the initial demand is coming from, Alanen noted. As part of that effort, an instance of the PagerDuty digital operations management platform running on AWS has been integrated with Amazon CloudWatch, AWS Personal Health Dashboard, Amazon EventBridge, AWS Security Hub, Amazon GuardDuty and AWS CloudTrail cloud services.

    Rather than replace all the capabilities already being provided by cloud service providers, Alanen said PagerDuty is committed to making them more accessible within the context of an incident management platform. With the rise of DevOps, the types of IT professionals who need to be able to access an incident management platform in the cloud era is also expanding. To resolve issues as quickly as possible, Alanen said it’s more critical than ever to make sure the right problems are being presented to the right teams within an IT organization.

    In terms of making that transition, however, IT organizations are all over the map. Many are trying to extend existing IT operations processes developed for on-premises IT environments. To the extent that organizations are essentially lifting and shifting monolithic applications running on-premises into a public cloud, that approach is feasible. However, once organizations also begin deploying cloud-native applications based on microservices and serverless computing frameworks, most will need to rely more on management and monitoring tools provided by cloud service providers.

    The next challenge then becomes defining workflows that automate responses to potential service disruptions or a security breach. At the same time, many organizations are adopting best DevOps practices that shift responsibility of many aspects of incident response on to the shoulders of developers who now need access to an incident management platform. The existing reactive approach to managing IT needs to be replaced with a proactive approach that automates responses to incidents whenever possible, Alanen said.

    To help organizations accelerate their move down that path, PagerDuty has crafted a Digital Operations Maturity Model, which provides benchmarking data and best practices for organizations looking to modernize their IT operations as part of a larger transition to increased reliance on automated workflow processes.

    Of course, once IT organizations define these workflows it won’t take long before IT organizations find themselves needing to extend more proactive approaches to incident management across multiple cloud computing environments. The challenge then becomes finding a way to weave all the application programming interfaces (APIs) and services exposed by multiple cloud service providers into what one day will become a truly hybrid cloud computing environment.

    — Mike Vizard