staging-devopsy.kinsta.cloud

Tag: devops

  • AI Automation Won’t Replace Jobs – It Will Replace Processes

    AI Automation Won’t Replace Jobs – It Will Replace Processes

    As Software 2.0 takes shape, how are AI processes—and the jobs that go with them—changing?

    When it comes to new AI automation, the Software 2.0 movement can be seen as akin to the auto industry’s recent evolution. Beginning in the 1990s and rapidly progressing into the 2000s, the auto industry saw a huge rise in AI automation such as digital vehicle diagnostics.

    Throughout this, people worried that digital transformation might take jobs away, when instead it led to “greater profits, productivity, and competitiveness,” according to a 2008 study by the journal of Technological Forecasting and Social Change. In short, these innovations didn’t replace jobs, per se; they simply changed the processes of jobs from the manufacturer’s assembly line down to the mechanic’s garage.

    An auto diagnostic specialist used to perform tedious tasks such as counting flashes and converting them to error codes on printed-out tables. Computerized diagnostic tools made this process faster, more reliable and, you might imagine, far less tedious. According to diagnostic specialists Helmut Frank and Uwe Schmidts, digital transformation went from being “a necessary evil to being a key to new, interesting and innovative functions.”

    You might liken the above example to the innovations that AI automation and machine learning (ML) have offered to software developers, who until recently, were tasked with a much more hands-on approach to deploying and maintaining applications. In many ways, DevOps practices have heroically come in to break them free of this minutiae, automating many of the repetitive tasks involved in application management. By design, it’s increasing efficiency and security while decreasing tedium and the chance for human error.

    Speaking to tedium, Mike Loukides and Ben Lorica, in their article “The road to Software 2.0” noted, “Up until now, we’ve built systems by carefully and painstakingly telling systems exactly what to do, instruction by instruction. The process is slow, tedious, and error-prone; most of us have spent days staring at a program that should work, but doesn’t.”

    In short: Much like digital transformation brought efficiency to the life cycle of automobiles, AI automation is offering efficiency to every stage of the software development cycle.

    Don’t Change the Players, Change the Game

    Indeed, a new generation of AI automation and ML tools is now emerging, shaping what some call a Software 2.0 movement. But before you begin to fear that these tools will lead to a “they took our jobs!” revolt, consider that they may be celebrated instead as a means of not only augmenting the work of software developers but also helping offset the ongoing developer drought. According to a 2019 Stack Overflow survey, over a third of developers named “not enough people for the workload” as their primary challenge to productivity. Rather than replacing jobs, automation should be viewed as offering efficiency in an industry that desperately needs it.

    As an example, the process of writing code, though more streamlined today than it was 25 years ago, has remained a mostly manual process. This has presented an excellent opportunity for the automated processes we are beginning to see, which bring in AI to automate aspects of code development such as program synthesis, smart code completion and static code analysis.

    Assaf Araki spoke to this in the article, “Software 2.0 takes shape.” Araki described how automation is filling gaps in many other areas, such as regression testing, code analysis, AI for vulnerability detection and observability tools to monitor software health.

    From Monolith to Microservices

    Just like the recent, mysterious disappearing monolith in the Utah desert, the relevancy of monolithic development may be on its way out.

    In the past, software cycles were longer. Teams updated monolithic-style deployments only monthly or yearly. Nowadays, companies are under pressure to push much more rapidly, often turning to microservices and progressive release strategies. This new paradigm requires much automation to increase release efficiencies and enable a seamless CI/CD.

    As Araki pointed out, “Automation in software is best viewed through the lens of Continuous Integration (CI) and Continuous Deployment (CD).”

    How to Know When AI Automation is Worth Its Salt

    It’s smart to consider the investment of both money and time when deciding whether an AI solution is worth adopting. “AI systems need to add value that exceed any new costs and requirements (such as the need for labeled data, hiring new staff, etc.),” Araki said.

    You should also be asking if the solutions are wide-ranging enough to meet the needs of your developers and anyone else on the team who may be employing them. Key considerations for AI automation, Araki noted, are focusing on tasks and systems that are frequently used, adjusted or tuned.

    4 Examples of Versatile AI Automation and ML:

    1. Data management systems from simple systems that employ ML to more complex systems that enable self-driving data management.
    2. Dashboards that are automatically created, ideally with intuitive and visually appealing UX, that use AI automation and ML to bring up the right elements—more progress is expected in these areas in the coming years.
    3. Websites and mobile apps that use AI automation to become highly personalized interfaces, such as those that employ real-time data for online and reinforcement learning—these are also expected to progress in the coming years.
    4. UX and design tools that use simple automation to design and revise digital media, with ML being a promising way to kick the automation up a notch, such as deep learning for game developers.

    AI Automation and ML to Grant Opportunities

    In short, Software 2.0 doesn’t currently appear poised to threaten jobs or decrease developer need. Instead, it seems to be changing the processes engineers are using and creating efficiencies in their work, at a time when their expertise is in increasing demand. Demand for novel roles, like the Cloud Economist, demonstrates the emergence of new opportunities for humans with cloud knowledge who work alongside AI.

    The new evolution of AI and ML also could create a unique subset of developers with skills keeping par with these new innovations—much like digital transformation in the auto industry led to evolving specialties in computerized vehicle systems.

    Not to mention that just as low-code has paved the way for citizen developers and AutoML led to citizen data scientists to fill large gaps in demand, AI and ML solutions have the potential to further broaden and democratize software and code development.

  • Demystifying the Low-Code Category

    Demystifying the Low-Code Category

    The low-code category is estimated to grow substantially in the coming years. By 2024, Gartner projects 75% of large enterprises will have adopted four low-code or no-code tools for both IT application and citizen development needs. But what exactly is low-code, and what are the scenarios in which it is needed?

    Low-code, like so much other software jargon, is somewhat ambiguous. Under this umbrella term exists a well of various tool types—website generators, form builders, API connectors, database builders, workflow automators and more, each requiring varying levels of technical proficiency. Some are 100% codeless and UI-based with drag-and-drop templates. Others need programming acumen to set up to complement a developer’s pre-existing workflow. When we talk about low-code, we’re typically referring to platforms that help create internal business utilities.

    I recently connected with Divyesh Kharade, co-founder and CEO at DronaHQ, to demystify the low-code category. Below, we define citizen developers, break apart low-code versus no-code and dig into specific types of low-code scenarios. Given this knowledge, technology leaders can better identify what tools they need to increase rapid development for their business needs.

    Hard Code Is to No-Code as Manual Is to Driverless

    So what’s the difference between traditional coding, low-code and no-code? To demonstrate the nuances between emerging code paradigms, Kharade compares them to automobile types.

    Manual Coding

    Traditional coding is like a manual car with a gearbox

    With a manual car, you must know how to switch between gears. Not to mention, you must understand driving itself and how to navigate busy streets, all while following traffic laws. We can compare manual driving to traditional coding. Manual coding is for trained developers.

    For example, consider we need to develop an application to interact with a database. To create this from scratch, this could mean coding a PHP application, connecting it to a back-end database management system such as PostgreSQL and hard-coding commands for ingesting data and manipulating entries.

    Manual coding requires an adept understanding of programming languages and development frameworks and libraries and collaborative development processes such as git. Naturally, traditional coding is in the professional developers’ realm and often necessary for software teams developing proprietary software-as-a-service offerings. It affords ultimate customizability.

    Low-Code

    Low-code is like an automatic car

    With an automatic car, you must still know how to drive, but gear-switching is automated. Cruise control is a nice perk, too. Similarly, low-code seeks to make development easier. It even enables some non-tech folks to construct applications. Low-code is used by professional developers and citizen developers.

    If we consider our database example, a low-code solution would likely involve a one-time registration of a database. This would require a little technical expertise to initiate, Kharade said. However, after which, it would be smooth riding—a user interface would be utilized to develop applications that interact with the database.

    Low-code platforms often incorporate various drag-and-drop building blocks to help develop applications. These pre-generated components may include tables, textboxes, dropdowns, charts, maps, forms, user directories and other elements. Low-code solutions typically make code under-the-hood accessible and portable, too. Degrees here vary. Some, such as some-code, are more geared toward alleviating operational tasks for skilled engineers.

    Low-code solutions also may assist with the surrounding operations necessary to create a fully functional app, such as hosting, deployment and monitoring. Example providers in the low-code spectrum include Appian, Claris Filemaker, Mendix, Outsystems, Quick Base and Salesforce Lightning.

    No-Code

    No-code is like a driverless car

    In an automated car, AI handles all. Driving through traffic and re-routing is automated. You don’t need your hands on the wheel—but you do need a destination. Similarly, no-code is the most usable, non-tech option for application development. No-code is geared toward citizen developers and non-tech users.

    No-code takes low-code usability to a new level—100% UI-driven, with pre-built templates and the ability to string together utilities. No-code is popular in connectivity and triggers, taking the burden out of integration. Of course, the more you automate, the more granular control you sacrifice. Many no-code options simply require logins and permissions for business applications you already use. Examples include Zapier, IFTTT, Kissflow, Airtable and Retool.

    The trend toward low-code and no-code is industry-agnostic, too, having seen adoption by banking, manufacturing, consumer goods, eCommerce, and other areas, Kharade said.

    5 Common Low-Code Scenarios

    Now that we understand the variants, what are enterprises building using low-code? Kharade sees four common scenarios:

    1. Building internal forms: Collecting data in a form, such as for an inspection or audit. May require PDF generation. Non-tech folks with an understanding of their industry could create forms.
    2. Extending with publicly available forms: Making the form public for registration or lead generation. Requires little added functionality and a no-code user could likely implement.
    3. Working with databases: Junior engineers will readily understand database mechanics, but it’s not entirely non-tech, said Kharade. For example, skills may be required to synchronize databases with other cloud operations.
    4. Business process management (BPM): Digitizing the traditional corporate project approval processes. What was once paper-based or email-based is streamlined. It could involve picking up an order, pushing to a manager and a manager approves. Digitizing this process requires a deep understanding of the data structure.
    5. Purely open applications: Perhaps used for onboarding or internal job postings. Requires a fuller amount of user interaction design, workflows, file hosting and API connectivity. May require skilled developers.

    “Building your data structure at the start is extremely critical,” Kharade said. To him, more complex projects will still likely require skilled engineers at some point. But, as you can see above, there are many areas where low-code and no-code could streamline the creation of simple apps and tools.

    Low-Code, Demystified

    In other pieces, I’ve attributed the rise of low-code to an ongoing developer drought and the emergence of a digital-first economy during COVID. Throughout the pandemic, “there has been a frenzy to digitize any manual process,” noted Kharade. “People are scared even to use a pen someone has used.”

    Low-code has in part become a critical option to stand up replacement business applications rather effortlessly, providing citizen developers or non-tech users new digital capabilities. With its surge in popularity, the marketing language around this area is a bit befuddling, and many applications claim to be low-code or codeless.

    Of course, you could call any tool with a UI “no-code.” Technically, your iPhone apps are codeless from a user’s point of view. However, what sets these low-code development styles apart is their use in forging internal business applications and automating repetitive software development tasks. By utilizing low-code, traditionally non-tech companies could wrap operational capabilities around basic applications and scale them like tech companies.

    Another key to low-code, which I will expand on in a follow-up piece, is solving the integration hurdle with third-party APIs. “It’s difficult for a non-tech user to register an API,” Kharade said. One must understand authentication, OAuth, payloads, HTTP methods, responses and headers just to make a single call.

    API-driven services have needed better usability for quite some time. Low-code layers could make consuming a vast array of SaaS more inclusive—a fascinating prospect to problem-solvers of all skillsets.

  • KubeCon Coverage: Incentivizing the DevSecOps Culture

    KubeCon Coverage: Incentivizing the DevSecOps Culture

    Just as DevOps set to de-silo development and operations teams, the DevSecOps movement is bringing security to the same table. A shift-left security mindset is permeating much discussion of late.

    Cyberattacks are on the rise in the era of COVID-19 and cybersecurity has become paramount to arm business-critical applications. Furthermore, new regulations have emerged to protect personally identifiable information (PII). A new cybersecurity awareness feels quite apparent throughout the tech spectrum. As a result, security policies are poised to integrate more directly within the process of engineering and deploying applications.

    Nonetheless, while DevSecOps feels necessary to implement at this critical hour, it does present challenges. Primarily of which is a significant cultural shift. Uniting security and development isn’t as easy as it sounds, especially when faced with rigid enterprise bureaucracy. Plus, how can an organization incentivize DevSecOps if developers fear security tools will slow down their workflows?

    The DevSecOps panel at the recent KubeCon + CloudNativeCon North America 2020 brought together leaders in the DevSecOps arena to advise introducing a security-first mindset to DevOps. Dan Papandrea, field chief technologist at Sysdig, led this super-informative panel. It featured:

    • Emily Fox, DevOps Security Lead, NSA
    • Peter Bosch, Cisco
    • Sunil James, HPE
    • Christian Weichel, Gitpod
    • Nicolas Chaillan, U.S. Air Force

    Below, I’ll cover some of the key insights garnered from the event. To summarize, the panel identified how policy-first tools and open source initiatives are placing security controls within IDEs and CI/CD pipelines. And what will it take to get everyone onboard? It appears that transparent tooling automation, making security adherence natural and a new wave of security-aware employees will be critical components in the future era of DevSecOps.

    What is DevSecOps?

    First off, to get a better picture, what exactly is DevSecOps? And why has it come about?

    In a previous article, we defined DevSecOps with six traits: security automation, security culture, de-siloing IT, shifting security left and security as an enabler (not a stalling force).

    The KubeCon panelists seemed to echo all of these points. Fox described DevSecOps as “a term used by organizations to ensure their DevOps approaches have a security-first mentality.” To her, this means placing security automation within workload environments.

    DevSecOps is something new. For most organizations, security isn’t baked into the standard development processes—it historically hasn’t been the first priority, Chaillan said. DevSecOps is changing that, adding continuous monitoring, zero trust enforcement and reducing the overall attack surface.

    The move toward continuous release is also an impetus for DevSecOps. “Our cadence of delivery has been quickened,” said James. All organizations now feel pressure to deliver value to customers rapidly. An interesting byproduct of building security-first, he said, is “the ability to create the consistency and speed with which you can actually offer these capabilities.”

    Another cause for DevSecOps is the sheer imbalance in the workforce. On average, the number of developers far outweigh the number of security folks, noted Bosch. So, DevSecOps could scale security across an organization more effectively than human power could.

    Bosch also is hopeful DevSecOps could put an end to time-consuming bureaucracies stalling an agile security process. “Developers used to write code and wait for free months to open a firewall.” This may have been standard operating procedure decades ago, but in the age of continuous everything, enterprises must evolve how they handle IT security policies. “By uniting security with developers, we bring speed, robustness, avoid OWASP issues, and build applications that better contribute to the bottom line,” he said.

    Finally, embracing DevSecOps is a significant commitment. The combination of culture and technology is essential, noted Weichel. As we infuse security testing into CI/CD pipelines and make everything security-first, he said, we indeed undergo a cultural shift—it’s something that permeates everything.

    Challenges in Implementation

    Companies attempting to introduce DevSecOps may run into a few challenges, including instilling the culture, implementing regulatory compliance and finding the right blend of open source and propriety tools to get the job done.

    Bosh is no stranger to sluggish enterprises and the systemic change required to introduce DevSecOps. In these environments, changing companywide antiquated processes may require executive sponsorship. “By and large, it’s mostly culture,” he said, yet acknowledged a vast divide between the two groups: “How do you convince security do get out of their comfort zone? And how do you convince developers to get out of theirs?”

    There is also developer experience to consider: Developers often view security tools as limiting their ability to deliver code quickly. Perhaps the biggest challenge to overcome is improving the experience of working alongside new security policies. If new practices are causing more pain, instead of automating pain points away, DevSecOps could suffer.

    Outside of team dynamics, there are also technical hurdles involved when introducing and enforcing legal regulatory requirements. “Getting any sort of compliance baked into CI/CD pipelines is a challenge,” said Fox. Part of the problem is that the tooling to enforce such requirements isn’t there yet. “Cloud-native is continually evolving and security is constantly playing catch up,” she noted. “It’s an ongoing challenge.”

    Incentivizing the DevSecOps Culture

    Employees need incentives to change established habits. With the above challenges in mind, how can organizations create incentives to adopt DevSecOps? Well, outside of monetary incentives or straight up demanding it, incentivizing that security culture could happen in a few different ways, the panelists noted.

    To Fox, it’s about “making security transparent to the developers.” You want developers to quickly build and deploy secure applications without the need for intensive cybersecurity training. Thus, the tools and processes must be as transparent and usable as possible. DevSecOps is possible “if you make it second nature,” she said.

    Another way to consider DevSecOps culture is the end performance. If customers are happy, this is a positive gauge. James said he has witnessed CIOs at a few organizations using customer satisfaction scores as a guiding light for not only the business but for security effectiveness well.

    Bosch similarly stressed an indirect incentive about overall company performance. “Companies that really embrace DevSecOps leads to better code,” said Bosch. This leads to more robustness, higher speed, and better-operating enterprises. Co-building a thriving success could be ample incentive alone for those who have the whole team in mind.

  • strongDM Extends Access Management as Code Efforts

    strongDM Extends Access Management as Code Efforts

    strongDM today announced it has added an application programming interface (API) and software development kits for Go, Java, Python, Ruby and other programming languages for involving the single sign-on (SSO) capabilities of its infrastructure access management platform, in addition to adding support for command-line interfaces (CLIs) exposed by cloud service providers.

    At the same time, strongDM has integrated its namesake platform with third-party secrets management tools such as Vault from HashiCorp as well as platforms such as PagerDuty’s incident management platform and Slack messaging tools.

    The access management platform can now also support any internal web applications and database platforms including DB2i, CitusDB, Oracle 11g, 12c, and 19c, Sybase ASE and IQ.

    Finally, the company has made enhancements to existing protocols including Mongo Atlas, SSH certificate-based authentication and SSH port forwarding, and added support for Windows remote desktop protocol (RDP) network-level authentication (NLA) and integration with other SSO tools such as Auth0 and VMware Identity One.

    Company CEO Elizabeth Zalman said strongDM, unlike other SSOs, is designed from the ground up to be incorporated within a developer’s workflow. The goal is to enable developers to manage access as code in the same way they manage infrastructure as code, she said.

    In general, access management is usually the last stage of automation to be addressed by DevOps teams. However, with most DevOps teams now working from home to help combat the spread of the COVID-19 pandemic, there has been a sharp rise in the number of organizations seeking to automate access management as an alternative to managing virtual private networking (VPN) software, said Zalman.

    Historically, many IT organizations have relied on Microsoft Active Directory (AD) or the Lightweight Directory Access Protocol (LDAP) to manage access. As an alternative, strongDM has been making a case for its platform since 2015. The company recently raised an additional $17 million in funding and claims to have more than 100 customers using its platform in production environments, including Peloton, SoFi and Betterment.

    It’s unclear whether DevOps teams are extending their existing workflows to include access management. However, as DevOps teams grow weary of using custom scripts to extend those workflows, Zalman said eventually they’ll look for a more consistent approach to automating access management. A major factor in that decision is the fact that most employees won’t be coming back to the office in the same numbers they did prior to the pandemic, even after a vaccine is discovered and distributed.

    At the same time, more DevOps teams are moving to embrace best DevSecOps practices, which inevitably will encompass access management. In fact, as part of that transition, it’s probable most DevOps teams will conclude the way access is currently managed is simply too antiquated to continue.

  • Survey Sees Surge in Demand for DevOps Skills

    Survey Sees Surge in Demand for DevOps Skills

    The Linux Foundation and edX, a provider of an online learning platform, jointly released an annual report today that for the first time finds organizations are looking for more IT personnel with open source DevOps skills (65%) than developers (59%).

    Clyde Seepersad, senior vice president and general manager for training and certification at The Linux Foundation, said the 2020 Open Source Jobs Report shows that as more workloads shift to the cloud in the wake of the COVID-19 pandemic, the number of organizations looking to embrace best DevOps practices is increasing substantially. The report finds 70% of respondents are more likely to hire an IT professional with cloud computing skills.

    Based on a survey of 175 hiring managers and 900 IT professionals with open source expertise, the report also finds that despite the economic downturn brought on by the pandemic, 37% of hiring managers say they will be looking to hire more skilled IT professionals in the next six months.

    The report also finds 93% of survey respondents report difficulty finding open source talent. An overwhelming 81% of hiring managers said recruiting open source talent is a priority for 2020, with 56% looking to recruit IT professionals with open source skills in the next six months. Nearly two-thirds (63%) said their organizations have begun to support open source projects with code or other resources for the explicit purpose of recruiting individuals with open source software skills.

    More than half of hiring managers (52%) said they are more likely to hire someone with a certification, while 80% of survey respondents reported providing online training courses for employees to learn open source software. Nearly three-quarters of organizations (74%) are now offering to pay for employee certifications, according to the survey results.

    An equal percentage of hiring managers also identified Linux as the most in-demand skill they seek in new hires, while 63% also said they are looking for IT architects who can build solutions based on open source software.

    Seepersad said given the difficulty attracting talent many organizations are focusing more on training their existing staff rather than trying to recruit talent in an open market where demand for open source expertise remains high.

    The talent pool is more global as more organizations become accustomed to working with staff who are now largely working from home, which, coupled with current economic realities, is limiting how much organizations are willing to pay for IT talent. More organizations are also keen to make sure there is a cultural fit with their organization when hiring, he added.

    In general, it’s clear more IT organizations will need to focus on growing their own farm teams with open source DevOps skills rather than hoping to lure away superstars from other organizations. Rather than simply trying to fill open positions, organizations need to think more in terms of acquiring and managing talent in the same way a general manager manages a sports franchise.

  • Winning DevOps in the Finance Industry

    Winning DevOps in the Finance Industry

    DevOps is making its impact on almost every software-powered company these days. Companies in the finance domain, however, have been perhaps the slowest to embrace DevOps because of concerns regarding governance, security, compliance and regulatory regulations. But it is safe to say that they are currently leading the digital innovation in this era of cloud-native technology.

    Why DevOps for the Finance Industry?

    Higher customer engagement and continuous transactions make the finance industry one of the busiest and naturally requires uninterrupted infrastructure. DevOps implementation helps companies develop fast, fail fast and learn fast so they can fulfill customer expectations and deliver features faster to market than their competitors. Technology in the finance sector has not evolved much, as many companies have assumed it is too risky to embrace DevOps because of the amount of legacy code and old methodologies still in practice. As a result, digital innovation has lagged.

    Challenges of Implementing DevOps in Finance

    Unlike other industries, regulated industries imply several challenges:

    • Strong restrictions on secured networks.
    • Fine-grained audit trails.
    • Strong ACLs models.
    • Full lifecycle governance.
    • Integration with third parties.

    Security and compliance are top concerns in financial companies, and DevOps practices are, in fact, viewed as risk factors to security. Plus, the increased frequency of software releases in DevOps is seen as a threat to governance and regulatory controls.

    Is the Needle Moving Toward DevOps?

    According to the “2020 State of Database DevOps report“ by Redgate, financial services organizations report the highest levels of database DevOps adoption this year, with proportionally higher rates than any other sector. That shows financial sector maturity in terms of digital innovation. 

    Image source credit: Redgate

    Case Studies of Financial Companies and Their DevOps Journey

    Barclays DevOps Adoption

    In 2015, Barclays announced it was adopting DevOps for its digital transformation journey. Barclays processes payments that equate to around 30% of the UK’s gross domestic product. The developers’ morale and quality of code increased because of DevOps and the leadership team at Barclays credited DevOps with significantly reducing the complexity of its codes, which has allowed the company to reduce delivery risk and ultimately, improve the quality of services.

    DevOps the Lunar Way

    Lunar Way’s journey shows you don’t have to be big to use Kubernetes. The company started its cloud-native DevOps journey by splitting its massive monolith application into smaller microservices. To spin up these microservices, Lunar Way used Ansible, Terraform and Jenkins and to deploy these microservices as a whole unit. 

    Then Lunar Way started to experience scaling issues with microservices and didn’t experience any of the microservices benefits. 

    The company started looking for ways to move past this complexity by shifting its focus from machine-oriented to application-oriented architecture. It chose Kubernetes as the abstraction layer along with AWS, not worrying about where the containers are running, and this is how it was able to manage and unlock the velocity of microservices. Lunar Way also chose Kubernetes from a security perspective and to specify how the applications should run. Now the company runs around 80+ microservices in production with the help of Kubernetes. Watch and learn how Lunar Way did it: “Two years in production with Kubernetes.”

    Italy’s Biggest Bank Embraces DevOps

    A conventional bank running its real business on such a young technology? No way, are you kidding me? Nope, I am not kidding. 

    Italy’s banking group, Intesa Sanpaolo, has made this transition. Banks still run their ATM networks on 30-year-old mainframe technology, so embracing the hottest trend and tech is nearly unbelievable. 

    Even though ING, the banking and financial corporation, changed the way the banks were seen by upgrading itself with Kubernetes and DevOps practices very early in the game, there was still a stigma attached to adopting Kubernetes in highly regulated and controlled environments including health care and financial services. The bank’s engineering team came up with an initiative strategy in 2018 to throw away the old way of thinking and started embracing microservices, container architecture and migration from monolithic to multi-tier applications. Today, the bank runs more than 3,000 applications. Of those, more than 120 are now running in production using the new microservices architecture, including two of the 10 most business-critical for the bank. 

    HSBC Going the DevOps Way

    HSBC is changing the way typical banking works with cloud-native technology and DevOps, as an early user of Google’s Cloud Services Platform to provide core banking services to its business customers. 

    HSBC plans to build its all-new business banking service to run on a Kubernetes-managed container infrastructure using Google’s toolset. HSBC also has excellent partnerships with AWS and Microsoft, favoring the multi-cloud strategy. The company’s engineering team believes that moving to a container model under the umbrella of Kubernetes is significant because it means the environment is similar across different clouds. HSBC also hired people from outside of the banking sector because it wanted to change its culture from within. HSBC is said to have invested $10 million so far in its DevOps initiative.

    Capital One’s Agile to DevOps Journey

    Capital One’s Agile journey started in late 2011, with just two teams, which expanded slowly as more teams were trained in Agile development. The developers at Capital One were following the Scaled Agile Framework (SAFe). Initially, since the teams were new to the automation methodologies, integration testing, security testing, unit and performance testing were all done outside of development sprints by separate test teams. Later, once they understood how important it is to have a collaborative culture, they integrated testing into the dedicated DevOps teams and automated it well. Then slowly, they moved all testing into development sprints, adopting a culture of DevOps and uniformity and wiring integration, security and performance testing into a continuous delivery pipeline. As of 2016, the company had more than 700 Agile teams following continuous delivery.

    DevOps at Bank of America

    Bank of America is one of the world’s biggest financial institutions following DevOps practices, and its digital enterprise operations support nearly 4,300 retail financial centers and approximately 16,600 ATMs worldwide. The award-winning digital bank has nearly 38 million active users and approximately 29 million mobile users.

    Automating Security and Compliance

    As DevOps continues to evolve and become mainstream, it will continue to be of interest for many senior managers across financial services. The above case studies show they are already making DevOps a priority. But security and compliance remain barriers slowing the innovation in the financial companies. 

    Organizations should start small and keep security high priority, first using continuous integration and then adding continuous delivery and deployment using the available tools in the market. Over time, the more a company automates, the higher return on investment it can achieve.

    Automating security permissions and controls will help financial companies remove some of their most common software barriers and get their releases out quicker than before, while still maintaining the necessary governance and compliance. 

    Take a Step Forward

    Financial services companies need complex and rigorous software development processes to meet intense corporate and regulatory requirements. Yet, they are expected to release software faster than ever to meet expectations and beat the competition. By starting small and keeping security and compliance in mind, DevOps in finance can be achieved with great success.

  • 3 Ways Atlassian Usage is Evolving in the New Normal

    3 Ways Atlassian Usage is Evolving in the New Normal

    Over the last half-year, the use of Atlassian’s tools is evolving and expanding

    COVID-19 has rapidly accelerated the digital transformation of businesses by several years. However, as companies shift to digital-only workflows, DevOps teams and the increasing number of departments they interact with face numerous challenges. Effective digital communication and collaboration are often vital to overcoming these challenges, and executives now pay a lot more attention to the tools enabling both.

    Millions of users globally rely on the Atlassian ecosystem every day. In particular, Atlassian tools such as Jira being used to improve software development, project management, team collaboration and code quality. As the world has shifted to more collaborative digital work over the last half-year, the use of Atlassian’s tools is evolving and expanding.

    We recently conducted research to understand more about how users globally are using their Atlassian tools. The “2020 State of the Atlassian Ecosystem Report” surveyed 700 professionals using these tools for their organization to find out how, where, when and with what other technology they are deploying Atlassian solutions. Many interesting findings emerged from the research, and here are three major takeaways.

    Atlassian Tool Usage Seeps Into the Enterprise

    A large proportion of Atlassian users are adopting its tools to achieve business goals from improving productivity and efficiency to change management as part of their digital transformation. This is driving the introduction of Atlassian tools beyond DevOps and into the hands of non-technical teams.

    For example, 80% of users said at least one business team in their organization uses Atlassian tools. These teams include customer support, design, sales and marketing and operations. Furthering this trend, Atlassian and Slack announced last month that they would be conducting joint app and integration development, allowing users to connect platform services. Our research found 46% of Atlassian users utilize both Atlassian and Slack solutions. These kinds of announcements will only increase the collaborative use of these tools between DevOps and the teams they collaborate with.

    Adopting Agile Development Methodologies & DevOps Strategies

    A key part of a company’s digital transformation is adopting agile methodologies and clear DevOps strategies. According to our research, 77% of Atlassian users believe their organization has implemented agile principles in some way. And 48% have a DevOps strategy in place, with an additional 29% planning to implement a clear DevOps strategy in the next three years. As workers are forced remote and developers are distributed virtually, iterative development between these teams is increasingly important. This can only happen when a clear DevOps strategy has been put in place and digital feedback can be gathered and implemented in a well-defined way.

    Taking a Hybrid Approach to Deployment

    There are several different deployment options available depending on the business needs of users of Atlassian tools. How companies choose to use, deploy and configure the tools that they use. In addition, parts of the same organization often have different business drivers for using Atlassian tools. For example, one function may need a public-facing help desk with access from anywhere in the world, while another may need a highly secure and internal-only solution.

    There isn’t a one-size-fits-all approach to address personalized organization needs. For instance, 9% of users are leveraging a hybrid model of Atlassian Cloud (SaaS) and a self-hosted solution (Server/data center), which means combining the advantages of cloud and self-hosting through multiple deployment models.

    These findings demonstrate how the Atlassian ecosystem is evolving and expanding in our new normal. As digital transformation takes hold across different organizations of every size, industry and level of technical sophistication, the need for flexibility is only going to increase to accommodate more digital-only workflows.

  • Survey Surfaces High Reliance on DevOps to Build and Deploy APIs

    A survey of 13,586 IT professionals who work with application programming interfaces (APIs) published today by Postman, provider of an API development platform, finds best DevOps practices are now widely employed to both produce and consume APIs.

    The primary method through which teams working on APIs collaborate is by sharing URLs (42%). However, publishing API artifacts to GitLab, GitHub, Bitbucket or other repositories came in a close second at 40%, followed by publishing API documentation to an API portal (39%) and instant messaging (28%).

    When queried about change management practices specifically, Git repositories topped the list at 63%, followed by versioning APIs (59%), versioning server code (34.6%), versioning client code (27.8%) and semantic versioning (21%).

    Respondents said when it comes to deploying APIs, the most widely used option was continuous integration/continuous delivery (CI/CD) pipelines (53%).

    A total of 77% of respondents listed their functional area as engineering/development, followed by DevOps/API Ops at 34% and developer relations/evangelism (13%).

    Among DevOps practitioners that participated in the survey, Jenkins is the most widely employed platform (41%), followed by AWS DevOps (30.7%) and Azure DevOps (26.1%).

    Overall, the survey finds REST is still the dominant architectural style for APIs (93%), followed by webhooks (34%), SOAP (33%) and GraphQL (23%). JSON Schema is the dominant API specification (76%), followed by Swagger 2.0 (44%), OpenAPI 3.0 (28%) and GraphQL (23 %).

    When asked to rate the percentage of their organization’s development effort spent working with APIs, 41% said more than half of the effort is spent on APIs.

    Kin Lane, chief evangelist for Postman, said the survey also makes it clear that more organizations have adopted an API-first philosophy to application development. Over 60% of respondents rated themselves as 5 or better out of 10 in terms of embracing an API-first philosophy.

    Nearly half of respondents said investment of time and resources into APIs will increase over the next 12 months, while another third stated that investments into APIs will stay the same. Eighty-five percent said APIs are playing a significant role in their digital business transformation initiatives, while nearly a third (31%) also noted APIs played a role in their organization’s ability to respond to COVID-19 by, for example, enabling customer communications or remote work options.

    APIs appear to have become a lot more stable in recent years, with more than half of respondents reporting APIs do not break, stop working or materially change specification often enough to matter.

    When it comes to producing APIs, the No. 1 obstacle by a wide margin is lack of time, while the No. 1 obstacle cited for consuming API is lack of documentation.

    The four most important factors individuals consider before integration with an API are reliability (72%), security (71%), performance (71%) and documentation (70%).

    Looking to future technologies, respondents appear to be most interested in microservices (49%), Kubernetes (44%) and containers (42%). Given the cloud-native attributes of those technologies, it appears all but certain the number of APIs strewn across the extended enterprise is about to increase exponentially.

  • Shifting Left With DevSecOps: ESG Report Exposes Difficulties

    Shifting Left With DevSecOps: ESG Report Exposes Difficulties

    A recent report asks the tough questions about DevSecOps adoption, and the results are surprising

    In a world of increasing development velocity, companies are placing more responsibility on developers to enact quick deployments. Naturally, security is “shifting left” as well.

    Theoretically, DevSecOps sounds great, but it opens up many questions. Who exactly is responsible for security knowledge? Are cybersecurity training measures effective? Which tools excel at decreasing risk while maintaining release fluidity? Are AppSec tools aiding or adding unneeded friction?

    A report conducted by Enterprise Strategy Group (ESG) sought to answer these pressing questions and expose trends around application development security. The study surveyed 378 IT, cybersecurity and application development professionals at organizations in North America.

    I recently met with Chris Eng, chief research officer at Veracode, and Dave Gruber, senior cybersecurity analyst at ESG, to dig deeper into the report’s findings.

    Security vs. Development

    There is still friction when attempting DevSecOps in practice. Eng recognizes a disconnect between security and development teams.

    “Most security teams lack an understanding of modern application development practices,” said Eng. Security may not realize how regularly code is pushed or understand the intricacies of tooling within a CI/CD pipeline. On the flip side, developers may not be well-versed in common container exploits and vulnerabilities.

    Organizations are pretty split when it comes to application security testing ownership. Thirty-one percent of organizations dedicate security analysts to manage application security. The development manager and security team jointly own responsibility at 29% of companies. Developer managers follow at 26%, and individual developers at 13%.

    To Eng, these figures reflect an evolution from traditional security testing to the way things are changing with faster development cycles. “Go back 10-plus years, and we had a centralized, dedicated security team responsible for managing AppSec,” described Eng.

    In modern times, security is shared or even furthered where developers themselves run tools by hand or automate security within CI/CD processes. Large companies, Eng noted, now often utilize a mix of those practices.

    Paradox: Secure Yet Vulnerable

    Seemingly, the most eye-opening part of the report was a paradox. The report found that most teams give their application security a high rating (36% gave their application security programs a 9 or 10, with a 7.92 mean). Yet, the majority say they regularly push vulnerable code (48% of respondents knowingly push vulnerable code regularly).

    Though it’s counterintuitive, organizations may knowingly push out vulnerable code for many reasons. “You’ve promised a release, customers are waiting, ” said Eng. “Revenue is waiting. What are the implications of pushing that release?” Teams may not have time to fix vulnerable code, or it may simply not be worth it.

    Publishing vulnerable code may also be a symptom of placing testing too late in the release process. “That’s why there is this trend to shift left,” added Gruber.

    Gauging DevSecOps Success

    The above findings underscore a reality of IT security: To keep pace, software providers must accept a level of risk. In fact, business success may depend on it.

    “The effectiveness of the programs isn’t necessarily measured with how many times you’ve shipped your code with no vulnerabilities,” said Gruber. “Success is that we understand the risk, we have mitigated the risk at the level we believe is right for the organization, and have the right level of control over the process.”

    Companies must weigh the cost of improper DevSecOps and mitigate imminent risk to maintain release agility.

    Application Security Tools: Finding Balance

    At modern companies with quick-release cadences, “CI integrative security tools are way more prevalent and baked into the process,” noted Eng. Yet, 26% of respondents indicate security tooling friction and ineffective tooling usage.

    Whereas InfoSec teams are usually the ones dictating policies, developers have much influence in the tooling selection. Thus, tools must appeal to the developer experience, fitting into their established workflows.

    “We have to build a habit around it,” said Eng. Frictionless tools that continually scan within an IDE, for example, can enable developers to fix an issue on the spot and foster future behaviors.

    At the same time, you must set standards and avoid governance issues. This requires control and observability. “The balance here is that you want developers to be able to integrate something tightly into their toolchain,” said Eng. “But InfoSec needs a bird’s eye view on the organization.”

    Developers Lack Knowledge to Mitigate Issues

    Though security tools may reveal vulnerabilities, they don’t always correlate with remediation. The ESG report found 29% of developers lack the knowledge to mitigate issues identified by security tools.

    There is a clear gap in the market when it comes to developer education around security. This could stem from computer science schooling, in which cybersecurity is rarely taught alongside programming languages.

    On the job, hands-on developer security training is also spotty. Thirty-five percent say that less than half of their development teams are participating in formal training. Only 15% of organizations have developers actively involved in security training.

    Just giving developers a laundry list of vulnerabilities doesn’t help mitigation. For Eng, what organizations require is habitual, recurring training with hands-on tools. “Try to simulate or provide a real-world environment for learning, identifying and patching these issues,” recommended Eng. Integrating tooling within the daily developer workflow can help with knowledge retention, because “if you don’t use it, you lose it.”

    Other best practices will only arise with a security habit. For example, most development environments ship with open defaults. “You need security training and tools to understand to restrict that appropriately,” noted Eng.

    That said, a wide variety of security tooling is already advancing this process. For example, 56% of organizations use API security vulnerability (ASV) scanning to identify and mitigate risk associated with API usage, helping secure applications as developers code dependencies.

    Final DevSecOps Thoughts

    The ESG study found 43% of participants believe DevOps integration is most important to improving AppSec programs. Yet, siloed roles and a lack of security education mean DevSecOps must still mature quite a bit to meet the security-shift-left vision.

    Other eye-opening findings included:

    • More than half of respondents (54%) use automated controls to identify and quarantine vulnerable images in their image repository.
    • The proliferation of AppSec tools on the market is a significant problem for 30% of companies.
    • AppSec spending is increasing, yet only 30% will protect more than three-quarters of their codebase 12 months from now.

    For more findings, check out the ESG Survey Report: Modern Application Development Security.

  • Z is for Zowe – the Open Path to Mainframe DevOps

    Z is for Zowe – the Open Path to Mainframe DevOps

    The energy and excitement in the IBM Z mainframe community during the two years since the announcement of Zowe have not been seen in a generation.  By enabling mainframers to work like technologists who work on the cloud, mobile and distributed platforms, Zowe unlocks the power of DevOps for mainframers, ensuring the vitality of the platform for the long-term.

    It accomplishes this by opening up the world of enterprise DevOps tools, both familiar and special-purpose ones, for mainframe applications, allowing users to easily create toolchains that include the mainframe. This freedom to leverage any enterprise DevOps tools across the entire deployment pipeline is achieved simply and elegantly.

    The Zowe Backstory

    The Zowe journey began when Broadcom, at the time CA Technologies, along with IBM and Rocket Software realized the challenges they were hearing from customers—IT isolation, need for increased delivery velocity and recruitment and retention problems—were bigger than any single vendor could solve on its own. The challenges of this scale required an ecosystem-wide response.

    So the vendors came together with the Linux Foundation’s Open Mainframe Project to create the open source framework known as Zowe (as in David Bowie, for the uninitiated). The Zowe community operates transparently and welcomes contributions from all in the form of code, enhancement requests, educational content and shared experiences. A virtual meetup held earlier in the year attracted participants from 47 companies from across the mainframe ecosystem.

    Traditionally, mainframe application development and operations have been separate from the rest of the IT organization due to specialized processes, roles and tools designed around the platform. While they have served the mainframe community extraordinarily well over the decades, the world outside has changed with demand for ever-shorter software delivery cycles resulting in the DevOps model.

    In short, the tipping point has arrived and the time to normalize is now.

    What Is Zowe?

    Zowe is a modern interface to the mainframe that extends z/OS-based services and tooling to the greater IT landscape, including DevOps. As a framework, it consists of four main components:

    • Zowe API Mediation Layer (APIML)
    • Zowe Command Line Interface (CLI)
    • Zowe Explorer (Visual Studio Code extension)
    • Zowe App Framework (UI)

    Broadcom is proud to have initially contributed the first three of these components. The first two were created as part of our CA Brightside offering, which won an award for the Most Innovative DevOps solution—yes, a mainframe offering won a DevOps innovation award!

    These components provide a modern way of working that supports core DevOps practices such as empowering developers with shift-left testing responsibilities and DevOps engineers with robust automation options.

    This article describes the framework’s ability to onboard the mainframe to enterprise DevOps, so developers, systems programmers and others who work with the mainframe can now do so the same way their peers do with other IT platforms (i.e., cloud, mobile, distributed). These shared experiences close the gap between mainframers and others while preserving the core advantages of the platform. Common tools fuel a common language that benefits all, especially when deploying hybrid applications (e.g., web front-end with mainframe back-end).

    Through automation and script libraries, Zowe helps reduce the range of mainframe-specific knowledge necessary to be productive, enabling organizations to more easily recruit, onboard and retain the next generation of talent. By using modern DevOps tools and techniques combined with access to “big iron” applications, the next-generation now sees mainframe as an exciting career choice.

    Recent milestones achieved by the Zowe community include the launch of the long-term support (LTS) release ensuring the stability and longevity of the framework and a Zowe Conformance program ensuring the interoperability of a growing ecosystem of conformant tools.

    The framework drives mainframe DevOps in several ways:

    Zowe API Mediation Layer – Discover and Securely Consume

    As RESTful APIs become table stakes, the greater the challenge becomes, making those services easily discoverable and consumable in a secure way. The Zowe API Mediation Layer (APIML) addresses this need by exposing mainframe services, ranging from z/OS system level to database/transaction services to tool-based services, through a centralized gateway and catalog.

    Conformance of these APIs with the Zowe LTS standard eliminates breaking changes while bringing additional value, including the usability and productivity benefits of single sign-on (SSO).

    At Broadcom, we’ve leveraged Zowe to achieve conformance across our product portfolio and have a healthy pipeline of Zowe-conformant products soon to be announced. By spanning the traditional Dev and Ops domains, we’re facilitating true mainframe DevOps adoption—teams can combine their preferred mainframe-native tools with off-platform ones, including ones used by their peers working on other platforms, to optimize speed, quality and productivity.

    Zowe CLI – Orchestrate and Automate

    For DevOps Engineers and DevOps-savvy self-service developers, CLIs are a “go-to” tool. They use CLIs for a wide range of platforms including AWS and Azure, Git, Kubernetes and so on … why not mainframe? The use of CLIs paves the way for robust, script-based automation, which is a cornerstone of DevOps. The Zowe community already offers some basic scripts, free for anyone to use.

    Using a CLI may be new for some, so community members often share tips and tricks to help these users navigate the productivity curve quickly.

    Broadcom currently offers a dozen or so CLI extensions to mainframe-native tools, once again spanning the traditional Dev and Ops domains to improve productivity across roles and processes.

    Zowe Explorer for VS Code – Visualize and Navigate

    Visual Studio Code is exciting for any mainframers editing code, especially developers and sysprogs. While ISPF and Eclipse will remain viable options for the foreseeable future, the extraordinary success of the lightweight VS Code, which is backed by Microsoft, and the rapid emergence of browser-based editors/IDEs expand the range of options. With 11 million users around the globe, VS Code provides an experience users love and a marketplace of 20,000 extensions enabling users to personalize their UX and optimize their workflows.

    Zowe Explorer, which streamlines interaction with mainframe data sets, USS files and jobs, is only one of the many VS Code extensions available in the VS Code marketplace for mainframe users (and a successful one with more than 12,000 downloads). Unlike Eclipse extensions, all of the mainframe extensions from Broadcom are free, and many are open-sourced.

    In the Queue: SSO/MFA, SDK, Mobile App

    The Zowe community has been working diligently on the implementation of SSO and multi-factor authentication (MFA), which eliminates the drudgery of constantly logging on to multiple systems.

    Also, software development kits (SDKs) are in the works, the goal of which is to make Zowe services (and by extension, the mainframe) easier to use by providing reusable code for common mainframe operations in popular languages and runtime libraries. The beginnings of a Zowe Node.js SDK and Zowe Python SDK already exist.

    Finally, a Zowe mobile app to help systems and operations support staff improve system availability is, like the SDK, being incubated by the community.

    Decision Time – Zowe Open Path or Closed, Proprietary

    Mainframe teams face a choice: Continue with a vendor-controlled model, in which vendors determine which tools you use and how you work or adopt the open path. While the prescriptive (aka opinionated) approach of the former may have been acceptable for a siloed mainframe, the open approach is the foundation of enterprise DevOps.

    As an indication of the early success of the open path, we’ve been delighted to learn about many mainframe shops adopting Zowe on their own, with CI/CD integration and code analysis a common entry point.

    For those with reservations about open source, consider the prevalence of Git, Visual Studio Code, Jenkins, SonarQube and so on in the DevOps landscape. Microsoft CEO Nadella recently stated, “We’re all-in on open source” and Zowe co-founder IBM recently acquired Red Hat—more shining examples that open source tools are a trusted part of the technology stack.

    Change is never easy, which is why no-fee Design Thinking workshops are available to help adopters prioritize the DevOps use cases enabled by Zowe. Because so many see “open” as the least costly, least risky and fastest path to mainframe modernization, familiarity with Zowe is quickly becoming an industry standard. Ask your vendors if they’re joining the Zowe community and ecosystem.

    With ongoing commitment from IBM, Rocket and Broadcom and the guidance and support of the Open Mainframe Project, the excitement around will Zowe continue so join the open mainframe movement!

    To learn more about Zowe, visit Zowe.org and the community blog site.