Author: Ericka Chickowski

  • 8 Hot Takes: Will We See a Monolithic Renaissance?

    8 Hot Takes: Will We See a Monolithic Renaissance?

    Is the DevOps world slowly backing away from microservices and to a monolithic renaissance? After years of proselytization about the benefits (Flexibility! Resilience! Support faster release speeds!) of microservices and serverless architecture, there are signs of a potential backlash against the singularly fragmented microservices architectural approach.

    The argument in favor of rolling back to monolithic architectures feels similar to the cloud repatriation debate. Just as with a lot of cloud-first transformations—often in lockstep with them, actually—many organizations moved headlong toward microservices without a case-by-case business or technical rationalization for the change. As the shiny newness of microservices has dulled under that approach’s daily exposure to real-world deployment and business challenges, the monolith isn’t looking so bad in a lot of situations.

    The monolithic versus microservices debate boiled over the last few weeks in the wake of the publication of a case study earlier this month from the Amazon Prime Video team. The analysis from Marcin Kolny, senior software development engineer for Prime Video, reported that his team was experiencing scaling bottlenecks in its audio/video quality inspection tooling, which was architected as a distributed system using serverless components. By rearchitecting the tool as a monolith, they were not only able to increase scaling capabilities but drastically reduced the infrastructure costs around that tool.

    “Moving our service to a monolith reduced our infrastructure cost by over 90%,” Kolny wrote.

    The announcement elicited a fair degree of hyperbole on the topic, but it also spurred a very organic airing of the pros and cons of monolithic, microservices and serverless architectures.

    The following are 10 different perspectives spurred on by this online debate from a range of CTO and engineering experts. Some of them are diametrically opposed to one another, many are overlapping in their opinions. But as a body, they point to the general (and very pragmatic) consensus that there simply will never be a one-size-fits-all approach to software architectures.

    Microservices a Siren Song for Complication

    “It’s clear that in practice, microservices pose perhaps the biggest siren song for needlessly complicating your system. And serverless only makes it worse. Replacing method calls and module separations with network invocations and service partitioning within a single, coherent team and application is madness in almost all cases.”

    —David Heinemeier Hansson, creator of Ruby on Rails and CTO at 37signals (maker of Basecamp and HEY)

    Architectures Aren’t Good or Bad

    “Application architectures are neither good nor bad, nor are they appropriate for every use case. Organizations need to step back and consider carefully what it is they’re trying to achieve and which architecture might best serve that need instead of choosing the most “modern” architecture because, well, it’s in vogue.”

    —Lori MacVittie, distinguished engineer in office of CTO at F5Networks

    Distributed Architectures Will Suit AI-Driven Future

    “In the ongoing debate between monolithic and serverless architectures, it is important to acknowledge that both approaches have their merits and considerations. Monolithic architectures have been prevalent in large human-centric organizations due to their management advantages and historical use cases. However, as the world continues its rapid progression towards a more AI-driven future, the landscape is shifting towards architectures that embrace fluidity, autonomy, and distribution.”

    — Reuven Cohen, veteran CTO and entrepreneur, mentor for TechStars

    Monolithic and Serverless Each Have Pros And Cons

    “Over the last 15 years, everyone has wanted to move to serverless microservices-based horizontally-scaled architecture – but it is not ideal for every application’s specific needs.  As with any choice in life and in business, there are always tradeoffs;  you must evaluate and choose the appropriate architecture based on your business and technology needs. Another important dimension to this is workforce skills for application architecture.

    —Hemang Davé, client technical leader and CTO at Kyndryl

    The Amazon Rearchitect Wasn’t Even Truly a Monolith

    “The problem is that they called this refactoring a microservice to monolith transition when it’s clearly a microservice refactoring step, and it is exactly what I recommend people do in my talks about serverless first. I don’t advocate “serverless only”, and I recommended that if you need sustained high traffic, low latency and higher efficiency, then you should re-implement your rapid prototype as a continuously running autoscaled container as part of a larger serverless event-driven architecture, which is what they did. If you built it as a microservice to start with, it would probably take longer (especially as you have to make lots of decisions about how to build and run it) and be less able to iterate as you figure out exactly what you are trying to build.”

    —Adrian Cockcroft, longtime DevOps advocate and veteran architect with stints at eBay, Netflix, Amazon, and Battery Ventures, among others

    One Tool as a Monolith, Maybe, But Not All of Amazon

    “Microservices, while duplicative and inefficient from a compute standpoint, allow you to release stuff really quickly, which is a tradeoff that most organizations at scale are willing to make.

    Microservices exist to reduce communication pathways—not to solve technical scaling challenges. At a certain point, teams stop functioning due to the communication overhead. A seven-person team has 21 communication pathways, whereas a 100-person team has 4,950. I work with large retailers selling billions of dollars a year online, and putting everything into a single monolith just doesn’t work. Could the entirety of Amazon.com be deployed as a single monolith? No way.”

    —Kelly Goetsch, chief strategy officer at commercetools

    Architectural Tension Always Exists Between Speed and Maintainability

    “Speed to market and speed to develop does not always equate to long-term scalability and maintainability. You must actively balance your investments across these two critical pillars to build a viable product. Active rebalancing and reconsideration of the stack and technology mix for your FinOps and BizOps context is both art and science.

    Cloud-native distributed systems paradigms are here to stay. The Prime Video folks have mistakenly labeled a properly factored, highly coupled data-intensive processing step a monolith—it’s at best a rocky outcropping in their distributed microservices forest. “

    —Ajish George, managing director of cyber data & analytics for State Street

    The Shiny Thing is Not Always the Best Thing

    “The news that AWS discovered that serverless and microservices aren’t for everything is surprising. Not because it didn’t work out for that use case; but because the wrong use cases have been so well-known for the serverless/microservices pattern and should have been obvious to Prime Video. It’s worth remembering: That shiny new thing isn’t necessarily a better way of trapping mice. Right tool for the right job.”

    —Doug Vanderweide, director of cloud architecture for Avanade

  • 4 Risk Factors Devs Should Consider for ChatGPT Integrations

    4 Risk Factors Devs Should Consider for ChatGPT Integrations

    It’s only been a couple of months since OpenAI introduced its latest low-cost API for developers to leverage ChatGPT in their applications and already many engineering teams have jumped headlong into new integrations. But developers experimenting with the possibilities of how ChatGPT can boost their software should temper that with some risk analysis and threat modeling before deployment. So say a range of cybersecurity and risk experts, who warn organizations that leverage the ChatGPT API in their software that they’ll need to navigate a minefield of privacy, data governance, and security operational risks.

    “I use ChatGPT Pro and think it’s an exciting tool for holiday cards, poems for my kid, writing inspiration, first drafts and search,” says Patrick Hall, data scientist and co-founder of BNH.AI, a boutique law firm focused on minimizing legal and technical risks of AI. He’s also the author of the upcoming book Machine Learning for High-Risk Applications. “I get concerned about generative AI used as a consumer product in high-risk domains, such as consumer finance or the law.”

    With that framing in mind, the following are a few of the areas that experts like Hall suggest that developers, DevOps teams and CTOs start thinking about to avoid some potentially costly unintended consequences of ChatGPT integration in their software.

    Data Privacy and Governance Concerns

    One of the most immediate and obvious risk concerns IT leadership, DevOps teams and privacy experts are worried about is what happens to sensitive data when users enter it into a ChatGPT-backed prompt—whether directly or through an app using the API. Be it a customer entering personally identifiable information (PII) into an external-facing app or an employee entering proprietary company information in an internal tool using ChatGPT’s API, these use patterns open up a world of data exposure and regulatory compliance concerns. The good news is that OpenAI does not use data submitted through the API to train its models, but this shouldn’t automatically give devs peace of mind. They’ll need to be sure that using generative AI in an app truly aligns with their company’s privacy policies and with any local, federal or international data privacy laws their firm is beholden to, said Hall.

    “Although OpenAI appears to maintain a robust security and privacy program and there are steps you can take on your own to limit data sharing, some people are still worried,” said Walter Haydock, founder and CEO of StackAware.

    Among Haydock’s side projects, he is currently developing a tool for sanitizing data before sending it to OpenAI. Called GPT-Guard, it’s still in beta and it would need to be run on a local machine or in the cloud to sanitize data before sending it via API. But it provides some rapid prototyping of the kind of methods and tools developers may need to think about as they design their applications to interact with ChatGPT.

    Intellectual Property Infringement

    Developers should also be thinking through intellectual property infringement concerns when it comes to the responses generated by an app tapping into ChatGPT, Hall said.

    “Devs need to consider whether a generative AI system may be coughing up other people’s licensed or copyrighted intellectual property as responses,” he said.

    This is likely going to be a tricky situation that will likely be tested by the courts in the coming years. For example, will applications using ChatGPT be required to compensate licensed copyright holders for certain kinds of responses using their material? These are the kinds of questions that developers and the organizations they work for should be asking themselves, said Frank Huerta, CEO of Curtail Security and a longtime DevOps expert.

    “For example, what’s the mechanism for resolving IP and copyright disputes?” he asked. “Is ChatGPT going to have its own IP? Is it going to generate it—and will it put that in the public domain? Who owns the copyright to a generated response? None of that has been considered yet, and it’s going to come up fast.”

    Adding New Cybersecurity Threat Vectors

    In addition to all of the concerns about the data, developers integrating ChatGPT into their applications also need to consider what that does to the software’s attack surface. ‘Never trust user input’ has always been a good rule of thumb for developers, and that advice is just as true for large language model (LLM) prompts as anything else.

    A recent example scenario laid out by a data scientist on Twitter showed one of the myriad ways that malicious actors could carry out prompt injections in the same way they’d carry out a SQL injection. In that scenario, if a recruiter used ChatGPT to crawl LinkedIn profiles for a certain keyword and send an email to the people whose profiles included that keyword, it’d be possible to add text to a profile that sends an email chiding the recruiter for using an automated tool to send out emails. That’s a fairly low-stakes example, but it shows the possibility for injection attacks, said Andy Patel, a security researcher for WithSecure who has been digging into AI risks.

    “So, if you are integrating it, you should think about things like that. You should think about the fact that people will determine that you are using a large language model to do something, to automate something, and then they’ll essentially SQL injection attack it,” he said.

    Problems With Bias

    Finally, developers and software design teams should be cognizant of the potential problems that AI bias can introduce into their software if they lean on the generated output from ChatGPT to trigger actions or to inform the user.

    “Generative AI systems continue to make wrong, offensive or otherwise problematic content. Being wrong can give rise to negligence issues and biased content can lead to discrimination issues,” Hall warned. “Don’t embarrass yourself, offend others or run afoul of the law by not testing and further protect yourself by screening the heck out of generated content.”

  • Are Self-Service Machine Learning Models the Future of AI Integration?

    Are Self-Service Machine Learning Models the Future of AI Integration?

    DevOps teams seeking to step up their mojo in developing cutting-edge artificial intelligence (AI) features are facing a big skills bottleneck when it comes to data analytics and machine learning modeling. As a result, the market is seeing an influx of self-service machine learning models and machine learning-as-a-service offerings designed to help development teams more easily integrate AI capabilities into their software.

    This is coming in direct response to an explosion in demand for AI capabilities in the enterprise. According to Gartner analysts, AI adoption in the enterprise tripled in the past year. A report last fall from MIT Sloan Management Review and Boston Consulting Group found that 91% of enterprises believe that AI will deliver new business growth to them by 2023.

    The trouble is that folding AI functions and predictive analytics into software requires a whole new level of expertise in data science and machine learning from cross-functional DevOps teams. They need added skills to choose the right algorithmic approaches, acquire and manage data, train the models and integrate them into the code base and underlying infrastructure so that everything works properly under the hood.

    “We have heard from customers everywhere that they want to adopt machine learning but struggle to actually get models into production,” said Eric Boyd, vice president of cognitive and AI for Microsoft.

    Ali Farhadi Xnor
    Ali Farhadi, Xnor.ai CEO

    DevOps leaders today already face an uphill battle to keep their developer ranks staffed with well-trained software engineers. Piling on additional requirements for very specialized machine learning and data science expertise further strains those recruitment efforts. According to the most recent Harvey Nash/KPMG CIO Survey, the top one and two technical realms that suffer the biggest skills shortages today are in data science and AI, with 46% and 38% of CIOs respectively reporting recruiting pain in those areas.

    “There’s a huge imbalance between the demand in the market and the supply of the very best AI experience to actually train these models,” said Ali Farhadi, CEO of Xnor.ai, a Seattle firm that just last week released a self-service platform for developers. Called AI2Go, the platform provides pre-trained deep learning models to quickly integrate AI features such as facial recognition and object classification directly into their software.

    “We want to enable anyone who can code to benefit from AI,” Farhadi said.

    The Xnor release comes just a few weeks on the heels of a big launch from Microsoft of several self-service machine learning model tools aimed firmly at MLOps teams—those DevOps teams implementing machine learning capabilities. The new services from Microsoft roll out capabilities to manage the code, data and environments used by MLOps teams throughout the machine learning deployment life cycle.

    “Azure Machine Learning service’s MLOps capabilities provide customers with asset management and orchestration services, enabling effective ML lifecycle management,” explained Jordan Edwards, senior program manager for Azure MLOps, in a blog post on the topic. “With this announcement, Azure is reaffirming its commitment to help customers safely bring their machine learning models to production and solve their business’s key problems faster and more accurately than ever before.”

    Tools such as these are not always going to obviate the need for data scientists and machine learning/AI specialists on DevOps teams. As Farhadi noted, even a simple service like AI2Go requires development teams to choose the right models for the right situations. Nevertheless, this burgeoning market could take the pressure off teams by minimizing the scale of specialized recruiting, while reducing bottlenecks and dreaded re-work.

    “Machine learning projects usually involve several people—data engineers, data scientists, DevOps and others. Conventionally, at each stage of the machine learning life cycle, most, if not all, of these people end up doing an excess of custom coding, redundant work and manual trial and error,” said Craig Stewart, senior vice president of product management and product marketing for SnapLogic, which last fall introduced a self-service machine learning add-on for its integration platform. He noted that one of the primary goals is speed, to help cross-functional teams “build, train, validate and deploy high-performing models faster.”

    — Ericka Chickowski

  • When DevOps and ITSM Collide: Mistakes to Avoid

    When DevOps and ITSM Collide: Mistakes to Avoid

    In many ways the DevOps movement evolved as a reaction against the worst elements of IT service management (ITSM). IT leaders wanted to get from under the yoke of overly complex processes, bureaucracy and red tape to speed up software delivery, and to just get stuff done. Now that DevOps has kicked into full gear at so many organizations, the question remains: What do we do about ITSM?

    While the instinct in some may be to completely ditch ITSM or to just let the ITSM team operate in a separate orbit from DevOps practitioners, experts say that would be a big mistake.

    “DevOps aims to deliver high-quality software to the business faster and in shorter cycles,” explained Krishna Mohan Reddy, vice president and deputy global head of Cognitive Business Operations at Tata Consultancy Services. “However, it still does not eliminate the need for controls.”

    In many ways ITSM is the yin to DevOps’ yang—the two operating philosophies do offer contrasting viewpoints but when the operate in a single system they can be extremely complementary and synergistic.

    “Maintaining the proper balance between the ITSM and DevOps helps the organizations to scale and grow and become more organized,” said Sachin Agarwal, DevOps engineer and solution architect for Mantra Labs. Operating at DevOps speeds with ITSM frameworks in place keeps finger-pointing at a minimum and reduces the risks of failed deployments. “Both in harmony will increase the efficiency and reduce the blame-game within the organization.”

    As organizations try to figure out how to achieve that harmony, here are some pitfalls they should seek to avoid:

    Letting Each Tribe’s Preconceived Notions Hold Back Overall IT Progress

    It can be easy to get wrapped up into dogmatic fervor that your particular philosophy is the One True Path for IT success, but that really does nobody any favor. Organizations that want to pick up the best of both ITSM and DevOps need to encourage stakeholders to approach the table with an open mind.

    “You need to compromise. A lot of the new, young practitioners in DevOps and Agile will pretty much automatically discount opinions coming from people who have got ITIL certifications,” said Andi Mann, chief technology advocate for Splunk. “And vice versa—a lot of traditional operations and development teams will look at the people in DevOps and Agile and think they’re all cowboys. But they all have something to learn from each other.”

    For DevOps practitioners, it’s important to keep in mind that ITSM controls such as service requests, change management or structured management of incidents and problems don’t have to stop a team from delivering quality software at speed, said Troy Vetter, COO of Coda Global.

    “In fact, speed and quality are actually helped by being closely monitored. In IT, just like in science, if it isn’t seen, measured, and “reproducible, it doesn’t really count,” he noted. “So, ITSM is actually the perfect complement to DevOps because it aids in the visibility, measurement and reproducibility of key DevOps processes.”

    Failing to Address the Elephant in the Room: Change Management

    Analyst Roy Atkinson agrees that ITSM has a lot to bring to the table for DevOps workflows. ITSM’s rich history in thinking about and tracking operational best practices offers tons of value in improving the customer experience with the software being delivered. But on the flip side, ITSM teams can’t continue to operate as they did in the past. They need to compromise and learn new ways of accommodating DevOps approaches—particularly in the area of change management.

    “Change management has often been where DevOps and ITSM collide,” said Atkinson, who analyzes the service management market for HDI. “The stereotypical Change Advisory Board must radically revise its approach to be in sync with the enormous amount of change that DevOps can produce in a day.”

    The good news is that thought leaders in the ITSM world have been thinking hard about this issue and have been evolving practices so they can embed change management controls into a DevOps framework without holding back continuous delivery.

    “There is some excellent recent thinking in the ITSM world about how change management can serve its intended purpose without being a roadblock by acting as an enabler of change and being involved all along the way, rather than a bolt-on quality check at the end of the line before the production environment,” Atkinson said.

    Maintaining Separate Toolchains

    Often times one of the big reasons ITSM will remain a roadblock for DevOps isn’t because the ITSM folks aren’t willing to collaborate, but simply because each team relies on completely separate toolchains.

    “You might have a tool that developers use for tracking bugs or product backlog and then there’s a completely separate IT service management tool,” said Donna Knapp, curriculum development manager for ITSM Academy and author of the DevOps Institute’s course on ITSM for DevOps. “So what you end up doing is forcing developers to use some process that’s outside of their norm in order to address issues.”

    As Vetter explained, a lot of the knee-jerk DevOps reaction against ITSM has to do with an aversion to the historical view of ITSM requiring manual inputs in monolithic ticketing systems. But the technology of ITSM controls has advanced—it just needs to be appropriately implemented.

    “Self-service requests, automation flows and IT orchestration are now our front door to actions, and IT service management can and should be happening in the background, automagically created, maintained and measured by our DevOps tools,” he said.

    — Ericka Chickowski

  • Puppet Labs talks Docker and containerization

    Puppet Labs talks Docker and containerization

    Last week Puppet Labs released a new version of Puppet Enterprise that brought forward a number of new features, including those to Puppet Node manager that helps automate the provisioning of infrastructure from containers to bare metal, as well as a new AWS module that helps automate provisioning, configuration and management of AWS resources using Puppet. As a part of the release, Puppet announced it was officially supporting a Puppet module for Docker that has been kicking around Puppet Forge since the containerization tool went open source.  With 90,000 downloads behind it and many more customers clamoring for advice on how to better automate workflows in Puppet-controlled infrastructure while using Docker, the module had gained critical mass. staging-devopsy.kinsta.cloud took the announcement as an opportunity to catch up with Gareth Rushgrove, senior software engineer with Puppet and author of the Docker module, to discuss containerization and how Puppet is working to help customers take better advantage of container environments.

    What do you consider some of the biggest challenges when deploying containers in the real world? And how is Puppet Labs hoping to help customers get the most out of Docker?

    I think the first thing is really just…

     

    –THIS STORY HAS BEEN MOVED TO CONTAINER JOURNAL–


  • Shooting a Rocket Over Docker’s Bow

    Shooting a Rocket Over Docker’s Bow

    As momentum continues to build for the Docker containerization platform, some of the open source project’s biggest contributors argue that it has lost its way. Priorities have shifted at Docker from the ideal of building a standard container to instead supporting an entire containerization ecosystem. According to the developers at Linux disto company Core OS, it’s this move has left the entire Docker process “fundamentally flawed.” It’s why they announced this week a competing container runtime called Rocket that they hope will become the “simple composable building block” they’d originally envisioned Docker would be.

    “Unfortunately, a simple re-usable component is not how things are playing out,” wrote Alex Polvi of CoreOS in a blog about the launch. “Docker now is building tools for launching cloud servers, systems for clustering, and a wide range of functions: building images, running images, uploading, downloading, and eventually even overlay networking, all compiled into one monolithic binary running primarily as root on your server.”

    That’s an inherently risky prospect for security-conscious organizations, Polvi argues. What’s more, as Docker starts moving toward expanding the capabilities of its platform it will only increase its complexity and diminish its ability to integrate containers elsewhere. Which is why CoreOS is leading the charge for Rocket. Introduced as a prototype on GitHub this week, Rocket is a command line tool designed to run what it calls ‘App Containers.’ These can be a specification of an image format, container runtime or discovery mechanism.

    “Rocket is the first implementation of an App Container, but we do not expect it to be the only one,” Polvi says. “An open specification allows other systems do their own implementation of App Container without using Rocket at all. CoreOS fully supports and embraces alternative implementations.”

    It’s a big move considering that the company’s co-founder and CTO, Brandon Philips, is a huge Docker contributor and on the Docker governance board. However, CoreOS argues that the clients that run Docker on CoreOS are clamoring for something with the kind of flexibility that Docker seemed to promise early on.  So while CoreOS will continue to support Docker long into the future it hopes Rocket can help solve some problems that many of its clients have with the platform.

    “Our primary users have existing platforms that they want to integrate containers with,” Polvi says. “We need to fill the gap for companies that just want a way to securely and portably run a container.”

    Polvi says that as CoreOS designed Rocket around four major factors that its team believes are key for good design of a standard container. First, it should be independent and composable. Second, container images should be designed for easy, distributed retrieval. Third, it should be open so that independent implementations of tools can run in the same container consistently every time. That’s why CoreOS says that it’s developing an open specification so that other systems can implement an App Container without even using Rocket.

    And, fourth, security should be tantamount.

    “Isolation should be pluggable, and the crypto primitives for strong trust, image auditing and application identity should exist from day one,” Polvi explains. “Security primitives are very important to us, so we added an identity feature to the meta-data service. This means every instance of a running container is given a unique identity, coupled with a lightweight HSM-like service for signing.”

    This is a new approach for container environments, so CoreOS is especially seeking community feedback for this design. However, the entire project is essentially ready and waiting for community support. Those interested should  out the details on GitHub.