staging-devopsy.kinsta.cloud

Tag: Decentralized

  • How do we Bring APIs Into Decentralized Web3?

    How do we Bring APIs Into Decentralized Web3?

    Blockchain-based applications are growing in popularity and underscore the decentralized Web3 movement. Simultaneously, web APIs are helping port data and reusable functionality across application types and are driving microservices development practices. So, how do we bring these two paradigms together?

    There are many cases where a blockchain-based smart contract may require data updates that only a web API request can fulfill. For example, perhaps the smart contract needs to access the latest price of a given stock ticker. Or, perhaps on-chain activities are contingent on the weather at a particular time. The ubiquity of APIs means there are thousands of helpful features available for the task at hand. Yet, integrating them into Web3 applications is tricky.

    Interestingly, the API3 Alliance is making strides to introduce an industry-standard solution to bring APIs to Web3 applications. API3 provides a vendor-neutral standard gateway to apply on top of existing APIs that helps them appeal to decentralized apps. The vendor-neutral API Alliance already consists of many APIs across various verticals and appears to be gaining more traction.

    I recently met with API3’s co-founder, Heikki Vänttinen, to learn more about the challenges of introducing traditional web APIs into blockchain development. According to Vänttinen, whereas previous solutions relied on a third-party middleware or ‘oracle,’ the best way to incorporate an API request and response system into smart contracts is through a node operated by the API provider themselves. Such a system could help blockchain-based applications leverage myriad software-as-a-service solutions, essentially uniting the two worlds.

    The State of APIs in Web3

    The number of APIs is growing exponentially. As companies reinvent themselves with digital transformation, APIs have become the glue for interconnecting the applications we use on a daily basis. When it comes to APIs, there are a growing number of potential use cases for blockchain—the technology is not just relegated to cryptocurrency; rather, it can power a number of scenarios that require smart ledgers.

    Since APIs drive much of the functionality in modern software development, it makes sense that they may be useful in decentralized app development. For example, a blockchain-based decentralized finance (DeFi) app that handles lending and borrowing may want to call an API to collect raw data for a liquidation price, says Vänttinen. Or, a smart contract may wish to acquire the cost of an asset as a conditionality for making a purchase. Hooking into public APIs to check local weather or flight arrival information could affect how smart contracts respond.

    There are many use cases for incorporating APIs into a distributed ledger system. However, “blockchain applications are not able to call APIs natively,” Vänttinen explained. The incompatibility is due to the nature of blockchain technology, said Vänttinen, as it requires consensus from multiple parties to maintain the smart contracts. For this reason, blockchain applications require a middleware piece to connect a blockchain node to an API, relaying data responses from outside of the chain.

    Incorporating APIs Into Blockchain

    As Vänttinen described, previous attempts to connect APIs to Web3 architectures have revolved around a third-party middleware node. Yet, externalizing integration responsibility in this way creates a centralized point of failure, which defeats the purpose of a decentralized environment altogether.

    Instead of relying on such a dependency, Vänttinen advocates for a “first-party oracle approach,” in which API providers operate a layer in front of their APIs. This enables a request-response mechanism for smart contracts, allowing blockchain applications to initiate API requests from on-chain and incorporate the responses into the contracts. This method retains immutability and trustlessness, which is inherent to the distributed ledger.

    One solution enabling such an architecture is Airnode, a Web3 middleware that connects any web API directly to any blockchain application. Companies that have adopted Airnode make up the API3 Alliance, which is “a collective of API providers who have signed up to enable this first-party use of their API for Web3 applications,” said Vänttinen. “Adopting Airnode is a pledge from API companies to make their API blockchain-compatible.”

    The API3 Alliance is one significant initiative aiming to extend the capabilities of smart contracts and blockchain applications. This could enable such environments to call an API to send a text message or email, or grab sports scores, collect maps data and so on. Thus far, participating APIs in the API3 Alliance come from sectors such as agriculture, biotechnology, communications, open banking, insurance, supply chain management and many others. The initiative also accepts applications from API providers that want to join the marketplace.

    Benefits of a Common Web3 Integration Layer

    So, what are the benefits of embracing an industry-standard middleware for web-API-to-blockchain connectivity? Well, according to Vänttinen, it is the best way to enable complex off-chain computation. Enacting intricate computing on the chain is quite resource-intensive, resulting in high transaction fees. By extending particularly complicated calculations externally, blockchain applications could reduce operational overhead. “The industry is long overdue for connection between the real world and blockchain,” said Vänttinen. (By real-world, I assumed he meant other architectural styles that pre-date blockchain.)

    Having a standard implementation could also reduce the development burden for the API provider to support nuanced technologies. Whereas other solutions would require someone in-house to operate blockchain nodes, Airnode is more of a plug-and-play API gateway that could be deployed on AWS as a Lambda function, Vänttinen said. This usability makes it easy for API-based services to appeal to Web3 applications, thus quickly opening a new potential growth segment.

    Of course, Vänttinen admitted, the concept of syncing with off-chain processes is still at a very early stage. There are competing philosophies about how to address it on the market, and some foundational issues have not yet been resolved. For example, most efforts have focused on Read operations to acquire raw data. The next step will be having the middleware ‘oracle’ incorporate off-chain Write capabilities, too. Furthermore, more PubSub-like formats will be important to enable event listeners capable of reactive qualities.

    Final Thoughts

    The Web3 trend is bringing more decentralized, blockchain-based applications to fruition. But, such applications can’t operate in a closed-off world. To experience the full breadth of digital possibilities, they’ll likely need to figure out how to incorporate off-chain computations and mesh with alternative communication software libraries. Naturally, this will require integrations with web APIs, which have become an important method of cloud-based communication.

    The API3 Alliance is “here to propose a solution to the API connectivity problem for Web3,” said Vänttinen. “When a data provider is blockchain-compatible, you can apply the reputation of the data provider in this decentralized data feed … The importance of this becomes more clear when you talk about connecting off-chain services and microservices with smart contracts.”

    Airnode and the surrounding API3 Alliance is one initiative spearheading the connection of APIs and smart contracts. But, this is just one method in a maturing field. What do you think about bridging the two paradigms? Comment on your experience below!

  • Pendulums and DevOps

    Pendulums and DevOps

    I have long noted the trend of pendulums in IT, particularly in organizations with longer histories. Centralized IT will be achieved, its weaknesses noticed and a movement will begin to decentralize. Decentralized IT is achieved, and people remember its weaknesses, causing centralization.

    The same is true with speed of delivery versus quality. Faster will be more important for a while, then the reality that faster causes quality concerns will surface, and reliable becomes the keyword, causing slower rollouts of better product.

    This has been going on since I started in IT, and while it is more evident in enterprises, any company that lasts long enough will see these cycles run through.

    Agile and DevOps, as the Great Disruptors, stood a chance of redefining the tensions that cause these pendulum swings. Centralized versus decentralized is less important if stakeholders are all truly involved and informed—something that the culture side of the DevOps crowd aimed to achieve. Similarly, fast and quality were promised, as the ability to recover from inevitable mistakes would be rapid, while integrated testing would uncover most issues.

    We have enough years under our belts to know at this point that these issues are, at best, resolved with a resounding asterisk that is clarified sometimes.

    One nice thing about DevOps (more than agile) is that it has not stopped evolving yet, which means there is still hope that rapid delivery and high quality can both be answered while involvement of business owners and all the various IT groups will reduce the centralized/de-centralized cycle. But we’re not there yet.

    Split.io—purveyor of feature flagging/tagging tools—has a pretty good blog about what they discovered when they went out and talked to hundreds of shops. It shows that failures are happening at an alarming rate, and at some number of shops, for your average bug, they will still be fixing this released issue when the next release goes out. That is asking for trouble, in my experience.

    Better integrated testing is certainly part of the answer. TDD is good, but not enough, since it is more granular than some tests require. Split believes that feature flags are part of the answer, and I believe they’re right. But this approach is a band-aid to the problem.

    By the time you flip a feature flag to turn off a bad feature, you’ve already released a bad feature. And you will be spending time researching and fixing it. I believe feature flags should be used to catch it when this (inevitably) happens in a complex coding environment, but at the same time the source of issues should be looked into. An ounce of prevention is indeed worth a pound of cure.

    Figuring out what is allowing bugs to slip through into production rather than counting on remediating them (using precious time to do re-work) should be a priority moving forward. Is there a process/policy that is at the source? Is it simply a training issue? Training issues are repetitive—teaching today does nothing for new algorithms/tools put to use after today, and turnover can create a training gap if you’re not staying on top of it. For some shops, is speed of delivery faster than some piece (normally testing, but some shops suffer for development on certain platforms also) of the delivery chain can adequately respond to? Do you need to re-evaluate toolchain and delivery timelines?

    We are delivering more code, faster than ever, and that is exciting. But quality matters, and anyone who relies on end-users to test is playing a very dangerous game. Find ways to ensure quality without slowing down the process more than necessary. At this point, I honestly would implement runtime-flippable feature flags as a starting point, simply to be more responsive to issues. From there, work to resolve the underlying reasons that bugs are cropping up.

    Until then, keep cranking it. Keep working with the business to get quality systems in front of users, and enjoy doing some great work.

    — Don Macvittie

  • With DevOps Culture, It’s Mind Over Matter

    With DevOps Culture, It’s Mind Over Matter

    There is quite a bit of conversation about tools versus culture, and sometimes, those constantly involved in the back-and-forth forget that not everyone is so steeped in the finer points of the conversation.

    Tools is the easy part. There are a ton of tools available to optimize IT operations, from small bespoke products such as the Antsle Private Cloud Server to CA’s broad suite of Dev/Test/DevOps/ARA tools. There are plenty of places to find lists or charts of tools, and I’ll leave you to them, just mentioning the two I’ve written about in the not-too-distant past: Xebia Labs’ Periodic Table of DevOps, and CA/Automic’s Continuous Delivery Landscape.

    My youngest just built a Cube-solving robot, so this picture inspired me.

    But what of culture? When pundits and old hands talk of culture, what exactly are they talking about?

    Well, it still varies a bit from organization to organization, but there are some definite underlying foundations.

    Increased Cross-Team Communications

    Even if an org is full-on DevOps, there is still the requirement of cross-team communications to make sure everyone understands what is going on.

    One of the things that this does not make abundantly clear, I will belabor a bit (sorry, old hands). Communications is not just talking. It is not just email exchanges, it is not just sitting in another team’s meetings while researching your current problem on your phone.

    It can be these things, but it is a breadth of options. People are using Slack channels to keep everyone updated, common notifications or even pub-sub systems to make certain information about the state of IT, operations and builds are getting wide distribution. Increase communication, not chatter.

    Organizational Change

    You hear that a lot, and it can mean complete overhaul of an IT organization, but in an enterprise setting, it rarely does. The reason is simple. There may only be one or two Cisco specialists, or one or two EMC specialists, and these people don’t properly belong on one or two product/project teams in the long term. The same goes for security and a whole further swath of operations team members.

    So in the enterprise, what “organizational change” often means is moving those responsible for development and core deployment closer together on small, product- or portfolio-based teams. Networking staff, security staff and infrastructure staff will join the teams as needed, but are not part of the core team.

    Management Change

    This is another often spoken-of mantra that easily can be misunderstood. In short, managing a small, agile, entire product lifecycle team is not the same as managing a pool of resources that pop from project to project and are either (a) not involved in the operations portion at all, or (b) not involved in the development portion at all.

    For a great set of perspectives on this topic, I’ll recommend our partner The DevOps Institute. It has an eBook and a certification course (DevOps Leader) on this topic.

    Decentralization, Yet Increased Oversight

    This is often a confusing pair of messages we send when talking about what DevOps culture change does. It really is not that complex—pundits just assume everyone already knows, and, honestly, those knee-deep in DevOps do know.

    Decentralization, as taught in the DevOps Leader course mentioned above, is about driving as much decision-making as possible down to the team. When teams are built around a product or group of products, then those with the most pertinent information about what is best for the product quite often can be made within the team, speeding decision-making and implementation.

    Increased oversight is an acknowledgement of two things. First, there is more information available in an agile environment, and that information is available more frequently. Standup meetings offer some, while milestones offer other data. Second, tools are starting to allow rollup of information in a uniform fashion for CIOs, CTOs and business leaders to look at. This means they can quickly see the status of projects they have a vested interest in and provide feedback into the process. To go from, “Oh, this roadmap item is going to take a lot more work than we thought,” to, “Great, we have a plan, let’s keep moving,” is shorter when there is more information available to the decision-makers on both the business side and the IT side.

    There is no doubt plenty I have missed, but these seem to be the core things that those of us discussing this topic a lot assume others know, and that those just looking into DevOps really don’t. Hopefully it will help someone push through the clutter and come to a decision on DevOps in general, or culture changes in particular.

    — Don Macvittie