Tag: encryption

  • How Encryption Innovation Will Power Cloud Privacy for Developers in 2024

    How Encryption Innovation Will Power Cloud Privacy for Developers in 2024

    For developers, the surge in cloud computing adoption has brought both new opportunities and challenges. On the one hand, developers are turning to cloud services for data storage, processing and accessibility due to their flexibility, convenience and cost-effectiveness. And because the cloud takes care of these infrastructure and operational concerns, it’s allowing developers to focus on coding, innovation and delivering value to end-users. It also allows companies to scale more easily when their market grows or when new hardware becomes available.

    However, this paradigm shift has led to the accumulation of vast volumes of sensitive data on remote cloud servers—including personal, health, financial and proprietary business information. As the volume of sensitive data on the cloud continues to grow, it has naturally raised mounting security concerns, from data breaches and cyberattacks, to unauthorised access and the complexities in complying with specific industry regulations, such as GDPR or CCPA.

    For some, having to rely on the cloud provider to maintain robust security practices and data protection has also led to hesitancy in sharing data, both within and between organizations. This not only limits the potential for data-driven collaboration, research, and innovation—essentially halting progress in fields like health care, science, and social sciences—but businesses of all types and sizes could miss opportunities for cost savings, scalability, and efficiency if they’re not willing to adopt innovative technologies. There are also cases when companies are forbidden to do something by law because of privacy, again causing roadblocks.

    All of this has meant that up until now, maintaining data privacy while making the most of cloud computing has been somewhat of a balancing act for developers and companies. Companies want to fulfill their clients’ needs, while developers have to find technical solutions that provide the level of privacy required.

    Traditional Encryption methods Vs. Fully Homomorphic Encryption (FHE)

    Currently, there are several technologies and practices employed to protect data in the cloud, one of the most commonly used being traditional data encryption, which provides protection only during data transmission and storage.

    However, when data is processed, it must be decrypted—at which point it can expose data to eavesdropping, rendering it susceptible to potential threats. The aim of FHE is to ensure that the computations can be carried out on cloud-based machines while the data itself remains encrypted. With FHE, the data is really encrypted end-to-end: not only is it encrypted during transport and storage, but also it is encrypted during computations. It is only when the data or results are sent back to the client side that they are decrypted.

    This means that in the event of a data leak, the exposed information would be rendered meaningless without the decryption key, which stays securely in the hands of the owner. In other words, neither the cloud provider nor any potential spyware on the cloud machine would have the capability to decrypt the encrypted data.

    This approach, which ensures sensitive data remains encrypted throughout its entire life cycle on the cloud, even during processing and analysis, significantly enhances data security and privacy.

    Current Limitations and FHE’s Future Developments

    While developers using traditional cloud services previously had to decide whether to use the cloud with some privacy concerns or forgo its benefits altogether, with FHE we are starting to see a possible shift in attention from users’ reliance on the cloud provider to a more user-controlled approach.

    As FHE becomes more widespread, users gain the ability to make their computations over any cloud provider, even those they may not fully trust. This means the upfront costs of building new cloud platforms can be lowered by providing only FHE computation.

    While the encryption of data ensures that computations are secure and no unauthorized access or breaches can occur, this new tech is not without its limitations. For the enhanced security FHE offers, there may come a trade-off of longer execution times. Presently, we are mainly reliant on CPUs for computations. However it’s worth noting that as the technology advances and hardware accelerators become available, the execution time is expected to improve in the future. In fact, we anticipate that by 2026, FHE implementations will see a considerable increase in speed, potentially by factors of 100x or 1,000x, expanding the applicability of FHE to a wider range of use cases.

    Other benefits currently being worked on include usability. FHE had a reputation of being very complex and so used to be reserved for those with cryptography PhDs. By developing open source tools and extending the community’s knowledge around FHE, the aim is to democratize FHE so that anyone can use it, companies can experiment free of charge and check if it fulfills their needs.

    Looking to the future, privacy will likely remain high on the agenda for almost everyone. Thankfully, with new encryption technologies rapidly developing in terms of speed and usability, developers look set to achieve this, ensuring their cloud-based applications and services are as secure as possible.

  • Implement Object-Level Encryption and Policy With OpenTDF

    Implement Object-Level Encryption and Policy With OpenTDF

    With an exponential increase in data being generated, stored and shared online, the issue of data security no longer belongs to IT alone. Software and application developers have been put on notice along with business owners and legal teams worldwide to not only maintain data privacy but to build more secure products. There’s no easy solution. However, there are resources readily available for devs to build security into the fabric of their products. One is called OpenTDF, an open source project that lets you integrate encryption and data policy controls into your new and existing apps to safeguard your data, and the sharing of it, for you as well as your users.

    What’s TDF, and Why Should I Use OpenTDF?

    TDF, the “trusted data format,” was originally developed by the United States National Security Agency (NSA). It’s an open standard for object-level encryption that keeps data protected and under the data owner’s control, wherever it’s created or shared. TDF includes cryptographically secured metadata that ensures consistent policy control throughout the data life cycle. Picture this: You can grant, revoke or “turn off” data access at any time, even if the data has left your network or application.

    OpenTDF is an open source project that evolves the open TDF specification and provides a blueprint for getting started. There are a multitude of example applications that demonstrate the implementation logic, as well as streaming video and IoT use cases.

    The OpenTDF project is based on Kubernetes and OCI containers, and there is a quick start guide to get you up and running with a development environment. The quick start process will install supplemental services like Keycloak as well as project-specific services like key access service (KAS) and Abacus (an ABAC front end for configuration and management of attribute-based access control). Once you’ve completed a quick start installation, you’ll have a basic OpenTDF cluster with a Keycloak identity provider, PostgreSQL data store and a single entry point at localhost with an Nginx ingress controller.

    There’s an architectural diagram available on GitHub to see all of the services and service interactions.

    Several SDKs are available for building on the OpenTDF framework, including JavaScript, Python, C++ and Java. The client SDKs generally include basic examples for identity auth and creating a TDF-protected encrypted object.

    Take OpenTDF for a Spin: Secure Remote Storage

    Have you ever wondered how to make data access secure and simple? Using OpenTDF, developers can create that experience for their users. Let’s walk through a sample web application that uses OpenTDF to encrypt and upload data to cloud storage.

    The application we’ll be using is called OpenTDF Secure Remote Storage. It’s a react-based example that shows developers how to create encrypted data streams. These streams allow you to upload and download files from S3-compatible remote data stores while maintaining data protection. You can even remove encryption if needed.

    To make things easy, we’ll be using OpenTDF’s client-web SDK, which authenticates against Keycloak using OpenID Connect (OIDC). Keep in mind that this example runs on your local machine. It’s not designed for cloud or enterprise services.

    Prerequisites: You’ll need an S3-compatible storage object, like an Amazon S3 bucket. (You can create one for free here.)

    Set it Up

    First, install two CLI tools: Kind and Tilt. These will be used to deploy the OpenTDF services to your local machine. If you’re on macOS, you can install them with a simple Homebrew command: brew install kind tilt.

    Next, you will need the sample code on your local machine. You can either download the zip or clone the OpenTDF GitHub Repository using the following command: git clone git@github.com:opentdf/opentdf.git. This will create a directory called “opentdf” in your current location.

    Now, navigate to the root directory of the sample application: cd opentdf/examples/secure-remote-storage. To deploy OpenTDF, you’ll need a local Kubernetes cluster. Use the Kind CLI to create one: kind create cluster --name opentdf.

    Finally, start the application using Tilt: tilt up. This will launch the necessary OpenTDF services.

    Steps to Run the Application

    1. To begin, go to http://localhost:65432/secure-remote-storage in your web browser. This is where the Secure Remote Storage webpage is hosted.

    2. Now, it’s time to log in. Use the following credentials (defined in the bootstrap’s config file):
    â—‹ Username: user1
    â—‹ Password: testuser123

    3. Then, choose a file from your computer to upload. It can be anything – a text file, a PDF, or an image. Let your creativity flow! If you don’t have a file handy, no worries. You can download an image by right-clicking on it and selecting “Save As” from this link.

    4. Next, tell the application where to store your encrypted file. Provide the necessary JSON object that defines your S3-compatible object store. You can refer to the prerequisites for more details. (Optionally, you can save the configuration for future uploads. Just give it a name and click “Save.” This way, you won’t need to define the object store again in the future.)

    5. Now, it’s time to encrypt and upload. Click the “encrypt and upload” button, and watch the magic happen!

    What’s happening behind the scenes? When you click “encrypt and upload,” the application uses the OpenTDF API to convert your selected file into a .tdf file. It applies AES-GCM encryption and attaches access controls to ensure that only authorized users, like you (in this case, testuser123), can access the data. Even if your data store is public, your data remains secure.

    6. Ready to view and download your uploaded file? The table on the webpage lists all the files you’ve successfully uploaded. Each file has a download button next to it. Click that button, and the hosted file will be downloaded and decrypted on your local file system.

    Excited to explore more? Now that you’ve seen OpenTDF in action, you can dive into the source code of this application. Use this sample application as a starting point to integrate OpenTDF into your own secure applications.

    What Else is Possible With OpenTDF?

    From a secure webcam app to a privacy-forward menstrual tracking app and more, the possibilities are endless with OpenTDF and the future is in developers’ hands. By building on OpenTDF, the valuable data flowing through your applications will be protected forever.

    Learn more about OpenTDF at openTDF.io, and get the full quick-start guide, including more detailed instructions and other sample apps at the OpenTDF GitHub.


    Cassandra Zimmerman, technical product manager at Virtru, contributed to this article.

  • Developers’ Role in Protecting Privacy

    Developers’ Role in Protecting Privacy

    We live in a world where we always expect things to happen seamlessly. Every time somebody walks into a café and signs in to the free WiFi and opens their Instagram or TikTok to stay up-to-date with trends and friends or retrieves a discount from a retailer by joining their newsletter circulation, rarely do they stop to think about how that convenience and value is derived.

    Technology is the key enabler working away in the background, and developers and DevOps teams are making that happen. Underneath the slick interfaces of the apps and websites that provide these highly efficient and personalized experiences is a complex machine that relies on data to fuel it, built by developers and delivered by DevOps teams.

    In many ways, data is a two-way exchange, and those end-users that are aware of this give and take appear to be happy with how the relationship works.

    Indeed, several surveys in recent years suggested that the majority of consumers are happy to share their data with companies if it improves their experience. Furthermore, a significant minority are still comfortable if they have no control over who sees their data, there is no clarity around how data is stored and used, even if the organization in question has a poor reputation for data privacy.

    The Privacy Balancing Act

    So, if most people already accept data trade-offs in some shape or form, should developers still care about building applications and solutions that guarantee data privacy?

    The short answer is yes, but as developers, that involves a tricky balancing act.

    Although sharing data has become commonplace in exchange for benefits and value, consumers are becoming more aware of privacy issues. Take the EU’s General Data Protection Regulations (GDPR) as an example. Over the past five years, awareness has more than doubled in notable European markets such as the UK, Spain, Germany, the Netherlands and France.

    Meanwhile, there is also commercial pressure, as employers rely on developers to innovate to remain profitable. At the same time, customers expect brands to be responsible with their data, and failure to do so at the expense of trying to commercialize a new application could be detrimental. Indeed, while the pandemic may have ushered in significant changes and altered consumers’ attitudes toward data privacy, end users remain unwavering about the importance of security.

    Maintaining this balancing act is becoming increasingly complex to achieve. However, the question of data privacy is becoming a key business priority, and that means developers have a big opportunity to show their commercial value to their organizations. Indeed, companies cannot afford to get the balance wrong, and developers are central to creating the perfect formula.

    In fact, the privacy conundrum has been around for a long time.

    When the internet was created, privacy was not considered an issue because all content accessed online was deemed to be public. Nothing was encrypted; the introduction of email complicated things, not least because anybody on the internet could, theoretically, intercept personal data and read private messages.

    This prompted the emergence of the cypherpunks, a group of cryptographers who began to contemplate what the future of digital privacy might and should look like. They started to create the tools they believed the world needed to safeguard data privacy–today, their mission is carried on by developers who value user privacy.

    Why Developers Need to Be Familiar With FHE

    However, securing the transmission of data is not enough, as many breaches still occur because data isn’t encrypted during processing.

    So, in a world where more and more data needs to be shared to deliver the sort of experiences consumers expect, is it inevitable that privacy and security will be compromised as the volume of data exchanges grows? Not necessarily.

    Fully homomorphic encryption (FHE) offers a solution, allowing developers to build applications and run services without needing to see or secure the underlying data. This is because FHE enables data to be processed blindly without having to decrypt it at any stage. FHE offers a way for developers to maintain that all-important balancing act between maintaining privacy and offering value to both business and customer.

    In FHE, to guarantee a user’s data privacy, all data is encrypted by the user with their own secret key. Users of a particular service, such as an online medical portal, will use a secret key to send their encrypted data to a server where blind processing takes place. The result is also encrypted and sent back, the user again using their secret key to decrypt the information.

    What this means is that, because data is encrypted end-to-end, the company providing the service can do so in a way that keeps the same customer experience, even though they cannot see the data that is being shared and sent back. From a security perspective, this is also critical. Governments, attackers and service providers cannot see the exchanged information because they do not possess the key–they also cannot break it, as the encryption used in FHE can resist attempts made even by powerful quantum computers.

    Privacy can also be protected during larger-scale data exchanges that underpin collaboration between organizations and nation-states on numerous strategic programs, with each contributor able to add their value knowing that their intellectual property and sensitive information is not compromised.

    Looking ahead, if developers can embed FHE into their practices on a broad scale, then a key milestone for privacy is within reach. Rather than being viewed as an objective relating to digital interactions, privacy can be treated as a naturally occurring by-product of every application, piece of software and service that the developer community creates.

  • FAA Ground Stop due to Technical Debt? | Don’t Do DIY Crypto!

    FAA Ground Stop due to Technical Debt? | Don’t Do DIY Crypto!

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: The FAA’s NOTAM database gets corrupted, and Threema shows why DIY cryptography is bad. (more…)

  • IBM z16 Delivers AI and Quantum-Safe Levels of Security

    IBM z16 Delivers AI and Quantum-Safe Levels of Security

    Business transformation and disruption is all about data. But it’s not just about collecting huge amounts of data, it’s what you do with that data and how it can enable you to create powerful insights.

    While many people focus on the cloud unicorns, the reality is that industry incumbents have a huge advantage. These businesses have decades of experience which translates into massive amounts of data and business logic. Where do many of these businesses host their most important workloads and data? IBM zSystems—the system that is best known as “the mainframe.” 

    Today, IBM is announcing its latest IBM zSystem—IBM z16. Two weeks ago, IBM spent time briefing the industry analyst community on the business strategy and technical underpinnings of the IBM z16. We met with a variety of IBM business leaders including executives from the IBM Systems division, Distinguished Engineers and IBM cloud leaders. The most significant updates to the platform are the inclusion of a dedicated on-chip AI accelerator and the addition of quantum-safe cryptography.

    It’s important to understand the size of the IBM zSystems ecosystem. The platform is used by 45 of the 50 largest banks in the world, four of the top five airlines and eight of the top 10 insurers, among others. Likewise, the partner ecosystem is huge, ranging from the largest system integrators to small DevOps- and AI-focused technology vendors. Suffice it to say, some of the world’s largest businesses and governments run on IBM zSystems.

    In this article, I will briefly cover the five biggest takeaways from the pre-announcement industry analyst briefings.

    IBM is Bringing AI to the Mainframe

    Many cloud data warehouse and analytics vendors say “Bring me the data.” IBM is flipping the script, because IBM zSystems already houses clients’ most critical customer and operational data. IBM z16 will be shipped with an on-chip integrated accelerator for AI.

    Everyone knows that AI creates a competitive advantage, so why should you care about the on-chip AI accelerator? The last mile of deploying machine learning/artificial intelligence (ML/AI) models is still a huge challenge. Businesses are investing large amounts of money and skills in developing models; however, latency often kills. The difficulty is that ML/AI models need to be built into applications so that the model can be executed at the point of transaction with virtually no delay. This also must be done at a massive scale! Think about the latency and security concerns if near-real-time banking transactions needed to go from a back-end system to a cloud AI platform to check for fraudulent transactions.

    For those of you who have been following the IBM zSystems platform, IBM’s z14 unveiled a pervasive encryption capability. This encrypt-by-default approach without traditional encryption overhead was enabled through an encryption-dedicated chip. Similarly, AI workloads will be performed by the integrated AI accelerator.

    Your AI Infrastructure is Only as Powerful as Your Software Ecosystem

    Developers and data scientists want to use data, ML and AI tools that they know and trust. AI success requires an open stack of software options, not a walled garden. While IBM has a wide range of AI technologies, including Watson Machine Learning for z/os, IBM Cloud Pak for Data and IBM Cloud Pak for Watson AIOps, just to name a few, an IBM-only approach would be a non-starter for many data scientists.

    The IBM z16 AI accelerator supports a wide selection of ML libraries and compilers. Clients can build and train models in a variety of popular frameworks including TensorFlow, SAS, Microsoft CognitiveToolkit (Microsoft CNTK), PyTorch and Keras, along with others.

    The system uses ONNX, an open source tool to enable interoperability of ML/AI models to prepare and deploy the model on the dedicated AI accelerator.

    What does this mean? You can bring your existing data models to IBM z16. In addition, your teams can use their existing preferred vendors while also working with IBM zSystem’s integrated accelerator chip.

    Cybersecurity Arms Race: Quantum-Safe Cryptography is a Growing Need

    Cybercrime is a multibillion-dollar business. Although fairly routine exploits like phishing remain the number-one money generator for cybercriminals, these gangs and nation-state-sponsored organizations are getting more sophisticated. For example, cybercriminals are investing in AI and ML just like traditional businesses. In some cases, cybercrime syndicate engineering employees believe they’re working for a legitimate company.

    What is quantum-safe cryptography? The research is focused on identifying algorithms that are resistant to attacks by both classical and quantum computers. 

    Why is this a growing area of concern? IBM, along with Amazon Web Services (AWS), Microsoft Azure and Google Cloud Platform (GCP) all offer quantum computing-as-a-service. While quantum computing has massive potential for applications across a wide range of industries including drug discovery, chemical engineering and combating climate change, keep in mind that cybercriminals are looking at this technology to unlock complex cryptography algorithms. 

    IBM claims that IBM z16 is a “quantum-safe system.” The company uses a variety of different hardware, software and business process approaches to assure data security. For example, the system includes a new crypto express card with quantum-safe APIs to upgrade existing application security. At a firmware level, IBM z16 has a built-in dual signature scheme.

    In addition, for a number of years, the IBM zSystems platform has taken a zero-trust approach to security which includes a bring-your-own-key (BYOK) framework. IBM can never access the keys to your encrypted data–and neither can cybercriminals and/or governmental organizations—instead, those encryption keys remain inside your firewall.

    IBM zSystems Tech Goes to the Cloud; Mainframe Remains On-Premises

    What a difference a nine can make! Five nines mean 99.999% availability while four nines (99.99% availability) result in more than 47 minutes of downtime per year.

    When I work with C-level executives that have a mainframe within their environment, the role of IBM zSystems is always discussed. In many cases, businesses have explored the possibility of transitioning off of the mainframe, but resilience, predictability and security are the reasons why businesses remain on the platform.

    IBM mainframe services are increasingly being offered in the cloud. Most recently, IBM announced IBM Z testing in the cloud. Likewise, IBM’s Cloud Hyper Protect Crypto Services, a key management approach using a specific security chip module, is enabled by IBM zSystems technology.

    However, you shouldn’t expect mainframe-as-a-service. Between latency, security, resilience, governance and compliance, it’s going to be a long time until you migrate and/or send your mainframe workloads to the cloud.

    As the author of Hybrid Cloud for Dummies and Cloud for Dummies 2nd Ed. (among other books and articles) I can confidently say that the cloud is not a panacea. The business case for moving many mainframe workloads to the cloud doesn’t exist. 

    Digital Transformation Means More Than Slick Apps–Plumbing Matters!

    Don’t get me wrong, mobile check deposits, airline ticketing and contactless hotel check-ins are great, but in many cases, they rely on older systems that have not been updated. 

    A few weeks ago while traveling for work, my flights were delayed for hours. I saw that another flight was leaving within the hour. I was already at the airport and figured I could rebook it via the airline’s app. Of course, it wasn’t that easy—I finally went to the ticketing desk and told the agent that I’d like to book a different flight. The agent then went to work; they spent a long time entering information into their system, talking to a colleague, asking for help and asking me several questions. After about ten minutes my flight was finally rebooked.

    This airline has a great app experience when things are running smoothly (so great, in fact, that I prefer to book my travel on the app rather than the computer). However, even a slight disruption caused everything to go off the rails. It’s clear to me that the company still relies on old, broken processes and outdated technology.

    My Perspective

    Moving AI closer to the data just makes sense. The last mile of AI—moving models into transactions—continues to be a huge challenge. I work with many businesses that have tons of data and do an excellent job retrospectively analyzing that data. However, real-time data-driven decision-making remains a challenge because of latency and the fact that moving data between on-premises and cloud systems raises security and governance risks (in addition to costs).

    Quantum-safe security is a growing concern for highly regulated organizations in the banking, insurance and governmental segments. Over the last several years, financial industry CIOs and CISOs/CSOs at large financial services organizations were focused on establishing a zero-trust approach to security. However, as attacks become more sophisticated, security teams are starting to perform risk assessments on their approach to cryptography. The inclusion of quantum-safe cryptography is important for IBM zSystems customers.

  • Lapsus$ Shames Okta/Sitel | Bitcoin Nukes Climate | EU DMA E2EE FAIL

    Lapsus$ Shames Okta/Sitel | Bitcoin Nukes Climate | EU DMA E2EE FAIL

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: Okta and Sitel under fire over Lapsus$ hack, Greenpeace and others call for bitcoin change, and Europe still hates encryption. (more…)

  • How Quantum Computing can Solve Real-World Problems

    How Quantum Computing can Solve Real-World Problems

    The history of technology can seem like it was very predictable in retrospect, but the future typically feels more uncertain. Nowhere is this uncertainty more evident than in the domain of quantum computing. When the spectrum of possible outcomes spans from “quantum computers will be one of the most important technology developments of all time” to “quantum computing may never really become practical enough to justify using over a classical computer alternative,” trying to make forecasts can seem futile. However, as a technologist, investor, consumer or just a curious observer, the boundaries and texture of the uncertainty itself can be very interesting, valuable and informative. 

    Quantum computing isn’t likely a replacement for today’s classical computing—at least not initially—but rather a potential complement to it for specific applications. A 2021 report by Deloitte lists dozens of applications that could be transformed by quantum computing approaches: Protein folding, fluid simulation, credit underwriting, financial risk analysis, supply chain optimization and forecasting, vehicle routing, fraud detection, fault analysis, weather forecasting, semiconductor chip design, product-portfolio optimization, consumer product recommendations—and this is only half the list. 

    Quantum computing could make previously intractable simulation, search and optimization calculations relatively easy and quick. For some kinds of calculations, a quantum computer could be mind-bendingly faster than a powerful classical computer could ever be. While today’s classical computers running machine learning systems can make incredibly accurate predictions, quantum-based systems can, in theory, far outstrip their speed at certain prediction-related tasks.

    The blend of physics, math and computer science behind quantum computers is fascinating, but complex. Computer scientist Scott Aaronson, one of the best science communicators, wrote a recent essay for Quanta magazine called, “Why Is Quantum Computing So Hard To Explain?” 

    To oversimplify, quantum computers are built around “qubits,” which are subatomic-scale components, often kept at incredibly cold temperatures (think: Absolute zero) to minimize interference from the rest of the universe. These components can be put into a quantum physics state called “superposition” that allows them to, in a sense, take on many potential values at once. Even though their value is uncertain, reliable calculations and transformations can be performed on these qubits while in a superposition before being reduced to a more classical state, effectively parallelizing computations. Qubits in a quantum computer can also be entangled with one another (a phenomenon that Einstein referred to as “spooky action at a distance”), harnessing them as a team to increase the variety of logical calculations that it’s possible to compute. For certain types of math—for example, searching for an optimal outcome from myriad options—quantum algorithms can seem close to instantaneous when compared to those that run on classical computing hardware. 

    Quantum Computing Challenges

    Publicly available quantum computing has been on the horizon for decades, yet there’s still no reliable forecast for when products capable of widespread adoption will get released, exactly what form the systems will take or how broad and deep quantum computing’s impact will be. There are many technical challenges when it comes to getting a broadly applicable quantum computer to be useful at scale. And quantum computers are racing against classical computers, which have also been improving very rapidly in power, versatility and ease-of-use. 

    One important bottleneck for quantum computers is that correctly reading the results of a quantum calculation is prone to a very high error rate—the delicate superposition state can deteriorate before the correct result is presented as an output.

    Aaronson explains it:

    The problem, in a word, is decoherence, which means unwanted interaction between a quantum computer and its environment — nearby electric fields, warm objects, and other things that can record information about the qubits … The only known solution to this problem is quantum error correction… But researchers are only now starting to make such error correction work in the real world. When you read about the latest experiment with 50 or 60 physical qubits, it’s important to understand that the qubits aren’t error-corrected. Until they are, we don’t expect to be able to scale beyond a few hundred qubits.

    Some theorists even believe a practical quantum computer is impossible to build because of the inherently high amount of “noise in the system.” Mathematician Gil Kalai predicts that doing error correction to reduce that noise to a usable level would, by basic computing theorems, restrict the computer’s number-crunching capacity so low as to be not worth using.

    Nevertheless, researchers continue to chisel away at the challenge of error correction and to pass key technical milestones. For example, in July, Google researchers published a paper in Nature explaining how they have found a promising error-reduction method that scales as more qubits are added to a system. 

    Hardware, Software and App Landscape

    There are several different approaches to quantum hardware. Rigetti and Oxford Quantum Circuits use superconducting quantum circuits, a more widespread approach favored also by Google, Microsoft, and IBM. IonQ (which is planning to go public via a SPAC transaction) and Honeywell (which recently announced a combination of its quantum computing unit with UK-based Cambridge Quantum Computing) use trapped ions. D-Wave uses quantum annealing, ColdQuanta is using cold atoms and Xanadu is using photonics. Each approach has its pros and cons, and each hardware technology faces a fluid set of shifting bottlenecks in reaching affordable scale. Work at the materials layer continues to advance as well, for example, Raicol Crystals is growing quantum quasi-phase matching crystals that can be used in applications as varied as quantum computing, sensing, encryption and communications. And it’s not certain that there won’t be another approach that leapfrogs them in solving the challenges of error correction and price.

    Other companies, such as Quantum Machines, Q-CTRL and Seeqc, are building control hardware that can operate in the space between classical and quantum hardware. Control systems can exist at temperatures warmer than absolute zero, but colder than room temperature. Some of these systems involve both software and hardware and can perform tasks like monitoring the quantum processor’s performance, performing error correction or serving as an abstraction layer connecting classical and quantum hardware. 

    There are also companies creating quantum computing application layer software, like Zapata Computing, Riverlane or QC Ware. This can include application frameworks, components, algorithms or even turnkey quantum computing applications. Classiq has software for quantum circuit synthesis. There’s also a lot of interesting fundamental research happening at the software intersection between quantum computing and artificial intelligence – e.g. Zapata published work recently using IonQ’s ion trap quantum system to train a machine learning system to generate high-quality handwritten digits with low error rates. 

    One of the first theoretical uses for quantum computing, proposed in 1994 by mathematician Peter Shor, would be to crack the RSA encryption widely used today to protect data. The threat spawned research into post-quantum cryptography, with researchers proposing multiple new approaches to encryption that would need to replace today’s algorithms to preserve the privacy of data. Companies such as ISARA (quantum-resistant encryption) and Quantum Xchange (an out-of-band, symmetric key delivery platform) emerged to help prepare governments and companies now for a world where quantum-based, encryption-cracking algorithms start to work on breaking today’s encryption.  

    There are several widely used software development kits (SDKs) that let developers write software for a quantum computer without actually having one to run it on. In fact, a kind of format war has already broken out between quantum computing development environments.

    One widely-used quantum SDK is IBM QISKit, an open-source Python-based development kit for prototyping software and devices on simulators or IBM Quantum Experience, a cloud-based quantum service. ExxonMobil, Goldman Sachs, Boeing, JP Morgan, Samsung and PayPal are just a few of IBM Quantum’s marquee customers. Google Cirq, another Python-based development kit, touts Volkswagen as a user, and Volkswagen is also a contributor to Google’s TensorFlow Quantum, an open source library that extends Google’s TensorFlow machine learning algorithms to quantum processors for rapid prototyping. Microsoft’s Quantum Development Kit (QDK) is based on Microsoft’s Q# programming language and brings quantum compatibility to Microsoft Azure cloud services. Early customers of Azure Quantum include Dow for chemical simulations and Toyota Tsusho, Toyota’s trading arm, developing traffic light optimizations to reduce urban traffic congestion.

    In a recent piece, The Information noted that “Amazon is looking to hire more than 100 new quantum computing scientists, hardware developers and engineers within its Amazon Web Services cloud business,” following the enormous amount of funding flowing to startups in the space, such as Quantum Motion, Atom Computing and PsiQuantum. Amazon also launched Amazon Braket, a fully managed quantum computing service that gives customers access to hardware from a diverse set of vendors such as IonQ, Rigetti and D-Wave. 

    It’s also possible that the most valuable quantum platforms won’t be versatile, horizontal platforms at all; rather, the space could evolve instead to include special-purpose machines, or quantum application-specific integrated circuits (ASICs). Unlike the way things have played out with classical computing, this could mean that quantum computers would be highly specialized integrated hardware and software machines tailored to specific applications, e.g. fluid simulation or fraud detection. 

    Early use cases may involve quantum systems taking specialized roles as part of a larger classical system, similar to how NVIDIA’s graphics processing units (GPUs), which often run alongside an advanced CPU, can be used to offload specific computations for which the GPU is well-suited. In this model, where quantum components are an “accelerator” to classical systems, a credit underwriter might use a quantum processor to handle one of the tasks involved in scoring applicants and might use a different special-purpose quantum processor to perform a piece of a calculation to optimize its borrower portfolio. 

    For many use cases, the error rates of calculations are still too high for developers to do more than hone their coding chops on quantum simulation platforms, waiting until quantum computers start to really work at scale. Quantum simulators, such as QMware, make quantum software algorithms and applications believe they are running on actual quantum hardware, but they are really running on classical computers pretending to be quantum computers. 

    In the meantime, to manage some of the timing uncertainty, quantum-related startups can pursue profitable projects with government, corporate and university buyers today as a source of funding and requirements. Like some biotech startups, they can do this while assembling a strong team and seeking an early adopter clientele, progressing through scientific milestones while keeping an eye on future commercial markets.  

    The hurdles to generally available quantum computers are much like those faced by alternative energy sources: Unlocking the potential will require breakthroughs in hardware design, lower manufacturing costs and a driving force to spur large-scale adoption. And many approaches won’t work. 

    And as with any technology, it is, of course, possible that software developers will find valuable applications for quantum processors that people haven’t thought of yet today—just as developers found that GPUs meant to handle graphics processing also excel at training highly parallelized machine learning models.

    That said, a broadly useful and effective quantum computer’s arrival has the potential to be as disruptive an opportunity as the creation of the Internet or the use of machine learning to make predictions.Â