staging-devopsy.kinsta.cloud

Tag: front end

Blockchains Done Right Are the Next Evolution in Open Source

Open source code is more than just a way to create new technology. It’s a disruptive force that changed the way software is built, from taking individual developers and turning them into thriving communities to changing how enterprises do business–building open ecosystems versus restricted walled-gardens.

The next evolution of this movement is blockchain technology. Decentralized groups of developers across the globe are contributing to greater data integrity for businesses, better guarantees to ecosystem partners and can work together on live instances including their data. And that was the missing piece–live instances. Up until now, we could only collaborate on writing the code for a system together. Today, with blockchain technology, we can also collaborate on running this system together.

Open Source is Decentralized

From its ideological beginnings in the 1980s, open source code changed how we do business around the world, all stemming from developers’ work in software creation. Developers have seen open source disrupting almost every field of infrastructure, from operating systems (Linux in the cloud) to databases (MySQL, MongoDB, Redis) to programming languages (JavaScript, Python, Java, C, PHP).

Over time, open source connected thousands of developers and established a corporate precedent. In parallel, the largest corporations in the world are also the biggest contributors to open source. It is with these two transformations open source is now over 50% of all code relied upon in the average proprietary system in enterprise.

Now, blockchain is disrupting existing business models by extending the implementations and possibilities of open software. It does so by providing a transparent and immutable record keeping method, auditability of algorithms, forking of live instances with the data and possibility facilitate governance participation, all derived from an open source, decentralized community.

Amazon recently added support for open source blockchain frameworks such as Hyperledger and Ethereum on AWS. This is a great first step toward making this technology widely available. As more AWS blockchain customers build on the open source frameworks to build for enterprise business, the powerful capabilities of open source will become a part of the blockchain movement. Transparency, auditability and forkability in code, all present enormous opportunities for developers while also demonstrating a precedent.

It should be noted where these instances run, whether on private permissioned networks or public permissionless ones, is material to their ability to provide external guarantees to their users. The highest standard of trust is permissionless–where any member of the public can participate in their operation and audit.

Decentralization Rewards Both Developers and Enterprise Business

Businesses have gained major advantages through the use of open source software. In the same way, decentralized blockchain infrastructure offers businesses increased security, robustness and most importantly, guaranteed transparency with its customers.

Prior to the open source software movement, traditional methods of programming were expensive, slow to develop and lacked insight into user experience until long after each new release came to market. Today, developer communities download, interact and actively contribute to blockchain projects in the same way as open source. Decentralized developers give businesses rapid ability to replace arbitrary features with intentional decision-making by a large, merit-based community. Decentralized blockchains are Open Source 2.0.

Today, most standard apps run with centralized back ends. The server hosts the back end and dictates how the database evolves, given users’ requests and transactions. End users interact through the front end and never see what’s going on behind their static experience. What if we gave users a way to shift the power structure inherent in this state? Apps could become better for their users and much more competitive.

Not All Blockchains Fit the Open Source Credo

There’s a wide spectrum in the industry with blockchain technologies ranging from fully public to completely private. There’s a lively debate whether the value given by these fundamentally different solutions is comparable. I argue that distributed ledger technologies or private blockchains have little technological innovation. Their validators are permissioned, which means they cannot provide unbiased external guarantees to their partners and users.

That said, there are some benefits to using private blockchains in certain use-cases such as coordination between a small number of competitors or WalMart onboarding all its external supply-chain services onto one coordinated platform, but they are limited in giving any guarantees to anyone not participating in the consensus (i.e. not running a node).

Public blockchains offer a business greater transparency, because there are neutral and external parties operating the blockchain outside of the company and its customers with an independent incentive system. This removes anyone’s opportunity inside the company to falsify records, as they could potentially do on a private blockchain, or have data integrity relying on business relationships and dependency. Public blockchains also live up to the ultimate test of trust–from the comfort of your home you can verify by yourself every guarantee provided by the network.

Where Hybrid Blockchain Comes In

Most public blockchains were designed for absolute decentralization, which makes them impractical for mainstream businesses. Private blockchains are highly practical but don’t offer the transparency and trust possible with public blockchains.

Hybrid blockchains, such as the one developed by Orbs, offer the best of both worlds. They’re perfectly designed to address fundamental issues with performance and interoperability while allowing for applications in highly distributed environments. They provide strong guarantees, verifiable by anyone, yet leave businesses in control of their business models, governance and security.

— Tal Kol

July 19, 2019
How to Create Bug-Free Blockchain Apps
While all developers strive for bug-free code, it’s particularly crucial in a blockchain deployment where sensitive data or other confidential info is being exchanged, such as in health care or finance. However, some businesses have learned that lesson the hard way. Cryptocurrency exchange Binance recently revealed a devastating security breach that resulted in a loss of more than $40 million. While this is a large sum, a security breach could be even more costly for businesses that lose private messages, confidential contracts and more.

Rebuilding (and regaining customer trust) is not where organizations should be spending their time. A better way forward is for company leadership to mandate a blockchain test automation strategy from the very beginning.

While most software testers are familiar with testing web applications, they may not be sure how to approach blockchain testing. The good news is, some of the same focus areas of web testing apply to blockchain testing, too (functional, performance and security). However, blockchain involves a greater number of tests, effort and focus areas, such as infrastructure orchestration, staging and simulating scenarios in a distributed environment, which requires an even greater level of expertise.

Because of these additions, fixing a bug in a blockchain app requires several extra steps and, as a result, it is more time-consuming and costly than fixing bugs in other apps. For example, to reproduce bugs or verify code, developers should perform these steps multiple times:
1. Form a new endpoint.
2. Deploy new code to the new endpoint.
3. Migrate the current data to the new back end.
4. Suspend the old back end.
5. Update all front ends to the current version.
After that, verifying the bug-fix performed by testers or continuous integration (CI) tools will require these steps again. This complexity makes it difficult to release bug-free blockchain applications, yet doing so puts organizations and their customers at risk of financial harm. Due to the high-risk nature of deploying blockchain applications, it’s critical that testing is included in the managerial vision and strategy.

To release blockchain apps with bug-free code, businesses should follow these guidelines:

Integrate Test Automation Into Your Blockchain Testing Strategy

Test automation can relieve testers from performing thousands of orchestrating tests manually, which is both difficult and inefficient. A good system architecture should be scalable, maintainable and testable, so teams can test components independently along with mocked components. If a test requires a different environment or infrastructure, infrastructure as code (IaC, such as Puppet, Chef, Ansible and AWS is invaluable in setting up appropriate test environments. Another crucial part of the strategy is to provide the infrastructure with declarative definitions and allow developers to implement codes that support test automation.

Run Unit and Integration Tests to Cover Both Back End and Front End Code

Blockchain applications contain at least a back end (a set of smart contracts or chain codes running on a blockchain) and a front end (where users can interact with data stored in the blockchain). Performing unit tests will cover both back end and front end code, and, because of its critical role to blockchain apps, its primary focus should be on the chain codes. Integration testing, however, should cover the various types of oracles and front end applications. Performing additional tests will eliminate any holes that can threaten the back end by verifying the integration between the UI and chain codes.

Perform Tests in a Local Infrastructure

Though public blockchains (such as Bitcoin or Ethereum) have several test-nets, it’s optimal to perform the tests in a local infrastructure. Local infrastructures are lightweight, faster and more stable for unit and integration tests. For example, in a Ganache local infrastructure, it’s possible to run more than 1,000 unit tests on five Ethereum smart contracts with approximately 3,000 deployments in an hour or less.

Continuously Update Security System Test Suites

Blockchain is a multi-party infrastructure and in some cases, it is a public network. As a result, these applications will be visible to many stakeholders and may even be open-sourced to the world. This level of transparency means that hackers are able to study the programs and plan attacks based on a particular system. Continuously updating security system test suites will ensure applications are protected against all common security vulnerabilities.

Minimize Continuous Testing Pipelines for Each Phase of the Development Process

Layering integration and unit tests in a local infrastructure with continuous updates to the security system test suites creates a continuous testing pipeline that is essential to high-quality blockchain code. Continuously verifying code changes in multiple environments, such as local blockchain and test-net, ensures high-quality code is delivered to the production blockchain. These tests, however, can last hours or days for each code change, so it’s important for teams to minimize and create lean pipelines that focus on test stages, test types and test environments, and develop multiple pipelines for code at each phase of the development process.

Build Customized Tests Suites to Eliminate Prospective Cheating Holes

Test teams also need to customize test suites in the pipelines to cover both functional and security testing. Covering security cases is important, but unit test modules should also exercise smart-contract-unit potential security threats, such as overflow and re-entrancy, to eliminate as many prospective cheating holes as possible.

Due to the nature of decentralization and orchestration of underlying infrastructure, developing bug-free code for blockchain applications presents many more challenges than other applications. By taking best practices from traditional software testing, especially test automation, and customizing them for blockchain development, businesses can achieve bug-free blockchain coding and protect their most vulnerable applications.

— Thong Nguyen
May 31, 2019
ABO and the Mystical Art of Source Code Compilation

Compilation can make or break the performance quality of an application. A developer might spend weeks creating the most elegant, efficient algorithm known to man, only to have it run at a snail’s pace in production because the compiler used to create the application’s executable binary is old, and not optimized to take advantage of the enhanced performance features offered by modern hardware.

When it comes to getting the most out of code, companies in the know figured out a while ago that compilers count. Yet, for the most part, they remain a complex—if not somewhat mystical—set of technologies. I’ll admit it, for the longest time my understanding of compilation and compilers went only so far as running a make or build command. I was woefully ignorant about what was going under the covers.

But that was then, and this is now. Recently, I’ve been taking an interest in IBM’s Automatic Binary Optimizer (ABO) technology. ABO is a technology that optimizes existing COBOL code to run better in mainframe environments, without having to alter source code or modify existing compilations.

ABO intrigues meet for two reasons. First, it spurs my curiosity about mainframe compilation in general. Second, it makes me wonder how it’s possible to refine code that is already compiled and operating in a production environment.

So, I decided to go beyond my comfort zone and find the answers to some of the fundamental questions around this mysterious process. Exactly how does a mainframe compiler work? What’s the difference between a good compiler and a bad one? And finally, how does a low-level optimization technology such as ABO do its work, given that it’s being applied to code already compiled?

The following is the result of my research.

Understanding Front-End and Back-End Compilation

Compilation is the process of taking text-based source code and transforming it into a binary file that is executable against a specific piece of hardware. Mainframe computing compilation is divided into two parts: front-end compilation and back-end compilation. Front-end compilation has absolutely nothing to do with clients or GUIs. Rather, the purpose of front-end compilation is surprisingly similar to that of linting, as applied to interpreted languages. Front-end compilation is the process of making sure that the text-based source code is grammatically correct in terms of syntax, semantics, type usage and function declaration.

Once source code is subjected to front-end compilation, it’s passed on to the back end compiler for further processing.

Back-end compilation is the place where the artifact created by the front end, called the intermediate representation, is made into the binary format specific to the particular operating system and CPU of the targeted hardware. This binary file is what gets run on the targeted system. (See Figure 1).

Figure 1: Compiling COBOL source code transforms it into an opaque binary.

Back-end compilation is where the rubber meets the road in terms of machine-specific performance. A well-designed back end compiler will create binary code that’s optimized to take full advantage of the CPU architecture of the target system, squeezing out every bit of performance benefit possible. A good compiler will create a binary file that is fast to load and fast to run. However, if the underlying source code is inefficient—for example, having code that makes unnecessary trips to the database, thus creating episodic increases in network latency—no amount of back-end compilation is going to improve such runtime inefficiency. However, given that the source code is well-written, a good back-end compiler will improve overall application performance significantly. A compiler that is just so-so can do more harm than good; thus, the importance of compiler technology in the software development life cycle.

Yet, for all the benefit that it provides, there is a potential problem that is inherent in any piece of compiled code. Once source code is compiled into an executable binary, it becomes difficult to maintain—particularly when the source code is no longer available.

Compilation Creates Challenges

Unlike interpreted languages such as Python or JavaScript, in which source code is stored as plain text on a machine and executed at runtime by an interpreter, compiled code exists in a binary format that is opaque to human inspection. It’s machine readable only. Once the code is compiled, it’s “locked down” and opaque.

This creates two challenges. First, because the binary is locked down, it uses only the instruction sets that were available on the targeted system at the time of compilation. Thus, even though hardware that supports more robust instruction sets evolves, older code compiled for older systems will not be able to take advantage of the features of the modern hardware. All the compiled code can do is execute the instruction sets that were available on the older systems. The result is new hardware, yet old performance. The compiled code just doesn’t know how to move any faster.

The second challenge is that while it’s easy to change programming behavior later on by rewriting the source code from which the compiled code is created, there’s no easy way that compiled code can be changed directly. Thus, maintenance developers are left in a lurch should they need to upgrade an application for which the source code is unavailable.

In the world of mainframe computing, this is a big problem. Don’t forget that 80 percent of corporate data still resides in mainframes. There’s a lot of code out there that needs to be upgraded.

More than one company has been in the situation of having to update code that is decades old, yet both the source code and the person who created the code is nowhere to be found. The result is that companies are left with an asset that is not realizing its full value. The code remains static while significant technological advances have taken place.

There are tools that can be used for decompiling the binaries back to the original source code against which changes can be made, but the process is time-consuming and error-prone. Even if the decompiler gets it right and manages to reproduce source code that accurately reflects the original version, you still need to find a developer who can make the required changes. Such developers are in high demand in the mainframe world. Many companies simply prefer to leave well enough alone and allocate their mainframe developers to projects that have the highest return on investment.

Clearly, having a way to transform old, legacy binaries into ones that are optimized to run on modern mainframes, without having to go through a complicated decompilation and recompilation process, will benefit any company that needs to get better code to market, faster and in a cost-effective manner. Fortunately, IBM has provided a way with its ABO technology.

ABO: Solving the Compilation Challenges

ABO is designed to modernize old code without having to go through an arduous compilation/recompilation process, as described above. The way it works is interesting.

ABO works directly on the compiled binary to re-create its program structure as represented in the intermediate representation. The program structure is then passed onto the optimizer, which recompiles a new binary. This new binary takes advantage of the additional instruction sets offered by new hardware. And, it does so in such a way that the original programming logic is not altered in any way. (See Figure 2).

Figure 2: ABO performs optimization on program structures extracted from the legacy binary without affecting program logic.

For example, ABO optimizes standard math operations in COBOL such as SUBTRACT, MULTIPLY, DIVIDE and EXPONENTIATION by creating instructions in machine language that are more elegant and execute faster. No doubt, improving math operations is significant considering that COBOL is primarily used for business-oriented number crunching and correlation (being able to do math faster really counts, no pun intended), yet the important thing to understand is that the essential value of ABO optimization is that it’s taking advantage of the advanced instructions sets offered by the new target system. Remember, running older compiled code on new systems is not going to make that much of a difference in performance. In the PC world, it’s the same as running 32-bit code on 64-bit machines. Sure, in most cases it will work, but the code will never go beyond the efficiency of the low-level instructions supported by 32-bit processors.

Yes, there is some performance boost that can take place without having to recompile due solely to the improved clock speed of the CPU. However clock speeds on new mainframe models are beginning to plateau. As clock speed levels off, so will the time it takes for code to execute on the processor. This fact has not been lost on mainframe designers. New ways of enhancing overall performance, beyond simply moving the bits through the processor faster, are being devised. Sadly, there is no magic in play; for legacy code to realize better performance that’s not dependent solely on clock speed, the code will need to be compiled.

Recompiling from scratch using the source code is viable, but doing so is expensive. And, if that recompilation affects any part of the source code, additional testing needs to take place. Thus, the cost of change is increased due to both the expense of hands-on development and the added testing.

ABO, on the other hand, works with existing binaries. The result is an upgraded binary that’s optimized to the latest mainframe technology. In addition to having state-of-the-art code that’s fast and efficient, a company saves time and money in the refactoring process. It’s pretty amazing when you think about it.

What does such increased efficiency look like? IBM is reporting 24 times the reductions in CPU consumption time. That’s a substantial number.

Now, granted we’re talking about optimizing existing functionality. Refactoring an existing application to add new features requires code rewrites, no question about it. Yet, there are a lot of applications in play today that are doing what’s required. The problem is that they’re performing poorly and slowing down workflows overall. Optimizing existing binaries means not only that applications can have better performance, but that entire workflows become more efficient as more room is made for other applications to run.

Putting It All Together

Making legacy code better is, and has been, a continuous burden for companies since the first days of computing. Updating code is difficult enough when the source code and developer who wrote it are available to do the work. When both are absent and all that’s left behind is an executable binary, refactoring the application becomes a herculean set of tasks. Decompilers will only get you so far.

Fortunately, an optimization technology such as ABO makes things easier. ABO modernizes legacy binaries without the need to have the source code on hand. The result is improved system performance, reduced costs and faster release cycles.

As those of us who have been out on the terrain have learned, the rate of change in computer technology never slows down. It only goes faster. Companies that can adapt to the breakneck speed of technological innovations will prosper. This includes companies that rely upon mainframe technology to get work done. Reducing the time it takes to optimize legacy code is essential to stay competitive in today’s marketplace. For those companies that rely upon mainframe technology to get work done, ABO’s ability to optimize code directly against existing binaries provides a transformational advantage that is difficult to ignore.

— Bob Reselman

March 26, 2019
Building Amazing Apps, Part 1: Optimizing the Front End
There are millions of apps available today, and it’s incredibly easy for any user to download and install any of them with just a few taps on a screen or click of a mouse. If you want to build and deliver an app that will amaze users, you need to be able to differentiate from the rest of the pack.

Over the past 14 years, I’ve been working closely with the top companies delivering the most popular apps, helping them optimize their apps for speed and scale so users get a great experience. In this three-part series, we’ll explore the fundamental steps you must take to build and deliver your own amazing app at scale that will keep users coming back for more.

We’ll break these steps down into three key areas:
- Front end: Covers the parts of the app that end users interact with.
- Back end: Looks at the required origin infrastructure to support the app, including databases, application, web servers and more.
- Middle-mile: Explores the internet, where all the information needs to traverse back and forth between the front end and back end.
Let’s explore the front end, and the five things you need to take into consideration to build and deliver a world-class front end for an amazing app.

Beautiful User Interface

People love beautiful things. That’s why they enjoy looking at the orange colors of a sunset or why they pay exorbitant amounts of money for a work of art. Building an aesthetically pleasing interface must be a top priority during the design phase. The first sign of a good user experience is an attractive design, and users are more likely to spend more time with the app if it is appealing to the eye.

So where do you start when designing your app? Begin by reviewing the most popular apps on the app stores or the ones you use and enjoy the most and write down what design characteristics you like from each one. You should also follow industry best practices for app design such as:
- Pay attention to graphic design.
- Ensure you use curated images that look great.
- Pick beautiful fonts with curved edges, e.g., Open Sans.
- Manage white space effectively to display as much information as possible without having crowded text.
- Ensure your UI is intuitive and simple to reduce the learning curve.
- Avoid information overload through short and clear wording.
- Maintain consistency within all the screens in the UI.
- Select a distraction-free UI that avoids blinking text and excessive use of colors.
- Follow standards (follow UI designs already in place).
Accessibility

At a fundamental level, it is a social responsibility to ensure your application can be used by people with impaired mobility, vision, hearing, cognition or language understanding. In some context, it is also a legal requirement.

An excellent accessibility reference for developers is the Web Content Accessibility Guidelines (WCAG), which is sponsored by the World Wide Web Consortium (W3C). This reference includes accommodations for blindness and low vision, deafness and hearing loss, limited movement, speech disabilities, photosensitivity and more. For example, you should provide text alternatives for non-text content such as an image or a graph using the alt text HTML tag that accessibility software can use to describe what the image or the graph is. This is as simple as <img src=”smiley.jpg” alt=”Smiley face“>.

Accessibility is too often forgotten or left to the later stages of app design. However, it is much more difficult to implement accessibility functions later on than if you plan for it upfront.

Speed and Responsiveness

User patience is shrinking and first impressions matter, so your app needs to start as quickly as possible. But it’s often easier said than done to get everything ready for user interaction within 1 second flat as certain operations, such as sending data to a server over a slow mobile network, can take a long time.

So, given the challenges, how can you overcome them to provide a fast experience in your app?

Start with performance-oriented design in mind, building the app logic in a way your users will perceive it as being fast at all times. You can, for example, implement caching and store the information presented to the user in the local cache. You can then use the cache to restore the last status almost instantaneously during startup while the app opens the network connection to the server and requests the data. Another characteristic of performance-oriented design is performance budget and code optimization. Performance budget means you set a performance baseline for a specific piece of code or functionality and put tools in place to ensure that baseline is not exceeded when releasing new code. Code optimization means trying new approaches to make the code run faster.

Do this in the following cycle: optimize -> deploy -> measure.

Repeat this cycle several times until your performance gains start to flatline. After this point, you can apply two concepts from the psychology world to make your app “feel faster” (aka “perceived performance” and “active waiting”).

Perceived performance refers to how users perceive whether something is slow or fast, while active waiting refers to the fact that users perceive time as passing more quickly when they are engaged while they wait. So, when there is an operation where the user has to wait, you can incorporate some movement on the screen to draw the user’s attention—or, better yet, get them to interact with the app by providing a suggestion for optimizing their usage of the app or a curated quote. The chat app Slack does this pretty well.

Network Awareness

Your app should keep track of network status, monitoring the available bandwidth and identifying when the network is not available at all. With this information, the app should adapt its behavior to improve the overall user experience. For example, let’s say the user is trying to send some data to the server while the network is not available. One solution is to switch the user to an offline mode, and let the user know they are offline and that the operation they are trying to do will be processed as soon as they are back online. At the same time, you could offer a cached version of the information, so the user can partially interact with the app.

Network monitoring in your app should be planned carefully to avoid consuming too much power, which may impact the device’s battery life. For example, when the network is already active, you could send a few bytes to measure the latency to the server. You should avoid a constant network keep-alive, as actively maintaining the network even when there is no data to send or receive may impact the device’s energy resources.

There are also intelligent edge solutions you can apply to your app that automatically adapt the compression of images depending on the network conditions. This means that the edge server can deliver a smaller version of the image for devices that have a slow network connection, without having to implement any logic on the app.

Security

Security is not an option, but a requirement that you should keep top of mind while building your application. The last thing you want is a security breach that compromises your users’ data or the integrity of your servers.

For starters, ensure all your network connections are encrypted end to end and use secure protocols such as TLS v1.2 at the minimum or v1.3, which offer better security and a faster connection establishment. Ensure that your certificate offers strong ciphers that implement “perfect forward secrecy” (which makes it more difficult to be caught by man-in-the-middle attacks) and implement login techniques within the app that verify the user through secure techniques such as two-factor-authentication.

You can also enhance your application security by implementing security measures at the edge such as automated bot protection. In this instance, machine learning technology in your edge servers can use device sensors such as the accelerometer to differentiate a real user from a bot that has stolen credentials and is trying to perform a malicious action.

Coming up next in this three-part series: Everything you need to know about optimizing the back end.

— Javier Garza
January 9, 2019