Tag: IDE

  • How Can DevOps Evolve? Platform Engineering

    How Can DevOps Evolve? Platform Engineering

    Kaylee Van De Voorde co-authored this article.

    Has your team adopted a DevOps mindset? Are your engineers using tools like Docker containers, Kubernetes and Terraform? Over the last decade, many engineering teams that adopted DevOps culture and pivoted to cloud-native applications that required rapid scaling may face a new set of problems. 

    DevOps

    DevOps practices have brought operations and software engineering teams together throughout the entire development life cycle, from initial concept through the engineering process to production support. This collaboration paved the way for iterative development: Improving velocity, providing rapid feedback and consequently improving application quality by uncovering bugs and environment requirements earlier without user impact. DevOps automation created a more “blameless” culture that improved trust, collaboration and innovation, which boosted team morale and increased engagement. 

    That said, DevOps teams have encountered many obstacles, including fears of job obsolescence and steep learning curves for every member. Teams that learned how to implement DevOps practices on their projects may have moved more slowly at first while learning how to work together in new ways and adopting new tools.

    What is Platform Engineering?

    So, what is platform engineering? Platform engineering bridges the existing gaps between software, hardware and infrastructure that come with adopting DevOps practices: Automated horizontal scaling, service discovery and cost optimization. Platform engineering makes it even easier and simpler for engineers produce high quality applications quickly by standardizing CI/CD deployments, ensuring that infrastructure is auditable and monitored, automating deployment methods and processes, providing accurate documentation and incorporating FinOps.

    Platform engineering teams can make these improvements available to their teams by creating tools such as an internal developer platform (IDP) that includes APIs that expose internal tools and features within the platform. 

    The Next Evolution

    Evolution does not mean starting from nothing.

    The core tenets of DevOps practices still hold their value; aligning software engineering and operations teams to better create applications that embody business and user functionality and run smoothly within an incorporated infrastructure, automation, security and dedicated site reliability engineering (SRE). Many DevOps teams may already be moving toward an overarching platform engineering culture and simply need to refine their standards. Distinct application teams may already be building similar features to provide platform functionality; a strong indication that a platform engineering team would be beneficial.

    Choosing to create a dedicated platform engineering team has tradeoffs: Building a platform engineering team pulls resources from building applications and new features that the business needs, but application teams could experience an increase in productivity if a separate platform engineering team is created. A dedicated platform engineering team can also provide clear, unified direction and nuanced goals for application teams; this is incredibly important when creating standards and internal tools.

    A platform engineering team is just one way that to advance and improve engineering culture. DevOps teams that may not require platform engineering can continue their growth by refining their existing practices and patterns in their everyday work. 

  • flox Unveils More Accessible Instance of Nix Framework

    flox Unveils More Accessible Instance of Nix Framework

    This week, flox launched a more opinionated implementation of the open source Nix framework that programmatically automates package management and system configuration.

    Fresh from raising an additional $16.5 million in funding, flox CEO Ron Efroni said the company will also make an enterprise edition of its namesake flox framework available later this year.

    The Nix framework has been available since 2003 and provides DevOps teams with a portable, fully isolated development, test and production environment along with a catalog and tools for managing software packages. There are more than 5,000 contributors to the platform, but Nix is based on a unique programming language that many DevOps teams have found challenging to master.

    The flox framework provides a higher layer of abstraction for invoking Nix, and provides a set of guardrails to make the framework more accessible to a wider range of DevOps teams, said Efroni.

    That approach will make it easier to share development environments and packages across teams, machines, individual developers and entire organizations. The framework also provides visibility into the software bill of materials (SBOM) for applications used in production environments to enable real-time vulnerability management and detection without scanning, noted Efroni. That capability makes it easier to discover dependencies spanning multiple architectures; many organizations today still try to track dependencies using spreadsheets and tools that don’t serve to make DevSecOps processes simpler to manage, he added.

    It’s not clear how many organizations are using Nix today, but Efroni said that as software development management continues to become more complex across multiple platforms and architectures, there is a clear need for a different approach to enabling portability at scale. The biggest obstacle is overcoming the psychological biases many DevOps teams have developed toward Nix over the last two decades, he noted. A more opinionated framework should reduce the overall fear factor associated with Nix, noted Efroni.

    Regardless of the frameworks employed, it’s become apparent that DevOps teams are struggling under the weight of all the code they must manage, much of it written in multiple programming languages. Many organizations that embraced DevOps early are now finding the platforms they deployed have become challenging to manage and maintain. As application environments become more complex, there is a need to modernize DevOps workflows. Less clear is whether those efforts might be focused on, for example, containers and serverless computing frameworks versus a framework such as flox that is designed to span multiple architectures and platforms.

    It might be a while before IT organizations develop a unified approach to application development and deployment, given all the preferences of individual development teams. However, with the rise of platform engineering, it’s apparent more organizations are trying to strike a balance between developer freedom and the need to reduce the total cost of DevOps. That cost continues to rise as additional software architectures and platforms are employed.

    In the meantime, flox plans to contribute to both Nix and its own more opinionated version of the framework in the hopes of one day making it easier to achieve that goal.

  • AWS Previews Amazon CodeCatalyst to Automatically Create Dev Environments

    AWS Previews Amazon CodeCatalyst to Automatically Create Dev Environments

    Amazon Web Services (AWS) this week launched a preview release of the Amazon CodeCatalyst framework that leverages blueprints to automatically provision and configure developer environments in its cloud.

    Announced at the AWS re:Invent 2022 conference, Amazon CodeCatalyst takes advantage of a Project Blueprints tool to launch a source code repository and a project management tracker that are then used to create build and release pipelines. DevOps teams can use either the tools provided by AWS or substitute an instance of, for example, GitHub or the Jira project management software.

    Amazon CTO Werner Vogels told conference attendees the goal is to reduce the friction that developers currently encounter whenever they need to launch a new development project. As part of that effort, Amazon CodeCatalyst promises to make it simpler to consistently replicate development environments using scaffolds in addition to providing dashboards and search capabilities for tracking changes across commits, pull requests and deployments.

    Amazon CodeCatalyst uses a devfile to define the configuration of an environment that comes in four resizable instance options made up of two, four, eight or 16 virtual CPUs. The devfile defines and configures all the resources needed to code, test and debug a project. Devfiles, which are added to the source code repository by the selected blueprint, can also be modified.

    Collectively, those capabilities make it possible to deploy sample TypeScript, Python, Java, .NET, JavaScript, React, Angular and Vue code running in a few minutes. Developers can work with the AWS Cloud9 integrated development environment (IDE) or use tools such as JetBrains IntelliJ IDEA Ultimate, PyCharm Pro, GoLand and Visual Studio Code on their local machine. Developers, as a result, can simply switch between multiple codebases or collaborate with other developers as needed.

    The pipelines can be run on pre-provisioned builds in the cloud or organizations can opt to deploy their own container environments. Developers can incorporate build actions that are built in or provided by AWS partners in addition to incorporating GitHub Actions to compose fully automated pipelines that can be configured using either a visual editor or YAML files. Build and release pipelines enable deployment on Amazon Elastic Container Service (Amazon ECS), AWS Lambda and Amazon Elastic Compute Cloud (Amazon EC2) services.

    In general, Vogels advocated that IT teams should build event-driven applications that are loosely coupled to make it simpler to add new capabilities to existing applications running at scale without impacting the entire codebase. To make it simpler to build those types of applications, AWS added an Amazon EventBridge Pipes tool that creates point-to-point integrations between event producers and consumers in a way that eliminates the need to write that code. AWS also made available in preview a graphical AWS Application Composer tool that makes it simpler to design, configure and build serverless applications using infrastructure-as-code (IaC) definitions.

    It’s not clear to what degree developers will be embracing event-driven applications. As a programming model, event-driven architectures have been around for decades. The issue has been those applications also have tended to be among the most challenging to build and maintain. AWS is now clearly signaling its intent to make building those applications a lot easier.

  • Bringing Development Environments Into the Modern Age

    Bringing Development Environments Into the Modern Age

    It’s time to rethink how we deliver distributed access to software development environments.

    We know how to write code. And there are plenty of well-established tools, techniques and processes for doing so.

    We know what kind of code to write: Functional, scalable and efficient code that’s inherently secure and that we can readily modify whenever and however we need to modify it.

    We know why we write code. We write it to enhance the value we offer customers, to expand our market reach, outperform the competition, cut costs, minimize risks and otherwise engage in what we call “digital transformation.”

    We know who writes code—or, more accurately, the various “whos” who write it. We need developers with a diverse range of skills. And as our needs keep changing, we keep adjusting the mix of in-house and contract developers accordingly.

    We also know when to write code—which is now and for the foreseeable future.

    The one thing we still struggle with is where to write code. Not ‘where’ in the physical sense; we’re way past that. We know that for development teams to be optimally productive, we have to enable everyone to work from any physical location at any time. COVID-19 has dramatically reinforced this principle.

    The ‘where’ we struggle with is the virtual ‘where’ of the developer workspace. Yes, we already provide developers with remote access to development resources. Sort of. But there are a lot—a lot—of problems with the way we do that. These problems adversely impact development every day by, among other things,

    • Bogging developers down in annoying technical problems that hamper their ability to set up their environments.
    • Forcing developers to wrestle with the maddening complexity of configuring and managing the multiple dependencies associated with each project.
    • Trapping developers in an endless cycle of building, terminating and recreating environments as projects begin, end and are resurrected for updates and enhancements.

    Unfortunately, we’ve become so accustomed to these problems that we tend to accept them as normal. But we cannot afford to accept them any longer. Here’s why.

    Development Environments: Insecurity, Lost Productivity and Technical Debt

    One big problem for both developers and the enterprise is that developer workspaces aren’t fully portable. The methods we use to give developers remote access to their environments invariably come with unacceptable insecurity. There’s also the issue of onboarding. When a developer joins a team—whether as a new hire, temporary contractor or current employee switching to another project—it can take days for them to get them fully up to speed with access to all the project-specific resources they need. That ramp-up time invariably frustrates developers.

    On the other hand, offboarding developers can be a security and compliance issue. Piecemeal provisioning of the multiple resources developers need inherently results in the piecemeal deactivation of those same resources. So when a developer quits or is fired—or when a contractor’s contract is up—we can’t be 100% certain that all their permissions have been revoked.

    Moreover, what happens when a developer needs to go back into a project that no one has touched in months or even years? How does a developer recreate the full workspace associated with that project and have high confidence that they’re doing it right?

    The answer is that we can’t. Instead, some unfortunate developer inevitably has to spin their wheels trying to piece together a long-dormant project. All your aging code is thus guaranteed to become “legacy” (which is just another name for ‘code no one is quite sure how to work on’), thereby guaranteeing that your organization will forever be burdened with an ever-growing mountain of resource-draining technical debt.

    The Shoemaker’s Children

    So how can we re-engineer the delivery of remote workspaces so we make life easier for developers while also fulfilling the security and compliance requirements of the enterprise?

    Here are a few ideas to consider:

    • Host developer workspaces in the cloud. Developers provide everybody else with a wealth of cloud resources, yet rarely take full advantage of the cloud themselves. Some resources they need may reside in the cloud, but the complete workspaces they require to be fully productive do not.
    • But why not? Everything a developer needs for every project should exist as a complete, unified, project-specific workspace that’s available to them anywhere—just like all the other SaaS solutions developers build for end-users every day.
    • And given the sophistication of today’s browsers, cloud-based workspaces can be made available to developers without requiring them to install a desktop client. The cloud also enables workspaces to be hosted anywhere to avoid latency delays.
    • Nothing should leave your secure workspace repository. Ever. To ensure the security of all developer work-product, no actual code or data should ever leave the hosting environment.
    • Only screen pixels and keystrokes should move across the network. This remote-access model makes developers’ workspaces fully secure and fully portable. It also provides the best protection against data exfiltration.
    • Simplify the definition and administration of project-specific workspaces. You don’t want developers looking around for tasks in progress, relevant datasets or process workflows. Instead, administrators should be able to configure workspaces that give developers access to all the resources they need for each project via a single sign-on (SSO).
    • Leverage containerization into your workspace hosting. Better, safer enterprise development isn’t just about re-engineering remote workspace access. It’s also about making those workspaces easier to manage on the back end. The best way to do this is containerization, which allows you to flexibly mix-and-match workspace elements. That way, you don’t have to build every workspace from scratch.

    And those workspaces don’t exist as monolithic, “all or nothing” environments that you pray you’ve gotten right. Instead, with containerization, you can build project-specific workspaces more quickly, at less cost and with less risk.

    Effective hosting of fully portable developer workspaces requires other important functionality as well: Inactivity time-outs, cloud cost/utilization controls, etc. Before addressing these details, however, development leaders must first come to terms with the fact that developers need a much better way to work remotely—and that every remote workspace must be a secure one.

  • …And all Those Other DevOps Tools

    …And all Those Other DevOps Tools

    When we started going through the DevOps toolchain for our new project, we found a ton of bits that normally we (and most likely you) wouldn’t pay too much attention to, because they don’t matter—until they do. Then we all pay attention. That’s why we added them to the toolchain list of options before they became a problem.

    It is time to stop, take a look around, make sure you have an inventory of what tools are going into the process of making your secret sauce. The easy one is IDE. For the last little while—say, a decade—we’ve let this largely be a team or an individual decision, but the steps required to do things and the add-ons/libraries used make this less and less about individual choice and more of at least a team, often an organizational decision. We’re doing it at the project level, so functionally equivalent to team. Our Java/Spring app is done in Eclipse because most of you are doing the same for that architecture. But our new Godot project will be in the built-in Godot editor (for Godot Script) and VS Code for C#. Again, because that’s all the rage in several languages, including C#.

    Here is the thing though – if you are using VS Code, you should keep a detailed and current list of plug-ins. I’m not a fan of their whole extension installing methodology through the command palette and want to make sure everyone has the tools needed at hand. In an enterprise setting, the ROI is there to just set it up and distribute to developers what they need in a package. But this is only a real possibility if most developers are on the same platform.

    What Git Desktop client is in use? While all of them support Git command line in the same format, helping each other use the UI for different operations kind of begs for standardization. Our team doesn’t change much, so I’m not terribly worried about this particular issue. We’re officially using GitHub Desktop, but as long as I’m not doing tech support (because you have a tool that you can’t use well), I don’t care if people stick to it. You might care more about that than we do—in this regard, we’re more like a startup than an enterprise. Since they share a command line, we can afford not to care. If you, as an enterprise, are providing ready-to-code environments, that is a different story.

    A couple of years ago we would have had to worry about containers, but these days we’re okay saying Docker locally and Kubernetes for production. While the Godot app doesn’t have a server component (hence the reason we’re keeping the Java/Spring app around), the build environment will be containers because we want to be able to build wherever we need to.

    Keep rocking it. I just thought I’d drop a list of what we’re going over—there is more, of course; this bit was focused on the developer tools side of things. I’ll eventually update with the final full stack, but for now, this bit is being put on the back burner while I look at some other areas. ::waves at security peeps::

  • How to Secure Online Coding Platforms

    How to Secure Online Coding Platforms

    Threat Modeling for Online Coding Platforms

    The evolution of DevOps teams and a greater reliance on cloud-based computing has completely changed the coding process. Now, with Integrated Development Environments (IDEs), coding can be done entirely online. This is convenient, but are online IDEs secure? To answer this, we will focus on two popular cloud-based IDEs: AWS Cloud9 and Visual Studio Online.

    In online coding, the IDE is rendered inside your browser, the browser JavaScript engine uses WebSockets to initiate an Secure Shell (SSH) connection to your linked device in the background—e.g. virtual private server (VPS)—providing you with the familiar terminal interface to execute commands. The environment—meaning the linked cloud VM/VPS or SSH-accessible device—includes tool configurations (like which tokens are included, or cloud configuration files), a copy of the source code, compilers and other tools that you want to use.

    Figure 1. Local versus cloud-based IDE. Source: Trend Micro.

    The main difference between local and cloud-based coding platforms is where the bulk of the environment resides. In cloud IDEs, the environment is a virtual machine instance inside your cloud provider. In terms of security, this means you are delegating trust to the cloud provider, but you are responsible for preventing any backdoors from opening or introducing misconfiguration issues to the VM. In the case of using your own device with AWS Cloud9, you are responsible for configuring the device securely.

    Linked Devices/VMs Are Not Always Secure

    What is running on the backend often dictates whether linked devices and VMs are secure. As previously described, an SSH-linked device is necessary to use these online coding platforms. In the case of Visual Studio Online (still in preview mode as of this writing), we can find a Visual Studio (VS) Code server running on the machine.

    The code server itself is a Node.js application to which your browser will connect. You can also download the whole ~/.vscode-remote folder and run the server inside your local environment.

    As the owner of the linked device, you have permission to promote yourself to root and install or configure whatever you think you will need. You also have, by default, both Docker and Git pre-installed.

    In the case of AWS-hosted Cloud9, the situation is a little more complicated. The backend necessary for platform communication is located inside the linked device while the frontend remains hosted on a different location. But, like in VS, you can also promote yourself to root.

    This brings us to our first security concern: How secure or private are linked devices/VMs?

    The linked device contains confidential information—access tokens, application configurations, source code, etc., which should be protected against unauthorized access.

    By default, the platform is secured by the provider. However, you should keep in mind that, together with your root access availability, you are also responsible for preventing misconfiguration issues, especially when using third-party plugins. AWS does not provide plugin support for plugins not developed by them.

    There are possible misconfigurations you might make that can lead to security issues. For example, if you set your IDE to be accessible from outside, either intentionally for ease of sharing, or unintentionally, that change can have major consequences.

    In addition, it is not common for access tokens to be stored in encrypted token vaults. A lot of them can be viewed via plaintext configuration files. Without additional security measures, your access tokens can end up exposed to outsiders.

    Once unauthorized access is acquired by cybercriminals, they can compromise your code for their own gain. An example would be the supply chain attack against a software company that modified software updates with malware.

    Browsers Can Introduce Malicious Extensions or Vulnerabilities

    The next security concern is the browser itself. As online coding platforms are accessed from web browsers, you should be very careful when accessing these sites from public, non-domain, shared or unprotected computers.

    Malicious browser extensions are a well-known phenomenon. While an untrusted computer possibly infected with malware presents an obvious risk, it is also possible for an attacker to steal code using a malicious web browser extension.

    Code Extensions and Plugins Can Also Contain Malware

    The main advantage of Visual Studio Online and, generally, the VS Code platform, is the number of extensions available. This in and of itself is yet another possible attack surface.

    Let’s imagine a malicious VS Code extension—a useful-looking extension with an embedded backdoor. The lack of permission checking (such as disk access, network access, process access, etc.) for extensions during installation or use, becomes a security problem. The extent of security checks during extension publishing is limited to having a valid publisher ID and a few image-related restrictions. This means you have to trust the extension developer entirely.

    No Software Application Is Error-Free

    The ever-present possibility of vulnerabilities in any kind of software must be considered as well. In the case of online coding platforms, they can be affected by web-class vulnerabilities since IDEs are web applications. For example, a vulnerability that allows an attacker to execute their own JavaScript code can have control of the IDE or remote linked devices.

    A recent vulnerability affecting the Visual Studio Live Share extension, which is also available for VS Code, shows that we can expect to see another vulnerability affecting online coding platforms in the near future.

    Securing Cloud IDEs Is a Must for DevOps

    Threat modeling allows us to understand the different factors at play that can affect the overall security of a computing environment. Cloud IDEs should be no different from other software, if not more critical to secure.

    Here are some recommendations for each of the security issues we raised:

    • For linked device security — Install trustworthy software. Keep that software updated. Don’t indiscriminately open ports to the internet.
    • For browsers — Work inside trusted and secured environments. Try to avoid using shared computers. Install browser extensions from trusted vendors only.
    • For VS Code extensions — Avoid installing extensions from unknown sources or authors.
    • For vulnerabilities in general — Make sure your environment is updated to the most recent version.

    — David Fiser

  • IBM Z Open Editor Support for Language Server Protocol is a Game Changer

    IBM Z Open Editor Support for Language Server Protocol is a Game Changer

    The integrated development environment (IDE) is an indispensable tool for software developers. Before it came along, coding was a laborious, detail ladened undertaking. We’ve become accustomed to the syntax checking and code completion features than even the most basic IDEs provide. These days we tend to forget how hard it was programming with nothing but a rudimentary text editor. Something as simple as finding a missing comma or a misplaced curly bracket that was causing a compilation error could take hours, maybe days should the codebase be big enough. When it came to tracing your way through a seemingly endless chain of functions and classes in order to find the culprit of a runtime error, well…fuggedaboutit! Without the modern IDE, we’d be sunk.

    Well, that was then and this is now. A few years back, a new technology emerged on the technical landscape that’s boosting the power of the IDE, and subsequently the scope of work a developer does and the decisions he or she makes by an order of magnitude. This new technology is Language Server Protocol (LSP).

    LSP Takes the IDE to a Whole New Level

    LSP is a specification developed by Microsoft that standardizes the way certain typical features are implemented within a programming IDE. LSP attempts to consolidate the implementation of IDE commonplace features such as code completion, hover-over tips, jump-to-definition, find-references and syntax checking into a standard format that can be applied to any IDE in an agnostic manner.

    LSP is a technology designed to address the following problem. Imagine that a tools vendor has an idea for implementing a new plugin that will automatically do code completion for programs written in the COBOL programming language. Such a feature might be useful to any number of COBOL programmers. However, not every COBOL programmer is programming using the same tool. Some might be using an IDE such as IBM Developer for z/OS. Another might be using a text editor such as Sublime Text, Atom or even something more basic such as vim. In earlier times, in order to get the feature to market the vendor needed to implement different versions of the plugin in a way that was special to the given development environment. There’d be a team working on a Sublime Text version of the plugin, another team working on the vim plugin so forth and so on. Not only did the tool developer need to accommodate different operating systems, but each team might come up with its own, particular way to implement the feature. As team members left the company and new members arrived to replace them, those new developers needed to learn each particular way the feature was realized. As you can see, not only did the variety breed complexity, but things got very expensive very quickly.

    LSP does away with the need for such specialized tools development. LSP provides a standard protocol that can be implemented in any tool. The protocol defines a set of standard methods, objects and communication methods that provide a common set of services in any IDE.

    Thus, the only work that needs to be done to get the previously described COBOL code complete feature to market is to express it in a way that is compatible with the LSP. The actual work of realizing the feature then involves writing a client that supports the standard. It’s a “one ring to rule them all” process so to speak.

    The LSP specification was originally published by Microsoft specific. But the standard has been embraced as an industry-wide. There are implementations in Java, C++, C#, Python, Scala, Swift and PHP, to name a few. Also, there is an implementation of LSP for COBOL under z/OS, which brings us to IBM Z Open Editor.

    Bringing the Use of LSP to the Mainframe

    IBM Z Open Editor is a free Visual Studio Code extension that provides a state of the art programming experience for COBOL and PL/1 developers. Also, IBM Z Open Editor integrates with Eclipse Che and Eclipse Theia. Integration with Eclipse Che and Theia is particularly interesting because it enables browser-based editing. It’s a powerful tool, even more so given that historically COBOL and PL/1 programmers have lived in the backwaters of modern IDE development. IBM Z Open Editor makes IDE-based software development a general programming experience. Developers can use the same tool to program COBOL as they do in Java or even HTML for that matter.

    LSP Enabled Features in IBM Z Open Editor

    The following is a list of tool features available to mainframe programmers using Visual Studio Code with IBM Z Open Editor enabled:

    • Code completion.
    • Hover over instructions and snippets.
    • Jump to definitions.
    • Workspace symbols.
    • Find references.
    • Diagnostics and syntax error detection.
    • Outline view.
    • Previewing of included copybooks on hover.
    • Navigate to copybooks.
    • Code template snippets.
    • Refactoring such as “rename symbol.”
    • Variable completion.

     Using a single IDE to program in a variety of languages is a trend that’s been evolving for a while. However, adding LSP into the mix allows any IDE to present a consistent set of design time features. For example, one IDE is now able to do code-completion, hover-over reporting and variable tracing regardless of the programming language in play.

    For example, Figure 1 below shows the COBOL code completion feature of IBM Z Open Editor plugin for Visual Studio Code. IBM Z Open Editor uses LSP to facilitate code completion.

    Figure 1: IBM Z Open Editor uses the Language Server Protocol to facilitate context-sensitive code completion.

    Figure 2 shows the search and sort features of IBM Z Open Editor, which again implemented according to the LSP.

    Figure 2: Using LSP to implement IBM Z Open Editor’s sortable and searchable outline view for navigation.

    Finally, Figure 3 shows how the IBM Z Open Editor takes advantage of the extensibility capabilities of LSP to implement a feature that resolves and loads copybooks for preview. A copybook is code that can be copied from a master source and inserted into several different programs or into a variety of places in a single program. A copybook is used to define the physical layout of program data, variables for example, as well as instances of procedural code.

    Figure 3: IBM Z Open Editor extends LSP to review and load a copybook within Visual Studio Code.

    Clearly, LSP brings a lot of power to the mainframe programming experience. Now tools developers can create a unified programming experience using a common methodology. But where things really get interesting is when LSP is used to report data that goes beyond standard programming and code management activities, to a place where the real-time behavior of those using deployed code is being reported. In other words, LSP makes it possible not only to get information about code that is local to the developer’s desktop but also about the code as it is being used at runtime out on the internet.

    Beyond the Mainframe: The Long Term Significance of LSP

    LSP is extensible by design. As you read above, IBM Z Open Editor extends LSP to allow programmers to preview and load copybooks. This is done by extending the protocol.

    The typical IDE to server interaction under LSP is that the LSP client reports to the LSP server when a code document has been opened, changed, saved and closed. Then, once the LSP server is aware that a code document is in play, the server can inspect the document and report back events that are relevant to the programming language (Figure 4).

    Figure 4: LSP uses a predefined set of commands and request/response objects to support programming IDE is a language and operating system agnostic manner.

    For example, the LSP server can inspect the document to make sure the code is syntactically correct. If an error is encountered the server reports back the starting and ending location of the error, along with a message. In the case of the copybook review feature, the client reports back the location of the cursor in the code window and the server determines an opportunity for a copybook review and informs the client accordingly.

    While the copybook preview feature is powerful, it’s confined in scope to the immediate document. This is good. But, imagine what it would be like if the server had knowledge of all instances of the code deployed and running on computers connected to the internet?

    Suppose for example, that your code was powering an API running on the internet and you had the ability to learn about the number of calls to a particular function backing the API or better yet, imagine that you had the ability to learn about function calls that encountered particular problems, such as a pattern of mistyped parameters or high latency calls. Then imagine if that global runtime behavior was reported right in the IDE? Such information would change the very nature of how a programmer approaches his or her work. More importantly, it would change the way an enterprise manages programmers.

    What if a programmer could decide what to work on based on global runtime behavior reported into the IDE via an LSP extension? For example, going back to the imagined pattern of mistyped parameters, let’s say that when a developer hovers over the related function in IDE a warning icon appeared informing the developer of 1000 instances of mistyped called were made at the global level. The developer decides that such an error rate is unacceptable. Then, he/she examines the developer documentation and discovers that the parameter type definitions for the function are poorly described and that the document needs to be corrected. The developer then makes updating that developer documentation a critical path, part of the day’s work that needs to be delivered immediately.

    Given the power that LSP brings to the programming experience, such a scenario is entirely possible. In fact, there are companies out there such as Apollo Graph Manager that presently allow developers and third parties to register their production APIs in order to collect the sort of global information described above and have it reported in an IDE via LSP. As those of us who have been in the software business for a while have come to learn, once a feature such as global reporting catches on, it’s only a matter of time until it becomes an industry-wide practice. It’s only a matter of time before globally collected runtime metrics delivered to a programmers LSP become commonplace.

    Again, the ramifications are significant. Once a programmer has the ability to make fundamental decisions about feature development and deployment schedules, what then becomes the role of the project manager and the product manager?

    To take the issue one step further, how much power should a programmer be given? And in terms of general business practices, how much should the programmer know about a company’s products? These questions might seem fantastic, but keep in mind at one time companies guarded their source code as a trade secret. Today, more code than not is shared between companies as part of the open-source movement. As a result, the way code is made and a developers work has changed. The same possibility exists due to the growing trend to extend the capabilities of the LSP. The ramifications of LSP go well beyond simply reporting syntax errors and doing code completion. LSP will change the way the entire industry works, from mainframe programming to developers working in the cloud or on commodity hardware.

    Putting it All Together

    Given current trends, there’s little doubt that LSP will make even the most rudimentary code editor a powerhouse tool for software development. Standardizing IDE features into a common format allows new features for any development tool to be implemented faster and at a reduced cost. In no time at all we might even figure out a way to use LSP to make is so large bodies of code can be written automatically with just a few guiding keystrokes from the developer’s keyboard no matter the programming language. It’s a real possibility.

    Not only will LSP making coding easier and faster, but it’s also going to change the role of the software developer. Once all the code running on the internet becomes LSP aware and accessible, developers will get runtime information about how their code is performing at runtime on a global level. The trend is already in play. As history has shown, the further down a technology goes into a given population, the more powerful that population becomes and the greater the ramifications when the power is exerted. There’s a very real possibility the information that LSP enabled IDEs provides will make the day-to-day programmer more prominent in the product decision-making processes that are presently the responsibilities of program and project managers. Once that shift takes place we’re likely to see another industry-wide transformation take place in the way software is made and delivered. History has shown that such transformations have long term benefit, but in the short term are disruptive and disturbing. We’ll do well to be prepared.

    — Bob Reselman

  • Eclipse Foundation Looks to Create Cloud-Based IDE Standards

    Eclipse Foundation Looks to Create Cloud-Based IDE Standards

    The Eclipse Foundation today announced the formation of a working group to create standards for cloud-based integrated development environments (IDEs) led by Broadcom, EclipseSource, Ericsson, IBM, Intel, Red Hat, SAP, Software AG and Typefox.

    Mike Milinkovich, executive director of the Eclipse Foundation, said the Eclipse Cloud Development Tools Working Group (ECD WG) will work to replicate the success the Eclipse Foundation had in establishing standards for desktop IDEs based on an Eclipse standard among a new generation of cloud-based IDEs.

    There already are several cloud-based IDEs available. However, because each of those IDEs is based on a codebase that differs from the codebase used to create the Eclipse desktop IDE, there is no interoperability across cloud-based IDEs. Milinkovich said that makes collaboration between developers using different cloud-based IDEs all but impossible as currently there is simply too much differentiation among providers of cloud-based IDEs.

    As standards for cloud-based IDEs emerge, the Eclipse Foundation also expects best DevOps practices revolving around the integration of cloud-based IDEs and continuous integration/continuous development tools to advance as well, Milinkovich added.

    Collectively, the ECD WG will drive adoption and standards for cloud-based developer tools, including language support, extensions and developer workspace definition. To achieve that goal, the ECG WD will pull together oversight of several existing open source projects being spearheaded by the Eclipse Foundation, including Eclipse Theia, an IDE framework for building TypeScript applications; Eclipse CodeWind, a project that adds support for containers to IDEs based on Eclipse; Eclipse Dirigible, a cloud-based development platform developed by SAP; Eclipse Sprotty, a diagramming framework; and Eclipse Orion, a browser-based integration platform.

    Milinkovich said via those efforts, the ECD WG hopes to accelerate the adoption of cloud integrated development environment (IDE) and container-based workspace management, including the recently launched Che 7 project that makes available a native instance of Eclipse to build Java applications for Kubernetes environments.

    It may be a while before the ECD WG can create a set of standards for cloud-based IDEs that will have a material impact on application development. The Eclipse Foundation still has to bring numerous application development powerhouses on board for this project, including Microsoft and, once the acquisition of Pivotal Software is completed, VMware. However, organizations that are shifting development to the cloud at the very least can take comfort in the fact that there is now some level of commitment to creating interoperability standards that one day could increase overall developer productivity as more applications are built in the cloud.

    In the meantime, it’s apparent the Eclipse Foundation is committed to remaining relevant in the age of the cloud by both fostering IDE interoperability and acting as the steward for Jakarta Enterprise Edition, the open source follow-on to Java. Of course, not every IDE may be employed to build applications using Java. But at the very least, many of the familiar interfaces and constructs developers know today on their desktops will be moving forward into the cloud.

    — Mike Vizard