Has your team adopted a DevOps mindset? Are your engineers using tools like Docker containers, Kubernetes and Terraform? Over the last decade, many engineering teams that adopted DevOps culture and pivoted to cloud-native applications that required rapid scaling may face a new set of problems.
DevOps
DevOps practices have brought operations and software engineering teams together throughout the entire development life cycle, from initial concept through the engineering process to production support. This collaboration paved the way for iterative development: Improving velocity, providing rapid feedback and consequently improving application quality by uncovering bugs and environment requirements earlier without user impact. DevOps automation created a more “blameless” culture that improved trust, collaboration and innovation, which boosted team morale and increased engagement.
That said, DevOps teams have encountered many obstacles, including fears of job obsolescence and steep learning curves for every member. Teams that learned how to implement DevOps practices on their projects may have moved more slowly at first while learning how to work together in new ways and adopting new tools.
What is Platform Engineering?
So, what is platform engineering? Platform engineering bridges the existing gaps between software, hardware and infrastructure that come with adopting DevOps practices: Automated horizontal scaling, service discovery and cost optimization. Platform engineering makes it even easier and simpler for engineers produce high quality applications quickly by standardizing CI/CD deployments, ensuring that infrastructure is auditable and monitored, automating deployment methods and processes, providing accurate documentation and incorporating FinOps.
Platform engineering teams can make these improvements available to their teams by creating tools such as an internal developer platform (IDP) that includes APIs that expose internal tools and features within the platform.
The Next Evolution
Evolution does not mean starting from nothing.
The core tenets of DevOps practices still hold their value; aligning software engineering and operations teams to better create applications that embody business and user functionality and run smoothly within an incorporated infrastructure, automation, security and dedicated site reliability engineering (SRE). Many DevOps teams may already be moving toward an overarching platform engineering culture and simply need to refine their standards. Distinct application teams may already be building similar features to provide platform functionality; a strong indication that a platform engineering team would be beneficial.
Choosing to create a dedicated platform engineering team has tradeoffs: Building a platform engineering team pulls resources from building applications and new features that the business needs, but application teams could experience an increase in productivity if a separate platform engineering team is created. A dedicated platform engineering team can also provide clear, unified direction and nuanced goals for application teams; this is incredibly important when creating standards and internal tools.
A platform engineering team is just one way that to advance and improve engineering culture. DevOps teams that may not require platform engineering can continue their growth by refining their existing practices and patterns in their everyday work.
This week, flox launched a more opinionated implementation of the open source Nix framework that programmatically automates package management and system configuration.
Fresh from raising an additional $16.5 million in funding, flox CEO Ron Efroni said the company will also make an enterprise edition of its namesake flox framework available later this year.
The Nix framework has been available since 2003 and provides DevOps teams with a portable, fully isolated development, test and production environment along with a catalog and tools for managing software packages. There are more than 5,000 contributors to the platform, but Nix is based on a unique programming language that many DevOps teams have found challenging to master.
The flox framework provides a higher layer of abstraction for invoking Nix, and provides a set of guardrails to make the framework more accessible to a wider range of DevOps teams, said Efroni.
That approach will make it easier to share development environments and packages across teams, machines, individual developers and entire organizations. The framework also provides visibility into the software bill of materials (SBOM) for applications used in production environments to enable real-time vulnerability management and detection without scanning, noted Efroni. That capability makes it easier to discover dependencies spanning multiple architectures; many organizations today still try to track dependencies using spreadsheets and tools that don’t serve to make DevSecOps processes simpler to manage, he added.
It’s not clear how many organizations are using Nix today, but Efroni said that as software development management continues to become more complex across multiple platforms and architectures, there is a clear need for a different approach to enabling portability at scale. The biggest obstacle is overcoming the psychological biases many DevOps teams have developed toward Nix over the last two decades, he noted. A more opinionated framework should reduce the overall fear factor associated with Nix, noted Efroni.
Regardless of the frameworks employed, it’s become apparent that DevOps teams are struggling under the weight of all the code they must manage, much of it written in multiple programming languages. Many organizations that embraced DevOps early are now finding the platforms they deployed have become challenging to manage and maintain. As application environments become more complex, there is a need to modernize DevOps workflows. Less clear is whether those efforts might be focused on, for example, containers and serverless computing frameworks versus a framework such as flox that is designed to span multiple architectures and platforms.
It might be a while before IT organizations develop a unified approach to application development and deployment, given all the preferences of individual development teams. However, with the rise of platform engineering, it’s apparent more organizations are trying to strike a balance between developer freedom and the need to reduce the total cost of DevOps. That cost continues to rise as additional software architectures and platforms are employed.
In the meantime, flox plans to contribute to both Nix and its own more opinionated version of the framework in the hopes of one day making it easier to achieve that goal.
Amazon Web Services (AWS) this week launched a preview release of the Amazon CodeCatalyst framework that leverages blueprints to automatically provision and configure developer environments in its cloud.
Announced at the AWS re:Invent 2022 conference, Amazon CodeCatalyst takes advantage of a Project Blueprints tool to launch a source code repository and a project management tracker that are then used to create build and release pipelines. DevOps teams can use either the tools provided by AWS or substitute an instance of, for example, GitHub or the Jira project management software.
Amazon CTO Werner Vogels told conference attendees the goal is to reduce the friction that developers currently encounter whenever they need to launch a new development project. As part of that effort, Amazon CodeCatalyst promises to make it simpler to consistently replicate development environments using scaffolds in addition to providing dashboards and search capabilities for tracking changes across commits, pull requests and deployments.
Amazon CodeCatalyst uses a devfile to define the configuration of an environment that comes in four resizable instance options made up of two, four, eight or 16 virtual CPUs. The devfile defines and configures all the resources needed to code, test and debug a project. Devfiles, which are added to the source code repository by the selected blueprint, can also be modified.
Collectively, those capabilities make it possible to deploy sample TypeScript, Python, Java, .NET, JavaScript, React, Angular and Vue code running in a few minutes. Developers can work with the AWS Cloud9 integrated development environment (IDE) or use tools such as JetBrains IntelliJ IDEA Ultimate, PyCharm Pro, GoLand and Visual Studio Code on their local machine. Developers, as a result, can simply switch between multiple codebases or collaborate with other developers as needed.
The pipelines can be run on pre-provisioned builds in the cloud or organizations can opt to deploy their own container environments. Developers can incorporate build actions that are built in or provided by AWS partners in addition to incorporating GitHub Actions to compose fully automated pipelines that can be configured using either a visual editor or YAML files. Build and release pipelines enable deployment on Amazon Elastic Container Service (Amazon ECS), AWS Lambda and Amazon Elastic Compute Cloud (Amazon EC2) services.
In general, Vogels advocated that IT teams should build event-driven applications that are loosely coupled to make it simpler to add new capabilities to existing applications running at scale without impacting the entire codebase. To make it simpler to build those types of applications, AWS added an Amazon EventBridge Pipes tool that creates point-to-point integrations between event producers and consumers in a way that eliminates the need to write that code. AWS also made available in preview a graphical AWS Application Composer tool that makes it simpler to design, configure and build serverless applications using infrastructure-as-code (IaC) definitions.
It’s not clear to what degree developers will be embracing event-driven applications. As a programming model, event-driven architectures have been around for decades. The issue has been those applications also have tended to be among the most challenging to build and maintain. AWS is now clearly signaling its intent to make building those applications a lot easier.
Weknowwhywewritecode.Wewriteittoenhancethevalueweoffercustomers, to expandourmarketreach,outperformthecompetition,cutcosts,minimizerisks andotherwiseengageinwhatwecall“digitaltransformation.”
We know whowrites code—or, more accurately, the various “whos” who write it. We needdeveloperswithadiverserangeofskills.Andasourneedskeepchanging,wekeep adjusting themixofin-houseandcontractdevelopersaccordingly.
The one thing we still struggle with is whereto write code. Not ‘where’ in the physical sense; we’rewaypastthat.Weknowthatfordevelopmentteamstobeoptimallyproductive,wehaveto enable everyone to work from any physical location at any time. COVID-19 has dramaticallyreinforcedthisprinciple.
The ‘where’westrugglewithisthevirtual ‘where’ofthedeveloperworkspace.Yes,wealreadyprovide developers with remote access to development resources. Sort of. But there are alot—a lot—of problems with the way we do that. These problems adversely impact developmenteverydayby,amongotherthings,
Boggingdevelopersdowninannoyingtechnicalproblems that hamper their ability tosetuptheirenvironments.
Trappingdevelopersinanendlesscycleofbuilding,terminatingandrecreatingenvironments as projects begin, end and are resurrected for updates andenhancements.
Unfortunately,we’vebecomesoaccustomedtotheseproblemsthatwetendtoacceptthem as normal.Butwecannotaffordtoacceptthemanylonger.Here’swhy.
Development Environments: Insecurity, Lost Productivityand Technical Debt
One bigproblem forboth developersandthe enterpriseis thatdeveloperworkspaces aren’tfully portable. The methods we use to give developers remote access to their environmentsinvariably come with unacceptable insecurity. There’s also the issue of onboarding. When adeveloper joins a team—whether as a new hire, temporary contractor or current employeeswitchingtoanotherproject—itcantakedaysforthemtogetthemfullyuptospeedwithaccessto all the project-specific resources they need. That ramp-up time invariably frustratesdevelopers.
Ontheotherhand,offboardingdeveloperscanbeasecurityandcomplianceissue.Piecemealprovisioning of the multiple resources developers need inherently results in the piecemealdeactivation of those same resources. So when a developer quits or is fired—or when acontractor’s contract is up—we can’t be 100% certain that all their permissions have beenrevoked.
Moreover, what happens when a developer needs to go back into a project that no one hastouchedinmonthsor even years?Howdoesadeveloperrecreatethefullworkspaceassociatedwiththatprojectandhavehighconfidencethatthey’redoingitright?
The answer is that we can’t. Instead, some unfortunate developer inevitably has to spin theirwheels trying to piece together a long-dormant project. All your aging code is thus guaranteed tobecome “legacy” (which is just another name for ‘code no one is quite sure how to work on’),thereby guaranteeing that your organization will forever be burdened with an ever-growingmountainofresource-drainingtechnical debt.
The Shoemaker’s Children
So how can we re-engineer the delivery of remote workspaces so we make life easier fordevelopers whilealsofulfillingthesecurityandcompliancerequirementsoftheenterprise?
Hereareafewideastoconsider:
Host developer workspaces in the cloud. Developers provide everybody else with awealth of cloud resources, yet rarely take full advantage of the cloud themselves. Someresourcestheyneedmayresideinthecloud,butthecompleteworkspacestheyrequiretobefullyproductivedonot.
Butwhynot?Everythingadeveloperneedsforeveryprojectshouldexistasacomplete,unified, project-specific workspace that’s available to them anywhere—just like all theotherSaaSsolutionsdevelopersbuildforend-userseveryday.
Andgiventhesophisticationoftoday’sbrowsers,cloud-basedworkspacescanbemadeavailable to developers without requiring them to install a desktop client. The cloud alsoenablesworkspacestobehostedanywheretoavoidlatencydelays.
Onlyscreenpixelsandkeystrokesshouldmoveacrossthenetwork.Thisremote-accessmodelmakes developers’ workspaces fully secure and fully portable. It also provides the bestprotectionagainstdataexfiltration.
Simplify the definition and administration of project-specific workspaces. You don’twant developers looking around for tasks in progress, relevant datasets or processworkflows. Instead, administrators should be able to configure workspaces that givedevelopersaccesstoalltheresourcestheyneedforeachprojectviaasinglesign-on (SSO).
Leverage containerization into your workspace hosting. Better, safer enterprisedevelopment isn’t just about re-engineering remote workspace access. It’s also aboutmaking those workspaces easier to manage on the back end. The best way to do this iscontainerization, whichallowsyoutoflexiblymix-and-matchworkspaceelements.Thatway,youdon’thavetobuildeveryworkspacefromscratch.
And those workspaces don’t exist as monolithic, “all or nothing” environments thatyouprayyou’vegottenright.Instead,withcontainerization,youcanbuildproject-specificworkspacesmorequickly,atlesscostandwithlessrisk.
Effectivehostingoffullyportabledeveloperworkspacesrequiresotherimportantfunctionalityaswell: Inactivity time-outs, cloud cost/utilization controls, etc. Before addressing these details,however, development leaders must first come to terms with the fact that developers need amuchbetterwaytoworkremotely—andthateveryremoteworkspacemustbeasecureone.
When we started going through the DevOps toolchain for our new project, we found a ton of bits that normally we (and most likely you) wouldn’t pay too much attention to, because they don’t matter—until they do. Then we all pay attention. That’s why we added them to the toolchain list of options before they became a problem.
It is time to stop, take a look around, make sure you have an inventory of what tools are going into the process of making your secret sauce. The easy one is IDE. For the last little while—say, a decade—we’ve let this largely be a team or an individual decision, but the steps required to do things and the add-ons/libraries used make this less and less about individual choice and more of at least a team, often an organizational decision. We’re doing it at the project level, so functionally equivalent to team. Our Java/Spring app is done in Eclipse because most of you are doing the same for that architecture. But our new Godot project will be in the built-in Godot editor (for Godot Script) and VS Code for C#. Again, because that’s all the rage in several languages, including C#.
Here is the thing though – if you are using VS Code, you should keep a detailed and current list of plug-ins. I’m not a fan of their whole extension installing methodology through the command palette and want to make sure everyone has the tools needed at hand. In an enterprise setting, the ROI is there to just set it up and distribute to developers what they need in a package. But this is only a real possibility if most developers are on the same platform.
What Git Desktop client is in use? While all of them support Git command line in the same format, helping each other use the UI for different operations kind of begs for standardization. Our team doesn’t change much, so I’m not terribly worried about this particular issue. We’re officially using GitHub Desktop, but as long as I’m not doing tech support (because you have a tool that you can’t use well), I don’t care if people stick to it. You might care more about that than we do—in this regard, we’re more like a startup than an enterprise. Since they share a command line, we can afford not to care. If you, as an enterprise, are providing ready-to-code environments, that is a different story.
A couple of years ago we would have had to worry about containers, but these days we’re okay saying Docker locally and Kubernetes for production. While the Godot app doesn’t have a server component (hence the reason we’re keeping the Java/Spring app around), the build environment will be containers because we want to be able to build wherever we need to.
Keep rocking it. I just thought I’d drop a list of what we’re going over—there is more, of course; this bit was focused on the developer tools side of things. I’ll eventually update with the final full stack, but for now, this bit is being put on the back burner while I look at some other areas. ::waves at security peeps::
The evolution of DevOps teams and a greater reliance on cloud-based computing has completely changed the coding process. Now, with Integrated Development Environments (IDEs), coding can be done entirely online. This is convenient, but are online IDEs secure? To answer this, we will focus on two popular cloud-based IDEs: AWS Cloud9 and Visual Studio Online.
In online coding, the IDE is rendered inside your browser, the browser JavaScript engine uses WebSockets to initiate an Secure Shell (SSH) connection to your linked device in the background—e.g. virtual private server (VPS)—providing you with the familiar terminal interface to execute commands. The environment—meaning the linked cloud VM/VPS or SSH-accessible device—includes tool configurations (like which tokens are included, or cloud configuration files), a copy of the source code, compilers and other tools that you want to use.
Figure 1. Local versus cloud-based IDE. Source: Trend Micro.
The main difference between local and cloud-based coding platforms is where the bulk of the environment resides. In cloud IDEs, the environment is a virtual machine instance inside your cloud provider. In terms of security, this means you are delegating trust to the cloud provider, but you are responsible for preventing any backdoors from opening or introducing misconfiguration issues to the VM. In the case of using your own device with AWS Cloud9, you are responsible for configuring the device securely.
Linked Devices/VMs Are Not Always Secure
What is running on the backend often dictates whether linked devices and VMs are secure. As previously described, an SSH-linked device is necessary to use these online coding platforms. In the case of Visual Studio Online (still in preview mode as of this writing), we can find a Visual Studio (VS) Code server running on the machine.
The code server itself is a Node.js application to which your browser will connect. You can also download the whole ~/.vscode-remote folder and run the server inside your local environment.
As the owner of the linked device, you have permission to promote yourself to root and install or configure whatever you think you will need. You also have, by default, both Docker and Git pre-installed.
In the case of AWS-hosted Cloud9, the situation is a little more complicated. The backend necessary for platform communication is located inside the linked device while the frontend remains hosted on a different location. But, like in VS, you can also promote yourself to root.
This brings us to our first security concern:How secure or private are linked devices/VMs?
The linked device contains confidential information—access tokens, application configurations, source code, etc., which should be protected against unauthorized access.
By default, the platform is secured by the provider. However, you should keep in mind that, together with your root access availability, you are also responsible for preventing misconfiguration issues, especially when using third-party plugins. AWS does not provide plugin support for plugins not developed by them.
There are possible misconfigurations you might make that can lead to security issues. For example, if you set your IDE to be accessible from outside, either intentionally for ease of sharing, or unintentionally, that change can have major consequences.
In addition, it is not common for access tokens to be stored in encrypted token vaults. A lot of them can be viewed via plaintext configuration files. Without additional security measures, your access tokens can end up exposed to outsiders.
Once unauthorized access is acquired by cybercriminals, they can compromise your code for their own gain. An example would be the supply chain attack against a software company that modified software updates with malware.
Browsers Can Introduce Malicious Extensions or Vulnerabilities
The next security concern is the browser itself. As online coding platforms are accessed from web browsers, you should be very careful when accessing these sites from public, non-domain, shared or unprotected computers.
Malicious browser extensions are a well-known phenomenon. While an untrusted computer possibly infected with malware presents an obvious risk, it is also possible for an attacker to steal code using a malicious web browser extension.
Code Extensions and Plugins Can Also Contain Malware
The main advantage of Visual Studio Online and, generally, the VS Code platform, is the number of extensions available. This in and of itself is yet another possible attack surface.
Let’s imagine a malicious VS Code extension—a useful-looking extension with an embedded backdoor. The lack of permission checking (such as disk access, network access, process access, etc.) for extensions during installation or use, becomes a security problem. The extent of security checks during extension publishing is limited to having a valid publisher ID and a few image-related restrictions. This means you have to trust the extension developer entirely.
No Software Application Is Error-Free
The ever-present possibility of vulnerabilities in any kind of software must be considered as well. In the case of online coding platforms, they can be affected by web-class vulnerabilities since IDEs are web applications. For example, a vulnerability that allows an attacker to execute their own JavaScript code can have control of the IDE or remote linked devices.
A recent vulnerability affecting the Visual Studio Live Share extension, which is also available for VS Code, shows that we can expect to see another vulnerability affecting online coding platforms in the near future.
Securing Cloud IDEs Is a Must for DevOps
Threat modeling allows us to understand the different factors at play that can affect the overall security of a computing environment. Cloud IDEs should be no different from other software, if not more critical to secure.
Here are some recommendations for each of the security issues we raised:
For linked device security — Install trustworthy software. Keep that software updated. Don’t indiscriminately open ports to the internet.
For browsers — Work inside trusted and secured environments. Try to avoid using shared computers. Install browser extensions from trusted vendors only.
For VS Code extensions — Avoid installing extensions from unknown sources or authors.
For vulnerabilities in general — Make sure your environment is updated to the most recent version.
The integrated development environment (IDE) is an indispensable tool for software developers. Before it came along, coding was a laborious, detail ladened undertaking. We’ve become accustomed to the syntax checking and code completion features than even the most basic IDEs provide. These days we tend to forget how hard it was programming with nothing but a rudimentary text editor. Something as simple as finding a missing comma or a misplaced curly bracket that was causing a compilation error could take hours, maybe days should the codebase be big enough. When it came to tracing your way through a seemingly endless chain of functions and classes in order to find the culprit of a runtime error, well…fuggedaboutit! Without the modern IDE, we’d be sunk.
Well, that was then and this is now. A few years back, a new technology emerged on the technical landscape that’s boosting the power of the IDE, and subsequently the scope of work a developer does and the decisions he or she makes by an order of magnitude. This new technology is Language Server Protocol (LSP).
LSP Takes the IDE to a Whole New Level
LSP is aspecification developed by Microsoft that standardizes the way certain typical features are implemented within a programming IDE. LSP attempts to consolidate the implementation of IDE commonplace features such as code completion, hover-over tips, jump-to-definition, find-references and syntax checking into a standard format that can be applied to any IDE in an agnostic manner.
LSP is a technology designed to address the following problem. Imagine that a tools vendor has an idea for implementing a new plugin that will automatically do code completion for programs written in the COBOL programming language. Such a feature might be useful to any number of COBOL programmers. However, not every COBOL programmer is programming using the same tool. Some might be using an IDE such asIBM Developer for z/OS. Another might be using a text editor such asSublime Text,Atom or even something more basic such asvim. In earlier times, in order to get the feature to market the vendor needed to implement different versions of the plugin in a way that was special to the given development environment. There’d be a team working on a Sublime Text version of the plugin, another team working on the vim plugin so forth and so on. Not only did the tool developer need to accommodate different operating systems, but each team might come up with its own, particular way to implement the feature. As team members left the company and new members arrived to replace them, those new developers needed to learn each particular way the feature was realized. As you can see, not only did the variety breed complexity, but things got very expensive very quickly.
LSP does away with the need for such specialized tools development. LSP provides a standard protocol that can be implemented in any tool. The protocol defines a set of standard methods, objects and communication methods that provide a common set of services in any IDE.
Thus, the only work that needs to be done to get the previously described COBOL code complete feature to market is to express it in a way that is compatible with the LSP. The actual work of realizing the feature then involves writing a client that supports the standard. It’s a “one ring to rule them all” process so to speak.
The LSP specification was originally published by Microsoft specific. But the standard has been embraced as anindustry-wide. There are implementations inJava,C++,C#,Python,Scala,Swift andPHP, to name a few. Also, there is an implementation of LSP forCOBOL under z/OS, which brings us to IBM Z Open Editor.
Bringing the Use of LSP to the Mainframe
IBM Z Open Editor is a free Visual Studio Code extension that provides a state of the art programming experience forCOBOL andPL/1 developers. Also, IBM Z Open Editor integrates with Eclipse Che andEclipse Theia. Integration with Eclipse Che and Theia is particularly interesting because it enables browser-based editing. It’s a powerful tool, even more so given that historically COBOL and PL/1 programmers have lived in the backwaters of modern IDE development. IBM Z Open Editor makes IDE-based software development a general programming experience. Developers can use the same tool to program COBOL as they do in Java or even HTML for that matter.
LSP Enabled Features in IBM Z Open Editor
The following is a list of tool features available to mainframe programmers using Visual Studio Code with IBM Z Open Editor enabled:
Code completion.
Hover over instructions and snippets.
Jump to definitions.
Workspace symbols.
Find references.
Diagnostics and syntax error detection.
Outline view.
Previewing of included copybooks on hover.
Navigate to copybooks.
Code template snippets.
Refactoring such as “rename symbol.”
Variable completion.
Using a single IDE to program in a variety of languages is a trend that’s been evolving for a while. However, adding LSP into the mix allows any IDE to present a consistent set of design time features. For example, one IDE is now able to do code-completion, hover-over reporting and variable tracing regardless of the programming language in play.
For example, Figure 1 below shows the COBOL code completion feature of IBM Z Open Editor plugin for Visual Studio Code. IBM Z Open Editor uses LSP to facilitate code completion.
Figure 1: IBM Z Open Editor uses the Language Server Protocol to facilitate context-sensitive code completion.
Figure 2 shows the search and sort features of IBM Z Open Editor, which again implemented according to the LSP.
Figure 2: Using LSP to implement IBM Z Open Editor’s sortable and searchable outline view for navigation.
Finally, Figure 3 shows how the IBM Z Open Editor takes advantage of the extensibility capabilities of LSP to implement a feature that resolves and loads copybooks for preview. Acopybook is code that can be copied from a master source and inserted into several different programs or into a variety of places in a single program. A copybook is used to define the physical layout of program data, variables for example, as well as instances of procedural code.
Figure 3: IBM Z Open Editor extends LSP to review and load a copybook within Visual Studio Code.
Clearly, LSP brings a lot of power to the mainframe programming experience. Now tools developers can create a unified programming experience using a common methodology. But where things really get interesting is when LSP is used to report data that goes beyond standard programming and code management activities, to a place where the real-time behavior of those using deployed code is being reported. In other words, LSP makes it possible not only to get information about code that is local to the developer’s desktop but also about the code as it is being used at runtime out on the internet.
Beyond the Mainframe: The Long Term Significance of LSP
LSP is extensible by design. As you read above, IBM Z Open Editor extends LSP to allow programmers to preview and load copybooks. This is done by extending the protocol.
The typical IDE to server interaction under LSP is that the LSP client reports to the LSP server when a code document has been opened, changed, saved and closed. Then, once the LSP server is aware that a code document is in play, the server can inspect the document and report back events that are relevant to the programming language (Figure 4).
Figure 4: LSP uses a predefined set of commands and request/response objects to support programming IDE is a language and operating system agnostic manner.
For example, the LSP server can inspect the document to make sure the code is syntactically correct. If an error is encountered the server reports back the starting and ending location of the error, along with a message. In the case of the copybook review feature, the client reports back the location of the cursor in the code window and the server determines an opportunity for a copybook review and informs the client accordingly.
While the copybook preview feature is powerful, it’s confined in scope to the immediate document. This is good. But, imagine what it would be like if the server had knowledge of all instances of the code deployed and running on computers connected to the internet?
Suppose for example, that your code was powering an API running on the internet and you had the ability to learn about the number of calls to a particular function backing the API or better yet, imagine that you had the ability to learn about function calls that encountered particular problems, such as a pattern of mistyped parameters or high latency calls. Then imagine if that global runtime behavior was reported right in the IDE? Such information would change the very nature of how a programmer approaches his or her work. More importantly, it would change the way an enterprise manages programmers.
What if a programmer could decide what to work on based on global runtime behavior reported into the IDE via an LSP extension? For example, going back to the imagined pattern of mistyped parameters, let’s say that when a developer hovers over the related function in IDE a warning icon appeared informing the developer of 1000 instances of mistyped called were made at the global level. The developer decides that such an error rate is unacceptable. Then, he/she examines the developer documentation and discovers that the parameter type definitions for the function are poorly described and that the document needs to be corrected. The developer then makes updating that developer documentation a critical path, part of the day’s work that needs to be delivered immediately.
Given the power that LSP brings to the programming experience, such a scenario is entirely possible. In fact, there are companies out there such asApollo Graph Manager that presently allow developers and third parties to register their production APIs in order to collect the sort of global information described above and have it reported in an IDE via LSP. As those of us who have been in the software business for a while have come to learn, once a feature such as global reporting catches on, it’s only a matter of time until it becomes an industry-wide practice. It’s only a matter of time before globally collected runtime metrics delivered to a programmers LSP become commonplace.
Again, the ramifications are significant. Once a programmer has the ability to make fundamental decisions about feature development and deployment schedules, what then becomes the role of the project manager and the product manager?
To take the issue one step further, how much power should a programmer be given? And in terms of general business practices, how much should the programmer know about a company’s products? These questions might seem fantastic, but keep in mind at one time companies guarded their source code as a trade secret. Today, more code than not is shared between companies as part of the open-source movement. As a result, the way code is made and a developers work has changed. The same possibility exists due to the growing trend to extend the capabilities of the LSP. The ramifications of LSP go well beyond simply reporting syntax errors and doing code completion. LSP will change the way the entire industry works, from mainframe programming to developers working in the cloud or on commodity hardware.
Putting it All Together
Given current trends, there’s little doubt that LSP will make even the most rudimentary code editor a powerhouse tool for software development. Standardizing IDE features into a common format allows new features for any development tool to be implemented faster and at a reduced cost. In no time at all we might even figure out a way to use LSP to make is so large bodies of code can be written automatically with just a few guiding keystrokes from the developer’s keyboard no matter the programming language. It’s a real possibility.
Not only will LSP making coding easier and faster, but it’s also going to change the role of the software developer. Once all the code running on the internet becomes LSP aware and accessible, developers will get runtime information about how their code is performing at runtime on a global level. The trend is already in play. As history has shown, the further down a technology goes into a given population, the more powerful that population becomes and the greater the ramifications when the power is exerted. There’s a very real possibility the information that LSP enabled IDEs provides will make the day-to-day programmer more prominent in the product decision-making processes that are presently the responsibilities of program and project managers. Once that shift takes place we’re likely to see another industry-wide transformation take place in the way software is made and delivered. History has shown that such transformations have long term benefit, but in the short term are disruptive and disturbing. We’ll do well to be prepared.
The Eclipse Foundation today announced the formation of a working group to create standards for cloud-based integrated development environments (IDEs) led by Broadcom, EclipseSource, Ericsson, IBM, Intel, Red Hat, SAP, Software AG and Typefox.
Mike Milinkovich, executive director of the Eclipse Foundation, said the Eclipse Cloud Development Tools Working Group (ECD WG) will work to replicate the success the Eclipse Foundation had in establishing standards for desktop IDEs based on an Eclipse standard among a new generation of cloud-based IDEs.
There already are several cloud-based IDEs available. However, because each of those IDEs is based on a codebase that differs from the codebase used to create the Eclipse desktop IDE, there is no interoperability across cloud-based IDEs. Milinkovich said that makes collaboration between developers using different cloud-based IDEs all but impossible as currently there is simply too much differentiation among providers of cloud-based IDEs.
As standards for cloud-based IDEs emerge, the Eclipse Foundation also expects best DevOps practices revolving around the integration of cloud-based IDEs and continuous integration/continuous development tools to advance as well, Milinkovich added.
Collectively, the ECD WG will drive adoption and standards for cloud-based developer tools, including language support, extensions and developer workspace definition. To achieve that goal, the ECG WD will pull together oversight of several existing open source projects being spearheaded by the Eclipse Foundation, including Eclipse Theia, an IDE framework for building TypeScript applications; Eclipse CodeWind, a project that adds support for containers to IDEs based on Eclipse; Eclipse Dirigible, a cloud-based development platform developed by SAP; Eclipse Sprotty, a diagramming framework; and Eclipse Orion, a browser-based integration platform.
Milinkovich said via those efforts, the ECD WG hopes to accelerate the adoption of cloud integrated development environment (IDE) and container-based workspace management, including the recently launched Che 7 project that makes available a native instance of Eclipse to build Java applications for Kubernetes environments.
It may be a while before the ECD WG can create a set of standards for cloud-based IDEs that will have a material impact on application development. The Eclipse Foundation still has to bring numerous application development powerhouses on board for this project, including Microsoft and, once the acquisition of Pivotal Software is completed, VMware. However, organizations that are shifting development to the cloud at the very least can take comfort in the fact that there is now some level of commitment to creating interoperability standards that one day could increase overall developer productivity as more applications are built in the cloud.
In the meantime, it’s apparent the Eclipse Foundation is committed to remaining relevant in the age of the cloud by both fostering IDE interoperability and acting as the steward for Jakarta Enterprise Edition, the open source follow-on to Java. Of course, not every IDE may be employed to build applications using Java. But at the very least, many of the familiar interfaces and constructs developers know today on their desktops will be moving forward into the cloud.