staging-devopsy.kinsta.cloud

Tag: development team

  • What Developers Must Know About Threat Modeling

    What Developers Must Know About Threat Modeling

    Threat modeling is a process that far few developers seem to pursue, but it is a process that helps you and your team to model all potential threats to your application. Essentially, threat modeling is your thinking through all of the potential threats against an application. Doing so is virtually as easy as putting together a list of threats in a structured manner that lets you assess any perceived risks, allowing you to formulate how you are going to respond to the threats.

    Ultimately, through threat modeling, you are optimizing network security by identifying objectives and vulnerabilities and creating measures to prevent the effects of threats to the system. Through this definition, anything described as a threat has the potential to be malicious to your organization and data systems; for example, incidental occurrences like the failure of a storage device–which can compromise the integrity of the entire enterprise.

    The reasons for threat monitoring may be pretty obvious, but any lack of progression here is quite common.

    Threats are one of the significant factors that led companies to spend $89.1 billion on enterprise security solutions. Gartner predicts threats will reach $96.3 billion by the end of 2018. According to Raytheon’s 2018 Study on Global Megatrends in Cybersecurity, 36% of senior IT and cyber professionals identified malicious or criminal insiders as a top cyber threat. Even with that knowledge, many organizations lack insider threat policies and policy enforcement tools, and they struggle to align security and IT responsibilities to tackle the threat effectively.

    Threat monitoring allows you the ability to determine which threats might exist for your application. Knowledge is power, and it’s always better to know beforehand what dangers might await you. Understanding these threats lets you decide whether you will accept the risk.

    For a threat to exist, there must be a combination of the following where the combined likelihood and impact are significant enough in which to do something. A framework for understanding threat modeling, includes: asking what you are working on, what might go wrong and what to do about it, as well as if a job is well done.

    According to the Open Web Application Security Project (OWASP), threat modeling is a process that lets you continually refine your processes based on what you have done so far: “Starting with all possible vulnerabilities is usually pointless, as most of them are not attackable by the threat agents, protected by a safeguard, or do not lead to a consequence. Therefore, it’s generally best to start with the factors that make a lot of difference.”

    Those steps include assessment scope, identity threat agents, existing countermeasures, identifying vulnerabilities, prioritizing identified risks and identifying countermeasures to reduce the threat.

    In the assessment scope, you’ve got to understand the price of what’s at stake. Identifying tangible assets, such as databases of information or sensitive files, is usually straightforward, and realizes the capabilities provided by the application.

    Next, identify possible threats or attacks. A crucial part of the threat model is characterizing different groups of people who might be able to attack your application–this may include insiders and outsiders, performing both inadvertent mistakes and malicious attacks. During this phase of development, consider existing countermeasures that allow you to determine what possible exploitable vulnerabilities exist that must be addressed. Through your search, look for weaknesses and connect any attacks you’ve identified to the negative consequences you’ve identified.

    When identified, risk management can be prioritized. For each threat, determine possible outcomes and the impact of those to understand how to mitigate these issues. Next, look for ways to reduce any future risks.

    While many of the threats might appear to be security related, many others have more natural explanations. Threat assessments can include power outages or even a flooded server room. These and other factors can lead to severe threats to your organization.

    When to Model the Threat?

    Threat modeling is often best at the beginning of a project, but it’s better to conduct a threat modeling project halfway through a project or even at the end than not at all. Threat modeling at the end of a project can have benefits, such as understanding the architecture and how data flows through it. However, threat monitoring at the end of a project can lead to finding more work may be required to fix poor design decisions not caught at the beginning.

    For those entering into threat modeling, conduct your first session on an existing project, letting you experiment with a threat modeling session so that you can familiarize yourself with everything required. This way, you’ll be better prepared to analyze all assets and flow of data to assess the risks. Once the process is pounded out, you’ll have a better chance of succeeding in your threat modeling of a new project. You’ll also be able to better prepare for threats that may exist in the design phase of the project. From here, over time, you’ll begin to take into account threats during the initial design of a new feature.

    Approaching the Threat Model

    Each method of approach depends on the organization and its goals. Large organizations usually have previously established processes for threat modeling. Smaller organizations new to threat modeling can begin by hiring in experience to initiate the task or seeking outside support from experts or those who can advise the program.

    Next, you need to understand who the users are within the system–regular users, administrators, outsourced contractors and perhaps hackers with apps or other tools looking for vulnerabilities within your network. There are also other actors, including disgruntled former employees.

    Defining the user actors is important since missing a group can mean you might be missing an entire category of threats. Also, consider the various types of hackers or groups of hackers in your models.

    Data and Information Flows

    At this point in the threat modeling process, track each of the various data flows running through the system. For each of these, you need to know where the data goes. What does it interact with or encounter along the way? Then look to identify where data can leak or where it might be exposed. More data components create more opportunities for a hacker to gain access to the information.

    Individual data means the information flows differently based on the architecture and the specific situation. One example might include a request from a user’s browser where the cookie is sent through and interacts multiple components as it does.

    For each data flow, search opportunities for any actors to gain hold of valuable information. If anything makes you frown, step back through the process and move that threat through until you can mitigate the frowning moment.

    Threat List Complete

    When the threat list is observed and complete, prioritize the list from very unlikely to improbable to not likely. No matter the probability of the risk, work through the risk as a real possibility. Do so to consider the real impact of the threat and what it may mean to the organization. Is the company going to crash and burn completely? Will every bit of data get pulled into a ransom situation? Place a number or priority of the impact, such as low, medium and high.

    When you have your risk list, manage the risks by:

    1. Accepting: If the risk is quite low, accept the potential impact. It’s also possible you’ll find the potential negative user impact is more important. You can also reassess later.
    2. Transferring: Perhaps another team is responsible for managing a particular risk. If that team can pick up the defense easily, outsource it.
    3. Avoiding: Don’t do this. If you must, consider complete structural changes to your data flows to avoid altogether avoiding risk. This might mean architectural changes are required or killing a particular feature altogether so as not to avoid danger.
    4. Reducing: Take actions to lower the risk by lowering the likelihood or impact of the risk. You may need to take multiple steps to reduce the threat. No matter the action, effort is required to reduce the risk.

    After threat modeling is complete, don’t stop. Keep working because threats and actors change. List all potential issues or risks that must be addressed and work them into your plan.

    Finally, remember you should be careful with whom you share your threat modeling plan. You never know where that information might end up otherwise, or who might be a threat to your organization and its data.

    One last note: The final result and usefulness of threat modeling depend almost entirely on the parameters established. Defining the values for the probability of a threat and impact may be subjective, but let your experience, input from colleagues and experts, and your gut guide you.

    — Jeroen Boks

  • From Automated Cloud Deployment to Progressive Delivery

    From Automated Cloud Deployment to Progressive Delivery

    Your team is on its agile journey and you can more or less track a deployment from a story, through git commits, to an automated builder, to an artifact repository, into a container and then onto the cloud. Or, in fact, any other of the many valid variants that lead you to believe your deployments are largely automated. What you can point to is a business request comes in and the result is a service or a new app. Your purview seems to stop before your users can even respond though. Is that good enough?

    Let’s go back to the early idea of a release. It was a collection of all the features and fixes the loudest stakeholders had persuaded the product owner to put at the top of the story list. A date was promised, but QA was late, and the persistent stakeholders squeezed a little more juice out of a story. Then it was stuffed into a ball and delivered to your servers overnight, lest anyone notice. The next day, the developers had to deal with the fallout as the support queues grew with confused users.

    While developers were getting the hang of the agile notion of automated deployment, the release lifecycle did pretty much end with a deployment going live. This gave certainty that what had been built and placed in the artifact repository was the same thing the testers had seen in the staging environment. And it also had the changes Jira said it had.

    This type of release was very much like a birthday present. Your uncle talked to your dad briefly, without your knowledge, and agreed you really would benefit from a new pair of socks. Then these were delivered on the day with the receipt in the bag, just in case you needed to take them back.

    The first evolution came with the concept that a release wasn’t exactly synonymous with a deployment. Finally, the end-user’s perspective nudged its way into corporate release strategy–updates could be deployed without necessarily impinging directly on every user. What if there were two identical release environments with clever network switching, meaning only one was truly live? This type of flip-flop arrangement (sometimes referred to as blue/green deployment) allowed changes to be tested internally in a realistic environment before going live.

    It was the advertising industry that prompted the idea of A/B testing. Instead of guessing one variation of a campaign might be superior to another, the surprisingly scientific method of using a controlled sample and a variation was used on audiences to see which they preferred. In the digital age, it is possible to deploy two variations of the same release to different server sets. This might mean each user session could be using either release. It would then be necessary to associate the resulting click-throughs, or whatever success measurement is chosen, to the specific release flavor. The worth of this kind of experimentation was only as good as the question you asked, but it was at least observing real user interaction.

    Moving on, DevOps environments began to use more feature flags or toggles. These were code paths that could deliver feature changes in live servers, controlled through configuration changes. This reduced the need to re-deploy to make certain changes. Release minded teams were coalescing around configuration, as opposed to just code. Whereas code once compiled and built was locked away in unreadable artifact files–a larger stakeholder community could administer configuration files, usually encoded in English. This helped to keep the feature changes closer to the stakeholders and less likely to slip back into silos.

    As a more defined understanding of delivery arrived, so did the understanding of the user community. It used to be the case that a user was a one-line entry in a database. Even firms whose business is not mining their user’s data understand there are different sets of users for products and services.

    Traditionally, there has always been internal users or beta testers–those within the firm whose job is to check the bits they understand, or are responsible for, are working as expected. Then outside the firewall, there are the tech savvy community who actively want the latest updates. These guys report bugs and often compare your product with the competition, perhaps using both. This audience will not lose their marbles if you deliver a bug to them. More to the point, they will notice rapidly–perhaps telling everyone on social media–about your failings.

    The disengaged users who may only have the free version or tier of your product come next. These people are more likely to work on impressions and are best not disappointed. They are also unlikely to be keen on updates. Finally, the large core of solid users who pay their way, just want things to work and need to understand a path to continuity will already appreciate the software and will understand the roadmap by osmosis. These will probably include your biggest users, too. Of course, if you have no way of distinguishing your users all the above is moot.

    With services sitting on a public cloud and CDN like edge services such as CloudFront, the ability to release by territory is much more straightforward. This is also increasingly necessary for legal reasons–EU regulations, for example. This allows for tactical releases to communities in different time zones and sizes. The practice of deploying changed services to a small group for risk mitigation, or “canarying,” is another method that takes advantage of smart routing. This requires the ability to quickly observe and rollback releases though.

    Today, most software has some degree of dark launch that restricts the visibility of a release to the appropriate community, until all is well. In this sense, a progressive delivery (defined as “continuous delivery with fine-grained control over the blast radius” by James Governor) can be seen as the virus-like spreading of your software or services from your development teams’ laptops to the last user.

    Observing service use is simply part of the core concern of virtually every company in the tech space. From this perspective, progressive delivery changes the way to see your development team as mere builders to user fulfillment specialists, which of course, they always were.

    The future will undoubtedly involve increasingly customer-driven releases, which goes hand in hand with the increasing use of data science to study user communities and their exhaust. The important take away is to make sure your team is reaching further to the right-hand side of the product journey, and keep the user’s reaction to what you make as a larger part of your sensory input. Think more about constant small changes and less about ceremonial releases.

    — David Eastman

  • Maintaining Exceptional Quality Despite Shrinking Delivery Deadlines

    Maintaining Exceptional Quality Despite Shrinking Delivery Deadlines

    My first article in this series focused on the need for continuous improvement across three key dimensions of software development–velocity, quality and efficiency. While these are all equally important, in this piece we’re going to focus more specifically on quality.

    The past few months—what you might call the summer of outages—have been troublesome for many of the internet’s biggest players. Google, Apple and Cloudflare, as well as Facebook, Twitter and LinkedIn, have all experienced bouts of unplanned downtime, sometimes even multiple bouts in quick succession. Is this all just a matter of coincidence and bad luck? Perhaps, but peeling back the layers reveals an increasingly painful reality of modern DevOps life.

    While the root causes of these outages were varied, several were blamed on software glitches. This suggests software rollouts may be happening before they are comprehensively battle-tested and ready for production-level systems. This is never a good idea, especially when you’re dealing with high-profile services supporting millions of users worldwide. In these cases, software-related stumbles are that much more damaging.

    When software is rolled out prematurely, a cardinal rule of development is broken in that we, the end-users, become the unwilling guinea pig for poor-quality applications. No matter the pace of rollouts, an unwavering focus on quality needs to be maintained, and automation, integration and measurement can be the keys to ensuring this.

    Automation

    Automation refers to a pre-programmed process that executes tasks with minimal human intervention. The more you automate, the higher your quality should be. Avoiding manual intervention decreases the likelihood of less optimal techniques or human mistakes, both of which can lead to poorer application quality and customer satisfaction. Essentially, automation enables the best techniques and practices to be quickly and easily replicated.

    Consider the example of automated unit and functional testing. If you can automate the execution of the best testing scripts incrementally across code units and functions, you can test more accurately, not to mention more quickly. The benefits will increase exponentially with the additional layers of testing you can automate. We recently worked with a customer who realized an overall ROI of 467% from their investment in automated unit testing.

    Integration

    Modern applications are increasingly componentized, spanning multiple platforms. Consider an e-commerce application, where the front-end may reside on a system of engagement (such as a web server) while the back-end transaction ultimately occurs on a system of record, such as a mainframe.

    High-quality, end-to-end digital services ultimately depend on all critical components being universally and automatically included and prioritized in DevOps pipelines. Similar to automation, this is because any time manual effort is introduced into the integration process, you increase the chances for error and consequently, the risk of a particular code segment lagging behind others.

    The end result? More defects making it into production in the sprint to the finish line. It’s like the famous I Love Lucy chocolate assembly line episode, which offers a humorous visual interpretation of the chaos that erupts once human intervention is introduced into a process.  Ideally, seamless toolchain integrations, across multiple application-leveraged platforms, can be utilized to maximize quality outputs leveraging DevOps technologies you’re already using, such as JIRA, Jenkins and SonarQube.

    Measurement

    Improving quality depends on development teams constantly improving their methods. This can be achieved through ongoing measurement against key performance indicators (KPIs).

    The challenge is establishing relevant KPIs, and we have seen many development teams fail at this. The most effective KPIs are those that translate to real business impact. For example, “number of bugs released into production” is a commonly used metric, but it does nothing to convey the bottom-line cost of those bugs. A more effective metric would be the cost of resolving defects, with success being declared when the organization reduces the amount of money spent on fixing bugs.

    The cost of fixing a bug is much less when it takes place earlier in the software development lifecycle (for example, in the coding and unit testing phases) than when the software is near or already in a production state. When a development team can consistently show it is reducing costs through earlier identification of bugs, this puts them in a stronger position to justify investments in better testing and analysis tools.

    Conclusion

    We’ve long known about the challenges DevOps teams face to balance velocity and efficiency with a high degree of software quality. It’s like asking a circus performer to sprint across a tight rope. It may seem unfair, but the need to release high quality software faster and faster, while staying within budget, is not going to abate any time soon.

    The recent spate of outages has taught us that even the biggest and the strongest in the industry are not immune to immense speed pressures, and the quality stumbles that often result. Automation, integration and measurement can help development teams in their ongoing quest to meet the seemingly conflicting requirements of improving quality, velocity and efficiency—at the same time.

    — Rick Slade

  • Top 3 Challenges of DevOps Adoption

    Top 3 Challenges of DevOps Adoption

    Customer expectations continue to relentlessly rise as new technological opportunities emerge. Today, organizations demand products and application features are delivered and implemented wherever and whenever needed. Everywhere you look, execution is at risk of falling behind imaginations and business imperatives. That is, of course, unless DevOps is starting to wield a strategic and philosophic influence.

    DevOps has been one of the hottest enterprise trends for some time now, and a crucial concept in helping companies satisfy customer demands by bringing teams together to fuel better collaboration and innovation.

    According to Deloitte, organizations adopting DevOps see an 18% to 21% reduction in time to market.

    By removing the waste that impedes software development, operations are streamlined, and businesses can react faster to market demands. At the same time, automation enables companies to align business objectives with processes to immediately recover from IT failures. Deloitte also reports the speedy, DevOps-influenced, customer-friendly release of new products and services led to a 20% revenue increase.

    While that is all good, DevOps is still a new topic to many. Large scale adoption is generally slow. It is important to note DevOps is not simply the implementation of new tools. It is about rebooting deep-rooted cultural habits, transforming old processes and ensuring all changes are made for the right reasons. Don’t embrace DevOps for the sake of it. Only proceed if you have clarity and purpose.

    Breaking of the Deep-Rooted, Cultural Habits

    DevOps is not a team. It is a culture of collaboration that yields a set of best practices and operational cultures. Sometimes companies wanting to adopt a DevOps approach are met with resistance from employees uncomfortable with change. Leadership needs to come from the top. Executives must support employees in dismantling the status quo and breaking down silos hindering successful implementation. Motivation rises when results are positive, concrete and observable. There is no magic formula but, if you get the right leadership and culture in place, chances are DevOps will land with resonance. Gartner reports 88% of businesses believe team culture is one of the top impacts on an organization’s ability to scale DevOps.

    Changing Well Defined Processes to More Efficient Ones

    The best way to identify inefficiencies is to map processes and recognize what is and isn’t working. Start by looking for waste–areas where resources are being exhausted without delivering real value. A silo-influenced lack of communication can significantly slow down businesses and there is a risk each department will think DevOps is not their responsibility. Some may be resistant to implementing change, which will cause unnecessary lags within the business. It is vital to understand it is not just the responsibility of software developers to ensure the processes work for them. DevOps is a process that brings development teams and other IT stakeholders together to achieve one common goal: delivering faster, higher quality work to market.

    Beware of DevOps for DevOps Sake

    With DevOps, companies can streamline and automate every task. That doesn’t mean it’s right for every element of every business. Often, companies will pick a couple of departments to trial a DevOps approach with–usually the development team. The temptation is then to continue scaling across the organization as soon as positive results are apparent. The issue here is that different departments all have different needs and challenges and, as we’ve already said, effective DevOps rollout requires an overhaul of processes and ways of working. Organizations need to be careful and think about specific business needs, instead of just being engaged in a rudderless trend chase. Know your objectives before engaging in sweeping overhauls. Keep learning and, above all, stay tuned to customer needs.

    — Rodrigo Albuquerque

  • Transforming the Security Team Into a DevOps Partner

    Transforming the Security Team Into a DevOps Partner

    Securing DevOps environments is an increasingly important concern for chief information security officers (CISOs) and security teams. While developers often recognize security is important, it is not their top priority. More typically, the DevOps team prioritizes delivering new capabilities and features to the business and customers, often as part of a larger digital transformation initiative. And, developers often view security as something that will slow down deployments.

    Security teams usually have limited DevOps knowledge or expertise. Too often the result is that DevOps adoption begins and even takes hold inside an organization before the security team gets involved. Consequently, security vulnerabilities are not always adequately addressed in DevOps environments and can drive unnecessary risk.

    Integrating Security in DevOps

    The priority is for the security team to take the lead in integrating security into the DevOps processes before poor practices become entrenched. But as both teams are often siloed and don’t tend to work collaboratively, how can security teams better engage, energize and collaborate with their DevOps counterparts to strike the right balance? In a nutshell, how can organizations bring their DevOps and security teams into alignment and establish collaboration for stronger overall security?

    There are a few crucial steps to take to achieve true integration of security and DevOps.

    1. Establish the Requisite Skills to Get in the Driver’s Seat. Effective collaboration requires effective communication. While developers write the actual code, it’s important for security teams to gain knowledge about programming languages along with how applications are built, tested and deployed automatically. This will help them have more meaningful discussions and credible conversations. Security professionals can start by learning some of the fundamentals: PowerShell, Python and Rust, as well as how DevOps tools use REST calls and containerization technologies–particularly Docker and Kubernetes.
    2. Make it Easy for Developers to Do the Right Thing. You can’t be the manual cog in their completely automated process. Make it easy for developers to do the right thing by training them in secure coding practices and implementing a self-service model for security capabilities. For example, you could provide security policy as code that can be integrated into the developers’ automated processes.
    3. Establish Effective Ways to Collaborate. Set up formal systems to ensure DevOps practitioners understand security risks and implement good security practices across the organization. Consider how best to deploy security resources into existing or new organizational models and structures. This includes establishing centers of excellence, community leaders, security champions and embedding security team members inside development teams.
    4. Get Developers to Think Like Attackers. Educate DevOps teams on specific attacker tactics, show how sample code modules could expose secrets and provide examples as user stories. For example, “As an attacker, I would scan the organization’s code repositories looking for secrets.” Take the team through a penetration testing exercise or engage a red team to demonstrate how an attacker would compromise a CI/CD pipeline.
    5. Adopt Agile and DevOps Methods. Security should begin utilizing agile and DevOps methods within their own teams, not only to gain a deeper understanding of DevOps methodologies but also to achieve greater efficiency by automating tasks or delivering capabilities in smaller increments more frequently.

    The bottom line is, it is crucial to understand how other enterprises approach secrets management challenges across DevOps and cloud environments. This can help encourage collaboration and help fast-track the security team’s own efforts. Ultimately, this will ensure agility is not just implemented for the sake of innovation, but companies reflect on their processes and prioritize security to make the most of their transformation.

    — Josh Kirkwood

  • A Golden Age for Developers

    A Golden Age for Developers

    There’s probably never been a time since the dawn of programming when there has been more opportunity for developers—individuals and teams—to imagine, create and be successful. Developers are able to do a lot more (and do things a lot faster) with more tools, a better development ecosystem and a tighter connection with the rest of the enterprise than was possible even a few years ago.

    To me, we are in a golden age for developers. And it comes just as the need for developers has never been greater, in every enterprise.

    Three things have come together to create this moment: Culture, code and the cloud.

    Changing Culture

    First, developers today operate in a culture that is far more collaborative than in years past, both internally and externally. A big part of that is the growing adoption of the DevOps methodology, which breaks down silos between business units, IT and development through partnerships between developers and the enterprise. It’s changing how we work and think.

    DevOps leads to a process of continuous integration of new ideas and features that meet business needs, and continuous deployment life cycles that get those features operational rapidly. Rather than taking weeks or months to get a new version of an application up and running, it’s possible to make constant incremental changes and have things running in days or even hours. The enterprise is able to be exceptionally agile responding to business changes and challenges—and, if there’s a problem or failure, the mean time to recovery is just as fast.

    Democratized Code, Empowering Cloud

    A major factor in enabling this approach is the widespread use of open source code. Open source has democratized access to powerful tools and platforms that can greatly accelerate work for any developer. Open source projects are mature, stable and growing, which provides equal access for citizen and enterprise developers alike. The size of your IT operation no longer matters: Developers everywhere can call up on their laptops the same tools that once were available only to web-scale unicorns. It removes limits to a developer’s imagination and the ability to create something new.

    Finally, the ever-expanding realm of cloud-native computing is putting incredible resources and computing ability within reach of every developer. Tools such as Kubernetes, which let you create and orchestrate applications in containers that can then be deployed and run in the cloud; serverless technologies; and other new ways of using distributed computing power remove barriers. Where a developer once might have been reluctant to create apps that required high levels of computing power, that’s not the case now with the cloud. What once may have taken access to a Cray supercomputer is now at a developer’s fingertips.

    Retraining, Not Replacement

    If there’s any challenge in this golden age, it’s helping the enterprise to make the transition—in particular, facilitating the leap that existing development teams must take in using these new approaches and new tools. In fact, the top issue for cloud developers is to make this cultural change.

    That takes both strong top-down leadership and an openness by veteran developers to new ways of working for developers, especially those who’ve been around for a while.

    Too often, we see companies trying to make the switch by offering early retirement to their veteran developers and trying to recruit a new team that’s already used to DevOps, open source and the cloud. That’s the wrong way to do it. Instead, even as they bring in newer developers, companies should look to retrain their existing team.

    That takes more than just telling people, “Here are the new ways we’re going to operate,” and then throwing them into it. There are a lot of tools and gateways available that can help people move gradually from the tools and languages they have been using. For example, in the last 12 months alone, numerous open source Kubernetes operators, service brokers and frameworks have been introduced that allow enterprise teams to leverage cloud-native technology from a context they already know, including WebLogic, database and Java. Look for combinations of people and projects that are likely to result in some early wins, and then have those people help train the rest of the organization.

    Make sure to take advantage, too, of the external collaborative environment that exists. Have veterans go to conferences or meet-ups and learn from the community—which, thanks to open source, is huge and growing.

    This will do more than give you a development team that’s built for today’s market, which demands that you deploy faster and react to failures in a more graceful way. It also will help you recruit and retain. Developers are twice as likely to recommend their workplace to if they are using DevOps principles—they are happier, easier to recruit and easier to retain.

    In this golden age for developers, the potentials are limitless—and every enterprise can participate. Making that cultural change, taking advantage of open source code and exploiting the power in the cloud—those are the keys to building the team and the applications that will spell success.

    — Bob Quillin

  • Textbook DevOps vs. Real-World DevOps

    Textbook DevOps vs. Real-World DevOps

    DevOps is one of those things—like Marxism or baking macarons—that is easy to understand in theory, but much messier to implement in practice. Real-world DevOps rarely aligns with textbook DevOps or the DevOps practices that organizations should follow in theory.

    That is true for a number of reasons. Let’s walk through some of the biggest below.

    Your Company Is Not Organized in the Way DevOps Theory Assumes

    According to DevOps theory, every company has an IT department and a development department. Doing DevOps boils down to making those departments work well together.

    The reality, of course, is that not all companies organize their staff or technical operations in this way. Some don’t silo developers and IT engineers by default. Others (especially smaller companies) don’t have developers or IT staff on hand at all; they outsource that work.

    These companies can still benefit from DevOps principles such as collaboration and automation. But they should avoid fixating on DevOps teachings regarding the organization of technical work.

    Real DevOps Skills are Hard to Find (And Expensive to Obtain)

    “DevOps engineer” is now a trending job title. But few technical training programs have caught up with the demand for DevOps.

    Most universities still silo majors and courses in software development away from those that focus on IT. Other types of technical training programs function similarly.

    What this means is that it is more difficult to hire a DevOps engineer, or other technical staff with a broad set of development and IT skills, than it may first appear. DevOps engineers exist, but they are usually experienced staff who learned their real-world DevOps skills on the job. They’re more expensive to hire and difficult to poach from their current gigs.

    Full Automation Is a Pipe Dream

    The DevOps mantra tells us that we should automate everything, from software testing and builds to deployment, production-environment monitoring and incident response.

    Automation is great in theory. But the reality is that you can never automate everything. Human engineers will still need to respond to problems occasionally, or to unanticipated issues that arise during testing, deployment and production management.

    An important point to emphasize here is that even if you successfully automate virtually all of your processes, the mere fact that human engineers must be available to respond when something does go wrong undercuts the idea that you can rely on automation alone. Humans still have to watch the automated processes, even if they rarely need to intervene. The simple act of watching and being available to respond when necessary is a significant burden.

    Agility Conflicts With Simplicity and Reproducibility

    In some respects, DevOps is a contradiction. On the one hand, it prioritizes agility and scalability. On the other, it emphasizes simplicity and reproducibility.

    What this means in practice is that DevOps teams adopt microservices and containers in the interest of maximizing agility. But microservices and containers are much more complex to manage than monoliths and virtual machines.

    I suspect the reality is that most DevOps organizations end up erring toward more complex technologies and architectures. They ultimately find greater value in them than they do in keeping things simple.

    Still, the idea that you can be agile and be simple at the same time doesn’t apply well in the real world. (And sure, automation, infrastructure as code and the like can help in simplifying the management of complex agile technologies, but they only go so far.)

    Most Organizations Still Aren’t Doing DevOps

    If you spend your time surrounded by other people who have adopted DevOps and implemented it successfully, it’s easy to assume that the rest of the world is like you.

    The reality, however, is that we’re now a decade into the age of DevOps, and yet a majority of organizations have made only halting steps toward implementing basic DevOps processes. If DevOps were half as easy to implement as it appears to be when you read the latest book about how to do DevOps, many more companies would be doing CI/CD by now.

    Conclusion

    DevOps delivers real benefits, and it’s important to continue preaching the DevOps gospel to organizations that can benefit from DevOps philosophy and practices.

    However, it’s important to balance the conversation about DevOps with the reality: Real-world DevOps is much messier than it often seems in theory. Not many organizations have the configuration or resources to do textbook DevOps.

    — Chris Tozzi

  • DevOps Security Champion: Who, What and Why?

    DevOps Security Champion: Who, What and Why?

    In general, DevOps is a process and culture of organizations to get applications out the door faster and with higher quality. To do so, security champions are essential. In DevOps, security champions work as a backup mechanism in various projects and take multiple leadership roles.

    Security champions make effective decisions and take projects forward while strengthening the best security practices. How do you enable security champions in DevOps? In this article, we will describe four practical ways of enabling them in DevOps. But first, we will discuss their role in more detail.

    Security Champions in DevOps

    Let’s suppose you are accountable for creating a mature security process across an organization. To fulfill this project, there are various teams working on different product and services, using multiple technologies and an active approach. However, some team approach security ad hoc security activities while others use existing security technologies and processes.

    After a few weeks of reading documents, learning about calendar releases and suggesting specific knowledge and best practices, you realize that your best efforts around security haven’t addressed myriad issues. You need help, but the central security team isn’t budgeted to help your team. This is where security champions come in.

    Security champions play a vital role in DevOps. They are dynamic members of a team who help in making decisions about the engagement of the security team.

    They act as a fundamental element of the security assurance process for the product and service, and are the main point of interaction within the entire team.

    By taking different leadership roles and decisions inside a smaller team, security champions form a shield of sorts for the team leaders. They give them the freedom needed to drive the project further while supporting and emphasizing the best security practices in the project.

    Enabling Security Champions in DevOps

    Here are four ways to build an efficient team of security champions.

    1. Identifying Teams

    The first and most important thing in building a competent team of security champions is to identify the team you are going to work with. In this phase, distribution of tasks occurs. It is important to note this step should be done before starting the project; otherwise, miscommunication and other problems could occur.

    It is essential to keep a record of the team so the other development departments also know who is reponsible for which of the various tasks.

    To implement this step, you need to talk with the decision-makers and technical managers, such as product owners and company heads to find out information such as the number of people working on different projects and how they can fit into the development teams already working on the project, what programming languages and various frameworks are being used and how they are being implemented.

    2. Role Distinction of Team

    It’s also important to define the role of each team member. Every team must have its security champion assigned to them and the security champions must know what they are supposed to do.

    The security champions should have well-defined goals and purpose, and the rest of the team should communicate with them to ensure best security practices are implemented. The security framework the project follows was selected during its planning phase. As such, the entire team must work with the security champions to ensure the security of the application.

    The security champion should also conduct security reviews regularly, especially at milestones during the project.

    It is important to note that protecting security best practices is not just the security champion. Rather, it is a collective effort of the entire project team.

    3. Select Champions

    Once you have identified the team and defined the roles, the next step is selecting the champions. To accomplish this step, you need approval from managers at all levels, from to-level to product owners and direct team managers. They can help you identify potential champions, choose eligible candidates and conduct interviews to judge their capabilities.

    Because this is the selection stage, you need to describe the role clearly to the selected candidate. You also may explain the strategy you plan to adopt as well as the advantages of becoming a security champion. These include:

    • The chance to attend various security conferences.
    • Being a part of the security meta-team.
    • Increased value in the marketplace.
    • Better product quality.
    • Personal development and the ability to look at things differently.

    4. Stay Connected and Communicate Clearly

    The security champions are required to communicate consistently with their team members and project leaders. They can use any of the various communication channels available to collaborate effectively:

    • Messaging apps
    • Email
    • VoIP apps
    • Mobile phone apps

    Security champions also need to combine as much of the technical data as possible to provide the team safe and secure access, thereby fostering a collective approach toward secure product development instead of keeping everyone in remote smaller groups who don’t exchange knowledge.

    Conclusion

    Security champions play a significant role in DevOps because they introduce a sense of responsibility of security within their department. Their participation starts small and increases gradually. Their presence benefits not only the company but also the application or service itself.

    There are different ways by which you can enable security champions in DevOps to add value to your project. By doing so, you are helping to create a culture of security awareness, which leads to secure applications and improved quality security features.

    — Zehra Ali

  • Understanding the Data Scientist’s Role in Cross-Functional Teams

    Understanding the Data Scientist’s Role in Cross-Functional Teams

    In our previous article, “Data Science Industry Perspectives in the Cloud,” we discussed that evolution is key if you plan to grow your business. You can start with ready-made solutions, then, in time, you can switch to in-house solutions created with the help of a group of data scientists.

    This time around, we’ll talk about the cases and solutions when machine learning as a service (MLasaS) doesn’t work. Your company’s first move is not to go out and hire a data scientist to solve your urgent business needs; rather, your company should invest in a custom-made solution. Only when you have a workable solution can you dive in deeper and create a proper team that can create an in-house data science solution.

    Custom Made End-to-End Solution as a Start

    Data Science in the classical sense involves data, goals and models to solve a pressing issue. The best way to jump start a data science process is by putting together various ready-made services into a single workable product to show your customers clear-cut results in short order. This can be done without any complex or global research, and you can comfortably formulate specifications taking into account all the feedback from your customers to create a more high-level data science offering later.

    One of the biggest issues for any data science project is in formulating the specifications for it. The usual request is, “Create something for my company using data science. Analyze data for me.” This type of a job has lots of trial and error. Having a custom end-to-end solution from the start lets you add on extra services as needed, and enables greater insights and predictions into a specific business workflow.

    I believe companies should build their data science system not from the data science research but from the point of view of an end-to-end solution. And then, bit by bit, they can adjust their services in the process.

    Moving on to Data Science Research

    Companies start data science research by employing a data scientist who matches their needs. You can use the standard data scientist qualifications: This employee should have a firm footing in the application environment, mathematics and programming. Also, their business skills are key to success—I believe an understanding of the topical area is key here because we are solving business issues. And a data scientist who solves problems that are more academic will be more focused on winning a Kaggle competition than on addressing the business needs.

    It is important this person understands the product development cycle. This way, the models and analyses data will be built in such a way that they could be deployed in production. For example, if a person is using R (the language and framework for data scientists), we should note that it is more research-oriented and it is not production-ready. Correspondingly, the results of such research cannot be deployed in any end-to-end solution. Therefore, the data scientist should work with a programmer.

    Although in the above diagram we see that the Data Scientist should have hacking skills, in reality, this is not the case. In their vast majority, Data Scientists are not able to write quality code. And if you need not just the research but a solution then process-wise you need to have a Data Scientist working together with a Data Engineer.

    CAP Theory as an Analogy

    It is impossible to have three database properties at once: consistency, availability and partition-tolerance. This basic rule is known to all developers. This is true for a data scientist as well: People envision a data scientist overlapping all three  properties, but in real life, you cannot find such an ideal candidate. Usually, people tend to lean one or another way in their work and keeping a balance is not always a priority.

    In principle, data scientist should have business insight, understand the math behind it and work in tandem with a skilled developer. Of course, a proper data scientist should be able to write any semblance of code, but a data scientist should work with a data engineer to write and realize code, being responsible for both the quality and sustainability of this solution.

    The classic tragedy of a company that decides to initiate any data science research is when a data scientist says, “I’ve got 40,000 lines of Python code on my PC. Can you make it work in production?” And, of course, this is virtually impossible to do. When that happens, all the research is simply wasted.

    Cross-Functional Teams

    Any team that deals with data science has to be cross-functional. In other words, it has to cover a whole stack of the solutions it writes. In a normal infrastructure there should be a DevOps engineer, data scientist, data engineer and a product developer writing the web app and/or mobile app. This single team is responsible for the result. The team members should work together and solve tasks that are interconnected in their interactions.

    All of this means that the whole team is responsible for the business result. This is also true for the transitionary research done by a data scientist that is impossible to use in production on its own.

    Old-School vs. Vertical Teams

    To dig in deeper, let’s take a classic old-school layered company organization structure when you have a department of data scientists, operations, UI developers, a big data department, QA engineers and so on. In this case, we have every project penetrating most of these teams. And the classic problem is that tickets and tasks are being thrown around by one team to another, and the real business goals are being watered down along the way and not solved in the end. So, instead of this horizontal division, we have divided the teams vertically. This allows us to create teams that see a clear-cut goal they need to achieve. And at the same time, they can improve their cross-skills and boost their responsibility levels.

    As a result, such teams began to deliver, Scrum and Agile properly. It is not directly related to data science, but companies need to realize the difference between a data scientist, which works in academia, and a production data scientist. Companies should aim at employing the latter one within their teams, and not let a data scientist work alone remotely.

    About the Author / Stepan Pushkarev

    Stepan Pushkarev is the Head of DevOps Practice at Squadex.com and CTO of Hydrosphere.io. Co-founded and managed engineering teams for eCommerce, IoT and Ad-tech companies. He has been responsible for the full products stack: math models, infrastructure & operations, enterprise applications as well as hiring, establishing engineering culture and delivery process. Stepan combines strong technology, management skills and entrepreneurial spirit. Connect with him on LinkedIn.