staging-devopsy.kinsta.cloud

Tag: continuous documentation

  • Documentation in Agile: Challenges and Trends in 2023

    Documentation in Agile: Challenges and Trends in 2023

    Agile documentation refers to the creation and maintenance of documentation in an agile software development process. It emphasizes “just enough” documentation that is necessary for the current iteration, preferring documentation that responds to specific needs over extensive upfront documentation. The goal is to provide clear and concise information to support the development team while keeping the documentation flexible and updating it as needed throughout the development process.

    In this article I’ll present the state of Agile documentation and how it is expected to evolve in 2023 and beyond.

    What Does an Agile Approach to Documentation Look Like?

    An agile approach to documentation looks different from traditional, waterfall-style documentation practices in several ways:

    • Emphasis on just-in-time documentation: Agile prioritizes documentation that is created and updated as needed, rather than extensive upfront documentation.
    • Collaborative and iterative: Agile documentation is created and maintained through collaboration between the development team and stakeholders, with a focus on continuous improvement.
    • Flexible and adaptive: Agile documentation is designed to be flexible and adaptable, allowing for changes and updates as the project progresses.
    • Minimalistic: Agile documentation focuses on providing just enough information to support the development process, avoiding unnecessary detail.
    • User-focused: Agile documentation is centered around the needs and perspectives of the users and stakeholders, rather than the development team.

    In practice, an agile approach to documentation might involve user stories, wikis, and documentation that is integrated into the development process rather than treated as a separate task.

    7 Agile Documentation Challenges

    Teams attempting to write and update agile documentation can face several challenges, including:

    1. Balancing speed and detail: Agile documentation is meant to be lightweight, but it also needs to be detailed enough to support the development process. Striking the right balance between speed and detail can be challenging, especially as the project evolves and requirements change.
    2. Keeping documentation up to date: Agile projects are highly iterative, which can make it difficult to keep documentation up to date. Documentation that is not updated regularly can become outdated or irrelevant, causing confusion and slowing down the development process.
    3. Maintaining consistency: In an Agile development process, multiple team members may contribute to documentation, which can make it challenging to maintain consistency in style and format. This can result in documentation that is difficult to navigate and understand, which can slow down the development process.
    4. Managing collaboration: Collaboration is a key part of the Agile development process, and it is important for team members to be able to share information and feedback on documentation. However, managing collaboration effectively can be challenging, especially when team members are located in different locations or time zones.
    5. Prioritizing documentation: In an Agile development process, there is often a lot of pressure to prioritize development work over documentation. This can make it difficult to allocate sufficient time and resources to documentation, which can result in documentation that is incomplete or of poor quality.
    6. Dealing with changing requirements: Agile projects are highly iterative, and requirements can change rapidly. This can make it challenging to keep documentation up-to-date and relevant, as documentation that was created to support one set of requirements may no longer be applicable as the project evolves.
    7. Ensuring documentation is accessible: Agile projects often involve multiple team members and stakeholders, and it is important for documentation to be accessible and usable by everyone. Ensuring that documentation is stored in a centralized, easily accessible location can be challenging, especially in large or complex projects.

    Agile Documentation Trends in 2023 and Beyond

    Here are some of the key trends in Agile documentation that can help your organization improve your process and overcome the challenges above.

    Documentation Automation

    Documentation automation refers to the use of technology to automate the creation, maintenance and updating of documentation. This can involve the use of tools that generate documentation based on code, data or other sources, as well as tools that streamline the collaborative creation and review of documentation.

    In an agile development context, documentation automation can help to increase the efficiency and accuracy of documentation while freeing up time for the development team to focus on other tasks. It can also facilitate the creation of consistent, up-to-date documentation that is always in sync with the latest changes to the code or other project assets.

    Some examples of documentation automation tools include:

    • Automatic code documentation generators that extract documentation from the source code and generate reference documentation.
    • Wiki and documentation management tools that allow for the creation, editing and collaboration on documentation.
    • Automated testing and reporting tools that generate documentation on test results and project progress.

    Documentation-as-Code

    Documentation-as-code (DaC) refers to the practice of treating documentation as a first-class artifact in the software development process alongside code and other project assets. This approach aims to integrate documentation into the development process, making it easier to manage, version and maintain.

    Documentation-as-code can be achieved through a variety of methods, including:

    • Storing documentation in version control systems, such as Git, alongside the code to ensure that documentation is versioned and backed up along with the code. This is especially compatible with a GitOps development process.
    • Using markup languages, such as Markdown or reStructuredText, to write documentation, making it easier to format and display the documentation and to convert it to other formats if needed.
    • Automating the generation of documentation, such as API reference documentation, based on the code.

    The goal is to make documentation a more integral part of the development process, reducing the effort required to create and maintain documentation and improving the accuracy and usefulness of the documentation. 

    Continuous and Collaborative Documentation

    Collaborative documentation refers to the practice of involving multiple stakeholders, including developers, product owners and end users in the creation, maintenance and review of documentation. This approach emphasizes collaboration and communication as key elements of the documentation process.

    In an agile development context, collaborative documentation is seen as a way to ensure that documentation is relevant, up-to-date and meets the needs of all stakeholders. This can be achieved through a variety of methods, including:

    • User stories and other agile documentation techniques that involve end users in the documentation process.
    • Collaborative authoring and review tools, such as wikis and online document editors that allow multiple stakeholders to contribute to and review the documentation in real-time.
    • Regular retrospectives and other agile processes that provide opportunities for the development team to review and improve the documentation.

    The goal is to create documentation that is accurate, relevant and useful to all stakeholders, while also fostering communication, collaboration and continuous improvement within the development team. 

    Interactive Documentation

    Interactive documentation refers to the practice of creating documentation that is designed to be interactive, allowing users to engage with the information in a dynamic and engaging way. This can include elements such as interactive examples, simulations and tutorials, as well as tools that allow users to experiment with and explore the documentation.

    Interactive documentation can provide a number of benefits, including:

    • Improving the user experience by making the documentation more engaging and accessible.
    • Facilitating a deeper understanding of the product by allowing users to experiment with and explore the documentation in a hands-on way.
    • Improving the accuracy and usefulness of the documentation by allowing users to provide feedback and suggestions for improvement.

    Interactive documentation can be achieved through a variety of methods, including:

    • Interactive tutorials and examples that demonstrate how to use the product.
    • Simulations and prototypes that allow users to experiment with the product in a safe and controlled environment.
    • Tools for creating and maintaining interactive documentation, such as online document editors and wikis.

    Conclusion

    In conclusion, documentation is an essential aspect of the software development process, and agile development has brought about new trends in the way documentation is created, maintained and used. From documentation automation to interactive documentation, the agile approach is focused on improving the quality and usefulness of documentation while also reducing the administrative burden on the development team. 

  • Turnover, Documentation and Missing Links

    Turnover, Documentation and Missing Links

    When employee turnover spikes, IT in general and DevOps teams in particular need to be prepared.

    At the time of this writing, the stock market is roiling, and DevOps compensation is bouncing up. This has an impact on turnover because many in DevOps (and nearly all senior positions) have compensation that includes stock—stock that may be down significantly or that is currently at high risk of going down. This causes two types of movement—those leaving an organization to avoid further loss in their compensation and those who are leaving their current gig to join an organization whose stock is down. Combined with those who leave for a role with a larger base salary, there are those who are leaving to earn more and those who are leaving to avoid more loss. IT budgets ballooned during the pandemic but those, too, are deflating, so while we have a worker, not a position shortage, there will be pressure from both sides.

    So, why is this geek going on about employment, compensation and available positions?

    Simple: Because DevOps is not any better at documentation than any other flavor of IT. In tightly-run shops, documentation is available and maintained. In most shops, it is spotty at best. In some, it is downright nonexistent. This has forever been true in IT and, if anything, it is more true because the early stages of DevOps implementation result in a massive amount of change.

    People leaving in an environment with less-than-stellar documentation can be a problem. What tools are used? Where are they located? What’s the login for these SaaS tools? Etc., etc., etc. If turnover means you’re losing entire DevOps teams in relatively short order, this can become a nightmare relatively quickly. And that nightmare is very real for everyone left behind and those that are coming in. Someone has to go figure it out; IT management has to let the rest of the business know that retention issues are causing backlogs. Everyone not tasked with figuring it out has to pick up slack for those who are … you get the picture.

    Take the hour or four and document what you have. It doesn’t have to be comprehensive, it just has to list what tools, where they are, who at the organization is responsible for credentials at any SaaS vendors and under what circumstances the tools are used. If each team documents their own stuff, even in a shared toolchain environment, comparing the documents will uncover little details one missed that another did not.

    Let’s face it, we have—for better or worse—moved largely to a world where loyalty is based on compensation. Much like mercenaries, most IT employees are always listening for a better offer. So the organization, while acknowledging this fact, needs to take steps to minimize the negative impacts of turnover. New ideas and suggestions to change things are all good and you don’t want to interfere with that aspect of turnover. But anything that impedes the organization’s ability to deliver should be addressed beforehand. And that means documentation so remaining and new employees can get up to speed quickly and take ownership rapidly and with confidence.

    And keep rocking it. Whether you are looking for a new gig or not, keep the apps you are responsible for running—and document them. You’ve contributed to the success of your organization; don’t slack off while moving along. And don’t hate those that are moving along. Not all of us base our loyalty on compensation, but we do all allow compensation to influence our decisions (unless you are working for free and I missed it?) Keep rocking, no matter your status. And don’t hesitate to tell interviewers that you helped your last organization achieve success.

  • Documenting: Don’t Build Your Replacement’s Nightmare

    Documenting: Don’t Build Your Replacement’s Nightmare

    Want to reduce headaches tomorrow? Document everything today

    About once a year I feel the need to remind you that you are creating technical debt. DevOps is good stuff, and most shops got over the “whatever a given project team decides” multiplication of services pretty quickly, but every line of DevOps code you write is creating technical debt.

    There is no way around this fact. You are writing code (scripts, if you prefer) and developing configs that make assumptions. Those assumptions will change—products will get incompatible upgrades, vendors will go under, libraries will become outdated … the list goes on.

    We have a long history with technical debt. DevOps partially grew out of the need to spend less time maintaining and more time releasing features, and yet we’re re-creating technical debt; in some cases at a much larger scale. It is unavoidable but can be managed. Keep a reference list of assumptions—“This app assumes environment contains X,” or “This app supports Cisco Network APIs version X.Y”—so you know where to look when things inevitably go wrong. The list conveniently doubles as a handy reference to what needs updating when you find out your middleware vendor is going under or that MySQL is out and MariaDB is in.

    The more cutting-edge tools you use, the more likely that you are baking future problems into your environment. This is not advice to avoid cutting-edge; it is advice to recognize reality and keep closer track as you move new things into your environment that might be half-baked or whose market is destined for consolidation.

    Our IT environment has become super-complex. DevOps helps us manage that complexity but discourages adequate documentation. We’ve all heard, “It’s self-documenting” (no, it’s not) or “There is basic documentation” (basic almost always means, “We scratched some notes when it was first set up”… and haven’t touched them since). Don’t do that. Make sure that the new guy after all of you have left can figure out what is going on in this highly complex environment. It is not up to new people to be super-learners with mind-reading skills; it is up to implementors to make certain new people can figure out what is going on without wasting 5,000 hours pawing through scripts.

    It does not take much time to document dependencies. It doesn’t take much more to document relationships. Future you will be grateful when they walk in and need to come up to speed in a short amount of time. Because it doesn’t matter how automated it is when the automation breaks—it matters how quickly staff on hand can fix it and get things going smoothly again.

    You’re keeping the world running. Take a second and make sure the next person can, too. And keep rocking it.

  • Don’t Expect Miracles

    Don’t Expect Miracles

    Let’s run some fun scenarios to start this one off, shall we?

    • “We’ll decide what language to use at a meeting next month. Until then, get to writing code!”
    • “We might deploy internally, though Azure is a strong option, and we’ve discussed GCP. Now get operations up and running, we’ll tell you when we’ve decided.”
    • “The code is changing constantly, get moving and write the documentation.”

    All three of these scenarios are equivalent. The difference is how we, as an industry, view them. The first two, we (collectively) would never tolerate. The third? We do that to the documentation team every day. In DevOps and agile, we do it every day in a never-ending iteration. And it has to stop.

    Two Different Documentations

    For those not aware, there is documentation for internal use and documentation for external use. While we’re certain you have internal covered with self-documenting code, this blog will focus on external use, because if you expose and API, documentation is mandatory. If you’re not an end-user product, any UI requires documentation, too. So to be clear, for this blog we are talking generally about external documentation for B2B systems, and specifically about APIs, though UIs can have the same issues, depending upon how complex your market is.

    We Generate Docs

    Ever tried to use auto-generated documentation? I’ve yet to try one generated set of API documentation that I liked (they say Google Places has it, but my understanding is devs actually write that documentation and it’s reviewed by tech writers, so maybe we count that as one I liked, but its process is not yours).

    Generated docs are only as good as the developers working on the code–very much like self-documenting code. If someone puts “This parameter is a placeholder for the spline development iteration” into a generatable comment, users are going to be totally clueless what that means. Heck, I just dreamed that statement up, and while we can all see it turning up in generated API docs, even I don’t know what it might mean.

    Documentation Slows Up Dev

    Duh? If you are loathe to commit to completing a project until you know the parameters, why would you expect a documentation team to be any different? Of course they need the final product for review, just as the business wants to see the final app before release. The product your docs team produces is detailed documentation about what is. If you change what is underneath them, they look bad for having inaccurate docs. When really it should be IT that looks bad for not considering the entire value chain when delivering. If you deliver an API that is poorly documented and people don’t use it, you have delivered nothing of value to the organization.

    What to Do?

    Put documentation on your DevOps teams. I mean an actual person whose responsibility is documenting. If you’ve got technical writers, use one of them. If you don’t, find someone who likes helping people use your toys, and give them time to do it. For some organizations, technical evangelists can fulfill this role if you don’t have them too busy helping active users. Have them sit in on meetings, treat them as just another piece of both agile and DevOps. They need to know when functionality, URI or whatever else changes, so they can update the docs. The best place to get that info is at stand-ups or design meetings.

    Just like always, they’ll want to be there for design, but they’ll also want to be there for implementation decisions/changes. Because current documentation is the only worthwhile documentation. Give them sign off on release–not release of code (that’s a done deal) but publication of APIs. Before you let users at the access points, make sure they have solid instructions in how to use them.

    Risks

    Everything is a trade-off, but this one has some pretty easy choices to make–fewer people use your API because your documentation isn’t great (outright sucks in many cases), or you run some risks of slower iterations and you dedicate resources to documentation. Given that if no one uses your API, it is all wasted time, a bit of slow-down and man-hour investment is an easy choice to make.

    Tech writers are generally cautious people who want perfection–because this is their product, and they want it right. This can cause unnecessary slow-downs in an agile/DevOps world. The easy answer here is to look for the same traits in documentation DevOps team members that you look for in other DevOps team members–ability to adapt, understanding that everyone is just trying to make it better and willingness to work through changes quickly to deliver.

    Just like you don’t want the DBA that says “no” to every request and you have to argue with just to move the project forward–I worked with this guy, genius, but really frustrating to have on an agile team–you don’t want the documentation person that adds weeks to timelines so they can get it right. This risk is easy to manage. You know how to choose people for a fast-moving environment, just follow your gut.

    Profit

    Well, okay, maybe not profit, but deliver a solid product quickly with sufficient documentation that users not steeped in your corporate shared knowledge will understand and be able to use. That’s the goal of public-facing APIs, to be used. And it’s the goal of docs people to make certain those calling the APIs understand what they’re getting into and how to make the most of it.

    Keep on cranking out cool solutions and crank out better docs. Your customers are counting on it. I’ve had my team go find alternative solutions based on shoddy API documentation. If I can’t use it, why would I keep trying if there are alternatives?

    — Don Macvittie

  • APIs: Automation is Great. So is Usability

    APIs: Automation is Great. So is Usability

    Many developers are not so great at documentation. Ask anyone who’s been thrown into a large code base. Changes aren’t documented well, tribal knowledge thrives in the code comments, and anyone who says, “I’m confused, what does this chunk of spaghetti code mean?” is often treated with annoyance or even disdain.

    That is something that DevOps doesn’t change. And there are, as with so many things in IT right now, complicating factors. First is APIs. In almost every shop, developers are responsible for the public-facing APIs. The same developers who don’t comment well are now documenting public-facing APIs. The other complicating factor is DevOps documentation mechanisms. Automated systems generate API documentation from developer comments. Again, the same developers that often don’t document their code well enough to efficiently on-board new developers.

    Those of us who have used more than a couple of APIs know the problem intimately. Dozens, or hundreds, or thousands of us have to search for what the allowable values for a parameter or JSON field are in an API call are because far too often that is not well-documented.

    The whole system grew naturally. APIs are the realm of developers; documentation was painful to get created because it required tech writers that were advanced enough to understand the purpose and function of the API; and development of documentation delayed API deployment—all are issues that lent themselves to automated generation and the developer in the code was best-situated to document it. In the API economy, and in light of both agile and DevOps, it just made sense to use generated documentation for this part of the project.

    But we’ve got enough experience now to know that “good enough” isn’t. Far too often users are left to figure out the details, and quite often, those users don’t bother. That’s not a benefit in the API economy; it’s a risk.

    Get Help With Your APIs

    And it’s pretty easily resolvable. Get someone not knee deep in the code to try and use your API as documented. Get their recommendations on usability, and fix the issues they find. I’ve used APIs that have a dozen JSON base parameters, none of which are documented as to what’s allowed or not. I’ve used APIs where the JSON field was text, but the values were from a specialized list that was not made public and users had to hunt for allowed values. We’ve all been through it, and it is a massive waste of time.

    If you have a customer base, use Alpha/Beta users to help with this process. Early access is generally people who know the project/product, so feedback will be more focused, but as outsiders they do not share in tribal knowledge. Without tribal knowledge, they will have solid input to improve API usability.

    And in the end, an API is a consumable that you want customers to use. Telling them how to do so is important. And the current trendy way of doing so is not good enough. So make certain your process is good enough, or accept user loss as a consequence.

    For those with established products/projects, do not wag your head and assume you have it covered. Some very cool, long-lived APIs have entire sets of JSON input values that are not well-documented. They were important enough to make them parameters (or return values for that matter), yet you didn’t document them well? Please, ask new users if they agree with your assessment of your API.

    Don’t have time/man hours/whatever to address documentation shortcomings in your API? Then don’t publish an API. The only thing worse than saying, “You can’t integrate via API with our system,” is saying, “You can integrate with our system via API,” then not documenting it well enough to do so.

    So keep the automation. But follow the new mantra:

    Now that you have generated documentation, check that you have usable documentation.

    And keep kicking it. APIs are the hard part, telling people how to use them is comparatively easy, so take the time to make your development efforts worthwhile.

    — Don Macvittie