How to Compare and Contrast APIs to Improve DevOps Processes

We know we need to eat our vegetables to keep our heart healthy, but sometimes it is easier to order takeout after a long day of work. Healthy API hygiene works the same way. In a perfect world, developers would register their API specifications and blueprints in an API catalog, and they would keep them up-to-date as the application changes. But this is a tedious manual process, and sometimes after a long day of work it is easier said than done.

An API catalog is critical for improving the DevOps process through quality assurance and testing. Furthermore, the production environment for APIs can be subject to even harsher and more hostile access and usage patterns compared to the Dev and Test environments, which tend to be more self-contained. The good news is that by maintaining an API catalog, DevOps can gain more visibility into the contrast between these environments to identify gaps that should be closed. The end result is an improved DevOps process that drives the production of better products and services.

There are a number of approaches to creating an API catalog, from manually logging APIs into Wikis, spreadsheets and vendor solutions, to more automated discovery tools. An API catalog should include meta information, such as the creator, the date it was last modified, which application it belongs to and so forth. At a deeper level, an API catalog should also include the parameters such as the data type, the bounds for the data type, the transport mechanism, the authentication and authorization mechanism, and a brief description about the data carried within the parameter.

Compare and Contrast APIs from Dev to Test and Test to Production

During the application testing phase, which is subject to quality assurance, regressions, performance and interaction cases, an API catalog should be updated and compared against its development state. The contrast between the development API catalog and those generated from the test environment can help identify:

• An API listed in the Dev catalog, but not in the Test catalog, which would indicate that QA has not tested the API or its payload; therefore, the QA process could be improved.
• An API listed in the Test catalog, but not the Dev catalog, which would indicate that the Dev catalog was incomplete; therefore, you cannot assume it is a “golden” image. This also suggests an area for improvement in the DevOps process because the developer has not maintained an up-to-date API catalog. Automated tools can help even more here since they generate an API catalog where there is none.

There is a third scenario, where an API isn’t recorded in the Dev catalog, and it doesn’t show up in the Test catalog, but may later show up in the production traffic. This is considered an unpublished or “out of spec” API.

Likewise, when moving from test to production environments, the API catalog should be updated to mirror real-world application access. The contrast between the Production API catalog and the Test API catalog can help identify:

An API listed in the Test catalog, but not the Production catalog, which could identify an API that is not being used. This would indicate that QA testing did not align with the real-world use of the application and suggest that development cycles could be prioritized elsewhere.

An API listed in the Production catalog, but not in the Test catalog, which would indicate the API catalog did not capture the most up-to-date specification. This is one example of a Shadow API, an API that was intended to be part of the application, but has not gone through the required processes related to testing and application security.

Additionally, identifying an API in the Production catalog, but not in the Test catalog can be useful in identifying threat actors searching for vulnerabilities to exploit an unpublished API.

Comparing and contrasting dev, test and production environments can help identify opportunities for improvement. However, manually creating and maintaining an API catalog is often neglected because of its tedious nature. DevOps needs automated tools to help discover APIs, generate blueprints and store them into the catalog. It is too much of a burden for developers to manually document APIs, but a continuously updated API catalog is the most important key to derive success using the compare and contrast methodology.