Powering API Governance with Spectral: Part One

Are you considering setting up an API Governance program, or have you thought of setting up an API Design Review process at your organization? How would that even work? Who does what, and can any of it be automated? Something about API Style Guides? In this first part of a three-part guide, we’re going to walk you through it, showing how tooling can help make things a little easier to make your API governance roles and responsibilities a breeze.

Why is governance increasingly important?

Many of us will know the pain of working at a company where every API is a unique snowflake, with custom data formats, subtly different error formats, BizArO_CaPs naming conventions, and custom security conventions that are not particularly secure. This is only becoming more of an issue over time as teams switch from a small number of APIs to more and more microservices.

There are various ways to go about solving this, and you might have heard the term “API Governance” bouncing around. Larger companies will often put together a team of experienced API developers, systems architects, etc., and get them to help with planning sessions, approve new API designs based on systems diagrams, review the OpenAPI before a new API is made, and keep on reviewing changes that are made to make sure the API is on point, and they’ll eventually write some sort of API Style Guide so that folks are not surprised when their API design is rejected for not using a certain standard or convention.

The existence of a team like this is expensive for large companies and can be tricky for smaller companies to dedicate resources to at all. Sometimes folks end up taking time out of their usual responsibilities to handle governance and reviews, which can cause a lot of stress. Whatever the team configuration, the more manual effort involved, the more likely this team is to become a bottleneck for API developers. It’s also rather tedious for that team, so however you look at it, automation is an important part of the process, making this quicker, more efficient, and generally less tedious.

Automating API Style Guides with Spectral

Spectral is increasingly being used to complement API Governance and in particular, API Design Reviews. It’s essentially an automated way to run an API Style Guide, by enforcing “rules” on a document like OpenAPI, to ensure things are conforming to the standards, conventions, and security practices you have chosen.

Because Spectral can be run in editors like Stoplight Studio or VS Code during the creation and update of an API design, in Continuous Integration to check pull requests, or even with a JavaScript API to create your own interface, it can be a powerful ally for anyone working in API Governance, whether it’s a single governor or a team.

There are a lot of companies with API style guides published on a website called API Style Book (another name for the same thing, and we just did a great podcast episode with Arnaud Lauret, who wrote the API Style Book). Let’s take a look at a company that has created a public API style guide and automated a bunch of those rules with Spectral. Then we can try and recreate it.

Let’s dive in: an example Style Guide with Adidas

The Adidas API Style Guide starts off with a great motivation:

The goal of this document is to facilitate the work and minimize the effort of all API users at Adidas while protecting their investment and encouraging API adoption.

The text-based API style guide has a lot of written content, which says things like:

Everyone MUST follow the API First principle. The API first principle is an extension of the contract-first principle. Therefore, the development of an API MUST always start with API design without any upfront coding activities. API design (e.g., description, schema) is the master of truth, not the API implementation.

Off to a strong start.

Approved API Design, represented by its API Description or schema, MUST represent the contract between API stakeholders, implementers, and consumers. An update to the corresponding contract (API Design) MUST be implemented and approved before any change to an API MUST.

Couldn’t agree more.

Every API design SHOULD be stored in a Version Control System (Bitbucket, GitHub). Where possible the API design SHOULD store in the same repository as the API implementation.

Absolutely, we recommend that here at Stoplight, and generally recommend you keep your OpenAPI in the Git repository so that when code exists, it can be changed along with the OpenAPI that is describing it, making every commit atomically correct to itself. This helps with contract testing and makes sure there’s just one source of truth instead of multiple sources that could fall out of sync.

These are all quite high-level ideas about the organization and workflow, which is more likely to be implicitly required by the tooling chosen than something that needs enforcing, but the rules then move onto naming conventions, which is definitely in Spectral’s wheelhouse.

• Use camelCase unless stated otherwise.
• Every identifier MUST be in American English and written in lowercase.
• Every URI MUST follow the General Rules except for the camelCase rule. Instead, a hyphen (-) SHOULD be used to delimit combined words (kebab-case). Besides, a URI MUST NOT end with a trailing slash (/).
• In addition to General Naming Rules, URI Template Variable names MUST follow the RFC6570. That is, the variable names can consist only from ALPHA / DIGIT / “_” / pct-encoded.
• Every HTTP Header should use Hyphenated-Pascal-Case. A custom HTTP Header SHOULD NOT start with X- (RFC6648).

Loads of this can be automated by Spectral. As Adidas has published the Spectral ruleset powering their API Style Guide, we can use this to quickly get an idea of how rules can be implemented to shape the design of an API.

Creating a Ruleset to power your API Style Guide

Using your favorite code editor, make a new directory to try all this out while we get the hang of all this, and create a file called .spectral.yaml. I’ll use vim just because it’s easy to write as a command in a blog post.

mkdir spectral-governance-p1
cd spectral-governance-p1
vim .spectral.yaml

Let’s take a swing at a naming conventions rule first. Pop this content into the YAML file.

extends: [spectral:oas]
rules:
  adidas-paths-kebab-case:
    description: All endpoints/paths MUST follow kebab-case
    severity: error
    given: $.paths[*]~
    then:
      function: pattern
      functionOptions:
        match: "^/([a-z0-9]+(-[a-z0-9]+)*)?(/[a-z0-9]+(-[a-z0-9]+)*|/{.+})*$"

In this YAML file, we can write rules in a special Domain Specific Language that powers Spectral. The first line is extending the OpenAPI ruleset so you can find out if your OpenAPI is valid or not, then we’ve got started defining our first rule below that.

The main idea of a ruleset is a set of rule objects that contain a rule name as the key. The description can be longer and contain more information that’s useful for helping people work out what to do, and for some rules, it can be a good place to explain the “why” it should be done too.

Then the next most important part is the path. This syntax is JSON Path Plus, which is similar in concept to the old XPath for XML. It’s used to help you navigate complex JSON/YAML documents, to target a specific piece of information. For Spectral, it’s used to find the part of the document to apply a rule. For OpenAPI, the API’s paths (a.k.a “endpoints”) which is an awkward coincidence, so we end up with a JSON Path containing $.paths[*]~… anyway. That’ll select an array of all the endpoints we define in OpenAPI, so let’s make an API and try to make this rule shout at us.

Let’s avoid writing a bunch of OpenAPI by hand by downloading Stoplight Studio (don’t worry it’s free). Open the spectral-governance-p1 directory like this:

Now create an API and call it whatever you like. Then we’ll add a New Endpoint, which is not following the rules.

As soon as we add an endpoint of /ignoringNamingConventions, Spectral will let us, the API designer, know that we’ve made a mistake, and tell us how to correct it.

If we rename the endpoint to /good-conventions then the error will go away, and now that is one less thing the API Governance team needs to worry about when doing a design review.

Another fantastic rule here is making sure successful responses are using the HAL data format.

adidas-oas3-response-success-hal:
    description: All success responses MUST be of media type application/hal+json
    severity: error
    given: $.paths..responses[?( @property >= 200 && @property < 300 )].content[*]~
    formats: [oas3]
    then:
      function: enumeration
      functionOptions:
        values:
          - application/hal+json

This means response bodies will need to look something like this, which is one way to standardize the payloads. There are several data formats with hypermedia support (a.k.a. “hypermedia formats”), and picking one can help offload a lot of the decision-making from your API Style Guide. Using standard data formats also opens up the world of generic tooling that knows how to interact with these formats.

Let’s throw in some more rules for fun.

Ban using GET with a Request Body

Some developers would never do this, they are surprised some tools support it, and can’t imagine anyone ever trying. Other developers absolutely love GET with a body, cannot believe there are tools that don’t support it, and will use it frequently. If you have tools that don’t allow GET bodies, set this rule to save everyone time and money, both in debugging and arguing.

  request-GET-no-body:
    description: A `GET` request MUST NOT accept a request body.
    severity: error
    formats: [oas3]
    given: $.paths..get.requestBody
    then:
      function: undefined

Ban HTTP Basic

It’s basically plain text. Even with SSL involved, it’s best to avoid using HTTP Basic in most situations.

  no-http-basic:
    description: "Consider a more secure alternative to HTTP Basic."
    message: "HTTP Basic is a pretty insecure way to pass credentials around, 
    please consider an alternative."
    severity: warn
    given: $.components.securitySchemes[*]
    then:
      field: scheme
      function: pattern
      functionOptions:
        notMatch: basic

No Numeric IDs

Auto-incrementing IDs makes it far too easy for somebody to loop through your API collections and save copies of all your data, so let’s use UUIDs or similar instead.

  no-numeric-ids:
    description: Please avoid exposing IDs as an integer, UUIDs are preferred.
    severity: error
    given: $.paths..parameters[*].[?(@property === "name" && (@ === "id" || 
    @.match(/(_id|Id)$/)))]^.schema
    then:
      function: schema
      functionOptions: 
        schema: 
          type: object
          not: 
            properties:
              type: 
                const: integer
          properties:
            format: 
              const: uuid

This can be a lot to take in at first, but take a look at the custom rulesets documentation and see if you can think up any more API Style Guide ideas.

For now, we have enough rules. Try messing around with your API design, triggering the errors, tweaking things a bit, and then let’s see how this works in the command line so we can start automating things.

Working in the CLI

The CLI enables usage of Spectral outside of Studio or VS Code, for uses like continuous integration, or maybe you just want to stay in the terminal instead of opening GUIs. Whatever the reason (probably both for some users), this is how it works.

npm install -g @stoplight/spectral-cli
spectral lint reference/Some-API.yaml

Running spectral in the CLI will look for a local ruleset, and it will find .spectral.yaml so long as you are inside spectral-governance-p1. If Spectral is being run from elsewhere, or the ruleset is named something else, you can pass it in as an argument:

spectral lint reference/Some-API.yaml --ruleset=some/path/.spectral.yaml

If Spectral was able to run in your environment, and the PATH is set up correctly for your NodeJS version manager, you should see some results like this if your OpenAPI document is breaking any of the rules.

/Users/phil/src/spectral-governance-p1/reference/Some-API.yaml
 16:16    error  no-numeric-ids         Please avoid exposing IDs as an integer, 
  UUIDs are preferred.                 paths./some-url/{someId}.parameters[0].schema
 22:19    error  request-GET-no-body    A `GET` request MUST NOT accept a request body                                                   
  paths./some-url/{someId}.get.requestBody
 63:15    error  no-http-basic          HTTP Basic is a pretty insecure way to pass 
 credentials around, please consider an alternative.  
  components.securitySchemes.User-Pass.scheme

Beautiful! Now we have feedback on not only making better quality OpenAPI, but we’re also getting our custom rules showing that we’re naming endpoints poorly.

If you’d like to get all the code used in these examples or are having any trouble getting things to work, pop over to this GitHub repo and see if that helps. I created the perfect “bad API” which breaks all of our rules.

Is this governance yet?

Not yet. This style guide still just lives on a computer somewhere and needs to be run manually. If everyone were to copy and paste this .spectral.yaml locally and run the CLI before committing any changes, things would all work out great, but that sounds like a lot of hassle, and it would be hard to maintain as updates are rolled out.

In the next part of this series, we’ll work on distributing our ruleset so folks can enable it in their various environments, and learn how to evolve these rulesets carefully over time to add new rules as we discover more problems that need avoiding.

For more resources on style guide creation and implementation, check out our style guide best practices, or feel free to tweet at me @philsturgeon.