API Endpoint Discovery, Part II

In Part I of this series, we explored the basics of how to automatically build endpoint definitions for an API you own. But what if you want to map out an API that you don't own, or that is otherwise private?

In this post, we're going to describe how to do API endpoint discovery with an API we don't own - the SendGrid API. You could also apply these methods in other scenarios, for example to map out private APIs that mobile apps are using.

SendGrid is a great candidate because their new dashboard is built on top of their API. For API endpoint discovery to work, we need to tap into HTTP traffic flowing to the API, and the SendGrid dashboard generates this traffic for us as we browse their UI.

Designer Setup

Note: API endpoint discovery only works with the app version of the API designer, not the web version.

First, we need to identify where the API is located. We can use Chrome's network inspector to view the HTTP traffic SendGrid's dashboard is creating. We quickly see that the API host is https://api.sendgrid.com, and the API base path is /v3.

host discovery

In the StopLight API Designer, create a workspace, project, and environment. In the environment's settings, set the API Host to https://api.sendgrid.com, and the base path to /v3.

The final part of the designer setup is to click on the environment editor proxy settings tab, and toggle the "API Discovery" option to on.

Click save at the top of the environment editor to persist our changes.

System Proxy Setup

Now, we need some way to proxy the http traffic generated from the SendGrid dashboard through our local Prism proxy instance. To do this, we'll set our system-wide OS X network settings to proxy all traffic on our computer through Prism.

  1. Open your network settings.
  2. Click the advanced button in the bottom left.
  3. Click the Proxies tab at the top.
  4. Toggle "Secure Web Proxy (HTTPS)" to true, and set the host to "localhost", and port to 4010 (or wherever prism is running on your computer).
  5. Click OK.
  6. Click Apply.

Note: Even though all of the network traffic on your computer is piped through the local proxy instance, Prism only processes http traffic that matches the current environment's API Host and base path! All other HTTP traffic is transparently proxied through and not changed, logged, or processed in any way.

network setup

System SSL Setup

The last piece of setup is to add the default SSL certificate that Prism proxy uses to your keychain. We need to do this because the API that we are targeting (SendGrid) runs over https only. Prism uses this SSL certificate to decrypt, and then re-encrypt, traffic that it processes. Again, it only processes traffic that matches the current environment's API host! Everything else passes through untouched.

  1. Visit http://localhost:4010/stoplight.cer (or wherever prism is running on your computer).
  2. Click the downloaded certificate to add it to your keychain.
  3. Find the "StopLight" certificate in your keychain list, right click it, and then click "More Info".
  4. Open up the "Trust" section at the top, and change the dropdown there to "Always Trust".
  5. Save these changes.

keychain ssl certificate

You can also use your own certificates! If you have a trusted certificate already installed on your computer, you can paste the csr and key data into the environment proxy settings in the designer. Prism will use those certificate details instead of it's default certificate.

Discovery!

Let's do this! In Chrome or Safari (Firefox has issues), visit https://sendgrid.com and login. As you browse their dashboard, you will see the discovery process happening in the designer endpoint list.

Prism is automatically processing HTTP traffic that the SendGrid dashboard is generating, and using that traffic to map out SendGrid's V3 API definitions.

Magic :).


In Part III, we will take a look at how to generate endpoint definitions from an existing test suite.

If you have not requested an invite to the StopLight beta, head on over to https://stoplight.io to get started.

Happy Designing!
The StopLight Team
Marc, Rana, Tom, and Vazha