Bi-Directional Contract Testing: a Basic Guide to API Contract Testing Compatibilities

While Stoplight focuses on mocking, API testing is different. But you need both to be successful. Below, in this guest blog from PandaDoc, we dive into what contract testing is for those who are curious.

What is API Contract Testing? API contract testing verifies the compatibility of two separate applications by checking the consistency with a specified set of rules. Furthermore, It does this by recording and storing the interactions between them in a contract that both parties must adhere to. The contract agrees on the allowable interactions while also providing for evolution over time.

API contract testing is often performed with Pact using consumer-driven contract testing (CDCT). In CDCT, the API consumer generates the contracts. This ensures an API provider is compatible with their expectations. For example, for an HTTP API, the provider would need to accept the expected requests and return the expected responses.

CDCT is ideal for environments with good collaboration between teams. But there are certain conditions that make it difficult to use, to name a few:

• The consumer lacks technical details about the provider’s setup.
• There is poor communication between teams.
• The provider already has an API testing suite.
• The workflow is more focused on the provider than the consumer.
• The provider has already produced an Open API specification (OAS).

In these situations, the newer method of bi-directional contract testing (BDCT) is the recommended approach. But what is bi-directional contract testing, and what are its benefits and uses? Read on to find out more.

What is bi-directional contract testing (BDCT)?

Bi-directional contract testing (BDCT) is a type of API contract testing unique to Pactflow. Pactflow is a testing platform that deploys and manages testing for APIs using Pact. BDCT compares two contracts to check their compatibility. One contract outlines the consumer’s expectations, while the other details the provider’s capabilities.

The consumer generates a contract using an API mock tool like Pact or Wiremock. Meanwhile, the API provider uses an API testing tool like ReadyAPI to verify their contract (such as an OAS). Then, both parties upload the contracts to Pactflow, which compares them and flags any inconsistencies.

For BDCT, you don’t need to implement Pact fully. Instead, you can “upgrade” your existing tools and specifications into an API contract testing solution. This makes it a much more versatile solution, though it’s important to note that BDCT and Pact testing aren’t mutually exclusive.

How is BDCT Different From traditional Contract Testing?

In the traditional approach of CDCT, a consumer generates a contract and uploads it to the Pact Broker. The Pact Broker sends the contract to a provider for verification (like a creditor sending a debt settlement agreement to a debtor for their signature).

The provider runs the necessary verifications and uploads the results to the Pact Broker. Both parties can then check if it’s safe to deploy the new version of their application using the command can-i-deploy.

In BDCT, consumers and providers generate their own contracts and upload them to Pactflow. Pactflow then compares them to see if they’re compatible. Both parties can check the results with can-i-deploy.

BDCT allows independence between consumers and providers. The providers don’t have to accommodate different consumers, and the consumers can choose providers that support their expectations.

Benefits of bi-directional Contract Testing

Bi-directional contract testing offers several benefits over traditional consumer-driven contract testing. This includes the following:

Easier to Learn and Deploy

BDCT is much easier to learn and understand than traditional customer-centered API testing. This is partly because it more closely resembles the kind of testing that teams are used to doing. Instead of having to learn a whole new testing framework, teams can use BDCT with their existing tools, such as Cypress, Wiremock, or Postman.

Furthermore, by reducing the dependence between providers and consumers, consumers can quickly determine if their changes are safe to deploy. Likewise, providers don’t have to worry about creating and managing a range of states to accommodate different consumers.

The Whole team can Take Part

CDCT requires direct access to the code base, restricting the types of roles that can contribute to it. This isn’t the case with BDCT. This means your whole team can participate in contract testing and quality control, including QA Engineers, SDETs, and Testers. This results in higher-quality contracts.

More Adaptable Workflows

BDCT allows providers to publish their contracts without having to wait for the consumer to start the testing process (and vice versa). This gives both parties the freedom to choose the workflow that best suits their organization, including API-first design (which CDCT doesn’t support).

Additional Use Cases

Because BDCT allows more independence between providers and consumers, it can support a wider range of use cases than traditional Pact testing. Examples include working with API gateways and integrating third-party software like PandaWaiver into your tech stack.

Rapid Scaling and Faster Time-To-Value

CDCT leads to scaling issues when a provider caters to many consumers, each with its own contract. BDCT avoids this issue by testing the provider contract rather than the provider application. This simplifies the testing process, making it easier for the provider to scale their operations.

Also, since BDCT works with your existing tools, you can rapidly scale contract testing across your systems. This also reduces your time-to-value.

How to Start Bi-Directional Contract Testing

As mentioned previously, bi-directional contract testing is unique to Pactflow. Using it development teams can start API contract testing from the consumer side (consumer-driven) or the provider side (specification-first).

Consumer-Driven

To start BDCT on the consumer side, the consumer takes the following steps.

1. The consumer tests the behavior of their application using a mock service like Pact or Wiremock.
2. Pact produces a Consumer Contract in the form of a .pact file, which captures the interactions produced by the consumer’s code.
3. The consumer uploads the Consumer Contract to Pactflow.
4. The consumer calls can-i-deploy, triggering Pactflow’s contract validation process.
5. Pactflow generates a Verification Result, which determines if the Consumer Contract is compatible with the Provider Contract.
6. If the contract passes the verification test, the consumer can deploy their application and record the deployment using the command pact-broker-record-deployment.

Specification First

Starting bi-directional contract testing on the provider’s side involves the following steps.

1. The provider creates their OpenAPI specification (OAS) by hand or using a code generator tool. This specification is the Provider Contract.
2. The provider tests the Provider Contract using a functional API testing tool (such as ReadyAPI or Postman).
3. The provider uploads the Provider Contract to Pactflow.
4. The provider calls can-i-deploy to trigger the contract verification process.
5. Pactflow generates a Verification Result, which determines if the Provider Contract is compatible with the Consumer Contract.
6. If the contracts are compatible, the provider deploys their interface and records the deployment using pact-broker-record-deployment.

Bi-Directional Contract Testing Responsibilities

Each party involved in bi-directional contract testing has certain responsibilities they must fulfill. This includes the consumer, the provider, and the central contract repository.

Consumer Responsibilities

The consumer’s responsibilities are to:

• Write a high-quality contract that contains only what is currently used by the consumer application.
• Test their application against the contract to ensure the contract is an accurate representation of their app.
• Publish the contract to the central contract repository in Pactflow so it’s available for testing with the provider contract.
• Only deploy their application if the central contract repository determines the contracts are compatible (to avoid production issues).

Provider Responsibilities

The responsibilities of the provider are to:

• Create a specific, high-quality specification (contract) that caters to both humans and computers.
• Test their contract with their interface to ensure they align.
• Publish the contract to the central contract repository to test it against the consumer contracts.
• Only deploy their interface if the contracts are compatible一or, one or more consumers will experience production issues.

Responsibilities of the Central Contract Repository

Pactflow’s central contract repository stores the provider and consumer contracts. This ensures all parties use the latest contracts. With BDCT, the central contract repository is also responsible for testing contract compatibility.

Conducting API contract testing in the central contract repository means all the tests are done with the same software. This ensures consistency.

Common Use Cases of bi-Directional Contract Testing

Bi-directional contract testing is ideal for a range of use cases and helps to solve many of the challenges of traditional Pact testing. Let’s look at some examples.

Takeaway

Bi-directional contract testing (BDCT) reduces the dependence between API consumers and providers. This offers several benefits over consumer-driven contract testing (CDCT). For example, BDCT offers simpler deployment, rapid scaling, adaptable workflows, and faster time-to-value.

Bi-directional contract testing offers an alternative to traditional Pact testing in use cases where CDCT is challenging, such as testing with API gateways, web apps, or third-party APIs. BDCT works alongside Pact testing to give both parties the flexibility to test their API integrations in the way that suits them. Ultimately making BDCT a valuable API contract testing solution.

Author Bio

Yauhen Zaremba is the Director of Demand Generation at PandaDoc, an all-in-one document management tool for almost all types of documents including PandaDoc eSignatures in Canada. He’s been a marketer for 10+ years, and for the last five years, he’s been entirely focused on the electronic signature, proposal, and document management markets. Yauhen has experience speaking at niche conferences where he enjoys sharing his expertise with other curious marketers. And in his spare time, he is an avid fisherman and takes nearly 20 fishing trips every year.