The concept of idempotency is quite important when writing APIs. It refers to the capability of an operation to be called multiple times and produce the same state always.

State here means any data or other side effects that might be altered or introduced by calling this API.

API calls might get interrupted due to many different reasons. Connections might be dropped unexpectedly, external services might be offline or servers might crash. As a client, one usually has no certainty why a particular request stopped and if it is safe to repeat the same request.

Web developers usually come into contact with this concept when working on REST APIs. While POST requests usually are non-idempotent, PUT and DELETE are. Updating an entity multiple times — with the same parameters — will always produce the same result (or state).

This post explains the logic of using idempotency keys, the next post will dive into the implementation details.

Idempotency keys

One way to solve idempotency is to introduce so-called idempotency keys, sufficiently random strings usually sent as part of a header. Those uniquely identify the request and can be used to retransmit the request when an issue occurs. Keys are stored temporarily and get recycled, for example, every 24 hours.

Stripe is a good example of a popular API that uses idempotency keys, see Designing robust and predictable APIs with idempotency.

A call to an operation that allows for idempotency can be in one of four states:

• NONE: No idempotency key was provided → the operation is executed as normally
• STARTED: The server has never seen this key → the operation is executed as normally
• RECOVERABLE: The server has seen this key before and the operation was left in a recoverable state (e.g. a network issue happened) → the server recovers the request and continues its execution
• FINISHED: The server has seen this key before and the last operation has completed (successfully or with an unrecoverable error) → the operation is not executed and the server sends the previous response

Flowchart of a request using an idempotency key.

Atomic actions

Implementing idempotent operations is hard. One way to do so is to use transactions in an ACID-compliant database. We could wrap the complete operation inside a single transaction.

If any failure happens during our operation we can roll back the complete transaction and try again. This assumes that we don’t mutate any external state, though! Once we leave our local system — our database — we potentially lose the ability to roll back any changes we have made. Think, for example, about charging a customer; once we have executed the charge, we cannot roll it back.

There are inherently idempotent mutations of external state, for example, changing a DNS record, sending a PUT request, etc. But not all systems behave that way.

To solve this issue of potentially mutating external state, we introduce the concept of atomic actions. An atomic action is any part of an operation between mutations of an external state. So, even when we do 20 database operations, all can be safely rolled back using transactions if something happens.

Recovery point

After each atomic action we create a recovery point and save it on the current idempotency key. Should an action fail, we can always recover from the last recovery point — if the error is recoverable. A recovery point will be identified by a simple string.

Action results

Each atomic action can return one of five results

• A recoverable error: in this case, we can roll back the last transaction and return the error to the client. The client can safely repeat the request and we will restart from the last recovery point.
• A non-recoverable error: here, we need to roll back and return the error to the client. Calling the operation again will short-circuit the operation and return the same error (the key is in status FINISHED).
• A recovery point: we update the recovery point on the key and the operation continues.
• A response: this is the happy path and we return the response to the client. If the client calls the API again (e.g. because the connection broke during the first request) we also short-circuit and return the same response.
• A no-op: here nothing happens and the operation continues.

Locking the key

When building the framework we also need to make sure that no two requests happen with the same idempotency key. The key should be locked immediately before starting any atomic actions. We release the lock on any action other than a no-op or a recovery point. We also introduce a timeout after which the key will be available for a new request again.

Threader

As an example of an app that requires idempotent operations let us imagine building a new Twitter — so that we might also get bought by some crazy billionaire. We call it Threader, and you can create Threads of short messages on the app. To reduce spam — and improve our bottom line — users need to pay $1 per Thread they create.

While our backend will offer many different operations, we only discuss the creation of Threads here:

Creating a Thread

Many different things need to happen when creating a Thread, those include

1. Create the idempotency key.
2. Save a ThreadRequest in the database to track the creation.
3. Charge the user’s credit card on Stripe.
4. Update the ThreadRequest with the stripe charge ID.
5. Add the Thread record to the main database.
6. Notify the user about the charge and send the receipt.

The bold actions are external state mutations. We cannot roll back a charge on stripe or send an email/push notification to users (easily).

Atomic actions when creating a thread

Atomic actions and recovery points

Following the above operations that need to happen to create a Thread, we can now separate them into distinct atomic actions:

• Creating an idempotency key always gets its action.
• The next external state mutation is the call to stripe, so saving the ThreadRequest is the next action.
• Charging the credit card can not be rolled back but Stripe can handle requests in an idempotent way as well. So even when something goes wrong we can safely retry.
• After adding the Thread to the database we can create a new recovery point since notifying the user is another external state mutation.

Ready to enhance your API’s reliability with idempotency keys? Contact inheaden.io today to discover how our expertise in Go and PostgreSQL can streamline your development process and ensure seamless user experiences.