Top 10 Design Tips for APIs

While building our new API, we learned several lessons on how to build a great API. We want to share Design Tips for APIs with you to maybe help you in designing or improving your API as well.

A few months ago we came to the conclusion that we had to introduce a completely new version of our API in order to keep up with all the new requirements we and our customers developed using Phrase. We had gained many insights since we launched the first Phrase API back in 2012.

We developed the old version of the API for internal use only. We did not worry too much about finding the perfect structure, naming or performance. After all, Phrase was an MVP and our main focus was having a working API for our very own In-Context Editor and the phrase command-line client.

However, as time passed and we started to gain customer after customer, the demand for a better-structured API grew up until the point where we had to start all over again. These are the 10 essentials on how to build an API, we learned in the process of developing our API v2:

Use Versioning

Even when you just started developing your product and are not sure if you will ever have the chance to create the next version of your API, version your API. It comes at no cost and will make it so much easier to add that new version when your product succeeds. If you have a successful product, you will almost always have to adapt your API interface and thus need to release a v2 of your API!

The background of this is simple: Changing a published and used API interface is bad. Really bad. Consumers of your API rely on its interface. They might have important business logic based on your API and it might fail we you change your endpoints. The consequences may long from minor inconvenience to customers suing your company for financial reimbursement.

When using Rails a simple namespace in your routing is enough to define the different versions:

namespace :api do

  namespace :v1 do

    resources :projects

  end

end

This will result in readable URLs like: /api/v1/projects

We recommend to set the version as part of the URL, since this is the most transparent method to its consumers. Github chose a different versioning approach by utilizing the HTTP Accept-Header.

Use HTTP Status Codes Correctly

HTTP provides you with a great mechanism to tell your API consumers if their request was successful or not: HTTP status codes.

Use these status codes correctly and provide your consumers with useful information. They will thank you. The following table notes our use of the HTTP status codes in a typical REST-API.

Rails controllers make it really convenient to use the correct status codes, i.e:

render json: @object, status: :created

JSON To The Rescue

There used to be a time where passing POST data as url-encoded was cool. But it is not any more. Do not consider XML - we do not live in the 90s and SOAP isn’t cool either. When possible, use JSON to send data to your endpoints. This makes any request more readable and can be assembled much easier by the consumer.

Rails makes it extremely easy to handle parameters that are JSON-encoded. Rails’ ActionController automatically unwraps the data from JSON so you can access them via the params hash easily.

Do You Speak HTTP?

Besides the incredibly useful HTTP status codes, you should also embrace the correct use of HTTP verbs. The most common you need in any REST-API are:

Using these verbs guides the consumer towards the correct usage of your API. It provides the consumer with an experience she’s familiar with.

Rails’ routing methods make implementing the correct HTTP verb for your action a breeze. See the Rails guide on routing for more information.

We All Make Mistakes…

Handling errors in an API sounds easy but requires careful planning. It is highly recommended to always return error messages in the same format so that the consumer does not have to handle different semantics on different resources. Also, consider that some client languages might not support arbitrary nesting in the JSON response and thus have more trouble parsing it.

Use HTTP status codes to top your error message handling off, e.g always return status 422 when a validation fails.

If you want to go the extra mile, you can include a link to a matching article from your documentation when an error occurs so that the user can debug the problem immediately. Nothing is worse than an error with no or even a wrong error message. Always try to be as helpful to your consumers as possible!

A good error message response might look something like this:

Status: 422

{

  "message": "Validation Failed",

  "errors": [

    {

      "resource": "Project",

      "field": "name",

      "message": "can't be blank"

    }

  ]

}

Less Is More: Always Paginate Your Results

When we first started developing Phrase, we could not imagine the amount of data our customers would want to manage with our platform. This caused problems eventually, since some of the endpoints we offered in our API did not offer pagination. Thus for larger customers they returned hundreds of thousands of records at once.

This can result in serious inconveniences for database, browser and web-server. A good way to prevent this is to paginate all results that return a list of items.

Pagination in Rails can be easily implemented by using an appropriate gem such askaminari, but you can of course also implement it yourself using LIMIT and OFFSETstatements in your queries or on your collections.

When returning your paginated results, make sure you display pagination meta information to the user. We found that the best way to do this is via HTTP Link-Headers in the response.

This header contains the full URL to the next, previous, first and last page of the result set and makes it really easy for clients to handle your paginated responses. This also doesn’t clutter the actual response body with meta information to keep parsing simple:

Link: <https://phrase.com/api/v2/projects?page=1>; rel="first", <https://phrase.com/api/v2/projects?page=3>; rel="prev", <https://phrase.com/api/v2/projects?page=5>; rel="next", <https://phrase.com/api/v2/projects?page=9>; rel="last"

Avoid providing meta stats like the total number of results or number of results on the currently returned page. Though these might be useful in some cases, most of the time they are a waste of computing time and increasing the query load on your database for good.

Knock, Knock: Authentication

If you’re building a non-public API, you will need some kind of authentication.

Until a while ago, Devise came with the TokenAuthenticatable strategy for this purpose, but removed it for various reasons. Plus, why add a complicated new mechanism if HTTP already provides a great authentication mechanism with Basic authentication?

Authenticating with username and password can be handled by any consumer. HTTP Basic authentication is supposedly implemented in every HTTP client. Therefore, it works out of the box.

On the server side, Rails provides great support for basic authentication that is very easy to implement into your API controllers.

Besides this easy-to-use authentication method you should also allow users to sign in using private access tokens.

These tokens can be used in CI-server setups or similar programmatic use cases. It is always recommended when you’re handing the access to your private resources to some external services in order to control the access and not publishing your password.

Rate Limiting

When your users begin to use your API initially, you probably don’t have to worry about performance or resource limitation.

However, if your application is a success and thousands of users begin to integrate your API into their infrastructure and workflows, things can and will go wrong: Unexperienced developers will call your endpoint in endless loops, with incredibly high concurrency and badly configured cron jobs will request the same URL over and over again, thousands of times per hour.

This is why you should consider implementing a rate-limit early on.

Not only can this prevent your servers being taken down by a CI-server, it also gives your users an indication on how to use the API properly by recommending a maximum number of requests for a given time interval.

We have implemented a very simple rate-limit to some of the most resource intensive endpoints using Redis. For even larger infrastructures, a more sophisticated approach (e.g. via Nginx limit_req) might make sense.

Whatever approach you choose, make sure you display rate-limit information clearly to your consumers, e.g. via response headers such as Github does:

X-Rate-Limit-Limit: 1000

X-Rate-Limit-Remaining: 1920

X-Rate-Limit-Reset: 2015-03-10T13:00:00Z

HTTP Caching: Conditional GET

For any successful web service, caching is important to ensure great performance. HTTP comes with it’s own caching mechanism that is pretty straightforward to use in Rails.

Learn more about HTTP caching in Rails: Conditional GET

You should implement the caching protocol early on, so your consumers can incorporate it into their API clients. Once necessary, you can implement the necessary server-side logic. Methods to consider are self-invalidating cache keys,Russian Doll Caching and Reverse-Proxy-Caches (Varnish, CDNs). In our experience caching data structures is often wasteful due to the necessary Marshalling. No matter the layer you cache or what your cache may be: Keep an eye on the cache effectivness. If your resources’ change frequency is high, cached objects might never be actually served because they changed in the mean time.

Provide Awesome Documentation

This might be the hardest job: You must provide a good documentation!

It should be precise, correct and easy to understand. A single error or misunderstanding in a documentation can drive developers nuts and make them hate your product :)

Be nice and provide them with a documentation that you would want to read.

Developers love examples, so don’t be stingy with code snippets (e.g. cURL calls) that ideally just work right out of the box. Try to include them in your test cases, so that you make sure the documentation isn’t outdated when you update your API.

There are some really great tools out there that can help you with this process.

For smaller Rails projects, Apipie can be a great solution. We at Phrase however, use a custom-made solution built on top of nanoc that generates our API documentation with a single click.

Building a rich, REST-API is not that hard given a powerful framework such as Rails. However, to make working with your API fun for developers, you should invest enough time to think things through beforehand: You won’t be able to introduce new principles into your API infrastructure once it is used by hundreds or thousands of customers.

We encourage you to take a look around and see how other successful (tech) companies build and document their APIs. Learn from stars like GitHub, Stripe and others how great APIs should look.

Knowing the basic mechanisms of HTTP will also vastly improve your API design and implementation.

These resources helped us developing our new API with great best practices:

• GitHub v3 API Documentation
• Best Practices for Designing a Pragmatic RESTful API

Be sure to subscribe and receive all updates from the Phrase blog straight to your inbox. You’ll receive localization best practices, about cultural aspects of breaking into new markets, guides and tutorials for optimizing software translation and other industry insights and information. Don’t miss out!