Best Practices: Working with Data and APIs

On this page

• Caching Data
  • Data Source Caching
  • Client (Application) Side Caching
• Error Handling

Caching Data

Caching is a broad topic and can be seen at the memory, storage or communication layers by either side of a client/server. In this context, the server side is the data source and the client side is the developer or user who is trying to download the data.

Data Source Caching

Some data sources may use caching to help deal with scalability. As previously mentioned, this caching may be found at different levels within their service. This is typically transparent to end users but it is important to understand these practices to efficiently and respectfully interact with the data source.

If your requests continuously yield the same response yet you are expecting an updated/newer dataset then consider the following:

• You have verified that there is no caching occurring within your tools or framework.
• There is a reasonable expectation that the data source should return a revised response.
• You have looked at the data source’s documentation for information regarding timestamps or cached response rules.
• You have tried changing your query to see if the response differs.
• Try waiting some time to see if the data gets refreshed at a later time.

In some cases, the data source might be using a Content Delivery Network (CDN) to cache common responses. Look at the HTTP headers to see hints of a CDN or other caching mechanisms being used.

Client (Application) Side Caching

You may find optimization opportunities within your application to cache data in part or in its entirety. Caching can dramatically increase the performance of your application and deliver significant improvements to user experiences. Of course, caching on the client-side requires a balance between performance and accuracy as client-side caches can become outdated.

Error Handling

As a general rule for datasets and APIs, it is important to recognize and respectfully react to HTTP response codes. The following is a summary of some of the common HTTP response codes you may encounter.