Reference
More
Rate Limiting

Rate Limiting

Oso Cloud sandbox accounts all run on shared infrastructure. As such, there are controls in place to ensure that all sandbox users have a good experience using the platform. Among these are limits on the number of concurrent requests for any Environment.

Types of limits

There are two different limits in place: one on the number of overall requests, and another on the number of write requests. Note that the latter is a subset of the former, meaning that write requests count toward both limits. As such the limit on overall requests is much higher than the limit on writes.

Overall requests

This is simply a limit on the number of requests a particular Environment will handle before it begins to respond with 429 Too Many Requests.

Write requests

This limits the number of writes an Environment will handle before it begins to respond with 429 Too Many Requests. Fact addition and deletion are covered by this rate limit.

What to do if you've been rate limited

If you exceed either rate limit, you will begin to receive responses from Oso Cloud with the status code 429 Too Many Requests.

The easiest thing to do is reduce the volume of requests you are sending Oso Cloud and wait for the limit to expire! Rate limiting is very transient; it will expire in less than a minute and has no "memory" (i.e. you won't end up on some sort of list of users that are more likely to be rate limited in the future).

Obviously this isn't always possible, so other options include:

Use bulk_tell instead of tell

This is the most common cause of rate limiting. Thankfully it's also the easiest to fix.

Rather than distributing related facts across multiple requests, you can use the client bulk methods to atomically write multiple, related facts (limit of

  1. in a singular request. In addition to transactional semantics, this has the benefit of reducing the number of write requests you may need to send data to Oso Cloud.

Even though we are are writing three facts, this is just one write request:

NodePythonGoRuby
await oso.bulkTell([
  ["has_role", {type: "User", id: "1"}, "member", {type: "Organization", id: "acme"}], 
  ["has_role", {type: "User", id: "2"}, "member", {type: "Organization", id: "acme"}], 
  ["has_role", {type: "User", id: "3"}, "member", {type: "Organization", id: "acme"}]
]);

Talk to an Oso Engineer

Whether you want to pair on refactoring a policy or talk about caching strategies, there are lots of ways to reduce the number of requests your application sends Oso Cloud that we'd be more than happy to help with.

If you'd like help with rate limiting (or anything else!) schedule a 1x1 with an Oso engineer.

Last updated on September 30, 2022