Rate Limiter

Tarun Jain
3 min readMar 15, 2024

Rate Limiter?

  • First line of defense
  • Any incoming request is first consulted against rate limiter
  • if we are under limits, go through otherwise, reject with error (429)

Token Bucket implementation in Java

Fixed size rate limiter implemented using Redis

Source — https://youtu.be/gVVDo2h6DwA?si=qp8LQkLHYOZrTx6q

Distributed Rate limiter

Where to place RL service?

How to integrate rate limiting service


Functional Requirement

  • client can send a limited number of requests to a server within a window
  • client should get an error message if the defined threshold limit of request is crossed for a server single server or across different combinations of servers.

Non-Functional Requirements:

  1. The system should be highly available since it protects our service from external attacks.
  2. Performance is an important factor for any system. So, we need to be careful that rate limiter service should not add substantial latencies to the system.

Back of the envelope memory calculation

  • key value pair

key is userid/ip address

value is timestamp+count

Relational DB vs In Memory cache time