# 限流算法, 以 Golang 方式

## 速率限制

1. 恒定速率（Constant Bit Rate，CBR）
2. 变速速率（Varible Bit Rate，VBR）

## 实现方案

### 计数器方式

``````type counter struct {
Maximal int
Period  time.Duration
count   int
tick    int64 // in nanosecond
}

func (s *counter) Take(count int) bool {
if time.Now().UnixNano() > s.tick {
// if timeout, reset counter regally at first
s.count = 0
}

s.count += count            // it's acceptable in HPC scene
return s.count <= s.Maximal // it's acceptable in HPC scene
}

### 漏桶法（Leaky Bucket）

#### 基本原理

From: dev.to/swyx/networ…

#### 开源实现

``````// Makes it easy to test time based things.
var now = time.Now

// LeakyBucket represents a bucket that leaks at a constant rate.
type LeakyBucket struct {
// The identifying key, used for map lookups.
key string

// How large the bucket is.
capacity int64

// Amount the bucket leaks per second.
rate float64

// The priority of the bucket in a min-heap priority queue, where p is the
// exact time the bucket will have leaked enough to be empty. Buckets that
// are empty or will be the soonest are at the top of the heap. This allows
// for quick pruning of empty buckets that scales very well. p is adjusted
// any time an amount is added to the Queue().
p time.Time

// The index is maintained by the heap.Interface methods.
index int
}

func NewLeakyBucket(rate float64, capacity int64) *LeakyBucket {
return &LeakyBucket{
rate:     rate,
capacity: capacity,
p:        now(),
}
}

func (b *LeakyBucket) Add(amount int64) int64 {
count := b.Count()
if count >= b.capacity {
// The bucket is full.
return 0
}

if !now().Before(b.p) {
// The bucket needs to be reset.
b.p = now()
}
remaining := b.capacity - count
if amount > remaining {
amount = remaining
}
t := time.Duration(float64(time.Second) * (float64(amount) / b.rate))

return amount
}

#### 我们的版本

##### 非阻塞实现

``````func New(maxCount int64, d time.Duration) *leakyBucket {
return (&leakyBucket{
int64(maxCount),
make(chan struct{}),
int64(d) / int64(maxCount),
time.Now().UnixNano(),
0,
}).start(d)
}

type leakyBucket struct {
Maximal     int64
exitCh      chan struct{}
rate        int64
refreshTime int64 // in nanoseconds
count       int64
}

func (s *leakyBucket) Count() int64            { return atomic.LoadInt64(&s.count) }
func (s *leakyBucket) Available() int64        { return int64(s.count) }
func (s *leakyBucket) Capacity() int64         { return int64(s.Maximal) }

func (s *leakyBucket) Close() {
close(s.exitCh)
}

func (s *leakyBucket) start(d time.Duration) *leakyBucket {
if s.rate < 1000 {
log.Errorf("the rate cannot be less than 1000us, it's %v", s.rate)
return nil
}

// fmt.Printf("rate: %v\n", time.Duration(s.rate))

// go s.looper(d)
return s
}

func (s *leakyBucket) looper(d time.Duration) {
// nothing to do
}

func (s *leakyBucket) max(a, b int64) int64 {
if a < b {
return b
}
return a
}

func (s *leakyBucket) take(count int) (requestAt time.Time, ok bool) {
requestAt = time.Now()

s.count = s.max(0, s.count-(requestAt.UnixNano()-s.refreshTime)/s.rate*int64(count))
s.refreshTime = requestAt.UnixNano()

if s.count < s.Maximal {
s.count += int64(count)
ok = true
}

return
}

func (s *leakyBucket) Take(count int) (ok bool) {
_, ok = s.take(count)
return
}

func (s *leakyBucket) TakeBlocked(count int) (requestAt time.Time) {
var ok bool
requestAt, ok = s.take(count)
for !ok {
time.Sleep(time.Duration(s.rate - (1000 - 1)))
_, ok = s.take(count)
}
time.Sleep(time.Duration(s.rate-int64(time.Now().Sub(requestAt))) - time.Millisecond)
return
}

1. 首先你要通过非阻塞的 Take() 尝试获得准许
2. 如果未能获准通过：
1. 如果你不想抛弃请求，则自行建立一个 queue 来缓存和延后重新尝试获得准许。
2. 如果你需要抛弃越限请求，那么什么都不做好了
3. 对于获准的请求，在 Take() 返回后得到的是不经过匀速整形的出水。
##### 兼容于 uber-go/ratelimit

``````import "github.com/hedzr/rate/leakybucket"

func TestLeakyBucketLimiter(b *testing.T) {
var counter int
l := leakybucket.New(100, time.Second, false) // one req per 10ms
defer l.Close()
time.Sleep(300 * time.Millisecond)
prev := time.Now()
for i := 0; i < 20; i++ {
ok := l.TakeBlocked(1)
now := time.Now()
counter++
fmt.Println(i, now.Sub(prev))
prev = now
time.Sleep(1 * time.Millisecond)
}
b.Logf("%v requests allowed.", counter)
}

``````\$ go test -v -race -test.run='^TestLeakyBucketLimiter\$' ./leakybucket
==== RUN   TestLeakyBucketLimiter
0 9.468477ms
1 10.47512ms
2 12.589327ms
3 10.364861ms
4 11.820381ms
5 10.353834ms
6 12.504309ms
7 10.167151ms
8 11.623466ms
9 10.487361ms
10 10.72498ms
11 12.435839ms
12 11.73813ms
13 10.956558ms
14 11.893156ms
15 11.964504ms
16 11.856545ms
17 11.094344ms
18 10.978074ms
19 10.632583ms
lb_test.go:36: 20 requests allowed.
--- PASS: TestLeakyBucketLimiter (0.53s)
PASS
ok      github.com/hedzr/rate/leakybucket       0.957s

##### 面向 Public API - 简化的 Gin 中间件

``````func (r *Limiter) getRL(ctx gin.Context) rateapi.Limiter {
if r.limiter == nil {
r.limiter = leakybucket.New(100, time.Second, false)
}
return r.limiter
}

func (r *Limiter) SimpleMiddleware() gin.HandlerFunc {
return func(ctx *gin.Context) {
limiter := r.getRL(ctx)
if err != nil {
ctx.AbortWithError(429, err)
} else if limiter != nil {
if limiter.Take(1) {
ctx.Next()
} else {
err = errors.New("Too many requests")
ctx.AbortWithError(429, err)
}
} else {
ctx.Next()
}
}
}

##### Gin 中间件

``````import "github.com/hedzr/rate/middleware"

func engine() *gin.Engine {
r := gin.Default()
r.Use(middleware.NewLimiterForGin(time.Second, 100, func(ctx *gin.Context) (string, error){
if key != "" {
return key, nil
}
ctx.JSON(403, gin.H{"code": 2901, "message": "API key is missing"})
return "", errors.New("API key is missing")
}))
return r
}

### 令牌法（Token Bucket）

From: dev.to/swyx/networ…

``````func New(maxCount int64, d time.Duration) *tokenBucket {
return (&tokenBucket{
int32(maxCount),
d,
int64(d) / int64(maxCount),
make(chan struct{}),
int32(maxCount),
}).start(d)
}

type tokenBucket struct {
Maximal int32
period  time.Duration
rate    int64
exitCh  chan struct{}
count   int32
}

func (s *tokenBucket) Count() int32            { return atomic.LoadInt32(&s.count) }
func (s *tokenBucket) Available() int64        { return int64(s.count) }
func (s *tokenBucket) Capacity() int64         { return int64(s.Maximal) }

func (s *tokenBucket) Close() {
close(s.exitCh)
}

func (s *tokenBucket) start(d time.Duration) *tokenBucket {
if s.rate < 1000 {
log.Errorf("the rate cannot be less than 1000us, it's %v", s.rate)
return nil
}

go s.looper(d)
return s
}

func (s *tokenBucket) looper(d time.Duration) {
ticker := time.NewTicker(d / time.Duration(s.Maximal))
// fmt.Printf("token building spped is: 1req/%v\n", d/time.Duration(s.Maximal))
defer func() {
ticker.Stop()
}()
for {
select {
case <-s.exitCh:
return
case <-ticker.C:
if vn < s.Maximal {
continue
}

vn %= s.Maximal
if vn > 0 {
atomic.StoreInt32(&s.count, s.Maximal)
}
}
}
}

func (s *tokenBucket) take(count int) bool {
if vn := atomic.AddInt32(&s.count, -1*int32(count)); vn >= 0 {
return true
}
atomic.StoreInt32(&s.count, 0)
return false
}

func (s *tokenBucket) Take(count int) bool {
ok := s.take(count)
return ok
}

func (s *tokenBucket) TakeBlocked(count int) (requestAt time.Time) {
requestAt = time.Now().UTC()
ok := s.take(count)
for !ok {
time.Sleep(time.Duration(s.rate - (1000 - 1)))
ok = s.take(count)
}
// time.Sleep(time.Duration(s.rate-int64(time.Now().Sub(requestAt))) - time.Millisecond)
return requestAt
}

##### 其它特性

``````package main

import (
"fmt"
"github.com/hedzr/rate"
"time"
)

func main() {
// rate.LeakyBucket, rate.TokenBucket, ...
l := rate.New(rate.LeakyBucket, 100, time.Second)
for i := 0; i < 120; i++ {
ok := l.Take(1)
if !ok {
fmt.Printf("#%d Take() returns not ok, counter: %v\n", i, rate.CountOf(l))
time.Sleep(50 * time.Millisecond)
}
}
}

``````import (
"github.com/hedzr/rate"
"github.com/hedzr/rate/middleware"
)

func engine() *gin.Engine {
config := &middleware.Config{
Name:          "A",
Description:   "A",
Algorithm:     stirng(rate.TokenBucket),
Interval:      time.Second,
MaxRequests:   1000,
ExceptionKeys: nil,
}
r := gin.Default()
rg := r.Group("/api")
rg.Use(middleware.ForGin(config))
// ...

config2 := ...
return r
}

##### 对 looper 做增强

1. logN 或者 ln 加速器方案
2. 平滑变比匀速：以一个时间片为统计单位，时刻调整 rate 值，从而在突发流量消耗了大部分令牌之后，降低今后一段时间的令牌发放速度。在 Google Guava 中它被称为 SmoothBusrty。
3. 平滑预热匀速：又被称作 SmoothWarmUp。在一个时间片的前一部分时间里以较快速度发放较多令牌，此时发放令牌的速度呈现出负加速度；随后，剩余时间内剩余令牌以匀速放出。
4. 其它更多方式：取决于具体的生产场景，可以通过定制 looper 算法来提供更多的策略（示例代码中未予以实现）。

## 比较

• 计数器方式：最容易实现，但应变能力最差，效率较高
• 滑动窗口法：理论易于理解，即尽可能地缩小计量粒度，从而将计数器的应变能力平滑地提高。然而很难以代码高效实现。
• 漏桶法：较好的取舍，针对进项流量进行整形，（通常）以匀速放出流量。
• 令牌法：相对最优的取舍，针对出口进行整形，（通常或可选地）以近似匀速放出流量。非常容易拓展为针对业务场景进行整形定制。

1. 考虑是否采用异步方式（另行处理被抛弃流量）/ 非阻塞模式（由调用者自行处理被抛弃流量）
1. 异步方式不适用于 API 接口的限流场所或者用作 web server middlerware
2. 异步方式、或者非阻塞方式有最好的进项流量接收性能
2. 非阻塞模式或阻塞方式的选择

## TODO

1. HPC 场景中，速率限制需要被分布式地实现
2. 在特定的分布式场景中，分布式的速率限制可以考虑被提升到入口点（通常可能是负载均衡器，或者是 Microservice API Gateway 处）去同一个处理。
3. 即使是由 API GW 处理，依然是一个分布式的速率限制器。