Some Nginx+Lua Script Practices

Total Traffic Limiting (Circuit Breaking) for an Interface

Some friends may think of Nginx’s built-in limit_req feature to limit access frequency. That approach is usually IP-based, meant to prevent users from maliciously flooding requests. What I want to introduce here is another situation: the interface’s qps is already very high and the backend cannot accept more requests. At this point what’s needed is rate limiting for the entire interface.

I’ll paste a fairly simple implementation:

local limit = {}

local function safe_incr(dict, key, timeout)
    local ok
    local newval, err = dict:incr(key, 1)
    if not newval and err == "not found" then
        newval = 1
        ok, err = dict:safe_add(key, newval, timeout)
        if not ok then
            if err == "exists" then
                newval, err = dict:incr(key, 1)
            elseif err == "no memory" then
                dict:add(key .. "|no memory", 0, timeout)
            end
        end
    end
    return newval
end

function limit.limit(dict, key, freq, err_code, err_msg, content_type)
    dict:set(key .. '|freq', freq)

    local time = ngx.time()
    local k = key .. '|' .. tostring(time)
    local newval = safe_incr(dict, k, 70)

    if newval > freq then
        -- local limit_info = ngx.shared.limit_info
        -- safe_incr(limit_info, k, 70)

        ngx.sleep(1)
        ngx.status = err_code
        ngx.header.content_type = content_type or 'application/json'
        ngx.print(err_msg)
        ngx.exit(ngx.status)
    end
end

return limit

Then use it in the Nginx configuration like this:

lua_shared_dict xxx_0 10M;
server {
  listen                80;
  server_name           aaa.bbb.com;

  location = /redis/incr {
      access_by_lua_block {
        local limit = require("limit")
        limit.limit(ngx.shared.xxx_0, 'PSAvcmVkaXMvaW5jcg==', 5, ngx.HTTP_FORBIDDEN, '{"reason": "Request queued, please retry later...", "code": 500}', 'application/json')
      }
      proxy_pass http://xxxxx;
  }
}

Implementing Forced https Redirection

For ordinary websites, https redirection is usually implemented with rewrite, like the configuration below:

server {
    listen 80;
    ...
    return 301 https://$server_name$request_uri;
}

server {
    listen 443;
    ...
}

Such configuration only suits the outermost Nginx. If it’s not placed at the entry Nginx users access, you’ll get a circular redirection problem like the one shown below.

So for multi-layer Nginx situations, the simplest solution is doing the https redirection at the first-layer Nginx and using http traffic internally.

But consider the following situation:

Moreover, for a PaaS platform, not all applications want forced https, and the first-layer Nginx isn’t controlled by us — but it passes us a header like this indicating the request is https: X-Forwarded-Proto: https.

At this time a more suitable choice is adding request-method judgment in the platform’s own Nginx layer. Based on the header’s forwarded-proto, there are two ways to solve it in the middle layer:

1. One uses Nginx’s if statement

server{
  listen *:80;
  server_name mydomain.com www.mydomain.com; 

  if ($http_x_forwarded_proto = "http") {
      return 301 https://$server_name$request_uri;
  }
  // location xxx
}

2. A more elegant solution (using Lua):

server {
  listen                80;
  add_header Strict-Transport-Security "max-age=31536000" always;
  # using lua makes the logic clearer
  rewrite_by_lua_block {
    local _request_uri = ngx.var.request_uri
    local _host = ngx.var.host
    if ngx.var.http_x_forwarded_proto == nil or ngx.var.http_x_forwarded_proto == 'http' then
      return ngx.redirect('https://'.._host.._request_uri, ngx.HTTP_MOVED_PERMANENTLY)
    end
  }
  // location xxx
}

Since our project already used Nginx+Lua, we adopted the second solution directly. If you use plain Nginx, just use the if statement.