HTTP : 413 Entity Too Large

Keep your headers (including cookies) short!

Sometime back, on a typical Monday morning, I faced a production issue where one of the user could not able to access our site.

The webserver was rejecting all requests with 413.

After analysis, we found out that we were using dynamic headers for passing list of feature togglers (to indicate what features are available for the users) as part of http headers. Turned out few additional features were turned on over the weekend and this user is kinda of super user to avail most of the features.

This resulted in header size being larger than the Apache allowd limit of 8kb.

As a quickfix, we increased the Apache header size limit to 16kb thus allowing requests with larger header to pass through. For permanent solution, we decided to create dedicated REST API to refer to the feature flags as needed instead of passing them through as part of headers.

Is it a limitation of HTTP? 

No. HTTP specification does not limit length of headers at all.However most web servers do limit size of headers they accept. Depending on web-server and their settings these limits vary from 4KB to 64KB (total for all headers). For example in Apache default limit is 8KB.Server will return 413 Entity Too Large error if headers size exceeds that limit.

What are the default limits for commonly used web servers? 

Apache 2.0, 2.2: 8K
nginx: 4K - 8K
IIS: varies by version, 8K - 16K
Tomcat: varies by version, 8K - 48K

Why web servers need to pose this limit? 

Uncapped HTTP header size keeps the server exposed to attacks and can bring down its capacity to serve organic traffic.

What we learned? 

Do not pass anything and everything as part of headers especially if you know the size is dynamic and can potentially increase over time.

Review the headers and the max combinations and have of sense of max length as for most servers, this limit applies to the sum of the request line and ALL header fields (so keep your cookies short).

Courtesy: https://stackoverflow.com/questions/654921/how-big-can-a-user-agent-string-get

Please share your thoughts.