Introducing a Failsafe Origin for 404s and Other Error Codes

Usually, the way to handle error codes when requesting the CDN file is by activating the “Use Stale” feature. This allows for expired content to be served when the origin returns one of the errors listed below:

• 500
• 502
• 503
• 504
• 403
• 404

Instead of activating the "Use Stale" feature, you can introduce a failsafe origin that will serve fresh content in the wake of 404 responses from the origin. You can do this by using MaxCDN EdgeRules to create a few rules to handle this behavior. Since this is a response with two rules in place, let's explain the use case first.

Use Case

We want to serve fresh content that was requested from the origin that does NOT have that file. So, by hitting origin #1 (the primary origin) through the CDN, we are being served with status code 404 and based on that, we want to use new origin #2 to actually serve this file. As mentioned, we’ll need two rules in place:  

What EdgeRules Are Needed?

The first rule will match on the default location (/) and have proxy_intercept_errors enabled so we can catch errors and have the error_page directive to locally rewrite the request to our “/fix” location. With this in place, we can fetch content from the new origin. When the error_page directive sends a 404 request to the "/fix" location, we are using two directives to pull from the new origin.

In a real life example that would look like this:

Rule #1 - Catching Errors

Nginx Configuration Block

location / {
		proxy_intercept_errors on;
		error_page 404 =200 /fix$request_uri;
}

Rule #2 - Alter proxy_pass

Nginx Configuration Block

location ~ /fix(.*)$ {
		proxy_set_header Host "neworigin.com";
		proxy_pass http://12.12.12.12$1;
}

 

How Does This Work?

1. A request for a file comes to the default location (/) asking for a non-existing file.
2. We intercept the 404 error and rewrite this request as the same, only with the “/fix” prefix.
3. When this (modified) request reaches the “/fix*” location, we call the new origin via proxy_pass and pass the modified Host header so we can properly reach the new origin server.
4. After this step, a file from the failsafe origin is served and the status code is recorded as a 200.

CURL Example Without Failsafe Origin

$ curl -I http://cdn.domain.com/image.jpg
HTTP/1.1 404 Not Found
Server: NetDNA-cache/2.2

CURL Example with Failsafe Origin

$ curl -I http://cdn.domain.com/image.jpg
HTTP/1.1 200 OK
Date: Fri, 18 Sep 2015 15:30:52 GMT
Content-Type: image/jpeg
Content-Length: 277077
Server: NetDNA-cache/2.2
X-Cache: HIT
Accept-Ranges: bytes

 

Using Failsafe Origin

For this particular case, we want to find all image files that have one or two of these substrings in the URI:

• substring1
• substring2

Regex that matches this case is as follows:

^\/((.*(substring2|substring2).*\.(jpg|jpeg|gif|png|bmp|ico)).*)$

The regex above matches the URI that starts with anything:

^\/((.*

It looks up the group of one or both under this array:

(substring2|substring2)

Then looks up any other string between the found group above and the file type (extension) defined as follows:

.*\.(jpg|jpeg|gif|png|bmp|ico)).*)

And treats what's found as the end of the URI string:

$

If we keep this setup and apply the scenario from the original case, this will produce a 404 status code. This is because when we rewrite the image URI locally to “/fix/*” it's STILL going to be handled by the same Nginx configuration block and NOT by the configuration block that searches for “/fix”. Here's how it works:

1. Request for a file comes to the default location (/) asking for a non-existing file.
2. We intercept the 404 error and rewrite this request the same only with the “/fix” prefix.
3. When rewritten, this request is handled by the same location block and not by the “/fix*” location block, thus causing a 404.

To make sure this doesn’t happen, we extend the regex so it matches everything the same way but exclude the “/fix” pattern:

^\/(((?!fix).*(wp|content).*\.(jpg|jpeg|gif|png|bmp|ico)).*)$

Rule #1 - Catching Error Code and Rewriting It Locally

Nginx Configuration Block

location ~ ^\/(((?!fix).*(wp|content).*\.(jpg|jpeg|gif|png|bmp|ico)).*)$ {                          
		error_page 404 =200 /fix$request_uri;
		proxy_intercept_errors on;
}

Rule #2 - Alter proxy_pass

Nginx Configuration Block

location ~ /fix(.*)$ {                          
		add_header ID $1;
		proxy_set_header Host "neworigin.com";
		proxy_pass http://12.12.12.12$1;
}