Help with Nginx GeoIP load balancing with failover

I am attempting to set up a test Nginx load balanced environment. So far I have sucessfully configured a load balancer nginx-balancer1 and 3 servers to serve webpages nginx1, nginx2 & nginx3.

I want to balance the load by region depending on the visitor's IP. I have configured my Nginx nginx-balancer1 to use the Maxmind GeoIP Country data.

So here is my configuration to the upstream servers:

### START

# Check where the user is coming from
server {
location / {
proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504 http_404;
proxy_connect_timeout 2;

if ($geoip_city_continent_code = "EU") {
proxy_pass http://ams1;
}
if ($geoip_city_continent_code = "NA") {
proxy_pass http://sfo1;
}
if ($geoip_city_continent_code = "AS") {
proxy_pass http://sgp1;
}

}
}

# Define upstream servers
upstream ams1 { server server1.example.com max_fails=3 fail_timeout=10s; }
upstream sfo1 { server server1.example.com max_fails=3 fail_timeout=10s; }
upstream sgp1 { server server1.example.com max_fails=3 fail_timeout=10s; }

### END

This seems to work well, however if I shutdown nginx on say ams1 (server1.example.com) and try to go to the main page I receive a 502 Bad Gateway error.

What I want to figure out is if a server is down, how can I get nginx-balancer1 to redirect to another server, either the next closest or the next functioning server.

Generic log error for this is: connect() failed (111: Connection refused) while connecting to upstream.

Can anybody help?

Thanks