Quantcast
Channel: Nginx Forum - How to...
Viewing all articles
Browse latest Browse all 4759

S3 Proxy/Cache: invalidating requests

$
0
0
I am having a strange issue. I am trying to create a caching proxy server between my apps and a private s3 bucket.

Everything works fine if I disable the "proxy_cache" line, but if it is enabled (which I want to cache requests/images) everything breaks. It seems the entire structure/flow of requests is being mutated when caching is enabled and I don't understand why. Perhaps I am implementing caching wrong?

Here is my config, and I have included some information from tcpdumps to give you an idea about the character of the problem.

I am using the aws sdk to form requests and authentication is sent in the headers.

http {

proxy_cache_path /home/cache levels=1:2 keys_zone=S3_CACHE:10m inactive=24h max_size=500m;
proxy_temp_path /home/cache/tmp;

server {

server_name my-cache-server.com;
listen 80;

proxy_cache S3_CACHE;

location / {

proxy_buffering on;
proxy_pass http://MY_BUCKET.s3.amazonaws.com/;
proxy_pass_request_headers on;
}
}
}

I have attached tcpdumps viewed in wireshark to explain the request flow in both situations... with proxy_cache disabled (working) is the larger image.

With the cache diasbled everything flows properly. First HEAD is sent from client -> proxy -> s3, then GET is sent from client -> proxy -> s3 and then the images are returned.

But if I turn on the cache, The HEAD is sent from client -> proxy, then it stops, then a GET is sent from proxy -> S3.

What is going on here, and why isn't the request flow the same when I turn the proxy on, and how can I make it so?

I have also posted this issue on stackoverflow - http://stackoverflow.com/questions/26825933/nginx-proxy-cache-invalidating-request-to-s3/26830981?noredirect=1#comment42245919_26830981

Viewing all articles
Browse latest Browse all 4759

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>