Misunderstanding of how ngx_http_access_module and ngx_http_auth_request_module work together

Hello guys!
I need to limit access to my API to the list of ip addresses and only for those users who successfully pass authentication held by third party module.

config file:
server {
listen 62081;
allow <ip_address>;
allow .......
deny all;
root /dev/null;

location / {
auth_request /auth;
.......
auth_request_set $usrcookie $upstream_http_set_cookie;
add_header Set-Cookie $usrcookie;
.......
proxy_pass http://<api_host_ip>;
}

location /auth {
proxy_pass http://127.0.0.1:20003/auth;
.......
}
}

I expected users not from the list of allowed ips to be rejected at the very beginning. Instead I see them accessing auth backend and, if authenticated successfully, being rejected later on with error in error.log:
[error] 28461#0: *7 access forbidden by rule while sending to client, client: <ip_address>

nginx version: nginx/1.4.7

Please give me some advice.
Thanks!