Hide Wordpress Page

Hello,

I have a wordpress site that I would like to restrict access to the admin page. The server sits in AWS behind an ELB. I am setting up the location directive in my .conf file between the server {}'s. The config that I have set is not working as I can still access the page from an ip address which should be denied.

Can someone please give a look and give some guidance? Thank you!

location ~ ^/(wp-admin\.php) {
real_ip_header X-Forwarded-For;
set_real_ip_from <my elb ip address>;
allow <my office ip address>;
deny all;
}