Per country IP filtering

havoc

Before I reinvent the wheel...anybody got tools/scripts to take raw ASN/ARIN data and turn it into something that can be used to geoblock?

Keen to avoid reliance on external database sites/APIs. I want to take the raw data an process it.

Gang here seems like this may be something y'all know about for fraud prevention etc.

Neoon

Any geo data on the ASN can be unreliable as fuck.
You really have to go per subnet, and even then, if the subnet is to big, you have to split it and go for each of the subnets again.

Good luck.

yoursunny

CF-IPCountry header automatically appears in every HTTP request to the origin server.
It contains the country code.
No need for any scripts or databases.

grasple

raw ASN/ARIN

Keen to avoid reliance on external database sites/APIs

that obviously won't work, since ARIN is but one RIR, and whois data is perhaps the most useless source of geoip data around.

you can process a dump of the Maxmind DB I guess?

havoc

Thanks gents (ladies).

Interesting. Seems I'm coming at this from a naive angle. Glad I asked

@Neoon noted on unreliability, but would that still hold true if I'm coming at this from a white list angle.

i.e. A VPS provider on here needs do deal with god knows what from god knows where and can't assume anything.

My incoming IPs is gonna be residential on name brand ISPs so if they're not from a pretty clean ASN I'll just bounce them. I don't need to deal with edge cases & my default case is obviously good or obviously bad not grey.

havoc

@yoursunny said:
CF-IPCountry header automatically appears in every HTTP request to the origin server.
It contains the country code.
No need for any scripts or databases.

Surely a bad actor would spoof?

MallocVoidstar

@havoc said:

@yoursunny said:
CF-IPCountry header automatically appears in every HTTP request to the origin server.
It contains the country code.
No need for any scripts or databases.

Surely a bad actor would spoof?

If you're behind Cloudflare that is a header added by Cloudflare based on the incoming IP. Only Cloudflare can set it (again, if you're behind Cloudflare).

tetech

@havoc said:
Before I reinvent the wheel...anybody got tools/scripts to take raw ASN/ARIN data and turn it into something that can be used to geoblock?

Keen to avoid reliance on external database sites/APIs. I want to take the raw data an process it.

Gang here seems like this may be something y'all know about for fraud prevention etc.

To clarify, are you wanting to block by ASN or wanting to geoblock? I do both (in different circumstances).

For ASN, basically

for asn in $(awk '{print $1}' banned_asn.txt | grep -v ^$)
do
  echo "Processing AS${asn}"
  wget -q https://raw.githubusercontent.com/ipverse/asn-ip/master/as/${asn}/ipv4-aggregated.txt -O - | grep -v "^#" >> badasn.txt
done

For geo, yeah stuff can be inaccurate but still quite OK for many situations.

for geo in ${countries[@]}
do
  wget "http://www.ipdeny.com/ipblocks/data/aggregated/${geo}-aggregated.zone" -q -O - >> geoban.txt
  sleep 1
done
sort geoban.txt | uniq > /etc/haproxy/geoban.lst

Typically I combine geo with some type of white list. Not good enough to satisfy a legal obligation (e.g. your stuff is banned in a certain country) but good enough to filter out 99% of malicious stuff in countries that shouldn't even be accessing your server.

somik

I remember some GeoIP lookup database that converts IP into a country. Here is something I had created long ago:

Github: https://github.com/somik123/IP-to-Country

Try it here: https://ip2c.somik.org/?ip=172.67.155.79
(replace IP with something else)

The script is OLD so do expect some bugs/issues. Also set php memory limit to at least 256MB as this loads the entire DB to memory when updating the database. Add this to the bottom of your php.ini file:
memory_limit = 128M

Feel free to port it to any other language you prefer.

Neoon

@havoc said:
Thanks gents (ladies).

Interesting. Seems I'm coming at this from a naive angle. Glad I asked

@Neoon noted on unreliability, but would that still hold true if I'm coming at this from a white list angle.

i.e. A VPS provider on here needs do deal with god knows what from god knows where and can't assume anything.

My incoming IPs is gonna be residential on name brand ISPs so if they're not from a pretty clean ASN I'll just bounce them. I don't need to deal with edge cases & my default case is obviously good or obviously bad not grey.

Why would that make a difference?
Just one ISP or multiple?

Easiest way, whitelist your ISP, done.

Kolin

Try BPFire project