HAProxy - forward client IP

AlwaysSkintAlwaysSkint OGSenpai
edited April 2020 in Help

After @Neoon kindly pointed me in the direction of haproxy, for reverse proxy use, I've nearly got a full setup. I'm struggling with the last piece of the puzzle, even after trying to comprehend the vast options in the documentation.
I'm trying to determine the original client IP. Straight http requests can be interrogated the through x-forward but this isn't much good for system monitoring, for example. Also, a control panel in a VM only sees access coming from the host IP.
Here's a snippet of what I've got, sorry for the formatting (need code tagging):

frontend webmin
bind *:10000
option tcplog
mode tcp
option http-server-close
option forwardfor header X-Client
acl tls req.ssl_hello_type 1
tcp-request inspect-delay 5s
tcp-request content accept if tls

acl host_ks.domain1.com_webmin req.ssl_sni -i ks.domain1.com
use_backend ks.domain1.com_webmin if host_ks.domain1.com_webmin

backend ukc.domain1.com_webmin
server ukc.domain1.com 10.0.0.100:10000
mode tcp

Note: I had to change the host webmin port (not such a bad idea), as I couldn't get haproxy to ignore/pass it though. Below is currently commented out..

( in frontend webmin )
acl host_d3.domain1.com_webmin req.ssl_sni -i d3.domain1.com
use_backend d3.domain1.com_webmin if host_d3.domain1.com_webmin

backend d3.domain1.com_webmin
server d3.domain1.com 178.123.123.90:10000
mode tcp

I have a further VM but left that out, for clarity.
All other aspects appear to be fine, with CSF redirecting various ports and VMs able to access the 'net.

Oh, and thanks again @Neoon B)

It wisnae me! A big boy done it and ran away.
NVMe2G for life! until death (the end is nigh)

Tagged:

Comments

  • skorousskorous OGSenpai

    I'm not an HAProxy expert but I don't think there's an easy way to do that with straight TCP requests. There's just no provision for it in TCP. There is the PROXY protocol if your application supports it but I'm not sure what services do to be honest. I think I remember reading something about how you can put the backends on NAT'ed addresses and convince HAProxy to effectively spoof the source address. This completely isolates the backends though as HAProxy has to be the gateway otherwise the clients simply reply directly to the original IP. It's equally possible I'm delusional. I use HAProxy for some simple HTTP proxying and that's it.

  • comicomi OG

    If I understand your setup correctly, haproxy has nothing to do with this.

    Haproxy just forwards tcp packet and everything within. If http inside tcp contains x-forward family header it will be there after haproxy forwards it. It means it is webmin that needs to pay attention to it.

  • skorousskorous OGSenpai

    @comi said: If I understand your setup correctly, haproxy has nothing to do with this.

    Both his frontend and backend are running in tcp mode so ... ( confused) there's ... nothing injecting an x-forward-for header. Umm, is option forwardfor even valid in tcp mode? I just accepted you needed tcp mode and didn't question it.

  • NeoonNeoon OGSenpai

    You can add headers in the HTTP mode, but not in TCP.
    Because the package contents cannot be manipulated since they are encrypted and the HAProxy has not the keys for it, it just checks if its valid TLS traffic, reads the SNI header and forwards it.

    But it can manipulate the headers of the packages such as the IP address, which it does by default.
    Either you terminate TLS on the HAProxy or you enable transparent mode, which is more complicated.

    Thanked by (2)skorous AlwaysSkint

    Free NAT KVM | Free NAT LXC | Bobr
    ITS WEDNESDAY MY DUDES

  • AlwaysSkintAlwaysSkint OGSenpai
    edited April 2020

    Thanks guys. I added the "http-server--close" and "forward for" later, to see if it made any difference, which it didn't for tcp mode. For completeness, a CWP installation (the other VM) on https port 2031 also displays the host IP as "your IP".
    Is it perhaps the redirect used in interface startup and/or the redirects in CSF that aren't passing through the source IP?

    Interfaces snippet:

    post-up iptables -t nat -A PREROUTING -d 178.123.123.90 -p tcp --dport 40000 -j DNAT --to 10.0.0.100:10000

    plus this:

    post-up iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1

    csf.redirect snippet:

    178.123.123.90|40000|10.0.0.100|10000|tcp

    If I run a phpinfo on the above VM (ports 443,80), then:

    REMOTE_ADDR 178.123.123.90

    though X-CLIENT reports correctly for port 80

    Haproxy was added after the interfaces/CSF setup, to (attempt to) allow accessing services using default ports. I do note that accessing CWP via the NAT port 43031 also displays the host IP, instead of the client one.

    It wisnae me! A big boy done it and ran away.
    NVMe2G for life! until death (the end is nigh)

  • comicomi OG
    edited April 2020

    @skorous said: nothing injecting an x-forward-for header

    Okay, so I misunderstood, I assumed the header must come from somewhere before it hits haproxy.

    @AlwaysSkint said: Haproxy was added after the interfaces/CSF setup, to allow accessing services using default ports.

    Why not use destination nat with default ports? Are there multiple instances of same service?

    Can you use haproxy in http mode?

  • AlwaysSkintAlwaysSkint OGSenpai
    edited April 2020

    @Neoon said: you enable transparent mode, which is more complicated.

    This appears to be the way forward. Goes off to Yahoo! (Google), after food.

    @comi said: Why not use destination nat with default ports? Are there multiple instances of same service?

    if I understand correctly http mode only works for non-SSL.
    I wish to be able to access the various VM services, without specifying obscure ports. For example: https://ks.domain1.com/nextcloud rather than the previously used https://ks.domain1.com:40443/nextcloud (which is still possible, of course).
    Extending this further, say three VMs serving standard port 443 https, would not be possible without some form of redirection per VM.

    Wholeheartedly admit my total lack of knowledge in this area. ;) :'(

    It wisnae me! A big boy done it and ran away.
    NVMe2G for life! until death (the end is nigh)

  • skorousskorous OGSenpai

    @AlwaysSkint said: if I understand correctly http mode only works for non-SSL.

    What do you mean only works for non-ssl?

  • skorousskorous OGSenpai

    @AlwaysSkint said: Extending this further, say three VMs serving standard port 443 https, would not be possible without some form of redirection per VM.

    You use SNI and set up ACLs to route traffic via the http header.

  • NeoonNeoon OGSenpai

    @skorous said:

    @AlwaysSkint said: if I understand correctly http mode only works for non-SSL.

    What do you mean only works for non-ssl?

    http should work if you terminate the TLS connections at HAProxy but that means HAProxy would need to have the keys.

    Free NAT KVM | Free NAT LXC | Bobr
    ITS WEDNESDAY MY DUDES

  • skorousskorous OGSenpai
    edited April 2020

    @AlwaysSkint said: Extending this further, say three VMs serving standard port 443 https, would not be possible without some form of redirection per VM.

    Something like this:

    frontend https
    bind *:443 ssl crt /etc/haproxy/certs/haproxy.pem

        acl sonarr_url          hdr_beg(host)        sonarr.
    acl radarr_url          hdr_beg(host)        radarr.
    acl lidarr_url          hdr_beg(host)        lidarr.

    use_backend     sonarr                  if sonarr_url
    use_backend     lidarr                  if lidarr_url
    use_backend     radarr                  if radarr_url

    ( Can't get the formatting right but you get the point )

  • AlwaysSkintAlwaysSkint OGSenpai

    Sounds like another can of worms, getting LE certs in the right place, renewed etc. :-|

    It wisnae me! A big boy done it and ran away.
    NVMe2G for life! until death (the end is nigh)

  • skorousskorous OGSenpai

    I'm used to doing it with dedicated certs but there are documents on how to set it up with HAProxy. Didn't look too bad to do but I, uh, might've cheated and used my webhost to create a wildcard. I'm waiting to see if when they renew it I have to refresh it my haproxy.pem. I'll know in June. :)

  • comicomi OG

    @AlwaysSkint okay, got you.

    I believe you need nginx here - much more natural to terminate all https on nginx and forward to whatevs.
    Things that are not http/s can be just dnatted since you don't need load balancing or smth I suppose.

  • AlwaysSkintAlwaysSkint OGSenpai
    edited April 2020

    @comi I started with nginx reverse proxy (not without some issues, in another thread *), then we can blame @Neoon for pointing me in the haproxy direction. :p

    It wisnae me! A big boy done it and ran away.
    NVMe2G for life! until death (the end is nigh)

  • skorousskorous OGSenpai

    @comi said: I believe you need nginx here - much more natural to terminate all https on nginx and forward to whatevs.

    If you're not using the caching pieces of nginx I they're about the same and personally I find HAProxy config files easier to read.

  • comicomi OG

    @AlwaysSkint said:
    @comi I started with nginx reverse proxy (not without some issues, in another thread *), then we can blame @Neoon for pointing me in the haproxy direction. :p

    @skorous said:

    @comi said: I believe you need nginx here - much more natural to terminate all https on nginx and forward to whatevs.

    If you're not using the caching pieces of nginx I they're about the same and personally I find HAProxy config files easier to read.

    Sorry, my bad, you can just as well terminate everything at haproxy, and setup the forward header. But everything behind haproxy need to read the header. If it can't you need to apply some jutsu for transparency.

    So yeah, basically what @Neoon said.

    Thanked by (1)skorous
Sign In or Register to comment.

© LowEndSpirit 2024

ipv6 ready