Issue with HAProxy as a forward proxy

Discussion:

Vijay Bais

2018-11-06 10:06:52 UTC

Hello,

I'm using HAProxy 1.8 as a forward proxy with below configuration

<snip>

defaults
mode tcp
log global
option tcplog
option dontlognull
option http-server-close
#option forwardfor except 127.0.0.0/8
option redispatch
retries 3
timeout http-request 10s
timeout queue 1m
timeout connect 10s
timeout client 1m
timeout server 1m
timeout http-keep-alive 10s
timeout check 10s
maxconn 3000
default-server resolvers dns

resolvers dns
nameserver local 127.0.0.1:53
nameserver ns1 10.0.0.2:53
hold valid 1s

listen c1
bind *10.0.0.26:10001 <http://10.0.0.26:10001>*
mode tcp
option tcplog
server r1 *ifconfig.co:80 <http://ifconfig.co:80>* source *<my Public
IP>*

</snip>

But this fails with below log lines for any internet destination (both in
TCP and HTTP mode):

10.0.1.79:47437 [06/Nov/2018:09:35:31.170] c1 c1/r1 1/-1/0 0 SC 1/1/0/0/3
0/0
Cannot bind to source address before connect() for backend c1.

Whereas, if the destination is under my control (with my source public IP
fully whitelisted), then the flow works perfectly.

Any help to know the actual issue would be great.

Thanks,
Vijay B

Vijay Bais

2018-11-07 10:53:33 UTC

Permalink

Thanks for the reply!

Here, the haproxy process runs as a *haproxy* user itself.

Let me try it as root and revert back.

Hi Vijay.

Post by Vijay Bais
Hello,
I'm using HAProxy 1.8 as a forward proxy with below configuration
<snip>
defaults
mode tcp
log global
option tcplog
option dontlognull
option http-server-close
#option forwardfor except 127.0.0.0/8 <http://127.0.0.0/8>
option redispatch
retries 3
timeout http-request 10s
timeout queue 1m
timeout connect 10s
timeout client 1m
timeout server 1m
timeout http-keep-alive 10s
timeout check 10s
maxconn 3000
default-server resolvers dns
resolvers dns
nameserver local 127.0.0.1:53 <http://127.0.0.1:53>
nameserver ns1 10.0.0.2:53 <http://10.0.0.2:53>
hold valid 1s
listen c1
bind *10.0.0.26:10001 <http://10.0.0.26:10001>*
mode tcp
option tcplog
server r1 *ifconfig.co:80 <http://ifconfig.co:80>* source *<my

Public IP>*

Post by Vijay Bais
</snip>
But this fails with below log lines for any internet destination (both

in TCP

Post by Vijay Bais
10.0.1.79:47437 <http://10.0.1.79:47437> [06/Nov/2018:09:35:31.170]

c1 c1/r1

Post by Vijay Bais
1/-1/0 0 SC 1/1/0/0/3 0/0
Cannot bind to source address before connect() for backend c1.
Whereas, if the destination is under my control (with my source public

IP fully

Post by Vijay Bais
whitelisted), then the flow works perfectly.
Any help to know the actual issue would be great.

The snipped does not show the global section.
I think you will need to run HAProxy as root to be able to do this.
Do you run HAProxy as root?

Post by Vijay Bais
Thanks,
Vijay B

Regards
Aleks

Vijay Bais

2018-11-08 02:51:37 UTC

Permalink

Hello Aleksandar,

I tried running haproxy as root, but it still failed with same logs.

Btw, here's the global section used,
<snip>

global
log 127.0.0.1 local2
chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 4000
user root
group root
daemon
stats socket /var/lib/haproxy/stats
stats timeout 2m
master-worker
nbthread 10

</snip>

Please let me know if anything is missing here.

Thanks & Regards,
Vijay B

Post by Vijay Bais
Thanks for the reply!
Here, the haproxy process runs as a *haproxy* user itself.
Let me try it as root and revert back.

Hi Vijay.

Post by Vijay Bais
Hello,
I'm using HAProxy 1.8 as a forward proxy with below configuration
<snip>
defaults
mode tcp
log global
option tcplog
option dontlognull
option http-server-close
#option forwardfor except 127.0.0.0/8 <http://127.0.0.0/8
option redispatch
retries 3
timeout http-request 10s
timeout queue 1m
timeout connect 10s
timeout client 1m
timeout server 1m
timeout http-keep-alive 10s
timeout check 10s
maxconn 3000
default-server resolvers dns
resolvers dns
nameserver local 127.0.0.1:53 <http://127.0.0.1:53>
nameserver ns1 10.0.0.2:53 <http://10.0.0.2:53>
hold valid 1s
listen c1
bind *10.0.0.26:10001 <http://10.0.0.26:10001>*
mode tcp
option tcplog
server r1 *ifconfig.co:80 <http://ifconfig.co:80>* source *<my

Public IP>*

Post by Vijay Bais
</snip>
But this fails with below log lines for any internet destination (both

in TCP

Post by Vijay Bais
10.0.1.79:47437 <http://10.0.1.79:47437>

[06/Nov/2018:09:35:31.170] c1 c1/r1

Post by Vijay Bais
1/-1/0 0 SC 1/1/0/0/3 0/0
Cannot bind to source address before connect() for backend c1.
Whereas, if the destination is under my control (with my source public

IP fully

Post by Vijay Bais
whitelisted), then the flow works perfectly.
Any help to know the actual issue would be great.

The snipped does not show the global section.
I think you will need to run HAProxy as root to be able to do this.
Do you run HAProxy as root?

Post by Vijay Bais
Thanks,
Vijay B

Regards
Aleks

Willy Tarreau

2018-11-08 08:06:03 UTC

Permalink

Hello Vijay,

Post by Vijay Bais
Hello Aleksandar,
I tried running haproxy as root, but it still failed with same logs.
Btw, here's the global section used,
<snip>
global
log 127.0.0.1 local2
chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 4000
user root
group root
daemon
stats socket /var/lib/haproxy/stats
stats timeout 2m
master-worker
nbthread 10

First, I'd be very surprised if you'd need 10 threads for only 4000
connections, please do not add stuff you don't need, this only
complicates the troubleshooting and help people can give you.

Post by Vijay Bais
defaults

(...)

Post by Vijay Bais
default-server resolvers dns
resolvers dns
nameserver local 127.0.0.1:53 <http://127.0.0.1:53>
nameserver ns1 10.0.0.2:53 <http://10.0.0.2:53>
hold valid 1s
listen c1
bind *10.0.0.26:10001 <http://10.0.0.26:10001>*
mode tcp
option tcplog
server r1 *ifconfig.co:80 <http://ifconfig.co:80>* source *<my Public IP>*

So what this means is that the name "ifconfig.co" will be periodically
resolved using the two resolvers above and that all your connections
will be sent there from your public address. Please make sure that the
source address you're forcing is properly bound on your server, and
usable to reach the address corresponding to "ifconfig.co".

Last, I'm a bit surprised by your statement that you're trying to make a
forward proxy because at least haproxy is not a forward http proxy, so
the term is confusing. Your config makes me think that you want to use
it to always reach ifconfig.co whose address may change overtime, am I
right ?

I'm also seeing that this host resolves both in IPv4 and IPv6. There's
something to put in resolvers sections if you want to force v4 only I
guess but I forgot the option name, you may want to take a look there.

Regards,
Willy

Vijay Bais

2018-11-08 09:38:24 UTC

Permalink

Thanks a lot, Willy!

Seems the issue was with preference of resolvers itself.
After configuring it to prefer IPv4 as stated by you; it starting working
like a charm.

Ref:
https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#5.2-resolve-prefer

Thanks again for your help.

Post by Willy Tarreau
Hello Vijay,

First, I'd be very surprised if you'd need 10 threads for only 4000
connections, please do not add stuff you don't need, this only
complicates the troubleshooting and help people can give you.

Post by Vijay Bais
defaults

(...)

Public IP>*
So what this means is that the name "ifconfig.co" will be periodically
resolved using the two resolvers above and that all your connections
will be sent there from your public address. Please make sure that the
source address you're forcing is properly bound on your server, and
usable to reach the address corresponding to "ifconfig.co".
Last, I'm a bit surprised by your statement that you're trying to make a
forward proxy because at least haproxy is not a forward http proxy, so
the term is confusing. Your config makes me think that you want to use
it to always reach ifconfig.co whose address may change overtime, am I
right ?
I'm also seeing that this host resolves both in IPv4 and IPv6. There's
something to put in resolvers sections if you want to force v4 only I
guess but I forgot the option name, you may want to take a look there.
Regards,
Willy