Discussion:
Generic backend in HAProxy config with server options as placeholders
Vijay Bais
2018-11-14 07:46:09 UTC
Permalink
Hello,

We have a requirement wherein a single generic backend with server options
configured as placeholders, which will resolve on the fly or at runtime.

Currently, we have to define multiple backends (has to be hardcoded) and
select them using the *use_backend* keyword.

Kindly help us with this generic backend implementation in HAProxy and let
us know if its possible OR any alternative way that this can be achieved.

Thank you in advance,
Vijay B
Vijay Bais
2018-11-14 09:14:34 UTC
Permalink
Hello Aleksandar,

We already considered using haproxy maps but we still have to define N
backends for corresponding N keys in the map file.
I'm looking more at an implementation with single backend definition with
the server options as placeholders.

Ex. Using maps would look something like this

frontend nat
bind *:10000
use_backend %[req.hdr(X-MyHeader), map(/etc/haproxy/my.map)]

backend example1.com:80
server myserver1 example1.com:80 source 10.0.0.1

backend example2.com:80
server myserver2 example2.com:80 source 10.0.0.2

backend example3.com:80
server myserver3 example3.com:80 source 10.0.0.3



Whereas, we are looking for something like below


frontend nat
bind *:10000
default_backend generic

backend generic
server myserver %[req.hdr(X-MyHeader)] source %[dst]


Thanks,
Vijay
Post by Vijay Bais
Hi.
Post by Vijay Bais
Hello,
We have a requirement wherein a single generic backend with server
options
Post by Vijay Bais
configured as placeholders, which will resolve on the fly or at runtime.
Currently, we have to define multiple backends (has to be hardcoded) and
select
Post by Vijay Bais
them using the /use_backend/ keyword.
Kindly help us with this generic backend implementation in HAProxy and
let us
Post by Vijay Bais
know if its possible OR any alternative way that this can be achieved.
Maybe you can use maps for your requirement.
https://www.haproxy.com/blog/introduction-to-haproxy-maps/
As an example can you take a look at the openshift router template ;-)
https://github.com/openshift/origin/blob/master/images/router/haproxy/conf/haproxy-config.template#L201-L202
Post by Vijay Bais
Thank you in advance,
Vijay B
Regards
Aleks
Igor Cicimov
2018-11-14 22:37:09 UTC
Permalink
Hi Vijay.
Post by Vijay Bais
Hello Aleksandar,
We already considered using haproxy maps but we still have to define N
backends
Post by Vijay Bais
for corresponding N keys in the map file.
I'm looking more at an implementation with single backend definition
with the
Post by Vijay Bais
server options as placeholders.
Ex. Using maps would look something like this
frontend nat
bind *:10000
use_backend %[req.hdr(X-MyHeader), map(/etc/haproxy/my.map)]
backend example1.com
server myserver1 example1.com:80 source 10.0.0.1
backend example2.com
server myserver2 example2.com:80 source 10.0.0.2
backend example3.com
server myserver3 example3.com:80 source 10.0.0.3
Whereas, we are looking for something like below
frontend nat
bind *:10000
default_backend generic
backend generic
server myserver %[req.hdr(X-MyHeader)] source %[dst]
Ah now concrete examples ;-)
Maybe you can use the server template?!
https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#4-server-template
So you would like to have something like this, is this possible, I don't
think so?
backend generic
server-template myserver 1-3 %[req.hdr(X-MyHeader)]:80 check source
0.0.0.0
usesrc %[dst]
Which version of HAProxy do you use?
haproxy -vv
Post by Vijay Bais
Thanks,
Vijay
Regards
Aleks
Post by Vijay Bais
Hi.
Post by Vijay Bais
Hello,
We have a requirement wherein a single generic backend with server
options
Post by Vijay Bais
Post by Vijay Bais
configured as placeholders, which will resolve on the fly or at
runtime.
Post by Vijay Bais
Post by Vijay Bais
Currently, we have to define multiple backends (has to be
hardcoded) and
Post by Vijay Bais
select
Post by Vijay Bais
them using the /use_backend/ keyword.
Kindly help us with this generic backend implementation in HAProxy
and let us
Post by Vijay Bais
Post by Vijay Bais
know if its possible OR any alternative way that this can be
achieved.
Post by Vijay Bais
Maybe you can use maps for your requirement.
https://www.haproxy.com/blog/introduction-to-haproxy-maps/
As an example can you take a look at the openshift router template
;-)
https://github.com/openshift/origin/blob/master/images/router/haproxy/conf/haproxy-config.template#L201-L202
Post by Vijay Bais
Post by Vijay Bais
Thank you in advance,
Vijay B
Regards
Aleks
You will need something dynamic like described here
https://www.haproxy.com/blog/haproxy-and-consul-with-dns-for-service-discovery/
Vijay Bais
2018-11-15 04:07:58 UTC
Permalink
Hi Vijay.
Post by Vijay Bais
Hello Aleksandar,
We already considered using haproxy maps but we still have to define N
backends
Post by Vijay Bais
for corresponding N keys in the map file.
I'm looking more at an implementation with single backend definition
with the
Post by Vijay Bais
server options as placeholders.
Ex. Using maps would look something like this
frontend nat
bind *:10000
use_backend %[req.hdr(X-MyHeader), map(/etc/haproxy/my.map)]
backend example1.com
server myserver1 example1.com:80 source 10.0.0.1
backend example2.com
server myserver2 example2.com:80 source 10.0.0.2
backend example3.com
server myserver3 example3.com:80 source 10.0.0.3
Whereas, we are looking for something like below
frontend nat
bind *:10000
default_backend generic
backend generic
server myserver %[req.hdr(X-MyHeader)] source %[dst]
Ah now concrete examples ;-)
Maybe you can use the server template?!
https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#4-server-template
So you would like to have something like this, is this possible, I don't
think so?
backend generic
server-template myserver 1-3 %[req.hdr(X-MyHeader)]:80 check source
0.0.0.0
usesrc %[dst]
@Aleksandar Lazic <al-***@none.at> tried using server-template, but
it won't work
Which version of HAProxy do you use?
We are using v1.8.13
haproxy -vv
HA-Proxy version 1.8.13 2018/07/30
Copyright 2000-2018 Willy Tarreau <***@haproxy.org>

Build options :
TARGET = linux2628
CPU = generic
CC = gcc
CFLAGS = -O2 -g -fno-strict-aliasing -Wdeclaration-after-statement
-fwrapv -fno-strict-overflow -Wno-unused-label
OPTIONS = USE_LINUX_TPROXY=1 USE_ZLIB=1 USE_REGPARM=1 USE_OPENSSL=1
USE_LUA=1 USE_PCRE=1

Default settings :
maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with network namespace support.
Built with zlib version : 1.2.3
Running on zlib version : 1.2.8
Compression algorithms supported : identity("identity"),
deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with PCRE version : 7.8 2008-09-05
Running on PCRE version : 8.21 2011-12-12
PCRE library supports JIT : no (USE_PCRE_JIT not set)
Built with multi-threading support.
Encrypted password support via crypt(3): yes
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT
IP_FREEBIND
Built with Lua version : Lua 5.3.4
Built with OpenSSL version : OpenSSL 1.0.1e-fips 11 Feb 2013
Running on OpenSSL version : OpenSSL 1.0.1e-fips 11 Feb 2013
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : SSLv3 TLSv1.0 TLSv1.1 TLSv1.2

Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.

Available filters :
[TRACE] trace
[COMP] compression
[SPOE] spoe
Post by Vijay Bais
Thanks,
Vijay
Regards
Aleks
Post by Vijay Bais
Hi.
Post by Vijay Bais
Hello,
We have a requirement wherein a single generic backend with
server options
Post by Vijay Bais
Post by Vijay Bais
configured as placeholders, which will resolve on the fly or at
runtime.
Post by Vijay Bais
Post by Vijay Bais
Currently, we have to define multiple backends (has to be
hardcoded) and
Post by Vijay Bais
select
Post by Vijay Bais
them using the /use_backend/ keyword.
Kindly help us with this generic backend implementation in
HAProxy and let us
Post by Vijay Bais
Post by Vijay Bais
know if its possible OR any alternative way that this can be
achieved.
Post by Vijay Bais
Maybe you can use maps for your requirement.
https://www.haproxy.com/blog/introduction-to-haproxy-maps/
As an example can you take a look at the openshift router template
;-)
https://github.com/openshift/origin/blob/master/images/router/haproxy/conf/haproxy-config.template#L201-L202
Post by Vijay Bais
Post by Vijay Bais
Thank you in advance,
Vijay B
Regards
Aleks
You will need something dynamic like described here
https://www.haproxy.com/blog/haproxy-and-consul-with-dns-for-service-discovery/
Thanks Igor! Let me explore this and get back :)

Loading...