Willy Tarreau
2016-11-09 23:18:23 UTC
Hi,
HAProxy 1.7-dev6 was released on 2016/11/09. It added 61 new commits
after version 1.7-dev5.
I must say I'm *really* happy because we managed to merge all the stuff
that was still pending in dev5 and all persons involved managed to
assign some time to get their stuff merged, which I do appreciate given
that we're all pretty busy at this period of the year.
There are still quite some important changes but with limited impact
on existing code since most of these changes were performed in side
areas. The 4 main changes are :
- ability to start a server whose address doesn't resolve, to decide how
it must resolve upon startup, and to let the DNS or the CLI set its IP
address later. This is achieved thanks to the new "init-addr" server
setting. This implies that server address resolution is now delayed to a
later moment in the boot sequence and that it will now be possible to
pre-configure pools of unused servers that can be populated at run time
when needed. This brings extra interesting improvements that we didn't
think about. The first one is that when a config has errors, now you get
all resolution errors at once instead of having to edit the file one
line at a time and to try again. The second one is that it's now trivial
to completely disable server address resolution upon failure, so we
added a new debug option for this (-dr). That's convenient for people
who, like me, often face configs which don't resolve in their
environment and still want to validate the parsing. Please refer to the
doc for this. [work done by Baptiste and me]
- a DNS resolution failure can now finally bring a server down once the
hold time is expired. This has been missing in 1.6 and had for
consequence that traffic could be sent to a wrong server if the address
was reassigned to someone else. This combined with init-addr above will
provide an interesting method to transparently enable/disable servers in
dynamic farms. [work done by Baptiste]
- initial support for OpenSSL 1.1.0 was added. It builds with some warnings
remining that parts of the old API are now deprecated, but it seems to
work. Compatibility with OpenSSL 1.0.1/1.0.2 was maintained and assured
via a compatibility file mapping the new API to the old one. At this
moment, OpenSSL 0.9.8 doesn't build anymore. It doesn't seem terribly
complicated to fix but as usual in this situations it's a painful
process and we preferred to focus on the other pending stuff given that
0.9.8 is not supported anymore. However if someone is willing to address
this, patches are more than welcome! I suggest to add a distinct section
in the openssl-compat file for 0.9.8 as its API differs from 1.0.x. Distro
maintainers might be interested in giving it a try on their next distros.
[work done by Dirkjan Bussink]
- and the new stream processing offload engine (SPOE). Yes, we had to give
it a name. And the protocol is called SPOP. This is what allows haproxy
to offload some of its processing to external processes which can apply
some actions and set variables. There are a few things that really
please me here. The first one obviously is that it was completed in
time. Kudos to Christopher on this one! The next one is that I
personally find the design quite clean and we left some room to improve
the protocol later if needed, and to improve our first implementation of
the protocol without breaking backwards compatibility. The next one is
that the code lies in its own file without affecting the code at all, it
solely relies on the new filters infrastructure, which at the same time
starts to proves its maturity, and this is great. The last one is that
there's quite an extensive doc and even an example of external agent to
be used as a starting point to move your processing outside. Most likely
the first use cases will be to implement various forms of authentication
or content inspection. We're obviously interested in feedback here.
Those not using it don't have to fear any side effect. More info here :
http://www.haproxy.org/download/1.7/doc/SPOE.txt
We also now have a third device detection engine, WURFL, contributed by
Scientiamobile. The code is clean and well isolated so it was not a problem
to merge it this late in the release process. I took this opportunity to
clean up our README by moving the parts specific to DeviceAtlas and
51Degrees to their own file as well because they used to represent 1/3 of
the whole file.
Aside this we fixed the last pending bugs around the systemd wrapper and
the issue I introduced in 1.6 when porting the peers to the new applet
subsystem causing some connections to stay there forever and prevening
old processes from disappearing sometimes upon reload. The drain state
is now properly restored upon reload when using the state-file.
Finally some minor performance improvements were brought to the HTTP parser
for large requests or responses (eg: long URLs, huge cookies). I've observed
up to 10% increase in request rate with 1kB cookies and 100-char URIs.
The goal now really is to test this version and to release it with minimal
changes in 1-2 weeks depending on feedback and bug reports. Yes that's short,
so if you have a few minor pending patches that you'd like to get merged in
1.7, send them NOW. There are still a number of things I'd like to see better
arranged, so cleanups and code moves may still happen, and still are welcome,
but we must not perform other important changes now. Please if you want to
touch anything in dumpstats.c, notify William who is trying to tidy all this
horrible mess by moving all non-stats parts to their relevant files (no code
change, just functions being reshuffled around).
#############################################################################################
Please find the usual URLs below :
Site index : http://www.haproxy.org/
Discourse : http://discourse.haproxy.org/
Sources : http://www.haproxy.org/download/1.7/src/
Git repository : http://git.haproxy.org/git/haproxy.git/
Git Web browsing : http://git.haproxy.org/?p=haproxy.git
Changelog : http://www.haproxy.org/download/1.7/src/CHANGELOG
Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/
Happy testing and thanks to participants!
Willy
---
Complete changelog :
- DOC: fix the entry for hash-balance-factor config option
- DOC: Fix typo in description of `-st` parameter in man page
- CLEANUP: cfgparse: Very minor spelling correction
- MINOR: examples: Update haproxy.spec URLs to haproxy.org
- BUG/MEDIUM: peers: on shutdown, wake up the appctx, not the stream
- BUG/MEDIUM: peers: fix use after free in peer_session_create()
- MINOR: peers: make peer_session_forceshutdown() use the appctx and not the stream
- MINOR: peers: remove the pointer to the stream
- BUG/MEDIUM: systemd-wrapper: return correct exit codes
- DOC: stats: provide state details for show servers state
- MEDIUM: tools: make str2ip2() preserve existing ports
- CLEANUP: tools: make ipcpy() preserve the original port
- OPTIM: http: move all http character classs tables into a single one
- OPTIM: http: improve parsing performance of long header lines
- OPTIM: http: improve parsing performance of long URIs
- OPTIM: http: optimize lookup of comma and quote in header values
- BUG/MEDIUM: srv-state: properly restore the DRAIN state
- BUG/MINOR: srv-state: allow to have both CMAINT and FDRAIN flags
- MINOR: server: do not emit warnings/logs/alerts on server state changes at boot
- BUG/MEDIUM: servers: properly propagate the maintenance states during startup
- MEDIUM: wurfl: add Scientiamobile WURFL device detection module
- DOC: move the device detection modules documentation to their own files
- CLEANUP: wurfl: reduce exposure in the rest of the code
- MEDIUM: ssl: Add support for OpenSSL 1.1.0
- MINOR: stream: make option contstats usable again
- MEDIUM: tools: make str2sa_range() return the FQDN even when not resolving
- MINOR: init: move apply_server_state in haproxy.c before MODE_CHECK
- MAJOR: server: postpone address resolution
- MINOR: new srv_admin flag: SRV_ADMF_RMAINT
- MINOR: server: indicate in the logs when RMAINT is cleared
- MINOR: stats: indicate it when a server is down due to resolution
- MINOR: server: make srv_set_admin_state() capable of telling why this happens
- MINOR: dns: implement extra 'hold' timers.
- MAJOR: dns: runtime resolution can change server admin state
- MEDIUM: cli: leave the RMAINT state when setting an IP address on the CLI
- MEDIUM: server: add a new init-addr server line setting
- MEDIUM: server: make use of init-addr
- MINOR: server: implement init-addr none
- MEDIUM: server: make libc resolution failure non-fatal
- MINOR: server: add support for explicit numeric address in init-addr
- DOC: add some documentation for the "init-addr" server keyword
- MINOR: init: add -dr to ignore server address resolution failures
- MEDIUM: server: do not restrict anymore usage of IP address from the state file
- BUG: vars: Fix 'set-var' converter because of a typo
- CLEANUP: remove last references to 'ruleset' section
- MEDIUM: filters: Add attch/detach and stream_set_backend callbacks
- MINOR: filters: Update filters documentation accordingly to recent changes
- MINOR: filters: Call stream_set_backend callbacks before updating backend stats
- MINOR: filters: Remove backend filters attached to a stream only for HTTP streams
- MINOR: flt_trace: Add hexdump option to dump forwarded data
- MINOR: cfgparse: Add functions to backup and restore registered sections
- MINOR: cfgparse: Parse scope lines and save the last one parsed
- REORG: sample: move code to release a sample expression in sample.c
- MINOR: vars: Allow '.' in variable names
- MINOR: vars: Add vars_set_by_name_ifexist function
- MEDIUM: vars: Add a per-process scope for variables
- MINOR: vars: Add 'unset-var' action/converter
- MAJOR: spoe: Add an experimental Stream Processing Offload Engine
- MINOR: spoe: add random ip-reputation service as SPOA example
- MINOR: spoe/checks: Add support for SPOP health checks
- DOC: update ROADMAP file
---
HAProxy 1.7-dev6 was released on 2016/11/09. It added 61 new commits
after version 1.7-dev5.
I must say I'm *really* happy because we managed to merge all the stuff
that was still pending in dev5 and all persons involved managed to
assign some time to get their stuff merged, which I do appreciate given
that we're all pretty busy at this period of the year.
There are still quite some important changes but with limited impact
on existing code since most of these changes were performed in side
areas. The 4 main changes are :
- ability to start a server whose address doesn't resolve, to decide how
it must resolve upon startup, and to let the DNS or the CLI set its IP
address later. This is achieved thanks to the new "init-addr" server
setting. This implies that server address resolution is now delayed to a
later moment in the boot sequence and that it will now be possible to
pre-configure pools of unused servers that can be populated at run time
when needed. This brings extra interesting improvements that we didn't
think about. The first one is that when a config has errors, now you get
all resolution errors at once instead of having to edit the file one
line at a time and to try again. The second one is that it's now trivial
to completely disable server address resolution upon failure, so we
added a new debug option for this (-dr). That's convenient for people
who, like me, often face configs which don't resolve in their
environment and still want to validate the parsing. Please refer to the
doc for this. [work done by Baptiste and me]
- a DNS resolution failure can now finally bring a server down once the
hold time is expired. This has been missing in 1.6 and had for
consequence that traffic could be sent to a wrong server if the address
was reassigned to someone else. This combined with init-addr above will
provide an interesting method to transparently enable/disable servers in
dynamic farms. [work done by Baptiste]
- initial support for OpenSSL 1.1.0 was added. It builds with some warnings
remining that parts of the old API are now deprecated, but it seems to
work. Compatibility with OpenSSL 1.0.1/1.0.2 was maintained and assured
via a compatibility file mapping the new API to the old one. At this
moment, OpenSSL 0.9.8 doesn't build anymore. It doesn't seem terribly
complicated to fix but as usual in this situations it's a painful
process and we preferred to focus on the other pending stuff given that
0.9.8 is not supported anymore. However if someone is willing to address
this, patches are more than welcome! I suggest to add a distinct section
in the openssl-compat file for 0.9.8 as its API differs from 1.0.x. Distro
maintainers might be interested in giving it a try on their next distros.
[work done by Dirkjan Bussink]
- and the new stream processing offload engine (SPOE). Yes, we had to give
it a name. And the protocol is called SPOP. This is what allows haproxy
to offload some of its processing to external processes which can apply
some actions and set variables. There are a few things that really
please me here. The first one obviously is that it was completed in
time. Kudos to Christopher on this one! The next one is that I
personally find the design quite clean and we left some room to improve
the protocol later if needed, and to improve our first implementation of
the protocol without breaking backwards compatibility. The next one is
that the code lies in its own file without affecting the code at all, it
solely relies on the new filters infrastructure, which at the same time
starts to proves its maturity, and this is great. The last one is that
there's quite an extensive doc and even an example of external agent to
be used as a starting point to move your processing outside. Most likely
the first use cases will be to implement various forms of authentication
or content inspection. We're obviously interested in feedback here.
Those not using it don't have to fear any side effect. More info here :
http://www.haproxy.org/download/1.7/doc/SPOE.txt
We also now have a third device detection engine, WURFL, contributed by
Scientiamobile. The code is clean and well isolated so it was not a problem
to merge it this late in the release process. I took this opportunity to
clean up our README by moving the parts specific to DeviceAtlas and
51Degrees to their own file as well because they used to represent 1/3 of
the whole file.
Aside this we fixed the last pending bugs around the systemd wrapper and
the issue I introduced in 1.6 when porting the peers to the new applet
subsystem causing some connections to stay there forever and prevening
old processes from disappearing sometimes upon reload. The drain state
is now properly restored upon reload when using the state-file.
Finally some minor performance improvements were brought to the HTTP parser
for large requests or responses (eg: long URLs, huge cookies). I've observed
up to 10% increase in request rate with 1kB cookies and 100-char URIs.
The goal now really is to test this version and to release it with minimal
changes in 1-2 weeks depending on feedback and bug reports. Yes that's short,
so if you have a few minor pending patches that you'd like to get merged in
1.7, send them NOW. There are still a number of things I'd like to see better
arranged, so cleanups and code moves may still happen, and still are welcome,
but we must not perform other important changes now. Please if you want to
touch anything in dumpstats.c, notify William who is trying to tidy all this
horrible mess by moving all non-stats parts to their relevant files (no code
change, just functions being reshuffled around).
#############################################################################################
Please find the usual URLs below :
Site index : http://www.haproxy.org/
Discourse : http://discourse.haproxy.org/
Sources : http://www.haproxy.org/download/1.7/src/
Git repository : http://git.haproxy.org/git/haproxy.git/
Git Web browsing : http://git.haproxy.org/?p=haproxy.git
Changelog : http://www.haproxy.org/download/1.7/src/CHANGELOG
Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/
Happy testing and thanks to participants!
Willy
---
Complete changelog :
- DOC: fix the entry for hash-balance-factor config option
- DOC: Fix typo in description of `-st` parameter in man page
- CLEANUP: cfgparse: Very minor spelling correction
- MINOR: examples: Update haproxy.spec URLs to haproxy.org
- BUG/MEDIUM: peers: on shutdown, wake up the appctx, not the stream
- BUG/MEDIUM: peers: fix use after free in peer_session_create()
- MINOR: peers: make peer_session_forceshutdown() use the appctx and not the stream
- MINOR: peers: remove the pointer to the stream
- BUG/MEDIUM: systemd-wrapper: return correct exit codes
- DOC: stats: provide state details for show servers state
- MEDIUM: tools: make str2ip2() preserve existing ports
- CLEANUP: tools: make ipcpy() preserve the original port
- OPTIM: http: move all http character classs tables into a single one
- OPTIM: http: improve parsing performance of long header lines
- OPTIM: http: improve parsing performance of long URIs
- OPTIM: http: optimize lookup of comma and quote in header values
- BUG/MEDIUM: srv-state: properly restore the DRAIN state
- BUG/MINOR: srv-state: allow to have both CMAINT and FDRAIN flags
- MINOR: server: do not emit warnings/logs/alerts on server state changes at boot
- BUG/MEDIUM: servers: properly propagate the maintenance states during startup
- MEDIUM: wurfl: add Scientiamobile WURFL device detection module
- DOC: move the device detection modules documentation to their own files
- CLEANUP: wurfl: reduce exposure in the rest of the code
- MEDIUM: ssl: Add support for OpenSSL 1.1.0
- MINOR: stream: make option contstats usable again
- MEDIUM: tools: make str2sa_range() return the FQDN even when not resolving
- MINOR: init: move apply_server_state in haproxy.c before MODE_CHECK
- MAJOR: server: postpone address resolution
- MINOR: new srv_admin flag: SRV_ADMF_RMAINT
- MINOR: server: indicate in the logs when RMAINT is cleared
- MINOR: stats: indicate it when a server is down due to resolution
- MINOR: server: make srv_set_admin_state() capable of telling why this happens
- MINOR: dns: implement extra 'hold' timers.
- MAJOR: dns: runtime resolution can change server admin state
- MEDIUM: cli: leave the RMAINT state when setting an IP address on the CLI
- MEDIUM: server: add a new init-addr server line setting
- MEDIUM: server: make use of init-addr
- MINOR: server: implement init-addr none
- MEDIUM: server: make libc resolution failure non-fatal
- MINOR: server: add support for explicit numeric address in init-addr
- DOC: add some documentation for the "init-addr" server keyword
- MINOR: init: add -dr to ignore server address resolution failures
- MEDIUM: server: do not restrict anymore usage of IP address from the state file
- BUG: vars: Fix 'set-var' converter because of a typo
- CLEANUP: remove last references to 'ruleset' section
- MEDIUM: filters: Add attch/detach and stream_set_backend callbacks
- MINOR: filters: Update filters documentation accordingly to recent changes
- MINOR: filters: Call stream_set_backend callbacks before updating backend stats
- MINOR: filters: Remove backend filters attached to a stream only for HTTP streams
- MINOR: flt_trace: Add hexdump option to dump forwarded data
- MINOR: cfgparse: Add functions to backup and restore registered sections
- MINOR: cfgparse: Parse scope lines and save the last one parsed
- REORG: sample: move code to release a sample expression in sample.c
- MINOR: vars: Allow '.' in variable names
- MINOR: vars: Add vars_set_by_name_ifexist function
- MEDIUM: vars: Add a per-process scope for variables
- MINOR: vars: Add 'unset-var' action/converter
- MAJOR: spoe: Add an experimental Stream Processing Offload Engine
- MINOR: spoe: add random ip-reputation service as SPOA example
- MINOR: spoe/checks: Add support for SPOP health checks
- DOC: update ROADMAP file
---