Willy Tarreau
2018-11-25 09:02:09 UTC
Hi,
HAProxy 1.9-dev8 was released on 2018/11/25. It added 56 new commits
after version 1.9-dev7.
This version fixes the issues that appeared with dev7 and the merge of
the pending features. It's getting better, much better.
H2 posts work again, though if the server responds before the end of
the transfer, it may still receive a truncated last chunk. This didn't
appear in 1.8 so it must be related to some recent changes that I will
investigate but I don't consider this critical.
We now have ALPN working on the server side, which will be needed to
negotiate H2 to the server. We also figured how to better deal with
some of the current limitations affecting the server-side keep-alive
when the client runs on H2.
The master-worker model got a few fixes as well as some improvements.
The master command "show proc" also displays each process' uptime. The
master now continues to accept commands while in "wait mode" (after a
failed reload).
The H1 mux (HTTP/1.1 on HTX) works much better now, and even splicing
was repaired. The performance is back to much better values after a
few fixes and the server side keep-alive was enabled (I didn't notice
it was not after I merged it). Stats now work fine over HTX as well.
We may get the compression and cache converted soon, and possibly even
Lua as well!
I finally decided to blow the dust on some goodies I had been leaving
in some old branches for several months. The first one concerns a better
observability, by reporting the maximum per-thread latency in "show
activity" (time to loop from poll to poll), the per-task average and
total CPU time consumed by the processing, as well as the average and
total latency inflicted by other tasks' processing. This is reported
in the logs when profiling is enabled in the global section, or enabled
at run time on the CLI (see show profiling). This will help indicate in
logs which request cost a lot to process and which ones suffer from the
other ones. Another very old patch was the busy polling, which allows
to reduce the request processing latency by 30 to 100 microseconds on
machines featuring frequency scaling or supporting deep idle states
(look for "busy-polling" in the doc).
Also, I was really really fed up with missing build warnings due to the
verbosity of the build process, so I implemented a quiet mode by default
which only shows "CC", "LD", "AR", ... and the name of the object file.
Just pass "V=1" to the make command to recover the original behaviour.
I must say that I'm quite happy to see how fast all pending issues were
resolved last week, this is an indication that the painful changes we've
been through are starting to pay off with a much less complex architecture
that is more understandable and straightforward. It also means that there
starts to be less core bugs as debugging sessions don't constantly end up
on another bug than then one being looked at.
At this point I feel reasonably confident in our ability to make the H2
mux use HTX and to support H2 on both sides in this mode, especially
since the idle connections are almost ready for this. I think it's worth
a try for this release as it doesn't affect all other areas that we
continue to review and test in parallel. I'm even willing to postpone
the release by an extra week if needed given that it brings some extra
value. If I face any painful showstopper, I won't take any risk and will
simply postpone this for 2.0.
We will get a dev9 next week, hopefully with H2 over HTX, maybe a dev10
the week after depending on what we'll discover, and then the release.
There are still some code cleanups I'd like to perform before the release
(functions and flags renaming, a bit of doc for example). This will not
impact the code and will be best done last.
Please find the usual URLs below :
Site index : http://www.haproxy.org/
Discourse : http://discourse.haproxy.org/
Sources : http://www.haproxy.org/download/1.9/src/
Git repository : http://git.haproxy.org/git/haproxy.git/
Git Web browsing : http://git.haproxy.org/?p=haproxy.git
Changelog : http://www.haproxy.org/download/1.9/src/CHANGELOG
Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/
Willy
---
Complete changelog :
Christopher Faulet (12):
MAJOR: mux-h1: Remove the rxbuf and decode HTTP messages in channel's buffer
BUG/MINOR: mux-h1: Enable keep-alive on server side
BUG/MEDIUM: mux-h1: Fix freeze when the kernel splicing is used
BUG/MEDIUM: mux-h1: Don't set the flag CS_FL_RCV_MORE when nothing was parsed
BUG/MINOR: stats/htx: Remove channel's output when the request is eaten
BUG/MINOR: proto_htx: Fix request/response synchronisation on error
MINOR: stream-int: Notify caller when an error is reported after a rcv_pipe()
MINOR: stream-int: Notify caller when an error is reported after a rcv_buf()
BUG/MINOR: stream-int: Don't call snd_buf() if there are still data in the pipe
MINOR: stream-int: remove useless checks on CS and conn flags in si_cs_send()
BUG/MINOR: config: Be aware of the HTX during the check of mux protocols
BUG/MINOR: mux-htx: Fix bad test on h1c flags in h1_recv_allowed()
Olivier Houchard (11):
BUG/MEDIUM: http_fetch: Make sure name is initialized before http_find_header.
MINOR: server: Add "alpn" and "npn" keywords.
MEDIUM: connection: Don't bother reactivating polling after connection retry.
MAJOR: connections: Defer mux creation for outgoing connection if alpn is set.
MEDIUM: ssl: Add ssl_bc_alpn and ssl_bc_npn sample fetches.
MINOR: servers: Free [idle|safe|priv]_conns on exit.
BUG/MEDIUM: connections: Don't reset the conn flags in *connect_server().
MINOR: server: Only defined conn_complete_server if USE_OPENSSL is set.
BUG/MEDIUM: servers: Don't check if we have a conn_stream too soon.
BUG/MEDIUM: sessions: Set sess->origin to NULL if the origin was destroyed.
MEDIUM: servers: Store the connection in the SI until we have a mux.
Tim Duesterhus (1):
BUG/MINOR: cli: Fix memory leak
William Lallemand (10):
MINOR: cli: format `show proc` to be more readable
MINOR: cli: displays uptime in `show proc`
MINOR: cli: show master information in 'show proc'
MEDIUM: mworker: wait mode use standard init code path
MINOR: log: introduce ha_notice()
MINOR: mworker: use ha_notice to announce a new worker
MINOR: cli: add mworker_accept_wrapper to 'show fd'
MEDIUM: signal: signal_unregister() removes every handlers
BUG/MEDIUM: mworker: unregister the signals of main()
BUG/MINOR: mworker: fix FD leak and memory leak in error path
Willy Tarreau (22):
REORG: config: extract the global section parser into cfgparse-global
REORG: config: extract the proxy parser into cfgparse-listen.c
BUILD: update the list of supported targets and compilers in makefile and readme
BUILD: reorder the objects in the makefile
BUILD: Makefile: make "V=1" show some of the commands that are executed
BUILD: Makefile: add the quiet mode to a few more targets
BUILD: Makefile: add "$(Q)" to clean, tags and cscope targets
BUILD: Makefile: switch to quiet mode by default for CC/LD/AR
BUG/MEDIUM: hpack: fix encoding of "accept-ranges" field
MINOR: cli: add a few missing includes in proto/cli.h
REORG: time/activity: move activity measurements to activity.{c,h}
MINOR: activity: report the average loop time in "show activity"
MINOR: activity: add configuration and CLI support for "profiling.tasks"
MEDIUM: tasks: collect per-task CPU time and latency
MINOR: sample: add cpu_calls, cpu_ns_avg, cpu_ns_tot, lat_ns_avg, lat_ns_tot
MINOR: cli/activity: rename the stolen CPU time fields to mention milliseconds
MINOR: poller: move the call of tv_update_date() back to the pollers
MINOR: polling: add an option to support busy polling
REGTEST: add the option to test only a specific set of files
REGTEST: add a test for connections to a "dispatch" address
BUG/MEDIUM: h2: wake the processing task up after demuxing
BUG/MEDIUM: h2: restart demuxing after releasing buffer space
---
HAProxy 1.9-dev8 was released on 2018/11/25. It added 56 new commits
after version 1.9-dev7.
This version fixes the issues that appeared with dev7 and the merge of
the pending features. It's getting better, much better.
H2 posts work again, though if the server responds before the end of
the transfer, it may still receive a truncated last chunk. This didn't
appear in 1.8 so it must be related to some recent changes that I will
investigate but I don't consider this critical.
We now have ALPN working on the server side, which will be needed to
negotiate H2 to the server. We also figured how to better deal with
some of the current limitations affecting the server-side keep-alive
when the client runs on H2.
The master-worker model got a few fixes as well as some improvements.
The master command "show proc" also displays each process' uptime. The
master now continues to accept commands while in "wait mode" (after a
failed reload).
The H1 mux (HTTP/1.1 on HTX) works much better now, and even splicing
was repaired. The performance is back to much better values after a
few fixes and the server side keep-alive was enabled (I didn't notice
it was not after I merged it). Stats now work fine over HTX as well.
We may get the compression and cache converted soon, and possibly even
Lua as well!
I finally decided to blow the dust on some goodies I had been leaving
in some old branches for several months. The first one concerns a better
observability, by reporting the maximum per-thread latency in "show
activity" (time to loop from poll to poll), the per-task average and
total CPU time consumed by the processing, as well as the average and
total latency inflicted by other tasks' processing. This is reported
in the logs when profiling is enabled in the global section, or enabled
at run time on the CLI (see show profiling). This will help indicate in
logs which request cost a lot to process and which ones suffer from the
other ones. Another very old patch was the busy polling, which allows
to reduce the request processing latency by 30 to 100 microseconds on
machines featuring frequency scaling or supporting deep idle states
(look for "busy-polling" in the doc).
Also, I was really really fed up with missing build warnings due to the
verbosity of the build process, so I implemented a quiet mode by default
which only shows "CC", "LD", "AR", ... and the name of the object file.
Just pass "V=1" to the make command to recover the original behaviour.
I must say that I'm quite happy to see how fast all pending issues were
resolved last week, this is an indication that the painful changes we've
been through are starting to pay off with a much less complex architecture
that is more understandable and straightforward. It also means that there
starts to be less core bugs as debugging sessions don't constantly end up
on another bug than then one being looked at.
At this point I feel reasonably confident in our ability to make the H2
mux use HTX and to support H2 on both sides in this mode, especially
since the idle connections are almost ready for this. I think it's worth
a try for this release as it doesn't affect all other areas that we
continue to review and test in parallel. I'm even willing to postpone
the release by an extra week if needed given that it brings some extra
value. If I face any painful showstopper, I won't take any risk and will
simply postpone this for 2.0.
We will get a dev9 next week, hopefully with H2 over HTX, maybe a dev10
the week after depending on what we'll discover, and then the release.
There are still some code cleanups I'd like to perform before the release
(functions and flags renaming, a bit of doc for example). This will not
impact the code and will be best done last.
Please find the usual URLs below :
Site index : http://www.haproxy.org/
Discourse : http://discourse.haproxy.org/
Sources : http://www.haproxy.org/download/1.9/src/
Git repository : http://git.haproxy.org/git/haproxy.git/
Git Web browsing : http://git.haproxy.org/?p=haproxy.git
Changelog : http://www.haproxy.org/download/1.9/src/CHANGELOG
Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/
Willy
---
Complete changelog :
Christopher Faulet (12):
MAJOR: mux-h1: Remove the rxbuf and decode HTTP messages in channel's buffer
BUG/MINOR: mux-h1: Enable keep-alive on server side
BUG/MEDIUM: mux-h1: Fix freeze when the kernel splicing is used
BUG/MEDIUM: mux-h1: Don't set the flag CS_FL_RCV_MORE when nothing was parsed
BUG/MINOR: stats/htx: Remove channel's output when the request is eaten
BUG/MINOR: proto_htx: Fix request/response synchronisation on error
MINOR: stream-int: Notify caller when an error is reported after a rcv_pipe()
MINOR: stream-int: Notify caller when an error is reported after a rcv_buf()
BUG/MINOR: stream-int: Don't call snd_buf() if there are still data in the pipe
MINOR: stream-int: remove useless checks on CS and conn flags in si_cs_send()
BUG/MINOR: config: Be aware of the HTX during the check of mux protocols
BUG/MINOR: mux-htx: Fix bad test on h1c flags in h1_recv_allowed()
Olivier Houchard (11):
BUG/MEDIUM: http_fetch: Make sure name is initialized before http_find_header.
MINOR: server: Add "alpn" and "npn" keywords.
MEDIUM: connection: Don't bother reactivating polling after connection retry.
MAJOR: connections: Defer mux creation for outgoing connection if alpn is set.
MEDIUM: ssl: Add ssl_bc_alpn and ssl_bc_npn sample fetches.
MINOR: servers: Free [idle|safe|priv]_conns on exit.
BUG/MEDIUM: connections: Don't reset the conn flags in *connect_server().
MINOR: server: Only defined conn_complete_server if USE_OPENSSL is set.
BUG/MEDIUM: servers: Don't check if we have a conn_stream too soon.
BUG/MEDIUM: sessions: Set sess->origin to NULL if the origin was destroyed.
MEDIUM: servers: Store the connection in the SI until we have a mux.
Tim Duesterhus (1):
BUG/MINOR: cli: Fix memory leak
William Lallemand (10):
MINOR: cli: format `show proc` to be more readable
MINOR: cli: displays uptime in `show proc`
MINOR: cli: show master information in 'show proc'
MEDIUM: mworker: wait mode use standard init code path
MINOR: log: introduce ha_notice()
MINOR: mworker: use ha_notice to announce a new worker
MINOR: cli: add mworker_accept_wrapper to 'show fd'
MEDIUM: signal: signal_unregister() removes every handlers
BUG/MEDIUM: mworker: unregister the signals of main()
BUG/MINOR: mworker: fix FD leak and memory leak in error path
Willy Tarreau (22):
REORG: config: extract the global section parser into cfgparse-global
REORG: config: extract the proxy parser into cfgparse-listen.c
BUILD: update the list of supported targets and compilers in makefile and readme
BUILD: reorder the objects in the makefile
BUILD: Makefile: make "V=1" show some of the commands that are executed
BUILD: Makefile: add the quiet mode to a few more targets
BUILD: Makefile: add "$(Q)" to clean, tags and cscope targets
BUILD: Makefile: switch to quiet mode by default for CC/LD/AR
BUG/MEDIUM: hpack: fix encoding of "accept-ranges" field
MINOR: cli: add a few missing includes in proto/cli.h
REORG: time/activity: move activity measurements to activity.{c,h}
MINOR: activity: report the average loop time in "show activity"
MINOR: activity: add configuration and CLI support for "profiling.tasks"
MEDIUM: tasks: collect per-task CPU time and latency
MINOR: sample: add cpu_calls, cpu_ns_avg, cpu_ns_tot, lat_ns_avg, lat_ns_tot
MINOR: cli/activity: rename the stolen CPU time fields to mention milliseconds
MINOR: poller: move the call of tv_update_date() back to the pollers
MINOR: polling: add an option to support busy polling
REGTEST: add the option to test only a specific set of files
REGTEST: add a test for connections to a "dispatch" address
BUG/MEDIUM: h2: wake the processing task up after demuxing
BUG/MEDIUM: h2: restart demuxing after releasing buffer space
---