Willy Tarreau
2018-10-28 20:01:29 UTC
Hi,
HAProxy 1.9-dev5 was released on 2018/10/28. It added 58 new commits
after version 1.9-dev4.
This version continues to merge new features and addresses some issues
that came in -dev4 regarding stream processing. For now it's working
rather well given the complexity of the changes, eventhough we still
expect to resurrect some deeply burried issues due to the significant
change of I/O scheduling.
Among the new features merged, I can list these ones :
- when running in master/worker mode, the master can now have its own
CLI socket, and implements a proxy able to connect to all worker
processes. It will even be able to reach older processes soon, so
that we can kill an old cnonection preventing an old process from
quitting, or simply figure why an old process doesn't quit. Some
more updates are coming on this part (prompt will be disabled by
default, older processes not joinable now, some doc etc).
- the HTTP small object cache can now cache objects larger than a
buffer. The new size limit defaults to 1/256 of the cache size but
can be changed with "max-object-size".
- the cache now implements the Age HTTP header field.
The rest is mostly infrastructure updates for the upcoming code, and
fixes for various issues. It's worth noting that Lukas has addressed
an interesting issue with HTTP authentication where the private
connection mistakenly had precedence over the load balancing algorithm
in order to cover NTLM/Negotiate. This one will be backported to 1.8.
Developers might like the addition of the ERR variable to the makefile
to automatically add -Werror.
For now what I'm seeing overall looks pretty good. We've again put the
finger on some old stuff around the stream interface flag SI_FL_WAIT_ROOM,
which we expected could easily replace channel_may_recv(), until the old
dirty zombies in the code decided to fight back :-) It's the first time
I've seen a 3-hours 3-person meeting dedicated to a single flag! But I
think we've found how to address this old crap so that we can rebase the
changes related to the internal native HTTP representation (codenamed HTX).
I'll try to issue -dev6 next week-end, eventhough this week will be short
for some of us. Ideally if we could merge the HTX code next week-end, we
could then switch to testing and debugging to stabilize all this stuff.
Please find the usual URLs below :
Site index : http://www.haproxy.org/
Discourse : http://discourse.haproxy.org/
Sources : http://www.haproxy.org/download/1.9/src/
Git repository : http://git.haproxy.org/git/haproxy.git/
Git Web browsing : http://git.haproxy.org/?p=haproxy.git
Changelog : http://www.haproxy.org/download/1.9/src/CHANGELOG
Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/
Willy
---
Complete changelog :
Christopher Faulet (3):
CLEANUP: http: Remove the unused function http_find_header
MINOR: h1: Export some functions parsing the value of some HTTP headers
BUG/MEDIUM: stream-int: don't set SI_FL_WAIT_ROOM on CF_READ_DONTWAIT
Frédéric Lécaille (12):
MINOR: shctx: Shared objects block by block allocation.
MINOR: cache: Larger HTTP objects caching.
MINOR: shctx: Add a maximum object size parameter.
MINOR: cache: Add "max-object-size" option.
DOC: Update about the cache support for big objects.
BUG/MINOR: cache: Crashes with "total-max-size" > 2047(MB).
BUG/MINOR: cache: Wrong usage of shctx_init().
BUG/MINOR: ssl: Wrong usage of shctx_init().
MINOR: cache: Avoid usage of atoi() when parsing "max-object-size".
MINOR: shctx: Change max. object size type to unsigned int.
DOC: cache: Missing information about "total-max-size" and "max-object-size"
MINOR: cache: Add "Age" header.
Ioannis Cherouvim (1):
CLEANUP: tools: fix misleading comment above function LIM2A
Lukas Tribus (2):
BUG/MINOR: only mark connections private if NTLM is detected
BUG/MINOR: only auto-prefer last server if lb-alg is non-deterministic
Olivier Houchard (1):
MINOR: stream_interface: Avoid calling si_cs_send/recv if not needed.
William Lallemand (20):
MINOR: mworker: number of reload in the life of a worker
MEDIUM: mworker: each worker socketpair is a CLI listener
REORG: mworker: move struct mworker_proc to global.h
MINOR: server: export new_server() function
MEDIUM: mworker: move proc_list gen before proxies startup
MEDIUM: mworker: add proc_list in global.h
MEDIUM: mworker: proxy for the master CLI
MEDIUM: mworker: create CLI listeners from argv[]
MEDIUM: cli: disable some keywords in the master
MEDIUM: mworker: find the server ptr using a CLI prefix
MEDIUM: cli: 'show proc' displays processus
MEDIUM: cli: implement 'mode cli' proxy analyzers
MINOR: cli: displays sockpair@ in "show cli sockets"
MEDIUM: cli: enable "show cli sockets" for the master
MINOR: cli: put @master @<relative pid> @!<pid> in the help
MEDIUM: listeners: set O_CLOEXEC on the accepted FDs
MEDIUM: mworker: stop the master proxy in the workers
MEDIUM: channel: reorder the channel analyzers for the cli
MEDIUM: cli: write a prompt for the CLI proxy of the master
MINOR: cli: helper to write an response message and close
Willy Tarreau (19):
BUILD: Makefile: add the new ERR variable to force -Werror
MINOR: freq_ctr: add swrate_add_scaled() to work with large samples
MINOR: proxy: add a new option "http-use-htx"
BUG/MEDIUM: pools: fix the minimum allocation size
MEDIUM: channel: merge back flags CF_WRITE_PARTIAL and CF_WRITE_EVENT
MINOR: stream: don't prune variables if the list is empty
MINOR: stream-int: add si_alloc_ibuf() to ease input buffer allocation
MEDIUM: stream-int: replace channel_alloc_buffer() with si_alloc_ibuf() everywhere
MEDIUM: stream: always call si_cs_recv() after a failed buffer allocation
MEDIUM: stream: don't try to send first in process_stream()
MEDIUM: stream-int: make si_update() synchronize flag changes before the I/O
MEDIUM: stream-int: call si_cs_process() in stream_int_update_conn
MINOR: stream-int: don't needlessly call tasklet_wakeup() in stream_int_chk_snd_conn()
MINOR: stream-int: make stream_int_notify() not wake the tasklet up
MINOR: stream-int: don't needlessly call si_cs_send() in si_cs_process()
REGTEST: make the IP+port logging test more reliable
BUG/MINOR: memory: make the thread-local cache allocator set the debugging link
BUG/MAJOR: http: http_txn_get_path() may deference an inexisting buffer
BUG/MINOR: backend: assign the wait list after the error check
---
HAProxy 1.9-dev5 was released on 2018/10/28. It added 58 new commits
after version 1.9-dev4.
This version continues to merge new features and addresses some issues
that came in -dev4 regarding stream processing. For now it's working
rather well given the complexity of the changes, eventhough we still
expect to resurrect some deeply burried issues due to the significant
change of I/O scheduling.
Among the new features merged, I can list these ones :
- when running in master/worker mode, the master can now have its own
CLI socket, and implements a proxy able to connect to all worker
processes. It will even be able to reach older processes soon, so
that we can kill an old cnonection preventing an old process from
quitting, or simply figure why an old process doesn't quit. Some
more updates are coming on this part (prompt will be disabled by
default, older processes not joinable now, some doc etc).
- the HTTP small object cache can now cache objects larger than a
buffer. The new size limit defaults to 1/256 of the cache size but
can be changed with "max-object-size".
- the cache now implements the Age HTTP header field.
The rest is mostly infrastructure updates for the upcoming code, and
fixes for various issues. It's worth noting that Lukas has addressed
an interesting issue with HTTP authentication where the private
connection mistakenly had precedence over the load balancing algorithm
in order to cover NTLM/Negotiate. This one will be backported to 1.8.
Developers might like the addition of the ERR variable to the makefile
to automatically add -Werror.
For now what I'm seeing overall looks pretty good. We've again put the
finger on some old stuff around the stream interface flag SI_FL_WAIT_ROOM,
which we expected could easily replace channel_may_recv(), until the old
dirty zombies in the code decided to fight back :-) It's the first time
I've seen a 3-hours 3-person meeting dedicated to a single flag! But I
think we've found how to address this old crap so that we can rebase the
changes related to the internal native HTTP representation (codenamed HTX).
I'll try to issue -dev6 next week-end, eventhough this week will be short
for some of us. Ideally if we could merge the HTX code next week-end, we
could then switch to testing and debugging to stabilize all this stuff.
Please find the usual URLs below :
Site index : http://www.haproxy.org/
Discourse : http://discourse.haproxy.org/
Sources : http://www.haproxy.org/download/1.9/src/
Git repository : http://git.haproxy.org/git/haproxy.git/
Git Web browsing : http://git.haproxy.org/?p=haproxy.git
Changelog : http://www.haproxy.org/download/1.9/src/CHANGELOG
Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/
Willy
---
Complete changelog :
Christopher Faulet (3):
CLEANUP: http: Remove the unused function http_find_header
MINOR: h1: Export some functions parsing the value of some HTTP headers
BUG/MEDIUM: stream-int: don't set SI_FL_WAIT_ROOM on CF_READ_DONTWAIT
Frédéric Lécaille (12):
MINOR: shctx: Shared objects block by block allocation.
MINOR: cache: Larger HTTP objects caching.
MINOR: shctx: Add a maximum object size parameter.
MINOR: cache: Add "max-object-size" option.
DOC: Update about the cache support for big objects.
BUG/MINOR: cache: Crashes with "total-max-size" > 2047(MB).
BUG/MINOR: cache: Wrong usage of shctx_init().
BUG/MINOR: ssl: Wrong usage of shctx_init().
MINOR: cache: Avoid usage of atoi() when parsing "max-object-size".
MINOR: shctx: Change max. object size type to unsigned int.
DOC: cache: Missing information about "total-max-size" and "max-object-size"
MINOR: cache: Add "Age" header.
Ioannis Cherouvim (1):
CLEANUP: tools: fix misleading comment above function LIM2A
Lukas Tribus (2):
BUG/MINOR: only mark connections private if NTLM is detected
BUG/MINOR: only auto-prefer last server if lb-alg is non-deterministic
Olivier Houchard (1):
MINOR: stream_interface: Avoid calling si_cs_send/recv if not needed.
William Lallemand (20):
MINOR: mworker: number of reload in the life of a worker
MEDIUM: mworker: each worker socketpair is a CLI listener
REORG: mworker: move struct mworker_proc to global.h
MINOR: server: export new_server() function
MEDIUM: mworker: move proc_list gen before proxies startup
MEDIUM: mworker: add proc_list in global.h
MEDIUM: mworker: proxy for the master CLI
MEDIUM: mworker: create CLI listeners from argv[]
MEDIUM: cli: disable some keywords in the master
MEDIUM: mworker: find the server ptr using a CLI prefix
MEDIUM: cli: 'show proc' displays processus
MEDIUM: cli: implement 'mode cli' proxy analyzers
MINOR: cli: displays sockpair@ in "show cli sockets"
MEDIUM: cli: enable "show cli sockets" for the master
MINOR: cli: put @master @<relative pid> @!<pid> in the help
MEDIUM: listeners: set O_CLOEXEC on the accepted FDs
MEDIUM: mworker: stop the master proxy in the workers
MEDIUM: channel: reorder the channel analyzers for the cli
MEDIUM: cli: write a prompt for the CLI proxy of the master
MINOR: cli: helper to write an response message and close
Willy Tarreau (19):
BUILD: Makefile: add the new ERR variable to force -Werror
MINOR: freq_ctr: add swrate_add_scaled() to work with large samples
MINOR: proxy: add a new option "http-use-htx"
BUG/MEDIUM: pools: fix the minimum allocation size
MEDIUM: channel: merge back flags CF_WRITE_PARTIAL and CF_WRITE_EVENT
MINOR: stream: don't prune variables if the list is empty
MINOR: stream-int: add si_alloc_ibuf() to ease input buffer allocation
MEDIUM: stream-int: replace channel_alloc_buffer() with si_alloc_ibuf() everywhere
MEDIUM: stream: always call si_cs_recv() after a failed buffer allocation
MEDIUM: stream: don't try to send first in process_stream()
MEDIUM: stream-int: make si_update() synchronize flag changes before the I/O
MEDIUM: stream-int: call si_cs_process() in stream_int_update_conn
MINOR: stream-int: don't needlessly call tasklet_wakeup() in stream_int_chk_snd_conn()
MINOR: stream-int: make stream_int_notify() not wake the tasklet up
MINOR: stream-int: don't needlessly call si_cs_send() in si_cs_process()
REGTEST: make the IP+port logging test more reliable
BUG/MINOR: memory: make the thread-local cache allocator set the debugging link
BUG/MAJOR: http: http_txn_get_path() may deference an inexisting buffer
BUG/MINOR: backend: assign the wait list after the error check
---