Willy Tarreau
2018-11-17 19:42:25 UTC
Hi Moemen,
so at the very least we're missing a test. However I suspect there
is a problem with the presence of the forward() call in your script,
because by doing this you're totally bypassing the HTTP engine, so
your script was called in an http context, it discretely stole the
contents under the blanket, and went back to the http engine saying
"I did nothing, it's not me!". The rest of the code continues to
process the HTTP contents from the buffer where they are, resulting
it quite a big mess. Ideally we should have a way to detect that
parts of the buffer were moved on return and immediately send an
error there. But there are some cases where it's valid if called
using the HTTP API. So I don't know for sure how to detect such
anomalies. Maybe buffer contents being smaller than the size of
headers known by the parser would already be a good step forward.
I remember Thierry recently had to try to strengthen a little bit
such use cases where tcp was used from within HTTP. We'll definitely
have to figure what the use cases are for this and to find a reliable
solution to this because by definition it will not work anymore with
HTX.
is totally bypassed and what you forward is not HTTP anymore, but bytes
from the wire, and that you may even expect that the HTTP parser appends
an error at some places and aborts if it discoveres the stream is
mangled.
I don't know if we can register filters from the Lua, but ideall that's
what should be the best option in your case : having a Lua-based filter
running on the data part would allow you to intercept the data stream
for each chunk decoded by the HTTP parser.
Willy
Hi,
I was playing with LUA, to configure a traffic mirroring behavior.
Basically I wanted HAProxy to send the http response of a request to a
3rd party before sending the response to the client.
So this is the stripped down version of the script to reproduce the
function mirror(txn)
local in_len = txn.res:get_in_len()
while in_len > 0 do
response = txn.res:dup()
-- sending response to 3rd party.
txn.res:forward(in_len)
core.yield()
in_len = txn.res:get_in_len()
end
end
core.register_action("mirror", { "http-res" }, mirror)
Then I use this script via "http-response lua.mirror"
I think problem here is that when I forward the response from the input
buffer to the output buffer and hand processing back to HAProyx, the
latter will try to send an invalid http request.
The request is invalid because HAProxy did not have the opportunity to
check the response and make sure there are valid headers because the
input buffer is empty after the core.yield().
So I was expecting an error and HAProxy telling me that this is an
invalid request but not a segfault.
I can't tell for sure, but I totally agree it should never segfault,I was playing with LUA, to configure a traffic mirroring behavior.
Basically I wanted HAProxy to send the http response of a request to a
3rd party before sending the response to the client.
So this is the stripped down version of the script to reproduce the
function mirror(txn)
local in_len = txn.res:get_in_len()
while in_len > 0 do
response = txn.res:dup()
-- sending response to 3rd party.
txn.res:forward(in_len)
core.yield()
in_len = txn.res:get_in_len()
end
end
core.register_action("mirror", { "http-res" }, mirror)
Then I use this script via "http-response lua.mirror"
I think problem here is that when I forward the response from the input
buffer to the output buffer and hand processing back to HAProyx, the
latter will try to send an invalid http request.
The request is invalid because HAProxy did not have the opportunity to
check the response and make sure there are valid headers because the
input buffer is empty after the core.yield().
So I was expecting an error and HAProxy telling me that this is an
invalid request but not a segfault.
so at the very least we're missing a test. However I suspect there
is a problem with the presence of the forward() call in your script,
because by doing this you're totally bypassing the HTTP engine, so
your script was called in an http context, it discretely stole the
contents under the blanket, and went back to the http engine saying
"I did nothing, it's not me!". The rest of the code continues to
process the HTTP contents from the buffer where they are, resulting
it quite a big mess. Ideally we should have a way to detect that
parts of the buffer were moved on return and immediately send an
error there. But there are some cases where it's valid if called
using the HTTP API. So I don't know for sure how to detect such
anomalies. Maybe buffer contents being smaller than the size of
headers known by the parser would already be a good step forward.
I remember Thierry recently had to try to strengthen a little bit
such use cases where tcp was used from within HTTP. We'll definitely
have to figure what the use cases are for this and to find a reliable
solution to this because by definition it will not work anymore with
HTX.
1/ Use mode tcp
2/ Use "get" and "send" instead of "forward", this way the LUA script
will send the response directly to the client, instead of HAProxy doing
that.
It should still cause the same problem which is that the HTTP parser2/ Use "get" and "send" instead of "forward", this way the LUA script
will send the response directly to the client, instead of HAProxy doing
that.
is totally bypassed and what you forward is not HTTP anymore, but bytes
from the wire, and that you may even expect that the HTTP parser appends
an error at some places and aborts if it discoveres the stream is
mangled.
I don't know if we can register filters from the Lua, but ideall that's
what should be the best option in your case : having a Lua-based filter
running on the data part would allow you to intercept the data stream
for each chunk decoded by the HTTP parser.
Willy