Jeremy Friesner
2018-10-11 01:03:29 UTC
Hi all,
First, a little background (and my apologies if my questions are very basic, I’m new to HAProxy) — I’ve got an embedded device with a built-in web server that allows the device to be controlled via HTTP requests and/or (more interactively) via WebSockets connections. This all works fine when the device’s web-server accessed directly from Safari, Chrome, Internet Explorer, Firefox, etc.
The problem is, this embedded device doesn’t implement any kind of security or access-control, so it would be a bad idea to put it directly on an untrusted network, since any random person could point their web browser at it and mess up its settings.
To work around that problem, I hide connect this embedded device directly to the second Ethernet port of a Mac running HAProxy 1.8.13, so that I can use HAProxy’s “http-request auth” feature (with or without SSH/TLS) to provide authentication. That way, nobody on the untrusted network can talk to my insecure embedded device directly; instead, they can point their web browsers to my Mac’s IP address, and HAProxy makes them enter the secret username-and-password before any of their connections can get forwarded on through to the embedded device’s web server.
This also works great — at least, it works great when the web browser is Chrome. If the web browser is Safari on the other hand, the vanilla http/https stuff works fine, but the WebSocket connections error-out when they hit HAProxy. In particular, the JavaScript scripts served from the device’s embedded web page can’t connect to the embedded device’s web-server (using either ws:// or wss:// protocol), and Safari’s JavaScript console shows this error message when the try:
[Error] WebSocket connection to 'wss://localhost:8080/' failed: Invalid HTTP version string: HTTP/1.0
My question is, does anyone know what might be going wrong here, or have any ideas about how I might get Safari’s WebSockets to play nicely with HAProxy’s client-username/password authentication feature? (Safari’s WebSockets do work fine through HAProxy if I comment out the “http-request auth” line in my haproxy.cfg file’s “frontend” section, but then accessing my embedded device no longer requires a password, which defeats the point of the exercise)
Thanks,
Jeremy
ps some hopefully-relevant debugging info follows...
I’m testing with Safari 12.0.1 (13606.2.104) running on the HAProxy-hosting Mac. haproxy is v1.8.13.
If I run haproxy with debugging output enabled, this is what I see when the JavaScript tries (and fails) to connect a WebSocket through HAProxy under Safari:
$ haproxy -dddd -f /usr/local/etc/haproxy.cfg
[WARNING] 282/174723 (9610) : parsing [/usr/local/etc/haproxy.cfg:32] : a 'http-request' rule placed after a 'reqadd' rule will still be processed before.
Available polling systems :
kqueue : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result FAILED
Total: 3 (2 usable), will use kqueue.
Available filters :
[SPOE] spoe
[COMP] compression
[TRACE] trace
Using kqueue() as the polling mechanism.
[WARNING] 282/174725 (9610) : Server galaxynodes/server1 is DOWN, reason: Layer4 timeout, check duration: 2002ms. 1 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
00000000:localnodes.accept(0004)=0009 from [127.0.0.1:63330] ALPN=<none>
00000000:localnodes.clireq[0009:ffffffff]: GET / HTTP/1.1
00000000:localnodes.clihdr[0009:ffffffff]: Upgrade: websocket
00000000:localnodes.clihdr[0009:ffffffff]: Connection: Upgrade
00000000:localnodes.clihdr[0009:ffffffff]: Host: localhost:8080
00000000:localnodes.clihdr[0009:ffffffff]: Origin: https://localhost:8080
00000000:localnodes.clihdr[0009:ffffffff]: Pragma: no-cache
00000000:localnodes.clihdr[0009:ffffffff]: Cache-Control: no-cache
00000000:localnodes.clihdr[0009:ffffffff]: Sec-WebSocket-Key: sUs/WOhQoe4plAvU5HQ+MQ==
00000000:localnodes.clihdr[0009:ffffffff]: Sec-WebSocket-Version: 13
00000000:localnodes.clihdr[0009:ffffffff]: Sec-WebSocket-Extensions: x-webkit-deflate-frame
00000000:localnodes.clihdr[0009:ffffffff]: User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.0.1 Safari/605.1.15
00000000:localnodes.clicls[0009:ffffffff]
00000000:localnodes.closed[0009:ffffffff]
… and then, just for comparison, here is the output from haproxy when the client is Google Chrome (v69.0.3497.100), and the WebSocket connections succeeds:
Jeremys-Mac-Pro:specs jaf$ haproxy -dddd -f /usr/local/etc/haproxy.cfg
[WARNING] 282/175026 (9663) : parsing [/usr/local/etc/haproxy.cfg:32] : a 'http-request' rule placed after a 'reqadd' rule will still be processed before.
Available polling systems :
kqueue : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result FAILED
Total: 3 (2 usable), will use kqueue.
Available filters :
[SPOE] spoe
[COMP] compression
[TRACE] trace
Using kqueue() as the polling mechanism.
00000000:localnodes.accept(0004)=0009 from [127.0.0.1:63542] ALPN=<none>
00000000:localnodes.clicls[0009:ffffffff]
00000000:localnodes.closed[0009:ffffffff]
00000001:localnodes.accept(0004)=0009 from [127.0.0.1:63544] ALPN=<none>
00000001:localnodes.clireq[0009:ffffffff]: GET / HTTP/1.1
00000001:localnodes.clihdr[0009:ffffffff]: Host: localhost:8080
00000001:localnodes.clihdr[0009:ffffffff]: Connection: Upgrade
00000001:localnodes.clihdr[0009:ffffffff]: Pragma: no-cache
00000001:localnodes.clihdr[0009:ffffffff]: Cache-Control: no-cache
00000001:localnodes.clihdr[0009:ffffffff]: Authorization: Basic bXl1c2VybmFtZTpteXBhc3N3b3Jk
00000001:localnodes.clihdr[0009:ffffffff]: User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36
00000001:localnodes.clihdr[0009:ffffffff]: Upgrade: websocket
00000001:localnodes.clihdr[0009:ffffffff]: Origin: https://localhost:8080
00000001:localnodes.clihdr[0009:ffffffff]: Sec-WebSocket-Version: 13
00000001:localnodes.clihdr[0009:ffffffff]: Accept-Encoding: gzip, deflate, br
00000001:localnodes.clihdr[0009:ffffffff]: Accept-Language: en-US,en;q=0.9
00000001:localnodes.clihdr[0009:ffffffff]: Sec-WebSocket-Key: fjfkDkUeLqZPGAuvKYYPqw==
00000001:localnodes.clihdr[0009:ffffffff]: Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits
00000001:galaxynodes.srvrep[0009:000a]: HTTP/1.1 101 Switching Protocols
00000001:galaxynodes.srvhdr[0009:000a]: Upgrade: websocket
00000001:galaxynodes.srvhdr[0009:000a]: Connection: Upgrade
00000001:galaxynodes.srvhdr[0009:000a]: Sec-WebSocket-Accept: MJ31YajGyhHwHUYCYeGeQw1V5c8=
00000001:galaxynodes.srvcls[0009:adfd]
00000001:galaxynodes.clicls[0009:adfd]
00000001:galaxynodes.closed[0009:adfd]
Also, here are the contents of my haproxy.cfg file:
global
log 127.0.0.1 local0
log 127.0.0.1 local1 debug
tune.ssl.default-dh-param 2048
maxconn 4096
defaults
log global
mode http
option httplog
option dontlognull
retries 3
option redispatch
maxconn 2000
timeout connect 5s
timeout client 1h
timeout server 1h
userlist MyDeviceUsers
user myusername insecure-password mypassword
# This section governs how web browser connect to HAProxy running on the Mac
frontend localnodes
bind *:8080 ssl crt /etc/ssl/private/my_combined_file.pem
reqadd X-Forwarded-Proto:\ https
mode http
default_backend mydevicenodes
acl ValidMyDeviceUser http_auth(MyDeviceUsers)
http-request auth realm MyDevice if !ValidMyDeviceUser # Commenting out this line allows Safari's WebSockets to connect through HAProxy, at the expense of no authentication
# This section governs how HAProxy connects to webd running on the MyDevice modules
backend mydevicenodes
mode http
balance roundrobin
option forwardfor
http-request set-header X-Forwarded-Port %[dst_port]
http-request add-header X-Forwarded-Proto https if { ssl_fc }
server server1 fe80::21c:abff:fe00:55e4%en1:8080 check # IPv6 address of my first MyDevice module
server server2 fe80::21c:abff:fe00:594c%en1:8080 check # IPv6 address of my second MyDevice module
First, a little background (and my apologies if my questions are very basic, I’m new to HAProxy) — I’ve got an embedded device with a built-in web server that allows the device to be controlled via HTTP requests and/or (more interactively) via WebSockets connections. This all works fine when the device’s web-server accessed directly from Safari, Chrome, Internet Explorer, Firefox, etc.
The problem is, this embedded device doesn’t implement any kind of security or access-control, so it would be a bad idea to put it directly on an untrusted network, since any random person could point their web browser at it and mess up its settings.
To work around that problem, I hide connect this embedded device directly to the second Ethernet port of a Mac running HAProxy 1.8.13, so that I can use HAProxy’s “http-request auth” feature (with or without SSH/TLS) to provide authentication. That way, nobody on the untrusted network can talk to my insecure embedded device directly; instead, they can point their web browsers to my Mac’s IP address, and HAProxy makes them enter the secret username-and-password before any of their connections can get forwarded on through to the embedded device’s web server.
This also works great — at least, it works great when the web browser is Chrome. If the web browser is Safari on the other hand, the vanilla http/https stuff works fine, but the WebSocket connections error-out when they hit HAProxy. In particular, the JavaScript scripts served from the device’s embedded web page can’t connect to the embedded device’s web-server (using either ws:// or wss:// protocol), and Safari’s JavaScript console shows this error message when the try:
[Error] WebSocket connection to 'wss://localhost:8080/' failed: Invalid HTTP version string: HTTP/1.0
My question is, does anyone know what might be going wrong here, or have any ideas about how I might get Safari’s WebSockets to play nicely with HAProxy’s client-username/password authentication feature? (Safari’s WebSockets do work fine through HAProxy if I comment out the “http-request auth” line in my haproxy.cfg file’s “frontend” section, but then accessing my embedded device no longer requires a password, which defeats the point of the exercise)
Thanks,
Jeremy
ps some hopefully-relevant debugging info follows...
I’m testing with Safari 12.0.1 (13606.2.104) running on the HAProxy-hosting Mac. haproxy is v1.8.13.
If I run haproxy with debugging output enabled, this is what I see when the JavaScript tries (and fails) to connect a WebSocket through HAProxy under Safari:
$ haproxy -dddd -f /usr/local/etc/haproxy.cfg
[WARNING] 282/174723 (9610) : parsing [/usr/local/etc/haproxy.cfg:32] : a 'http-request' rule placed after a 'reqadd' rule will still be processed before.
Available polling systems :
kqueue : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result FAILED
Total: 3 (2 usable), will use kqueue.
Available filters :
[SPOE] spoe
[COMP] compression
[TRACE] trace
Using kqueue() as the polling mechanism.
[WARNING] 282/174725 (9610) : Server galaxynodes/server1 is DOWN, reason: Layer4 timeout, check duration: 2002ms. 1 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
00000000:localnodes.accept(0004)=0009 from [127.0.0.1:63330] ALPN=<none>
00000000:localnodes.clireq[0009:ffffffff]: GET / HTTP/1.1
00000000:localnodes.clihdr[0009:ffffffff]: Upgrade: websocket
00000000:localnodes.clihdr[0009:ffffffff]: Connection: Upgrade
00000000:localnodes.clihdr[0009:ffffffff]: Host: localhost:8080
00000000:localnodes.clihdr[0009:ffffffff]: Origin: https://localhost:8080
00000000:localnodes.clihdr[0009:ffffffff]: Pragma: no-cache
00000000:localnodes.clihdr[0009:ffffffff]: Cache-Control: no-cache
00000000:localnodes.clihdr[0009:ffffffff]: Sec-WebSocket-Key: sUs/WOhQoe4plAvU5HQ+MQ==
00000000:localnodes.clihdr[0009:ffffffff]: Sec-WebSocket-Version: 13
00000000:localnodes.clihdr[0009:ffffffff]: Sec-WebSocket-Extensions: x-webkit-deflate-frame
00000000:localnodes.clihdr[0009:ffffffff]: User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.0.1 Safari/605.1.15
00000000:localnodes.clicls[0009:ffffffff]
00000000:localnodes.closed[0009:ffffffff]
… and then, just for comparison, here is the output from haproxy when the client is Google Chrome (v69.0.3497.100), and the WebSocket connections succeeds:
Jeremys-Mac-Pro:specs jaf$ haproxy -dddd -f /usr/local/etc/haproxy.cfg
[WARNING] 282/175026 (9663) : parsing [/usr/local/etc/haproxy.cfg:32] : a 'http-request' rule placed after a 'reqadd' rule will still be processed before.
Available polling systems :
kqueue : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result FAILED
Total: 3 (2 usable), will use kqueue.
Available filters :
[SPOE] spoe
[COMP] compression
[TRACE] trace
Using kqueue() as the polling mechanism.
00000000:localnodes.accept(0004)=0009 from [127.0.0.1:63542] ALPN=<none>
00000000:localnodes.clicls[0009:ffffffff]
00000000:localnodes.closed[0009:ffffffff]
00000001:localnodes.accept(0004)=0009 from [127.0.0.1:63544] ALPN=<none>
00000001:localnodes.clireq[0009:ffffffff]: GET / HTTP/1.1
00000001:localnodes.clihdr[0009:ffffffff]: Host: localhost:8080
00000001:localnodes.clihdr[0009:ffffffff]: Connection: Upgrade
00000001:localnodes.clihdr[0009:ffffffff]: Pragma: no-cache
00000001:localnodes.clihdr[0009:ffffffff]: Cache-Control: no-cache
00000001:localnodes.clihdr[0009:ffffffff]: Authorization: Basic bXl1c2VybmFtZTpteXBhc3N3b3Jk
00000001:localnodes.clihdr[0009:ffffffff]: User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36
00000001:localnodes.clihdr[0009:ffffffff]: Upgrade: websocket
00000001:localnodes.clihdr[0009:ffffffff]: Origin: https://localhost:8080
00000001:localnodes.clihdr[0009:ffffffff]: Sec-WebSocket-Version: 13
00000001:localnodes.clihdr[0009:ffffffff]: Accept-Encoding: gzip, deflate, br
00000001:localnodes.clihdr[0009:ffffffff]: Accept-Language: en-US,en;q=0.9
00000001:localnodes.clihdr[0009:ffffffff]: Sec-WebSocket-Key: fjfkDkUeLqZPGAuvKYYPqw==
00000001:localnodes.clihdr[0009:ffffffff]: Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits
00000001:galaxynodes.srvrep[0009:000a]: HTTP/1.1 101 Switching Protocols
00000001:galaxynodes.srvhdr[0009:000a]: Upgrade: websocket
00000001:galaxynodes.srvhdr[0009:000a]: Connection: Upgrade
00000001:galaxynodes.srvhdr[0009:000a]: Sec-WebSocket-Accept: MJ31YajGyhHwHUYCYeGeQw1V5c8=
00000001:galaxynodes.srvcls[0009:adfd]
00000001:galaxynodes.clicls[0009:adfd]
00000001:galaxynodes.closed[0009:adfd]
Also, here are the contents of my haproxy.cfg file:
global
log 127.0.0.1 local0
log 127.0.0.1 local1 debug
tune.ssl.default-dh-param 2048
maxconn 4096
defaults
log global
mode http
option httplog
option dontlognull
retries 3
option redispatch
maxconn 2000
timeout connect 5s
timeout client 1h
timeout server 1h
userlist MyDeviceUsers
user myusername insecure-password mypassword
# This section governs how web browser connect to HAProxy running on the Mac
frontend localnodes
bind *:8080 ssl crt /etc/ssl/private/my_combined_file.pem
reqadd X-Forwarded-Proto:\ https
mode http
default_backend mydevicenodes
acl ValidMyDeviceUser http_auth(MyDeviceUsers)
http-request auth realm MyDevice if !ValidMyDeviceUser # Commenting out this line allows Safari's WebSockets to connect through HAProxy, at the expense of no authentication
# This section governs how HAProxy connects to webd running on the MyDevice modules
backend mydevicenodes
mode http
balance roundrobin
option forwardfor
http-request set-header X-Forwarded-Port %[dst_port]
http-request add-header X-Forwarded-Proto https if { ssl_fc }
server server1 fe80::21c:abff:fe00:55e4%en1:8080 check # IPv6 address of my first MyDevice module
server server2 fe80::21c:abff:fe00:594c%en1:8080 check # IPv6 address of my second MyDevice module