Discussion:
HAproxy ssh connection closes fast , after logon
Imam Toufique
2018-10-19 18:21:36 UTC
Permalink
Hi,

I am working on setting up ssh / sftp capability with HAProxy,
initial setup is done ( thanks to some of the members in the haproxy email
list for help! ) . I ran into an issue -- as soon as I ssh via the proxy
node, within a minutes or so, ssh connection closes on me.



here is my config file:
---------------------------------

global
log /dev/log local0
log /dev/log local1 notice
chroot /var/lib/haproxy
stats timeout 30s
user haproxy
group haproxy
daemon

defaults
log global
mode http
option tcplog
option dontlognull
timeout connect 5000
timeout client 50000
timeout server 50000

frontend http_front
bind *:80
stats uri /haproxy?stats
default_backend http_back
mode http
option forwardfor # forward IP
http-request set-header X-Forwarded-Port %[dst_port]
http-request add-header X-Forwarded-Proto https if { ssl_fc }

backend http_back
balance roundrobin # roundrobin is rotate customers into backend server
server web1 10.1.100.156:80 <http://10.1.100.156/> check inter 2000
cookie w1
server web2 10.1.100.160:80 <http://10.1.100.160/> check inter 2000
cookie w1
timeout connect 900000
timeout server 900000

frontend www-ssh-proxy
bind *:2200
mode tcp
default_backend www-ssh-proxy-backend

backend www-ssh-proxy-backend
mode tcp
balance roundrobin
stick-table type ip size 200k expire 30m
stick on src
default-server inter 1s
server web1 10.1.100.156:22 check id 1
server web2 10.1.100.160:22 check id 2


Here is the ssh related message with debug enabled:

[***@web1 ~]$ debug3: send packet: type 1
debug1: channel 0: free: client-session, nchannels 1
debug3: channel 0: status: The following connections are open:
#0 client-session (t4 r0 i0/0 o0/0 fd 4/5 cc -1)

Connection to crsplab2 closed by remote host.
Connection to crsplab2 closed.
Transferred: sent 3312, received 3184 bytes, in 50.3 seconds
Bytes per second: sent 65.8, received 63.3
debug1: Exit status -1

I am not sure why this is happening. I did not change anything in
sshd_config files anywhere.

can you guys help me with some hints?

thanks, as always!
--
Regards,
*Imam Toufique*
*213-700-5485*
Jarno Huuskonen
2018-10-20 07:10:41 UTC
Permalink
Hi,
Post by Imam Toufique
I am working on setting up ssh / sftp capability with HAProxy,
initial setup is done ( thanks to some of the members in the haproxy email
list for help! ) . I ran into an issue -- as soon as I ssh via the proxy
node, within a minutes or so, ssh connection closes on me.
Is the connection idle ? "in 50.3 seconds" matches your
timeout client/server 50000(ms)
(Your (haproxy)logs should give more information why the connection
was closed).
Post by Imam Toufique
---------------------------------
...
Post by Imam Toufique
timeout client 50000
timeout server 50000
...
Post by Imam Toufique
backend http_back
timeout connect 900000
timeout server 900000
900000(ms)(=15minutes) connect timeout for http seems long ...
(AFAIK this is how long haproxy will wait for tcp connection
to a backend http server).
Post by Imam Toufique
backend www-ssh-proxy-backend
mode tcp
balance roundrobin
stick-table type ip size 200k expire 30m
stick on src
default-server inter 1s
server web1 10.1.100.156:22 check id 1
server web2 10.1.100.160:22 check id 2
Try using longer timeout server on www-ssh-proxy-backend.
(and/or longer timeout client on www-ssh-proxy).

(You could also try to play with sshd_config: ClientAliveInterval and
TCPKeepAlive)
Post by Imam Toufique
Transferred: sent 3312, received 3184 bytes, in 50.3 seconds
-Jarno
--
Jarno Huuskonen
Imam Toufique
2018-10-20 07:48:07 UTC
Permalink
Thanks, Jarno!

Brian Gibson helped me out! thanks, Brian!
Post by Jarno Huuskonen
Hi,
Post by Imam Toufique
I am working on setting up ssh / sftp capability with HAProxy,
initial setup is done ( thanks to some of the members in the haproxy
email
Post by Imam Toufique
list for help! ) . I ran into an issue -- as soon as I ssh via the
proxy
Post by Imam Toufique
node, within a minutes or so, ssh connection closes on me.
Is the connection idle ? "in 50.3 seconds" matches your
timeout client/server 50000(ms)
(Your (haproxy)logs should give more information why the connection
was closed).
Post by Imam Toufique
---------------------------------
...
Post by Imam Toufique
timeout client 50000
timeout server 50000
...
Post by Imam Toufique
backend http_back
timeout connect 900000
timeout server 900000
900000(ms)(=15minutes) connect timeout for http seems long ...
(AFAIK this is how long haproxy will wait for tcp connection
to a backend http server).
Post by Imam Toufique
backend www-ssh-proxy-backend
mode tcp
balance roundrobin
stick-table type ip size 200k expire 30m
stick on src
default-server inter 1s
server web1 10.1.100.156:22 check id 1
server web2 10.1.100.160:22 check id 2
Try using longer timeout server on www-ssh-proxy-backend.
(and/or longer timeout client on www-ssh-proxy).
(You could also try to play with sshd_config: ClientAliveInterval and
TCPKeepAlive)
Post by Imam Toufique
Transferred: sent 3312, received 3184 bytes, in 50.3 seconds
-Jarno
--
Jarno Huuskonen
--
Regards,
*Imam Toufique*
*213-700-5485*
Imam Toufique
2018-10-20 08:43:56 UTC
Permalink
Hi Aleks,

Basically, I needed to bind to a different port then 22. and , with
Brian's help, i ended up with the following config:

frontend www-ssh-proxy
bind x.x.x.x:22
mode tcp
default_backend www-ssh-proxy-backend

backend www-ssh-proxy-backend
mode tcp
balance source
server web1 10.1.100.156:22 check weight 1
server web2 10.1.100.160:22 check weight 1

I have now changed the port 2200, to 22 -- by using 2200 for sshd itself
and 22 for binding.

This is not the perfect set up yet, but this is better then what I had
before.

thanks.
Hi.
Post by Imam Toufique
Thanks, Jarno!
Brian Gibson helped me out! thanks, Brian!
Would be nice to know what's the solution was, thanks.
Regards
Aleks
Post by Imam Toufique
Hi,
Post by Imam Toufique
I am working on setting up ssh / sftp capability with HAProxy,
initial setup is done ( thanks to some of the members in the
haproxy email
Post by Imam Toufique
Post by Imam Toufique
list for help! ) . I ran into an issue -- as soon as I ssh via
the proxy
Post by Imam Toufique
Post by Imam Toufique
node, within a minutes or so, ssh connection closes on me.
Is the connection idle ? "in 50.3 seconds" matches your
timeout client/server 50000(ms)
(Your (haproxy)logs should give more information why the connection
was closed).
Post by Imam Toufique
---------------------------------
...
Post by Imam Toufique
timeout client 50000
timeout server 50000
...
Post by Imam Toufique
backend http_back
timeout connect 900000
timeout server 900000
900000(ms)(=15minutes) connect timeout for http seems long ...
(AFAIK this is how long haproxy will wait for tcp connection
to a backend http server).
Post by Imam Toufique
backend www-ssh-proxy-backend
mode tcp
balance roundrobin
stick-table type ip size 200k expire 30m
stick on src
default-server inter 1s
server web1 10.1.100.156:22 <http://10.1.100.156:22> check id 1
server web2 10.1.100.160:22 <http://10.1.100.160:22> check id 2
Try using longer timeout server on www-ssh-proxy-backend.
(and/or longer timeout client on www-ssh-proxy).
(You could also try to play with sshd_config: ClientAliveInterval and
TCPKeepAlive)
Post by Imam Toufique
Transferred: sent 3312, received 3184 bytes, in 50.3 seconds
-Jarno
--
Jarno Huuskonen
--
Regards,
*/Imam Toufique/*
/*213-700-5485*/
--
Regards,
*Imam Toufique*
*213-700-5485*
Loading...