Discussion:
[Gluster-users] glusterfs 4.1.5 - SSL3_GET_RECORD:wrong version number
Davide Obbi
2018-10-09 13:27:38 UTC
Permalink
Hi,

i have enabled SSL/TLS on a cluster of 3 nodes, the server to server
communication seems working since gluster volume status returns the three
bricks while we are unable to mount from the client and the client can be
also one of the gluster nodes iteself.
Options:
/var/lib/glusterd/secure-acceess
option transport.socket.ssl-cert-depth 3

ssl.cipher-list:
HIGH:!SSLv2:!SSLv3:!TLSv1:!TLSv1.1:TLSv1.2:!3DES:!RC4:!aNULL:!ADH
auth.ssl-allow:
localhost,glusterserver-1005,glusterserver-1008,glusterserver-1009
server.ssl: on
client.ssl: on
auth.allow: glusterserver-1005,glusterserver-1008,glusterserver-1009
ssl.certificate-depth: 3

We noticed the following in glusterd logs, the .18 address is the client
and one of the cluster nodes glusterserver-1005:
[2018-10-09 13:12:10.786384] D [socket.c:354:ssl_setup_connection]
0-tcp.management: peer CN = glusterserver-1005

[2018-10-09 13:12:10.786401] D [socket.c:357:ssl_setup_connection]
0-tcp.management: SSL verification succeeded (client: 10.10.0.18:49149)
(server: 10.10.0.18:24007)
[2018-10-09 13:12:10.956960] D [socket.c:354:ssl_setup_connection]
0-tcp.management: peer CN = glusterserver-1009

[2018-10-09 13:12:10.956977] D [socket.c:357:ssl_setup_connection]
0-tcp.management: SSL verification succeeded (client: 10.10.0.27:49150)
(server: 10.10.0.18:24007)
[2018-10-09 13:12:11.322218] D [socket.c:354:ssl_setup_connection]
0-tcp.management: peer CN = glusterserver-1008

[2018-10-09 13:12:11.322248] D [socket.c:357:ssl_setup_connection]
0-tcp.management: SSL verification succeeded (client: 10.10.0.23:49150)
(server: 10.10.0.18:24007)
[2018-10-09 13:12:11.368753] D [socket.c:354:ssl_setup_connection]
0-tcp.management: peer CN = glusterserver-1005

[2018-10-09 13:12:11.368770] D [socket.c:357:ssl_setup_connection]
0-tcp.management: SSL verification succeeded (client: 10.10.0.18:49149)
(server: 10.10.0.18:24007)
[2018-10-09 13:12:13.535081] E [socket.c:364:ssl_setup_connection]
0-tcp.management: SSL connect error (client: 10.10.0.18:49149) (server:
10.10.0.18:24007)
[2018-10-09 13:12:13.535102] E [socket.c:203:ssl_dump_error_stack]
0-tcp.management: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong
version number
[2018-10-09 13:12:13.535129] E [socket.c:2677:socket_poller]
0-tcp.management: server setup failed

I believe that something has changed since version 4.1.3 cause using that
version we were able to mount on the client and we did not get that SSL
error. Also the cipher volume option was not set in that version. At this
point i can't understand if node to node is actually using SSL or not and
why the client is unable to mount

thanks
Davide
Davide Obbi
2018-10-09 15:10:21 UTC
Permalink
Hi,

after running volume stop/start the error disappeared and the volume can be
mounted from the server.

Regards
Post by Davide Obbi
Hi,
i have enabled SSL/TLS on a cluster of 3 nodes, the server to server
communication seems working since gluster volume status returns the three
bricks while we are unable to mount from the client and the client can be
also one of the gluster nodes iteself.
/var/lib/glusterd/secure-acceess
option transport.socket.ssl-cert-depth 3
HIGH:!SSLv2:!SSLv3:!TLSv1:!TLSv1.1:TLSv1.2:!3DES:!RC4:!aNULL:!ADH
localhost,glusterserver-1005,glusterserver-1008,glusterserver-1009
server.ssl: on
client.ssl: on
auth.allow: glusterserver-1005,glusterserver-1008,glusterserver-1009
ssl.certificate-depth: 3
We noticed the following in glusterd logs, the .18 address is the client
[2018-10-09 13:12:10.786384] D [socket.c:354:ssl_setup_connection]
0-tcp.management: peer CN = glusterserver-1005
[2018-10-09 13:12:10.786401] D [socket.c:357:ssl_setup_connection]
0-tcp.management: SSL verification succeeded (client: 10.10.0.18:49149)
(server: 10.10.0.18:24007)
[2018-10-09 13:12:10.956960] D [socket.c:354:ssl_setup_connection]
0-tcp.management: peer CN = glusterserver-1009
[2018-10-09 13:12:10.956977] D [socket.c:357:ssl_setup_connection]
0-tcp.management: SSL verification succeeded (client: 10.10.0.27:49150)
(server: 10.10.0.18:24007)
[2018-10-09 13:12:11.322218] D [socket.c:354:ssl_setup_connection]
0-tcp.management: peer CN = glusterserver-1008
[2018-10-09 13:12:11.322248] D [socket.c:357:ssl_setup_connection]
0-tcp.management: SSL verification succeeded (client: 10.10.0.23:49150)
(server: 10.10.0.18:24007)
[2018-10-09 13:12:11.368753] D [socket.c:354:ssl_setup_connection]
0-tcp.management: peer CN = glusterserver-1005
[2018-10-09 13:12:11.368770] D [socket.c:357:ssl_setup_connection]
0-tcp.management: SSL verification succeeded (client: 10.10.0.18:49149)
(server: 10.10.0.18:24007)
[2018-10-09 13:12:13.535081] E [socket.c:364:ssl_setup_connection]
10.10.0.18:24007)
[2018-10-09 13:12:13.535102] E [socket.c:203:ssl_dump_error_stack]
0-tcp.management: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong
version number
[2018-10-09 13:12:13.535129] E [socket.c:2677:socket_poller]
0-tcp.management: server setup failed
I believe that something has changed since version 4.1.3 cause using that
version we were able to mount on the client and we did not get that SSL
error. Also the cipher volume option was not set in that version. At this
point i can't understand if node to node is actually using SSL or not and
why the client is unable to mount
thanks
Davide
--
Davide Obbi
System Administrator

Booking.com B.V.
Vijzelstraat 66-80 Amsterdam 1017HL Netherlands
Direct +31207031558
[image: Booking.com] <https://www.booking.com/>
The world's #1 accommodation site
43 languages, 198+ offices worldwide, 120,000+ global destinations,
1,550,000+ room nights booked every day
No booking fees, best price always guaranteed
Subsidiary of Booking Holdings Inc. (NASDAQ: BKNG)
Loading...