[Gluster-users] Issues with SSL connections

[t***@baesystems.com]

Hi all,

I posted earlier concerning rolling back on the version of gluster we are using.

After being successful in doing so I've come to realise the issue we were facing may not have been introduced by the latest versions of gluster but are instead an issue with configuration.

We haven't changed how we were installing and setting up glusterFS but recently we are having issues with SSL, NFS and in turn the log file filling up the root partition.

I've attached some relevant extracts from logs below of the issue we are facing (some may not be relevant) - I haven't been able to find anything in the mailing list that solves the issue.


From glusterd log:

```
[2016-07-20 10:25:31.700017] I [rpc-clnt.c:1004:rpc_clnt_connection_init] 0-snapd: setting frame-timeout to 600
[2016-07-20 10:25:31.700076] I [socket.c:3927:socket_init] 0-snapd: SSL support for glusterd is ENABLED
[2016-07-20 10:25:31.700191] E [socket.c:4005:socket_init] 0-snapd: failed to open /etc/ssl/dhparam.pem, DH ciphers are disabled
```
```
[2016-07-20 10:25:31.742672] W [socket.c:589:__socket_rwv] 0-socket.management: writev on 127.0.0.1:65510 failed (No data available)
[2016-07-20 10:25:31.742715] E [socket.c:2497:socket_poller] 0-socket.management: poll error on socket
```
```
[2016-07-20 10:25:31.777638] I [MSGID: 106502] [glusterd-handler.c:2821:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2016-07-20 10:25:31.777925] I [MSGID: 106006] [glusterd-svc-mgmt.c:323:glusterd_svc_common_rpc_notify] 0-management: nfs has disconnected from glusterd.
[2016-07-20 10:25:31.777974] I [MSGID: 106006] [glusterd-svc-mgmt.c:323:glusterd_svc_common_rpc_notify] 0-management: glustershd has disconnected from glusterd.
[2016-07-20 10:25:31.778527] E [socket.c:2502:socket_poller] 0-socket.management: error in polling loop
```
And then the following streams and fills up the log file ~GBs in the matter of hours
```
[2016-07-20 10:25:31.865991] E [socket.c:2502:socket_poller] 0-socket.management: error in polling loop
[2016-07-20 10:25:32.670678] E [socket.c:352:ssl_setup_connection] 0-socket.management: SSL connect error
[2016-07-20 10:25:32.670726] E [socket.c:206:ssl_dump_error_stack] 0-socket.management: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number
[2016-07-20 10:25:32.670744] E [socket.c:2389:socket_poller] 0-socket.management: server setup failed
[2016-07-20 10:25:34.217120] I [MSGID: 106006] [glusterd-svc-mgmt.c:323:glusterd_svc_common_rpc_notify] 0-management: nfs has disconnected from glusterd.
[2016-07-20 10:25:34.301599] I [socket.c:347:ssl_setup_connection] 0-socket.management: peer CN = ip-172-31-169-227
[2016-07-20 10:25:34.303179] E [socket.c:2502:socket_poller] 0-socket.management: error in polling loop
[2016-07-20 10:25:34.378333] I [socket.c:347:ssl_setup_connection] 0-socket.management: peer CN = ip-172-31-169-165
[2016-07-20 10:25:34.379965] E [socket.c:2502:socket_poller] 0-socket.management: error in polling loop
[2016-07-20 10:25:35.389754] I [socket.c:347:ssl_setup_connection] 0-socket.management: peer CN = ip-172-31-169-65
[2016-07-20 10:25:35.390996] E [socket.c:2502:socket_poller] 0-socket.management: error in polling loop
[2016-07-20 10:25:35.642357] I [socket.c:347:ssl_setup_connection] 0-socket.management: peer CN = ip-172-31-169-59
[2016-07-20 10:25:35.642815] E [socket.c:2502:socket_poller] 0-socket.management: error in polling loop
[2016-07-20 10:25:35.671336] E [socket.c:352:ssl_setup_connection] 0-socket.management: SSL connect error
[2016-07-20 10:25:35.671376] E [socket.c:206:ssl_dump_error_stack] 0-socket.management: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number
```

I have also attached nfs.log and glustershd.log

Thanks in advance for all the help!
Tommy
Please consider the environment before printing this email. This message should be regarded as confidential. If you have received this email in error please notify the sender and destroy it immediately. Statements of intent shall only become binding when confirmed in hard copy by an authorised signatory. The contents of this email may relate to dealings with other companies under the control of BAE Systems Applied Intelligence Limited, details of which can be found at http://www.baesystems.com/Businesses/index.htm.

[Atin Mukherjee]

+ Mohit

We are aware of this issue and there is a RHBZ [1] filed for it.

We have a plan to fix that in coming 3.7.x release, probably 3.7.15 if not
3.7.14.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1309215

Thanks,
Atin

Post by t***@baesystems.com
Hi all,
I posted earlier concerning rolling back on the version of gluster we are using.
After being successful in doing so I’ve come to realise the issue we were
facing may not have been introduced by the latest versions of gluster but
are instead an issue with configuration.
We haven’t changed how we were installing and setting up glusterFS but
recently we are having issues with SSL, NFS and in turn the log file
filling up the root partition.
I’ve attached some relevant extracts from logs below of the issue we are
facing (some may not be relevant) – I haven’t been able to find anything in
the mailing list that solves the issue.
```
[2016-07-20 10:25:31.700017] I [rpc-clnt.c:1004:rpc_clnt_connection_init]
0-snapd: setting frame-timeout to 600
[2016-07-20 10:25:31.700076] I [socket.c:3927:socket_init] 0-snapd: SSL
support for glusterd is ENABLED
[2016-07-20 10:25:31.700191] E [socket.c:4005:socket_init] 0-snapd: failed
to open /etc/ssl/dhparam.pem, DH ciphers are disabled
```
```
[2016-07-20 10:25:31.742672] W [socket.c:589:__socket_rwv]
0-socket.management: writev on 127.0.0.1:65510 failed (No data available)
[2016-07-20 10:25:31.742715] E [socket.c:2497:socket_poller]
0-socket.management: poll error on socket
```
```
[2016-07-20 10:25:31.777638] I [MSGID: 106502]
Received my uuid as Friend
[2016-07-20 10:25:31.777925] I [MSGID: 106006]
[glusterd-svc-mgmt.c:323:glusterd_svc_common_rpc_notify] 0-management: nfs
has disconnected from glusterd.
[2016-07-20 10:25:31.777974] I [MSGID: 106006]
glustershd has disconnected from glusterd.
[2016-07-20 10:25:31.778527] E [socket.c:2502:socket_poller]
0-socket.management: error in polling loop
```
And then the following streams and fills up the log file ~GBs in the matter of hours
```
[2016-07-20 10:25:31.865991] E [socket.c:2502:socket_poller]
0-socket.management: error in polling loop
[2016-07-20 10:25:32.670678] E [socket.c:352:ssl_setup_connection]
0-socket.management: SSL connect error
[2016-07-20 10:25:32.670726] E [socket.c:206:ssl_dump_error_stack]
0-socket.management: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong
version number
[2016-07-20 10:25:32.670744] E [socket.c:2389:socket_poller]
0-socket.management: server setup failed
[2016-07-20 10:25:34.217120] I [MSGID: 106006]
[glusterd-svc-mgmt.c:323:glusterd_svc_common_rpc_notify] 0-management: nfs
has disconnected from glusterd.
[2016-07-20 10:25:34.301599] I [socket.c:347:ssl_setup_connection]
0-socket.management: peer CN = ip-172-31-169-227
[2016-07-20 10:25:34.303179] E [socket.c:2502:socket_poller]
0-socket.management: error in polling loop
[2016-07-20 10:25:34.378333] I [socket.c:347:ssl_setup_connection]
0-socket.management: peer CN = ip-172-31-169-165
[2016-07-20 10:25:34.379965] E [socket.c:2502:socket_poller]
0-socket.management: error in polling loop
[2016-07-20 10:25:35.389754] I [socket.c:347:ssl_setup_connection]
0-socket.management: peer CN = ip-172-31-169-65
[2016-07-20 10:25:35.390996] E [socket.c:2502:socket_poller]
0-socket.management: error in polling loop
[2016-07-20 10:25:35.642357] I [socket.c:347:ssl_setup_connection]
0-socket.management: peer CN = ip-172-31-169-59
[2016-07-20 10:25:35.642815] E [socket.c:2502:socket_poller]
0-socket.management: error in polling loop
[2016-07-20 10:25:35.671336] E [socket.c:352:ssl_setup_connection]
0-socket.management: SSL connect error
[2016-07-20 10:25:35.671376] E [socket.c:206:ssl_dump_error_stack]
0-socket.management: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong
version number
```
I have also attached nfs.log and glustershd.log
Thanks in advance for all the help!
Tommy
Please consider the environment before printing this email. This message
should be regarded as confidential. If you have received this email in
error please notify the sender and destroy it immediately. Statements of
intent shall only become binding when confirmed in hard copy by an
authorised signatory. The contents of this email may relate to dealings
with other companies under the control of BAE Systems Applied Intelligence
Limited, details of which can be found at
http://www.baesystems.com/Businesses/index.htm.
_______________________________________________
Gluster-users mailing list
http://www.gluster.org/mailman/listinfo/gluster-users

[t***@baesystems.com]

Ah great – good to see that it will be fixed soon.

Is this also what is causing the issues with NFS?

Thanks,
Tommy

From: Atin Mukherjee [mailto:***@redhat.com]
Sent: 20 July 2016 12:24
To: Yardley, Tommy (UK Guildford); Mohit Agrawal
Cc: gluster-***@gluster.org
Subject: Re: [Gluster-users] Issues with SSL connections

+ Mohit
We are aware of this issue and there is a RHBZ [1] filed for it.
We have a plan to fix that in coming 3.7.x release, probably 3.7.15 if not 3.7.14.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1309215
Thanks,
Atin

On Wed, Jul 20, 2016 at 4:16 PM, ***@baesystems.com<mailto:***@baesystems.com> <***@baesystems.com<mailto:***@baesystems.com>> wrote:
Hi all,

I posted earlier concerning rolling back on the version of gluster we are using.

After being successful in doing so I’ve come to realise the issue we were facing may not have been introduced by the latest versions of gluster but are instead an issue with configuration.

We haven’t changed how we were installing and setting up glusterFS but recently we are having issues with SSL, NFS and in turn the log file filling up the root partition.

I’ve attached some relevant extracts from logs below of the issue we are facing (some may not be relevant) – I haven’t been able to find anything in the mailing list that solves the issue.


From glusterd log:

```
[2016-07-20 10:25:31.700017] I [rpc-clnt.c:1004:rpc_clnt_connection_init] 0-snapd: setting frame-timeout to 600
[2016-07-20 10:25:31.700076] I [socket.c:3927:socket_init] 0-snapd: SSL support for glusterd is ENABLED
[2016-07-20 10:25:31.700191] E [socket.c:4005:socket_init] 0-snapd: failed to open /etc/ssl/dhparam.pem, DH ciphers are disabled
```
```
[2016-07-20 10:25:31.742672] W [socket.c:589:__socket_rwv] 0-socket.management: writev on 127.0.0.1:65510<http://127.0.0.1:65510> failed (No data available)
[2016-07-20 10:25:31.742715] E [socket.c:2497:socket_poller] 0-socket.management: poll error on socket
```
```
[2016-07-20 10:25:31.777638] I [MSGID: 106502] [glusterd-handler.c:2821:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2016-07-20 10:25:31.777925] I [MSGID: 106006] [glusterd-svc-mgmt.c:323:glusterd_svc_common_rpc_notify] 0-management: nfs has disconnected from glusterd.
[2016-07-20 10:25:31.777974] I [MSGID: 106006] [glusterd-svc-mgmt.c:323:glusterd_svc_common_rpc_notify] 0-management: glustershd has disconnected from glusterd.
[2016-07-20 10:25:31.778527] E [socket.c:2502:socket_poller] 0-socket.management: error in polling loop
```
And then the following streams and fills up the log file ~GBs in the matter of hours
```
[2016-07-20 10:25:31.865991] E [socket.c:2502:socket_poller] 0-socket.management: error in polling loop
[2016-07-20 10:25:32.670678] E [socket.c:352:ssl_setup_connection] 0-socket.management: SSL connect error
[2016-07-20 10:25:32.670726] E [socket.c:206:ssl_dump_error_stack] 0-socket.management: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number
[2016-07-20 10:25:32.670744] E [socket.c:2389:socket_poller] 0-socket.management: server setup failed
[2016-07-20 10:25:34.217120] I [MSGID: 106006] [glusterd-svc-mgmt.c:323:glusterd_svc_common_rpc_notify] 0-management: nfs has disconnected from glusterd.
[2016-07-20 10:25:34.301599] I [socket.c:347:ssl_setup_connection] 0-socket.management: peer CN = ip-172-31-169-227
[2016-07-20 10:25:34.303179] E [socket.c:2502:socket_poller] 0-socket.management: error in polling loop
[2016-07-20 10:25:34.378333] I [socket.c:347:ssl_setup_connection] 0-socket.management: peer CN = ip-172-31-169-165
[2016-07-20 10:25:34.379965] E [socket.c:2502:socket_poller] 0-socket.management: error in polling loop
[2016-07-20 10:25:35.389754] I [socket.c:347:ssl_setup_connection] 0-socket.management: peer CN = ip-172-31-169-65
[2016-07-20 10:25:35.390996] E [socket.c:2502:socket_poller] 0-socket.management: error in polling loop
[2016-07-20 10:25:35.642357] I [socket.c:347:ssl_setup_connection] 0-socket.management: peer CN = ip-172-31-169-59
[2016-07-20 10:25:35.642815] E [socket.c:2502:socket_poller] 0-socket.management: error in polling loop
[2016-07-20 10:25:35.671336] E [socket.c:352:ssl_setup_connection] 0-socket.management: SSL connect error
[2016-07-20 10:25:35.671376] E [socket.c:206:ssl_dump_error_stack] 0-socket.management: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number
```

I have also attached nfs.log and glustershd.log

Thanks in advance for all the help!
Tommy
Please consider the environment before printing this email. This message should be regarded as confidential. If you have received this email in error please notify the sender and destroy it immediately. Statements of intent shall only become binding when confirmed in hard copy by an authorised signatory. The contents of this email may relate to dealings with other companies under the control of BAE Systems Applied Intelligence Limited, details of which can be found at http://www.baesystems.com/Businesses/index.htm.

_______________________________________________
Gluster-users mailing list
Gluster-***@gluster.org<mailto:Gluster-***@gluster.org>
http://www.gluster.org/mailman/listinfo/gluster-users

[Atin Mukherjee]

Post by t***@baesystems.com
Ah great – good to see that it will be fixed soon.
Is this also what is causing the issues with NFS?

Could be, Mohit will get back.

Post by t***@baesystems.com
Thanks,
Tommy
*Sent:* 20 July 2016 12:24
*To:* Yardley, Tommy (UK Guildford); Mohit Agrawal
*Subject:* Re: [Gluster-users] Issues with SSL connections
+ Mohit
We are aware of this issue and there is a RHBZ [1] filed for it.
We have a plan to fix that in coming 3.7.x release, probably 3.7.15 if not 3.7.14.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1309215
Thanks,
Atin
Hi all,
I posted earlier concerning rolling back on the version of gluster we are using.
After being successful in doing so I’ve come to realise the issue we were
facing may not have been introduced by the latest versions of gluster but
are instead an issue with configuration.
We haven’t changed how we were installing and setting up glusterFS but
recently we are having issues with SSL, NFS and in turn the log file
filling up the root partition.
I’ve attached some relevant extracts from logs below of the issue we are
facing (some may not be relevant) – I haven’t been able to find anything in
the mailing list that solves the issue.
```
[2016-07-20 10:25:31.700017] I [rpc-clnt.c:1004:rpc_clnt_connection_init]
0-snapd: setting frame-timeout to 600
[2016-07-20 10:25:31.700076] I [socket.c:3927:socket_init] 0-snapd: SSL
support for glusterd is ENABLED
[2016-07-20 10:25:31.700191] E [socket.c:4005:socket_init] 0-snapd: failed
to open /etc/ssl/dhparam.pem, DH ciphers are disabled
```
```
[2016-07-20 10:25:31.742672] W [socket.c:589:__socket_rwv]
0-socket.management: writev on 127.0.0.1:65510 failed (No data available)
[2016-07-20 10:25:31.742715] E [socket.c:2497:socket_poller]
0-socket.management: poll error on socket
```
```
[2016-07-20 10:25:31.777638] I [MSGID: 106502]
Received my uuid as Friend
[2016-07-20 10:25:31.777925] I [MSGID: 106006]
[glusterd-svc-mgmt.c:323:glusterd_svc_common_rpc_notify] 0-management: nfs
has disconnected from glusterd.
[2016-07-20 10:25:31.777974] I [MSGID: 106006]
glustershd has disconnected from glusterd.
[2016-07-20 10:25:31.778527] E [socket.c:2502:socket_poller]
0-socket.management: error in polling loop
```
And then the following streams and fills up the log file ~GBs in the matter of hours
```
[2016-07-20 10:25:31.865991] E [socket.c:2502:socket_poller]
0-socket.management: error in polling loop
[2016-07-20 10:25:32.670678] E [socket.c:352:ssl_setup_connection]
0-socket.management: SSL connect error
[2016-07-20 10:25:32.670726] E [socket.c:206:ssl_dump_error_stack]
0-socket.management: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong
version number
[2016-07-20 10:25:32.670744] E [socket.c:2389:socket_poller]
0-socket.management: server setup failed
[2016-07-20 10:25:34.217120] I [MSGID: 106006]
[glusterd-svc-mgmt.c:323:glusterd_svc_common_rpc_notify] 0-management: nfs
has disconnected from glusterd.
[2016-07-20 10:25:34.301599] I [socket.c:347:ssl_setup_connection]
0-socket.management: peer CN = ip-172-31-169-227
[2016-07-20 10:25:34.303179] E [socket.c:2502:socket_poller]
0-socket.management: error in polling loop
[2016-07-20 10:25:34.378333] I [socket.c:347:ssl_setup_connection]
0-socket.management: peer CN = ip-172-31-169-165
[2016-07-20 10:25:34.379965] E [socket.c:2502:socket_poller]
0-socket.management: error in polling loop
[2016-07-20 10:25:35.389754] I [socket.c:347:ssl_setup_connection]
0-socket.management: peer CN = ip-172-31-169-65
[2016-07-20 10:25:35.390996] E [socket.c:2502:socket_poller]
0-socket.management: error in polling loop
[2016-07-20 10:25:35.642357] I [socket.c:347:ssl_setup_connection]
0-socket.management: peer CN = ip-172-31-169-59
[2016-07-20 10:25:35.642815] E [socket.c:2502:socket_poller]
0-socket.management: error in polling loop
[2016-07-20 10:25:35.671336] E [socket.c:352:ssl_setup_connection]
0-socket.management: SSL connect error
[2016-07-20 10:25:35.671376] E [socket.c:206:ssl_dump_error_stack]
0-socket.management: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong
version number
```
I have also attached nfs.log and glustershd.log
Thanks in advance for all the help!
Tommy
Please consider the environment before printing this email. This message
should be regarded as confidential. If you have received this email in
error please notify the sender and destroy it immediately. Statements of
intent shall only become binding when confirmed in hard copy by an
authorised signatory. The contents of this email may relate to dealings
with other companies under the control of BAE Systems Applied Intelligence
Limited, details of which can be found at
http://www.baesystems.com/Businesses/index.htm.
_______________________________________________
Gluster-users mailing list
http://www.gluster.org/mailman/listinfo/gluster-users
--
--Atin
Please consider the environment before printing this email. This message
should be regarded as confidential. If you have received this email in
error please notify the sender and destroy it immediately. Statements of
intent shall only become binding when confirmed in hard copy by an
authorised signatory. The contents of this email may relate to dealings
with other companies under the control of BAE Systems Applied Intelligence
Limited, details of which can be found at
http://www.baesystems.com/Businesses/index.htm.

[Mohit Agrawal]

Hi tommy,

Can you please share reproducer steps to reproduce the issue?


Regards
Mohit Agrawal

Post by Atin Mukherjee

Post by t***@baesystems.com
Ah great – good to see that it will be fixed soon.
Is this also what is causing the issues with NFS?

Could be, Mohit will get back.

Post by t***@baesystems.com
Thanks,
Tommy
*Sent:* 20 July 2016 12:24
*To:* Yardley, Tommy (UK Guildford); Mohit Agrawal
*Subject:* Re: [Gluster-users] Issues with SSL connections
+ Mohit
We are aware of this issue and there is a RHBZ [1] filed for it.
We have a plan to fix that in coming 3.7.x release, probably 3.7.15 if not 3.7.14.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1309215
Thanks,
Atin
Hi all,
I posted earlier concerning rolling back on the version of gluster we are using.
After being successful in doing so I’ve come to realise the issue we were
facing may not have been introduced by the latest versions of gluster but
are instead an issue with configuration.
We haven’t changed how we were installing and setting up glusterFS but
recently we are having issues with SSL, NFS and in turn the log file
filling up the root partition.
I’ve attached some relevant extracts from logs below of the issue we are
facing (some may not be relevant) – I haven’t been able to find anything in
the mailing list that solves the issue.
```
[2016-07-20 10:25:31.700017] I [rpc-clnt.c:1004:rpc_clnt_connection_init]
0-snapd: setting frame-timeout to 600
[2016-07-20 10:25:31.700076] I [socket.c:3927:socket_init] 0-snapd: SSL
support for glusterd is ENABLED
failed to open /etc/ssl/dhparam.pem, DH ciphers are disabled
```
```
[2016-07-20 10:25:31.742672] W [socket.c:589:__socket_rwv]
0-socket.management: writev on 127.0.0.1:65510 failed (No data available)
[2016-07-20 10:25:31.742715] E [socket.c:2497:socket_poller]
0-socket.management: poll error on socket
```
```
[2016-07-20 10:25:31.777638] I [MSGID: 106502]
Received my uuid as Friend
[2016-07-20 10:25:31.777925] I [MSGID: 106006]
[glusterd-svc-mgmt.c:323:glusterd_svc_common_rpc_notify] 0-management: nfs
has disconnected from glusterd.
[2016-07-20 10:25:31.777974] I [MSGID: 106006]
glustershd has disconnected from glusterd.
[2016-07-20 10:25:31.778527] E [socket.c:2502:socket_poller]
0-socket.management: error in polling loop
```
And then the following streams and fills up the log file ~GBs in the matter of hours
```
[2016-07-20 10:25:31.865991] E [socket.c:2502:socket_poller]
0-socket.management: error in polling loop
[2016-07-20 10:25:32.670678] E [socket.c:352:ssl_setup_connection]
0-socket.management: SSL connect error
[2016-07-20 10:25:32.670726] E [socket.c:206:ssl_dump_error_stack]
0-socket.management: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong
version number
[2016-07-20 10:25:32.670744] E [socket.c:2389:socket_poller]
0-socket.management: server setup failed
[2016-07-20 10:25:34.217120] I [MSGID: 106006]
[glusterd-svc-mgmt.c:323:glusterd_svc_common_rpc_notify] 0-management: nfs
has disconnected from glusterd.
[2016-07-20 10:25:34.301599] I [socket.c:347:ssl_setup_connection]
0-socket.management: peer CN = ip-172-31-169-227
[2016-07-20 10:25:34.303179] E [socket.c:2502:socket_poller]
0-socket.management: error in polling loop
[2016-07-20 10:25:34.378333] I [socket.c:347:ssl_setup_connection]
0-socket.management: peer CN = ip-172-31-169-165
[2016-07-20 10:25:34.379965] E [socket.c:2502:socket_poller]
0-socket.management: error in polling loop
[2016-07-20 10:25:35.389754] I [socket.c:347:ssl_setup_connection]
0-socket.management: peer CN = ip-172-31-169-65
[2016-07-20 10:25:35.390996] E [socket.c:2502:socket_poller]
0-socket.management: error in polling loop
[2016-07-20 10:25:35.642357] I [socket.c:347:ssl_setup_connection]
0-socket.management: peer CN = ip-172-31-169-59
[2016-07-20 10:25:35.642815] E [socket.c:2502:socket_poller]
0-socket.management: error in polling loop
[2016-07-20 10:25:35.671336] E [socket.c:352:ssl_setup_connection]
0-socket.management: SSL connect error
[2016-07-20 10:25:35.671376] E [socket.c:206:ssl_dump_error_stack]
0-socket.management: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong
version number
```
I have also attached nfs.log and glustershd.log
Thanks in advance for all the help!
Tommy
Please consider the environment before printing this email. This message
should be regarded as confidential. If you have received this email in
error please notify the sender and destroy it immediately. Statements of
intent shall only become binding when confirmed in hard copy by an
authorised signatory. The contents of this email may relate to dealings
with other companies under the control of BAE Systems Applied Intelligence
Limited, details of which can be found at
http://www.baesystems.com/Businesses/index.htm.
_______________________________________________
Gluster-users mailing list
http://www.gluster.org/mailman/listinfo/gluster-users
--
--Atin
Please consider the environment before printing this email. This message
should be regarded as confidential. If you have received this email in
error please notify the sender and destroy it immediately. Statements of
intent shall only become binding when confirmed in hard copy by an
authorised signatory. The contents of this email may relate to dealings
with other companies under the control of BAE Systems Applied Intelligence
Limited, details of which can be found at
http://www.baesystems.com/Businesses/index.htm.

--
Atin
Sent from iPhone

[t***@baesystems.com]

Hi all,

Any ideas on what is the cause of the NFS issues I’m having?

Thanks,
Tommy

From: Atin Mukherjee [mailto:***@redhat.com]
Sent: 20 July 2016 15:51
To: Yardley, Tommy (UK Guildford)
Cc: Mohit Agrawal; gluster-***@gluster.org
Subject: Re: [Gluster-users] Issues with SSL connections



On Wednesday 20 July 2016, ***@baesystems.com<mailto:***@baesystems.com> <***@baesystems.com<mailto:***@baesystems.com>> wrote:
Ah great – good to see that it will be fixed soon.

Is this also what is causing the issues with NFS?

Could be, Mohit will get back.

Thanks,
Tommy

From: Atin Mukherjee [mailto:***@redhat.com<javascript:_e(%7B%7D,'cvml','***@redhat.com');>]
Sent: 20 July 2016 12:24
To: Yardley, Tommy (UK Guildford); Mohit Agrawal
Cc: gluster-***@gluster.org<javascript:_e(%7B%7D,'cvml','gluster-***@gluster.org');>
Subject: Re: [Gluster-users] Issues with SSL connections

+ Mohit
We are aware of this issue and there is a RHBZ [1] filed for it.
We have a plan to fix that in coming 3.7.x release, probably 3.7.15 if not 3.7.14.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1309215
Thanks,
Atin

On Wed, Jul 20, 2016 at 4:16 PM, ***@baesystems.com<javascript:_e(%7B%7D,'cvml','***@baesystems.com');> <***@baesystems.com<javascript:_e(%7B%7D,'cvml','***@baesystems.com');>> wrote:
Hi all,

I posted earlier concerning rolling back on the version of gluster we are using.

After being successful in doing so I’ve come to realise the issue we were facing may not have been introduced by the latest versions of gluster but are instead an issue with configuration.

We haven’t changed how we were installing and setting up glusterFS but recently we are having issues with SSL, NFS and in turn the log file filling up the root partition.

I’ve attached some relevant extracts from logs below of the issue we are facing (some may not be relevant) – I haven’t been able to find anything in the mailing list that solves the issue.


From glusterd log:

```
[2016-07-20 10:25:31.700017] I [rpc-clnt.c:1004:rpc_clnt_connection_init] 0-snapd: setting frame-timeout to 600
[2016-07-20 10:25:31.700076] I [socket.c:3927:socket_init] 0-snapd: SSL support for glusterd is ENABLED
[2016-07-20 10:25:31.700191] E [socket.c:4005:socket_init] 0-snapd: failed to open /etc/ssl/dhparam.pem, DH ciphers are disabled
```
```
[2016-07-20 10:25:31.742672] W [socket.c:589:__socket_rwv] 0-socket.management: writev on 127.0.0.1:65510<http://127.0.0.1:65510> failed (No data available)
[2016-07-20 10:25:31.742715] E [socket.c:2497:socket_poller] 0-socket.management: poll error on socket
```
```
[2016-07-20 10:25:31.777638] I [MSGID: 106502] [glusterd-handler.c:2821:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2016-07-20 10:25:31.777925] I [MSGID: 106006] [glusterd-svc-mgmt.c:323:glusterd_svc_common_rpc_notify] 0-management: nfs has disconnected from glusterd.
[2016-07-20 10:25:31.777974] I [MSGID: 106006] [glusterd-svc-mgmt.c:323:glusterd_svc_common_rpc_notify] 0-management: glustershd has disconnected from glusterd.
[2016-07-20 10:25:31.778527] E [socket.c:2502:socket_poller] 0-socket.management: error in polling loop
```
And then the following streams and fills up the log file ~GBs in the matter of hours
```
[2016-07-20 10:25:31.865991] E [socket.c:2502:socket_poller] 0-socket.management: error in polling loop
[2016-07-20 10:25:32.670678] E [socket.c:352:ssl_setup_connection] 0-socket.management: SSL connect error
[2016-07-20 10:25:32.670726] E [socket.c:206:ssl_dump_error_stack] 0-socket.management: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number
[2016-07-20 10:25:32.670744] E [socket.c:2389:socket_poller] 0-socket.management: server setup failed
[2016-07-20 10:25:34.217120] I [MSGID: 106006] [glusterd-svc-mgmt.c:323:glusterd_svc_common_rpc_notify] 0-management: nfs has disconnected from glusterd.
[2016-07-20 10:25:34.301599] I [socket.c:347:ssl_setup_connection] 0-socket.management: peer CN = ip-172-31-169-227
[2016-07-20 10:25:34.303179] E [socket.c:2502:socket_poller] 0-socket.management: error in polling loop
[2016-07-20 10:25:34.378333] I [socket.c:347:ssl_setup_connection] 0-socket.management: peer CN = ip-172-31-169-165
[2016-07-20 10:25:34.379965] E [socket.c:2502:socket_poller] 0-socket.management: error in polling loop
[2016-07-20 10:25:35.389754] I [socket.c:347:ssl_setup_connection] 0-socket.management: peer CN = ip-172-31-169-65
[2016-07-20 10:25:35.390996] E [socket.c:2502:socket_poller] 0-socket.management: error in polling loop
[2016-07-20 10:25:35.642357] I [socket.c:347:ssl_setup_connection] 0-socket.management: peer CN = ip-172-31-169-59
[2016-07-20 10:25:35.642815] E [socket.c:2502:socket_poller] 0-socket.management: error in polling loop
[2016-07-20 10:25:35.671336] E [socket.c:352:ssl_setup_connection] 0-socket.management: SSL connect error
[2016-07-20 10:25:35.671376] E [socket.c:206:ssl_dump_error_stack] 0-socket.management: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number
```

I have also attached nfs.log and glustershd.log

Thanks in advance for all the help!
Tommy
Please consider the environment before printing this email. This message should be regarded as confidential. If you have received this email in error please notify the sender and destroy it immediately. Statements of intent shall only become binding when confirmed in hard copy by an authorised signatory. The contents of this email may relate to dealings with other companies under the control of BAE Systems Applied Intelligence Limited, details of which can be found at http://www.baesystems.com/Businesses/index.htm.

_______________________________________________
Gluster-users mailing list
Gluster-***@gluster.org<javascript:_e(%7B%7D,'cvml','Gluster-***@gluster.org');>
http://www.gluster.org/mailman/listinfo/gluster-users