Discussion:
FUSE incompatible on RHGS with centos/ubuntu open source clients?
(too old to reply)
Gluster Admin
2015-09-23 17:40:22 UTC
Permalink
Just curious here if RH is purposely trying to prevent anyone but RHEL
servers using their storage natively via the FUSE client?

with RHGS 3.1 I can mount via fuse with no issues on RHEL 6/7 clients but
no other variant of Gluster 3.6 or 3.7 client can connect to it. Even
tried installing the RHGS fuse and client libs with no success. Same if i
try to mount Gluster 3.7 from RHEL with FUSE so its both ways.

Just a bummer as we have tons of non RHEL clients and would prefer not to
setup NFS HA when the native client performs pretty well and covers all the
HA without the hassle.
Prasun Gera
2015-09-23 22:11:35 UTC
Permalink
Is this confirmed ? I have held off upgrading from 3.0 to 3.1. I have a lot
of Ubuntu clients who are using the ppa for the fuse clients. I don't want
to upgrade if it's known to break things.
Post by Gluster Admin
Just curious here if RH is purposely trying to prevent anyone but RHEL
servers using their storage natively via the FUSE client?
with RHGS 3.1 I can mount via fuse with no issues on RHEL 6/7 clients but
no other variant of Gluster 3.6 or 3.7 client can connect to it. Even
tried installing the RHGS fuse and client libs with no success. Same if i
try to mount Gluster 3.7 from RHEL with FUSE so its both ways.
Just a bummer as we have tons of non RHEL clients and would prefer not to
setup NFS HA when the native client performs pretty well and covers all the
HA without the hassle.
_______________________________________________
Gluster-users mailing list
http://www.gluster.org/mailman/listinfo/gluster-users
Gluster Admin
2015-09-23 22:20:14 UTC
Permalink
I would love to know if i am doing something wrong but as it stands I have
a redhat gluster 3.1 setup and the only native FUSE clients i can get to
work are redhat. I have had no success with Centos 6/7 and various gluster
clients or OEL 6 in same case.

I am hoping there is some option or something I am not setting and this
will actually work.
Post by Prasun Gera
Is this confirmed ? I have held off upgrading from 3.0 to 3.1. I have a
lot of Ubuntu clients who are using the ppa for the fuse clients. I don't
want to upgrade if it's known to break things.
Post by Gluster Admin
Just curious here if RH is purposely trying to prevent anyone but RHEL
servers using their storage natively via the FUSE client?
with RHGS 3.1 I can mount via fuse with no issues on RHEL 6/7 clients but
no other variant of Gluster 3.6 or 3.7 client can connect to it. Even
tried installing the RHGS fuse and client libs with no success. Same if i
try to mount Gluster 3.7 from RHEL with FUSE so its both ways.
Just a bummer as we have tons of non RHEL clients and would prefer not to
setup NFS HA when the native client performs pretty well and covers all the
HA without the hassle.
_______________________________________________
Gluster-users mailing list
http://www.gluster.org/mailman/listinfo/gluster-users
Prasun Gera
2015-09-23 22:44:43 UTC
Permalink
There were some posts regarding incompatibility between versions due to the
insecure ports option.
http://www.gluster.org/pipermail/gluster-users/2015-August/023207.html.
Perhaps worth a shot ?
Post by Gluster Admin
I would love to know if i am doing something wrong but as it stands I have
a redhat gluster 3.1 setup and the only native FUSE clients i can get to
work are redhat. I have had no success with Centos 6/7 and various gluster
clients or OEL 6 in same case.
I am hoping there is some option or something I am not setting and this
will actually work.
Post by Prasun Gera
Is this confirmed ? I have held off upgrading from 3.0 to 3.1. I have a
lot of Ubuntu clients who are using the ppa for the fuse clients. I don't
want to upgrade if it's known to break things.
Post by Gluster Admin
Just curious here if RH is purposely trying to prevent anyone but RHEL
servers using their storage natively via the FUSE client?
with RHGS 3.1 I can mount via fuse with no issues on RHEL 6/7 clients
but no other variant of Gluster 3.6 or 3.7 client can connect to it. Even
tried installing the RHGS fuse and client libs with no success. Same if i
try to mount Gluster 3.7 from RHEL with FUSE so its both ways.
Just a bummer as we have tons of non RHEL clients and would prefer not
to setup NFS HA when the native client performs pretty well and covers all
the HA without the hassle.
_______________________________________________
Gluster-users mailing list
http://www.gluster.org/mailman/listinfo/gluster-users
Gluster Admin
2015-09-24 02:00:56 UTC
Permalink
The RHS 3.1 gluster volume info:

[***@rhgluster1 ~]# gluster volume info

Volume Name: rhs_gluster
Type: Distributed-Replicate
Volume ID: ce5ebd81-35bf-40f6-bc53-04c494a8836f
Status: Started
Number of Bricks: 2 x 2 = 4
Transport-type: tcp
Bricks:
Brick1: rhgluster1:/rhs/brick1
Brick2: rhgluster2:/rhs/brick1
Brick3: rhgluster3:/rhs/brick1
Brick4: rhgluster4:/rhs/brick1
Options Reconfigured:
server.allow-insecure: off
performance.readdir-ahead: on
cluster.server-quorum-type: server
cluster.quorum-type: auto
cluster.server-quorum-ratio: 51%

This is what I get on a centos client with 3.7.4 glusterfs fuse installed
trying to mount a RHS Gluster 3.1 volume:

[2015-09-24 01:57:31.742054] I [MSGID: 100030] [glusterfsd.c:2301:main]
0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.4
(args: /usr/sbin/glusterfs --volfile-server=rhgluster1
--volfile-id=rhs_gluster /TEMP)
[2015-09-24 01:57:31.755132] I [MSGID: 101190]
[event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread
with index 1
[2015-09-24 01:57:31.756694] W [socket.c:588:__socket_rwv] 0-glusterfs:
readv on 172.18.60.100:24007 failed (No data available)
[2015-09-24 01:57:31.757143] E [rpc-clnt.c:362:saved_frames_unwind] (-->
/usr/lib64/libglusterfs.so.0(_gf_log_callingfn+0x1eb)[0x7f87cb0de63b] (-->
/usr/lib64/libgfrpc.so.0(saved_frames_unwind+0x1e7)[0x7f87caeaa1d7] (-->
/usr/lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7f87caeaa2ee] (-->
/usr/lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0xab)[0x7f87caeaa3bb]
(--> /usr/lib64/libgfrpc.so.0(rpc_clnt_notify+0x1c2)[0x7f87caeaa9f2] )))))
0-glusterfs: forced unwinding frame type(GlusterFS Handshake)
op(GETSPEC(2)) called at 2015-09-24 01:57:31.755686 (xid=0x1)
[2015-09-24 01:57:31.758030] E [glusterfsd-mgmt.c:1604:mgmt_getspec_cbk]
0-mgmt: failed to fetch volume file (key:rhs_gluster)
[2015-09-24 01:57:31.758095] W [glusterfsd.c:1219:cleanup_and_exit]
(-->/usr/lib64/libgfrpc.so.0(saved_frames_unwind+0x20e) [0x7f87caeaa1fe]
-->/usr/sbin/glusterfs(mgmt_getspec_cbk+0x3f2) [0x40d5d2]
-->/usr/sbin/glusterfs(cleanup_and_exit+0x65) [0x4059b5] ) 0-: received
signum (0), shutting down
[2015-09-24 01:57:31.758138] I [fuse-bridge.c:5595:fini] 0-fuse: Unmounting
'/TEMP'.
[2015-09-24 01:57:31.764511] W [glusterfsd.c:1219:cleanup_and_exit]
(-->/lib64/libpthread.so.0(+0x79d1) [0x7f87ca1c69d1]
-->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xcd) [0x405e4d]
-->/usr/sbin/glusterfs(cleanup_and_exit+0x65) [0x4059b5] ) 0-: received
signum (15), shutting down
Post by Prasun Gera
There were some posts regarding incompatibility between versions due to
the insecure ports option.
http://www.gluster.org/pipermail/gluster-users/2015-August/023207.html.
Perhaps worth a shot ?
Post by Gluster Admin
I would love to know if i am doing something wrong but as it stands I
have a redhat gluster 3.1 setup and the only native FUSE clients i can get
to work are redhat. I have had no success with Centos 6/7 and various
gluster clients or OEL 6 in same case.
I am hoping there is some option or something I am not setting and this
will actually work.
Post by Prasun Gera
Is this confirmed ? I have held off upgrading from 3.0 to 3.1. I have a
lot of Ubuntu clients who are using the ppa for the fuse clients. I don't
want to upgrade if it's known to break things.
Post by Gluster Admin
Just curious here if RH is purposely trying to prevent anyone but RHEL
servers using their storage natively via the FUSE client?
with RHGS 3.1 I can mount via fuse with no issues on RHEL 6/7 clients
but no other variant of Gluster 3.6 or 3.7 client can connect to it. Even
tried installing the RHGS fuse and client libs with no success. Same if i
try to mount Gluster 3.7 from RHEL with FUSE so its both ways.
Just a bummer as we have tons of non RHEL clients and would prefer not
to setup NFS HA when the native client performs pretty well and covers all
the HA without the hassle.
_______________________________________________
Gluster-users mailing list
http://www.gluster.org/mailman/listinfo/gluster-users
Atin Mukherjee
2015-09-24 02:28:41 UTC
Permalink
This has to be a problem with op version, nothing else. Its just that 3.7.4
clients are running with higher op version because of which you are unable
to connect. Could you check the behaviour with 3.7.2/3.7.3?

-Atin
Sent from one plus one
Post by Gluster Admin
Volume Name: rhs_gluster
Type: Distributed-Replicate
Volume ID: ce5ebd81-35bf-40f6-bc53-04c494a8836f
Status: Started
Number of Bricks: 2 x 2 = 4
Transport-type: tcp
Brick1: rhgluster1:/rhs/brick1
Brick2: rhgluster2:/rhs/brick1
Brick3: rhgluster3:/rhs/brick1
Brick4: rhgluster4:/rhs/brick1
server.allow-insecure: off
performance.readdir-ahead: on
cluster.server-quorum-type: server
cluster.quorum-type: auto
cluster.server-quorum-ratio: 51%
This is what I get on a centos client with 3.7.4 glusterfs fuse installed
[2015-09-24 01:57:31.742054] I [MSGID: 100030] [glusterfsd.c:2301:main]
0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.4
(args: /usr/sbin/glusterfs --volfile-server=rhgluster1
--volfile-id=rhs_gluster /TEMP)
[2015-09-24 01:57:31.755132] I [MSGID: 101190]
[event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread
with index 1
readv on 172.18.60.100:24007 failed (No data available)
[2015-09-24 01:57:31.757143] E [rpc-clnt.c:362:saved_frames_unwind] (-->
/usr/lib64/libglusterfs.so.0(_gf_log_callingfn+0x1eb)[0x7f87cb0de63b] (-->
/usr/lib64/libgfrpc.so.0(saved_frames_unwind+0x1e7)[0x7f87caeaa1d7] (-->
/usr/lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7f87caeaa2ee] (-->
/usr/lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0xab)[0x7f87caeaa3bb]
(--> /usr/lib64/libgfrpc.so.0(rpc_clnt_notify+0x1c2)[0x7f87caeaa9f2] )))))
0-glusterfs: forced unwinding frame type(GlusterFS Handshake)
op(GETSPEC(2)) called at 2015-09-24 01:57:31.755686 (xid=0x1)
[2015-09-24 01:57:31.758030] E [glusterfsd-mgmt.c:1604:mgmt_getspec_cbk]
0-mgmt: failed to fetch volume file (key:rhs_gluster)
[2015-09-24 01:57:31.758095] W [glusterfsd.c:1219:cleanup_and_exit]
(-->/usr/lib64/libgfrpc.so.0(saved_frames_unwind+0x20e) [0x7f87caeaa1fe]
-->/usr/sbin/glusterfs(mgmt_getspec_cbk+0x3f2) [0x40d5d2]
-->/usr/sbin/glusterfs(cleanup_and_exit+0x65) [0x4059b5] ) 0-: received
signum (0), shutting down
Unmounting '/TEMP'.
[2015-09-24 01:57:31.764511] W [glusterfsd.c:1219:cleanup_and_exit]
(-->/lib64/libpthread.so.0(+0x79d1) [0x7f87ca1c69d1]
-->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xcd) [0x405e4d]
-->/usr/sbin/glusterfs(cleanup_and_exit+0x65) [0x4059b5] ) 0-: received
signum (15), shutting down
Post by Prasun Gera
There were some posts regarding incompatibility between versions due to
the insecure ports option.
http://www.gluster.org/pipermail/gluster-users/2015-August/023207.html.
Perhaps worth a shot ?
Post by Gluster Admin
I would love to know if i am doing something wrong but as it stands I
have a redhat gluster 3.1 setup and the only native FUSE clients i can get
to work are redhat. I have had no success with Centos 6/7 and various
gluster clients or OEL 6 in same case.
I am hoping there is some option or something I am not setting and this
will actually work.
Post by Prasun Gera
Is this confirmed ? I have held off upgrading from 3.0 to 3.1. I have a
lot of Ubuntu clients who are using the ppa for the fuse clients. I don't
want to upgrade if it's known to break things.
Post by Gluster Admin
Just curious here if RH is purposely trying to prevent anyone but RHEL
servers using their storage natively via the FUSE client?
with RHGS 3.1 I can mount via fuse with no issues on RHEL 6/7 clients
but no other variant of Gluster 3.6 or 3.7 client can connect to it. Even
tried installing the RHGS fuse and client libs with no success. Same if i
try to mount Gluster 3.7 from RHEL with FUSE so its both ways.
Just a bummer as we have tons of non RHEL clients and would prefer not
to setup NFS HA when the native client performs pretty well and covers all
the HA without the hassle.
_______________________________________________
Gluster-users mailing list
http://www.gluster.org/mailman/listinfo/gluster-users
_______________________________________________
Gluster-users mailing list
http://www.gluster.org/mailman/listinfo/gluster-users
Prasun Gera
2015-09-24 03:10:08 UTC
Permalink
I think he mentioned 3.6 clients as well as 3.7 clients. The op version
should not affect 3.6 clients right ?
Post by Atin Mukherjee
This has to be a problem with op version, nothing else. Its just that
3.7.4 clients are running with higher op version because of which you are
unable to connect. Could you check the behaviour with 3.7.2/3.7.3?
-Atin
Sent from one plus one
Post by Gluster Admin
Volume Name: rhs_gluster
Type: Distributed-Replicate
Volume ID: ce5ebd81-35bf-40f6-bc53-04c494a8836f
Status: Started
Number of Bricks: 2 x 2 = 4
Transport-type: tcp
Brick1: rhgluster1:/rhs/brick1
Brick2: rhgluster2:/rhs/brick1
Brick3: rhgluster3:/rhs/brick1
Brick4: rhgluster4:/rhs/brick1
server.allow-insecure: off
performance.readdir-ahead: on
cluster.server-quorum-type: server
cluster.quorum-type: auto
cluster.server-quorum-ratio: 51%
This is what I get on a centos client with 3.7.4 glusterfs fuse installed
[2015-09-24 01:57:31.742054] I [MSGID: 100030] [glusterfsd.c:2301:main]
0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.4
(args: /usr/sbin/glusterfs --volfile-server=rhgluster1
--volfile-id=rhs_gluster /TEMP)
[2015-09-24 01:57:31.755132] I [MSGID: 101190]
[event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread
with index 1
readv on 172.18.60.100:24007 failed (No data available)
[2015-09-24 01:57:31.757143] E [rpc-clnt.c:362:saved_frames_unwind] (-->
/usr/lib64/libglusterfs.so.0(_gf_log_callingfn+0x1eb)[0x7f87cb0de63b] (-->
/usr/lib64/libgfrpc.so.0(saved_frames_unwind+0x1e7)[0x7f87caeaa1d7] (-->
/usr/lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7f87caeaa2ee] (-->
/usr/lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0xab)[0x7f87caeaa3bb]
(--> /usr/lib64/libgfrpc.so.0(rpc_clnt_notify+0x1c2)[0x7f87caeaa9f2] )))))
0-glusterfs: forced unwinding frame type(GlusterFS Handshake)
op(GETSPEC(2)) called at 2015-09-24 01:57:31.755686 (xid=0x1)
[2015-09-24 01:57:31.758030] E [glusterfsd-mgmt.c:1604:mgmt_getspec_cbk]
0-mgmt: failed to fetch volume file (key:rhs_gluster)
[2015-09-24 01:57:31.758095] W [glusterfsd.c:1219:cleanup_and_exit]
(-->/usr/lib64/libgfrpc.so.0(saved_frames_unwind+0x20e) [0x7f87caeaa1fe]
-->/usr/sbin/glusterfs(mgmt_getspec_cbk+0x3f2) [0x40d5d2]
-->/usr/sbin/glusterfs(cleanup_and_exit+0x65) [0x4059b5] ) 0-: received
signum (0), shutting down
Unmounting '/TEMP'.
[2015-09-24 01:57:31.764511] W [glusterfsd.c:1219:cleanup_and_exit]
(-->/lib64/libpthread.so.0(+0x79d1) [0x7f87ca1c69d1]
-->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xcd) [0x405e4d]
-->/usr/sbin/glusterfs(cleanup_and_exit+0x65) [0x4059b5] ) 0-: received
signum (15), shutting down
Post by Prasun Gera
There were some posts regarding incompatibility between versions due to
the insecure ports option.
http://www.gluster.org/pipermail/gluster-users/2015-August/023207.html.
Perhaps worth a shot ?
Post by Gluster Admin
I would love to know if i am doing something wrong but as it stands I
have a redhat gluster 3.1 setup and the only native FUSE clients i can get
to work are redhat. I have had no success with Centos 6/7 and various
gluster clients or OEL 6 in same case.
I am hoping there is some option or something I am not setting and this
will actually work.
Post by Prasun Gera
Is this confirmed ? I have held off upgrading from 3.0 to 3.1. I have
a lot of Ubuntu clients who are using the ppa for the fuse clients. I don't
want to upgrade if it's known to break things.
Post by Gluster Admin
Just curious here if RH is purposely trying to prevent anyone but
RHEL servers using their storage natively via the FUSE client?
with RHGS 3.1 I can mount via fuse with no issues on RHEL 6/7 clients
but no other variant of Gluster 3.6 or 3.7 client can connect to it. Even
tried installing the RHGS fuse and client libs with no success. Same if i
try to mount Gluster 3.7 from RHEL with FUSE so its both ways.
Just a bummer as we have tons of non RHEL clients and would prefer
not to setup NFS HA when the native client performs pretty well and covers
all the HA without the hassle.
_______________________________________________
Gluster-users mailing list
http://www.gluster.org/mailman/listinfo/gluster-users
_______________________________________________
Gluster-users mailing list
http://www.gluster.org/mailman/listinfo/gluster-users
Atin Mukherjee
2015-09-24 03:37:08 UTC
Permalink
The problem with the older releases was that the upstream op-versions were
different compared to respective downstream releases. This is why we never
encouraged to use a mix match of upstream and downstream. From RHGS 3.1
(3.7.2)we decided to have same op-version.

-Atin
Sent from one plus one
I think he mentioned 3.6 clients as well as 3.7 clients. The op version
should not affect 3.6 clients right ?
Post by Atin Mukherjee
This has to be a problem with op version, nothing else. Its just that
3.7.4 clients are running with higher op version because of which you are
unable to connect. Could you check the behaviour with 3.7.2/3.7.3?
-Atin
Sent from one plus one
Post by Gluster Admin
Volume Name: rhs_gluster
Type: Distributed-Replicate
Volume ID: ce5ebd81-35bf-40f6-bc53-04c494a8836f
Status: Started
Number of Bricks: 2 x 2 = 4
Transport-type: tcp
Brick1: rhgluster1:/rhs/brick1
Brick2: rhgluster2:/rhs/brick1
Brick3: rhgluster3:/rhs/brick1
Brick4: rhgluster4:/rhs/brick1
server.allow-insecure: off
performance.readdir-ahead: on
cluster.server-quorum-type: server
cluster.quorum-type: auto
cluster.server-quorum-ratio: 51%
This is what I get on a centos client with 3.7.4 glusterfs fuse installed
[2015-09-24 01:57:31.742054] I [MSGID: 100030] [glusterfsd.c:2301:main]
0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.4
(args: /usr/sbin/glusterfs --volfile-server=rhgluster1
--volfile-id=rhs_gluster /TEMP)
[2015-09-24 01:57:31.755132] I [MSGID: 101190]
[event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread
with index 1
readv on 172.18.60.100:24007 failed (No data available)
[2015-09-24 01:57:31.757143] E [rpc-clnt.c:362:saved_frames_unwind] (-->
/usr/lib64/libglusterfs.so.0(_gf_log_callingfn+0x1eb)[0x7f87cb0de63b] (-->
/usr/lib64/libgfrpc.so.0(saved_frames_unwind+0x1e7)[0x7f87caeaa1d7] (-->
/usr/lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7f87caeaa2ee] (-->
/usr/lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0xab)[0x7f87caeaa3bb]
(--> /usr/lib64/libgfrpc.so.0(rpc_clnt_notify+0x1c2)[0x7f87caeaa9f2] )))))
0-glusterfs: forced unwinding frame type(GlusterFS Handshake)
op(GETSPEC(2)) called at 2015-09-24 01:57:31.755686 (xid=0x1)
[2015-09-24 01:57:31.758030] E [glusterfsd-mgmt.c:1604:mgmt_getspec_cbk]
0-mgmt: failed to fetch volume file (key:rhs_gluster)
[2015-09-24 01:57:31.758095] W [glusterfsd.c:1219:cleanup_and_exit]
(-->/usr/lib64/libgfrpc.so.0(saved_frames_unwind+0x20e) [0x7f87caeaa1fe]
-->/usr/sbin/glusterfs(mgmt_getspec_cbk+0x3f2) [0x40d5d2]
-->/usr/sbin/glusterfs(cleanup_and_exit+0x65) [0x4059b5] ) 0-: received
signum (0), shutting down
Unmounting '/TEMP'.
[2015-09-24 01:57:31.764511] W [glusterfsd.c:1219:cleanup_and_exit]
(-->/lib64/libpthread.so.0(+0x79d1) [0x7f87ca1c69d1]
-->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xcd) [0x405e4d]
-->/usr/sbin/glusterfs(cleanup_and_exit+0x65) [0x4059b5] ) 0-: received
signum (15), shutting down
Post by Prasun Gera
There were some posts regarding incompatibility between versions due to
the insecure ports option.
http://www.gluster.org/pipermail/gluster-users/2015-August/023207.html.
Perhaps worth a shot ?
Post by Gluster Admin
I would love to know if i am doing something wrong but as it stands I
have a redhat gluster 3.1 setup and the only native FUSE clients i can get
to work are redhat. I have had no success with Centos 6/7 and various
gluster clients or OEL 6 in same case.
I am hoping there is some option or something I am not setting and this
will actually work.
Post by Prasun Gera
Is this confirmed ? I have held off upgrading from 3.0 to 3.1. I have
a lot of Ubuntu clients who are using the ppa for the fuse clients. I don't
want to upgrade if it's known to break things.
Post by Gluster Admin
Just curious here if RH is purposely trying to prevent anyone but
RHEL servers using their storage natively via the FUSE client?
with RHGS 3.1 I can mount via fuse with no issues on RHEL 6/7 clients
but no other variant of Gluster 3.6 or 3.7 client can connect to it. Even
tried installing the RHGS fuse and client libs with no success. Same if i
try to mount Gluster 3.7 from RHEL with FUSE so its both ways.
Just a bummer as we have tons of non RHEL clients and would prefer
not to setup NFS HA when the native client performs pretty well and covers
all the HA without the hassle.
_______________________________________________
Gluster-users mailing list
http://www.gluster.org/mailman/listinfo/gluster-users
_______________________________________________
Gluster-users mailing list
http://www.gluster.org/mailman/listinfo/gluster-users
Gluster Admin
2015-09-24 03:43:41 UTC
Permalink
Thanks Atin,

It did have an issue with an older 3.6 client but thats not important to
us. I manually forced install of 3.7.2 gluster client and now the redhat
gluster volume and it finally mounted on centos/sl. Although Redhat
identifies as 3.7.1 so maybe I should use that same client to be on safe
side.
Post by Atin Mukherjee
This has to be a problem with op version, nothing else. Its just that
3.7.4 clients are running with higher op version because of which you are
unable to connect. Could you check the behaviour with 3.7.2/3.7.3?
-Atin
Sent from one plus one
Post by Gluster Admin
Volume Name: rhs_gluster
Type: Distributed-Replicate
Volume ID: ce5ebd81-35bf-40f6-bc53-04c494a8836f
Status: Started
Number of Bricks: 2 x 2 = 4
Transport-type: tcp
Brick1: rhgluster1:/rhs/brick1
Brick2: rhgluster2:/rhs/brick1
Brick3: rhgluster3:/rhs/brick1
Brick4: rhgluster4:/rhs/brick1
server.allow-insecure: off
performance.readdir-ahead: on
cluster.server-quorum-type: server
cluster.quorum-type: auto
cluster.server-quorum-ratio: 51%
This is what I get on a centos client with 3.7.4 glusterfs fuse installed
[2015-09-24 01:57:31.742054] I [MSGID: 100030] [glusterfsd.c:2301:main]
0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.4
(args: /usr/sbin/glusterfs --volfile-server=rhgluster1
--volfile-id=rhs_gluster /TEMP)
[2015-09-24 01:57:31.755132] I [MSGID: 101190]
[event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread
with index 1
readv on 172.18.60.100:24007 failed (No data available)
[2015-09-24 01:57:31.757143] E [rpc-clnt.c:362:saved_frames_unwind] (-->
/usr/lib64/libglusterfs.so.0(_gf_log_callingfn+0x1eb)[0x7f87cb0de63b] (-->
/usr/lib64/libgfrpc.so.0(saved_frames_unwind+0x1e7)[0x7f87caeaa1d7] (-->
/usr/lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7f87caeaa2ee] (-->
/usr/lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0xab)[0x7f87caeaa3bb]
(--> /usr/lib64/libgfrpc.so.0(rpc_clnt_notify+0x1c2)[0x7f87caeaa9f2] )))))
0-glusterfs: forced unwinding frame type(GlusterFS Handshake)
op(GETSPEC(2)) called at 2015-09-24 01:57:31.755686 (xid=0x1)
[2015-09-24 01:57:31.758030] E [glusterfsd-mgmt.c:1604:mgmt_getspec_cbk]
0-mgmt: failed to fetch volume file (key:rhs_gluster)
[2015-09-24 01:57:31.758095] W [glusterfsd.c:1219:cleanup_and_exit]
(-->/usr/lib64/libgfrpc.so.0(saved_frames_unwind+0x20e) [0x7f87caeaa1fe]
-->/usr/sbin/glusterfs(mgmt_getspec_cbk+0x3f2) [0x40d5d2]
-->/usr/sbin/glusterfs(cleanup_and_exit+0x65) [0x4059b5] ) 0-: received
signum (0), shutting down
Unmounting '/TEMP'.
[2015-09-24 01:57:31.764511] W [glusterfsd.c:1219:cleanup_and_exit]
(-->/lib64/libpthread.so.0(+0x79d1) [0x7f87ca1c69d1]
-->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xcd) [0x405e4d]
-->/usr/sbin/glusterfs(cleanup_and_exit+0x65) [0x4059b5] ) 0-: received
signum (15), shutting down
Post by Prasun Gera
There were some posts regarding incompatibility between versions due to
the insecure ports option.
http://www.gluster.org/pipermail/gluster-users/2015-August/023207.html.
Perhaps worth a shot ?
Post by Gluster Admin
I would love to know if i am doing something wrong but as it stands I
have a redhat gluster 3.1 setup and the only native FUSE clients i can get
to work are redhat. I have had no success with Centos 6/7 and various
gluster clients or OEL 6 in same case.
I am hoping there is some option or something I am not setting and this
will actually work.
Post by Prasun Gera
Is this confirmed ? I have held off upgrading from 3.0 to 3.1. I have
a lot of Ubuntu clients who are using the ppa for the fuse clients. I don't
want to upgrade if it's known to break things.
Post by Gluster Admin
Just curious here if RH is purposely trying to prevent anyone but
RHEL servers using their storage natively via the FUSE client?
with RHGS 3.1 I can mount via fuse with no issues on RHEL 6/7 clients
but no other variant of Gluster 3.6 or 3.7 client can connect to it. Even
tried installing the RHGS fuse and client libs with no success. Same if i
try to mount Gluster 3.7 from RHEL with FUSE so its both ways.
Just a bummer as we have tons of non RHEL clients and would prefer
not to setup NFS HA when the native client performs pretty well and covers
all the HA without the hassle.
_______________________________________________
Gluster-users mailing list
http://www.gluster.org/mailman/listinfo/gluster-users
_______________________________________________
Gluster-users mailing list
http://www.gluster.org/mailman/listinfo/gluster-users
Prasun Gera
2015-09-24 03:49:57 UTC
Permalink
So how would one use ubuntu clients with RHGS 3.1 ? Specifically, some of
my ubuntu clients are 12.04, which will not get 3.7. Is there an op version
that can be set on the server that would make it compatible with ubuntu
clients (3.6/7) ?
Post by Gluster Admin
Thanks Atin,
It did have an issue with an older 3.6 client but thats not important to
us. I manually forced install of 3.7.2 gluster client and now the redhat
gluster volume and it finally mounted on centos/sl. Although Redhat
identifies as 3.7.1 so maybe I should use that same client to be on safe
side.
On Wed, Sep 23, 2015 at 7:28 PM, Atin Mukherjee <
Post by Atin Mukherjee
This has to be a problem with op version, nothing else. Its just that
3.7.4 clients are running with higher op version because of which you are
unable to connect. Could you check the behaviour with 3.7.2/3.7.3?
-Atin
Sent from one plus one
Post by Gluster Admin
Volume Name: rhs_gluster
Type: Distributed-Replicate
Volume ID: ce5ebd81-35bf-40f6-bc53-04c494a8836f
Status: Started
Number of Bricks: 2 x 2 = 4
Transport-type: tcp
Brick1: rhgluster1:/rhs/brick1
Brick2: rhgluster2:/rhs/brick1
Brick3: rhgluster3:/rhs/brick1
Brick4: rhgluster4:/rhs/brick1
server.allow-insecure: off
performance.readdir-ahead: on
cluster.server-quorum-type: server
cluster.quorum-type: auto
cluster.server-quorum-ratio: 51%
This is what I get on a centos client with 3.7.4 glusterfs fuse
[2015-09-24 01:57:31.742054] I [MSGID: 100030] [glusterfsd.c:2301:main]
0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.4
(args: /usr/sbin/glusterfs --volfile-server=rhgluster1
--volfile-id=rhs_gluster /TEMP)
[2015-09-24 01:57:31.755132] I [MSGID: 101190]
[event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread
with index 1
readv on 172.18.60.100:24007 failed (No data available)
[2015-09-24 01:57:31.757143] E [rpc-clnt.c:362:saved_frames_unwind] (-->
/usr/lib64/libglusterfs.so.0(_gf_log_callingfn+0x1eb)[0x7f87cb0de63b] (-->
/usr/lib64/libgfrpc.so.0(saved_frames_unwind+0x1e7)[0x7f87caeaa1d7] (-->
/usr/lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7f87caeaa2ee] (-->
/usr/lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0xab)[0x7f87caeaa3bb]
(--> /usr/lib64/libgfrpc.so.0(rpc_clnt_notify+0x1c2)[0x7f87caeaa9f2] )))))
0-glusterfs: forced unwinding frame type(GlusterFS Handshake)
op(GETSPEC(2)) called at 2015-09-24 01:57:31.755686 (xid=0x1)
[2015-09-24 01:57:31.758030] E [glusterfsd-mgmt.c:1604:mgmt_getspec_cbk]
0-mgmt: failed to fetch volume file (key:rhs_gluster)
[2015-09-24 01:57:31.758095] W [glusterfsd.c:1219:cleanup_and_exit]
(-->/usr/lib64/libgfrpc.so.0(saved_frames_unwind+0x20e) [0x7f87caeaa1fe]
-->/usr/sbin/glusterfs(mgmt_getspec_cbk+0x3f2) [0x40d5d2]
-->/usr/sbin/glusterfs(cleanup_and_exit+0x65) [0x4059b5] ) 0-: received
signum (0), shutting down
Unmounting '/TEMP'.
[2015-09-24 01:57:31.764511] W [glusterfsd.c:1219:cleanup_and_exit]
(-->/lib64/libpthread.so.0(+0x79d1) [0x7f87ca1c69d1]
-->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xcd) [0x405e4d]
-->/usr/sbin/glusterfs(cleanup_and_exit+0x65) [0x4059b5] ) 0-: received
signum (15), shutting down
Post by Prasun Gera
There were some posts regarding incompatibility between versions due to
the insecure ports option.
http://www.gluster.org/pipermail/gluster-users/2015-August/023207.html.
Perhaps worth a shot ?
Post by Gluster Admin
I would love to know if i am doing something wrong but as it stands I
have a redhat gluster 3.1 setup and the only native FUSE clients i can get
to work are redhat. I have had no success with Centos 6/7 and various
gluster clients or OEL 6 in same case.
I am hoping there is some option or something I am not setting and
this will actually work.
Post by Prasun Gera
Is this confirmed ? I have held off upgrading from 3.0 to 3.1. I have
a lot of Ubuntu clients who are using the ppa for the fuse clients. I don't
want to upgrade if it's known to break things.
Post by Gluster Admin
Just curious here if RH is purposely trying to prevent anyone but
RHEL servers using their storage natively via the FUSE client?
with RHGS 3.1 I can mount via fuse with no issues on RHEL 6/7
clients but no other variant of Gluster 3.6 or 3.7 client can connect to
it. Even tried installing the RHGS fuse and client libs with no success.
Same if i try to mount Gluster 3.7 from RHEL with FUSE so its both ways.
Just a bummer as we have tons of non RHEL clients and would prefer
not to setup NFS HA when the native client performs pretty well and covers
all the HA without the hassle.
_______________________________________________
Gluster-users mailing list
http://www.gluster.org/mailman/listinfo/gluster-users
_______________________________________________
Gluster-users mailing list
http://www.gluster.org/mailman/listinfo/gluster-users
Gluster Admin
2015-09-24 15:13:21 UTC
Permalink
Another update, after playing with our Oracle Enterprise Linux servers and
still not having luck I noted two things:

1) OEL 6 in Redhat Kernel works fine with 3.7.2 gluster fuse to RHS 3.1
2) OEL 6.6 UEK kernel does not work at all with 3.8.13-16 which is
installed by default. I updated to latest UEK kernel below and now gluster
fuse mounts to RHS 3.1 and Open Source Gluster 3.7.x work well

kernel-uek-3.8.13-98.2.2.el6uek.x86_64
kernel-uek-firmware-3.8.13-98.2.2.el6uek.noarch

[***@localhost ~]# uname -a
Linux localhost.localdomain 3.8.13-98.2.2.el6uek.x86_64 #2 SMP Tue Sep 15
16:20:45 PDT 2015 x86_64 x86_64 x86_64 GNU/Linux
Post by Gluster Admin
Just curious here if RH is purposely trying to prevent anyone but RHEL
servers using their storage natively via the FUSE client?
with RHGS 3.1 I can mount via fuse with no issues on RHEL 6/7 clients but
no other variant of Gluster 3.6 or 3.7 client can connect to it. Even
tried installing the RHGS fuse and client libs with no success. Same if i
try to mount Gluster 3.7 from RHEL with FUSE so its both ways.
Just a bummer as we have tons of non RHEL clients and would prefer not to
setup NFS HA when the native client performs pretty well and covers all the
HA without the hassle.
Prasun Gera
2015-11-17 23:28:23 UTC
Permalink
Can someone confirm if it's OK to upgrade from RHGS 3 to 3.1 if I have a
mix of RH and ubuntu clients ? All the ubuntu clients are on 3.6, and the
ones on 12.04 will remain on 3.6 because of lack of 3.7 on the platform.
Since setting the op version is irreversible, I would like to proceed with
caution.
Post by Gluster Admin
Another update, after playing with our Oracle Enterprise Linux servers and
1) OEL 6 in Redhat Kernel works fine with 3.7.2 gluster fuse to RHS 3.1
2) OEL 6.6 UEK kernel does not work at all with 3.8.13-16 which is
installed by default. I updated to latest UEK kernel below and now gluster
fuse mounts to RHS 3.1 and Open Source Gluster 3.7.x work well
kernel-uek-3.8.13-98.2.2.el6uek.x86_64
kernel-uek-firmware-3.8.13-98.2.2.el6uek.noarch
Linux localhost.localdomain 3.8.13-98.2.2.el6uek.x86_64 #2 SMP Tue Sep 15
16:20:45 PDT 2015 x86_64 x86_64 x86_64 GNU/Linux
Post by Gluster Admin
Just curious here if RH is purposely trying to prevent anyone but RHEL
servers using their storage natively via the FUSE client?
with RHGS 3.1 I can mount via fuse with no issues on RHEL 6/7 clients but
no other variant of Gluster 3.6 or 3.7 client can connect to it. Even
tried installing the RHGS fuse and client libs with no success. Same if i
try to mount Gluster 3.7 from RHEL with FUSE so its both ways.
Just a bummer as we have tons of non RHEL clients and would prefer not to
setup NFS HA when the native client performs pretty well and covers all the
HA without the hassle.
_______________________________________________
Gluster-users mailing list
http://www.gluster.org/mailman/listinfo/gluster-users
Loading...