[Gluster-users] Active-Active HA ganesha-nfs configuration

[Herb Burnswell]

All,

I would like to set up HA NFS (Active/Active) on our 2 node gluster
environment using NFS-Ganesha.

Specs:

- RHEL 7
- glusterfs 3.8.15 built on Aug 16 2017 14:48:01

I am following this process in this documentation, however it is confusing
to me:

https://docs.gluster.org/en/v3/Administrator%20Guide/NFS-Ganesha%20GlusterFS%20Integration/

I already had Pacemaker/Corosync up and running on the our 2 node gluster
environment with fence resources on each.

After the package installs and confirming "Pre-requisites to run
NFS-Ganesha", here is what I've done:

1. # gluster volume set all cluster.enable-shared-storage enable

2. Create /etc/ganesha/ganesha-ha.conf file (scrubbed. I'm also assuming
that HA_NAME should be equal to the already created pacemaker cluster name):

#
# HA File

HA_NAME="clustername"
HA_CLUSTER_NODES="server1,server2"

VIP_server1="10.19.3.66"
VIP_server2="10.19.3.67"

3. # gluster nfs-ganesha enable
Enabling NFS-Ganesha requires Gluster-NFS to be disabled across the trusted
pool. Do you still want to continue?
(y/n) y
This will take a few minutes to complete. Please wait ..
nfs-ganesha : success


4. # gluster volume set vol1 ganesha.enable on
volume set: success

At this point I can see the export available:

# showmount -e
Export list for server1:
/vol1 (everyone)

And I can successfully mount the export from another server.

However, nothing appears to be done regarding HA. nfs-ganesha is not
started on server2 and no additional resources are created in pacemaker.

Can anyone provide guidance as to what I may be doing incorrectly?

Thanks,

HB

[David Spisla]

Hello Herbs,
this setup is not as easy as it sounds. Here you can find additional setup
instructions (look for the chapter of NFS-Ganesha HA):
https://access.redhat.com/documentation/en-us/red_hat_gluster_storage/3.2/pdf/administration_guide/Red_Hat_Gluster_Storage-3.2-Administration_Guide-en-US.pdf

Maybe you should use Gluster3.10 because the setup for 3.9 and 3.8 is not
stable. With Gluster 3.10 I had a stable HA-Cluster.
Also you should be aware of the fact, that since Gluster 3.11 this setup
may not work because the Developers switching to storhaug:
https://github.com/linux-ha-storage/storhaug/blob/master/README.md

At the moment there is confusing in the coummunity about that issue.
Storhaug is not complete and still under development.
I don't know if my informations are up-to-date. Because of that fact I was
switching to Samba (SMB). You can also have a HA Cluster with Samba/CTDB.

Regards
David Spisla

Post by Herb Burnswell
All,
I would like to set up HA NFS (Active/Active) on our 2 node gluster
environment using NFS-Ganesha.
- RHEL 7
- glusterfs 3.8.15 built on Aug 16 2017 14:48:01
I am following this process in this documentation, however it is confusing
https://docs.gluster.org/en/v3/Administrator%20Guide/NFS-
Ganesha%20GlusterFS%20Integration/
I already had Pacemaker/Corosync up and running on the our 2 node gluster
environment with fence resources on each.
After the package installs and confirming "Pre-requisites to run
1. # gluster volume set all cluster.enable-shared-storage enable
2. Create /etc/ganesha/ganesha-ha.conf file (scrubbed. I'm also assuming
#
# HA File
HA_NAME="clustername"
HA_CLUSTER_NODES="server1,server2"
VIP_server1="10.19.3.66"
VIP_server2="10.19.3.67"
3. # gluster nfs-ganesha enable
Enabling NFS-Ganesha requires Gluster-NFS to be disabled across the
trusted pool. Do you still want to continue?
(y/n) y
This will take a few minutes to complete. Please wait ..
nfs-ganesha : success
4. # gluster volume set vol1 ganesha.enable on
volume set: success
# showmount -e
/vol1 (everyone)
And I can successfully mount the export from another server.
However, nothing appears to be done regarding HA. nfs-ganesha is not
started on server2 and no additional resources are created in pacemaker.
Can anyone provide guidance as to what I may be doing incorrectly?
Thanks,
HB
_______________________________________________
Gluster-users mailing list
https://lists.gluster.org/mailman/listinfo/gluster-users

[Herb Burnswell]

Thank you for the reply.

It is a bit concerning that you mention Gluster 3.8 and 3.9 set up of
NFS-Ganesha is not stable.

Regarding SMB, I am using SMB with CTDB and it works great. However, I
would also like the ability to export NFS mounts as well. I have read
about using CTDB for HA NFS. Is that a viable/better solution?

Thanks,

HB

Post by David Spisla
Hello Herbs,
this setup is not as easy as it sounds. Here you can find additional setup
https://access.redhat.com/documentation/en-us/red_hat_
gluster_storage/3.2/pdf/administration_guide/Red_Hat_Gluster_Storage-3.2-
Administration_Guide-en-US.pdf
Maybe you should use Gluster3.10 because the setup for 3.9 and 3.8 is not
stable. With Gluster 3.10 I had a stable HA-Cluster.
Also you should be aware of the fact, that since Gluster 3.11 this setup
https://github.com/linux-ha-storage/storhaug/blob/master/README.md
At the moment there is confusing in the coummunity about that issue.
Storhaug is not complete and still under development.
I don't know if my informations are up-to-date. Because of that fact I was
switching to Samba (SMB). You can also have a HA Cluster with Samba/CTDB.
Regards
David Spisla

Post by Herb Burnswell
All,
I would like to set up HA NFS (Active/Active) on our 2 node gluster
environment using NFS-Ganesha.
- RHEL 7
- glusterfs 3.8.15 built on Aug 16 2017 14:48:01
I am following this process in this documentation, however it is
https://docs.gluster.org/en/v3/Administrator%20Guide/NFS-Gan
esha%20GlusterFS%20Integration/
I already had Pacemaker/Corosync up and running on the our 2 node gluster
environment with fence resources on each.
After the package installs and confirming "Pre-requisites to run
1. # gluster volume set all cluster.enable-shared-storage enable
2. Create /etc/ganesha/ganesha-ha.conf file (scrubbed. I'm also assuming
#
# HA File
HA_NAME="clustername"
HA_CLUSTER_NODES="server1,server2"
VIP_server1="10.19.3.66"
VIP_server2="10.19.3.67"
3. # gluster nfs-ganesha enable
Enabling NFS-Ganesha requires Gluster-NFS to be disabled across the
trusted pool. Do you still want to continue?
(y/n) y
This will take a few minutes to complete. Please wait ..
nfs-ganesha : success
4. # gluster volume set vol1 ganesha.enable on
volume set: success
# showmount -e
/vol1 (everyone)
And I can successfully mount the export from another server.
However, nothing appears to be done regarding HA. nfs-ganesha is not
started on server2 and no additional resources are created in pacemaker.
Can anyone provide guidance as to what I may be doing incorrectly?
Thanks,
HB
_______________________________________________
Gluster-users mailing list
https://lists.gluster.org/mailman/listinfo/gluster-users

[Kaleb S. KEITHLEY]

Pacemaker and corosync running on both nodes?

ports in the firewall open for pacemaker and corosync?

selinux ?

`pcs status` shows what?

You should have a pair of ganesha_mon, ganesha_grace, ganesha_nfsd, and
VIP resource agents more or less matching your two hosts and the two VIPs.

Look for errors in /var/log/messages and /var/log/cluster/* for any
anomalies.

Might want to check with the real experts in #clusterlabs on freenode
too. Because when the stuff works, it's great, but when it doesn't work,
they're the experts.

--

Kaleb

Post by Herb Burnswell
All,
I would like to set up HA NFS (Active/Active) on our 2 node gluster
environment using NFS-Ganesha.
- RHEL 7
- glusterfs 3.8.15 built on Aug 16 2017 14:48:01
I am following this process in this documentation, however it is
https://docs.gluster.org/en/v3/Administrator%20Guide/NFS-Ganesha%20GlusterFS%20Integration/
I already had Pacemaker/Corosync up and running on the our 2 node
gluster environment with fence resources on each.
After the package installs and confirming "Pre-requisites to run
1. # gluster volume set all cluster.enable-shared-storage enable
2. Create /etc/ganesha/ganesha-ha.conf file (scrubbed.  I'm also
assuming that HA_NAME should be equal to the already created pacemaker
#
# HA File
HA_NAME="clustername"
HA_CLUSTER_NODES="server1,server2"
VIP_server1="10.19.3.66"
VIP_server2="10.19.3.67"
3. # gluster nfs-ganesha enable
Enabling NFS-Ganesha requires Gluster-NFS to be disabled across the
trusted pool. Do you still want to continue?
 (y/n) y
This will take a few minutes to complete. Please wait ..
nfs-ganesha : success 
4. # gluster volume set vol1 ganesha.enable on
volume set: success
# showmount -e
/vol1 (everyone)
And I can successfully mount the export from another server.
However, nothing appears to be done regarding HA.  nfs-ganesha is not
started on server2 and no additional resources are created in pacemaker.
Can anyone provide guidance as to what I may be doing incorrectly?
Thanks,
HB
_______________________________________________
Gluster-users mailing list
https://lists.gluster.org/mailman/listinfo/gluster-users