Discussion:
Problem manipulating files when a replica is down
Add Reply
Gary Lloyd
2017-07-17 09:10:33 UTC
Reply
Permalink
Raw Message
We currently have a small pilot gluster setup as Replica 2 + 1 (arbiter)

As a DR test we decided to see that to see what would happen when taking
one of the replicas offline. For the first couple of hours everything
seemed fine and then one of my colleagues tried to a directory with files
via samba from a windows client by using robocopy.

At this point it starting to go wrong. The brick went offline, I could
force it online but when trying to manipulate files on the volume the brick
kept going offline.

Some Examples of trying to manipulate the files from the mounted gluster
volume:

rm -rf a Directory: "Directory not empty"

on an empty directory ls -la : "total 0" ( didn't show "." or ".." )


After bringing the partner replica up the brick on the system that was
never shutdown kept going offline. In the end I ended up restoring the
volume from a gluster snapshot from the night before and then everything
seemed fine again.


This morning I was made aware that creating a new directory via samba
resulted in 5 New folders being created. Restarting glusterd has made the
issue go away for now.


This is on Centos 7.3 with gluster 3.10.3-1 and samba 4.4.4-14


Can anyone help ?

Thanks

*Gary Lloyd*
________________________________________________
I.T. Systems:Keele University
Finance & IT Directorate
Keele:Staffs:IC1 Building:ST5 5NB:UK
+44 1782 733063 <%2B44%201782%20733073>
________________________________________________
Ravishankar N
2017-07-17 10:17:35 UTC
Reply
Permalink
Raw Message
Post by Gary Lloyd
We currently have a small pilot gluster setup as Replica 2 + 1 (arbiter)
As a DR test we decided to see that to see what would happen when
taking one of the replicas offline. For the first couple of hours
everything seemed fine and then one of my colleagues tried to a
directory with files via samba from a windows client by using robocopy.
At this point it starting to go wrong. The brick went offline, I could
force it online but when trying to manipulate files on the volume the
brick kept going offline.
What do you mean by the brick going offline? The brick gets killed? What
does the brick log say?
Post by Gary Lloyd
Some Examples of trying to manipulate the files from the mounted
rm -rf a Directory: "Directory not empty"
on an empty directory ls -la : "total 0" ( didn't show "." or ".." )
After bringing the partner replica up the brick on the system that was
never shutdown kept going offline. In the end I ended up restoring the
volume from a gluster snapshot from the night before and then
everything seemed fine again.
This morning I was made aware that creating a new directory via samba
resulted in 5 New folders being created. Restarting glusterd has made
the issue go away for now.
Are you saying that each brick has the same directory created 5 times?
-Ravi
Post by Gary Lloyd
This is on Centos 7.3 with gluster 3.10.3-1 and samba 4.4.4-14
Can anyone help ?
Thanks
/Gary Lloyd/
________________________________________________
I.T. Systems:Keele University
Finance & IT Directorate
Keele:Staffs:IC1 Building:ST5 5NB:UK
+44 1782 733063 <tel:%2B44%201782%20733073>
________________________________________________
_______________________________________________
Gluster-users mailing list
http://lists.gluster.org/mailman/listinfo/gluster-users
Loading...