[Gluster-users] lstat & readlink calls during glusterfsd process startup

[Serkan Çoban]

Hi all,

I am on gluster 3.10.5 with one EC volume 16+4.
One of the machines go down previous night and I just fixed it and powered on.
When glusterfsd processes started they consume all CPU on the server.
strace shows every process goes over in bricks directory and do a
lstat & readlink calls.
Each brick directory is 8TB, %60 full. I waited for 24 hours for it to
finish but it did not.
I stopped glusterd and restarted it but same thing happens again. Why
on startup glusterfsd processes traverse brick directory? Is it
related to self heal?

This happened one time before and I somehow prevent it happening with
glusterd stop&start or some other way I cannot remember right now.

Any thoughts how to solve this issue?

[Serkan Çoban]

This is an example from one of the glusterfsd processes, strace -f -c
-p pid_of_glusterfsd

%time seconds usecs/call calls errors syscall
68 36.2 2131 17002 4758 futex
13 7 5783 1206 epoll_wait
11 5.4 360545 15 select
...

---------- Forwarded message ----------
From: Serkan Çoban <***@gmail.com>
Date: Mon, Apr 16, 2018 at 9:20 AM
Subject: lstat & readlink calls during glusterfsd process startup
To: Gluster Users <gluster-***@gluster.org>


Hi all,

I am on gluster 3.10.5 with one EC volume 16+4.
One of the machines go down previous night and I just fixed it and powered on.
When glusterfsd processes started they consume all CPU on the server.
strace shows every process goes over in bricks directory and do a
lstat & readlink calls.
Each brick directory is 8TB, %60 full. I waited for 24 hours for it to
finish but it did not.
I stopped glusterd and restarted it but same thing happens again. Why
on startup glusterfsd processes traverse brick directory? Is it
related to self heal?

This happened one time before and I somehow prevent it happening with
glusterd stop&start or some other way I cannot remember right now.

Any thoughts how to solve this issue?