[Gluster-users] glusterfs 3.13 repo unavailable and downgrade to 3.12.9 fails

Discussion:

Davide Obbi

2018-05-15 08:58:12 UTC

hi,

i noticed that this repo for glusterfs 3.13 does not exists anymore at:

http://mirror.centos.org/centos/7/storage/x86_64/

i knew was not going to be long term supported however the downgrade to
3.12 breaks the server node i believe the issue is with:

*[2018-05-15 08:54:39.981101] E [MSGID: 101019] [xlator.c:503:xlator_init]
0-management: Initialization of volume 'management' failed, review your
volfile again [2018-05-15 08:54:39.981113] E
[MSGID: 101066] [graph.c:327:glusterfs_graph_init] 0-management:
initializing translator
failed
[2018-05-15 08:54:39.981121] E [MSGID: 101176]
[graph.c:698:glusterfs_graph_activate] 0-graph: init
failed
*

Any help appreciated, thanks

Details:
*Installed Packages*
glusterfs.x86_64
3.12.9-1.el7
@gluster-3.12
glusterfs-api.x86_64
3.12.9-1.el7
@gluster-3.12
glusterfs-cli.x86_64
3.12.9-1.el7
@gluster-3.12
glusterfs-client-xlators.x86_64
3.12.9-1.el7
@gluster-3.12
glusterfs-events.x86_64
3.12.9-1.el7
@gluster-3.12
glusterfs-fuse.x86_64
3.12.9-1.el7
@gluster-3.12
glusterfs-libs.x86_64
3.12.9-1.el7
@gluster-3.12
glusterfs-server.x86_64
3.12.9-1.el7
@gluster-3.12
python2-gluster.x86_64
3.12.9-1.el7
@gluster-3.12

*# journalctl -u glusterd *

-- Logs begin at Wed 2018-03-21 15:06:46 CET, end at Tue 2018-05-15
10:48:01 CEST. --
Mar 21 15:06:58 glusternode-1003 systemd[1]: Starting GlusterFS, a
clustered file-system
server...

Mar 21 15:07:01 glusternode-1 systemd[1]: Started GlusterFS, a clustered
file-system
server.

Mar 21 15:25:07 glusternode-1 systemd[1]: Stopping GlusterFS, a clustered
file-system
server...

Mar 21 15:25:07 glusternode-1 systemd[1]: Starting GlusterFS, a clustered
file-system
server...

Mar 21 15:25:09 glusternode-1 systemd[1]: Started GlusterFS, a clustered
file-system
server.

May 10 12:29:11 glusternode-1 systemd[1]: Stopping GlusterFS, a clustered
file-system
server...

May 10 12:29:11 glusternode-1 systemd[1]: Starting GlusterFS, a clustered
file-system
server...

May 10 12:29:13 glusternode-1 systemd[1]: Started GlusterFS, a clustered
file-system
server.

May 15 10:21:01 glusternode-1 systemd[1]: Starting GlusterFS, a clustered
file-system
server...

May 15 10:21:01 glusternode-1 systemd[1]: glusterd.service: control process
exited, code=exited
status=1
May 15 10:21:01 glusternode-1 systemd[1]: Failed to start GlusterFS, a
clustered file-system
server.
May 15 10:21:01 glusternode-1 systemd[1]: Unit glusterd.service entered
failed
state.

May 15 10:21:01 glusternode-1 systemd[1]: glusterd.service failed.
May 15 10:22:14 glusternode-1 systemd[1]: Starting GlusterFS, a clustered
file-system
server...

May 15 10:22:14 glusternode-1 systemd[1]: glusterd.service: control process
exited, code=exited
status=1
May 15 10:22:14 glusternode-1 systemd[1]: Failed to start GlusterFS, a
clustered file-system
server.
May 15 10:22:14 glusternode-1 systemd[1]: Unit glusterd.service entered
failed
state.

May 15 10:22:14 glusternode-1 systemd[1]: glusterd.service failed.
May 15 10:22:49 glusternode-1 systemd[1]: Starting GlusterFS, a clustered
file-system
server...

May 15 10:22:49 glusternode-1 systemd[1]: glusterd.service: control process
exited, code=exited
status=1
May 15 10:22:49 glusternode-1 systemd[1]: Failed to start GlusterFS, a
clustered file-system
server.
May 15 10:22:49 glusternode-1 systemd[1]: Unit glusterd.service entered
failed
state.

May 15 10:22:49 glusternode-1 systemd[1]: glusterd.service failed.
May 15 10:23:36 glusternode-1 systemd[1]: Starting GlusterFS, a clustered
file-system
server...

May 15 10:23:36 glusternode-1 systemd[1]: glusterd.service: control process
exited, code=exited
status=1
May 15 10:23:36 glusternode-1 systemd[1]: Failed to start GlusterFS, a
clustered file-system
server.
May 15 10:23:36 glusternode-1 systemd[1]: Unit glusterd.service entered
failed
state.

May 15 10:23:36 glusternode-1 systemd[1]: glusterd.service failed.
[2018-05-15 08:54:38.354520] I [MSGID: 100030] [glusterfsd.c:2511:main]
0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.12.9
(args: /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO)
[2018-05-15 08:54:38.359267] I [MSGID: 106478] [glusterd.c:1423:init]
0-management: Maximum allowed open file descriptors set to
65536
[2018-05-15 08:54:38.359305] I [MSGID: 106479] [glusterd.c:1481:init]
0-management: Using /var/lib/glusterd as working
directory
[2018-05-15 08:54:38.359313] I [MSGID: 106479] [glusterd.c:1486:init]
0-management: Using /var/run/gluster as pid file working
directory
[2018-05-15 08:54:38.363363] E [rpc-transport.c:283:rpc_transport_load]
0-rpc-transport: /usr/lib64/glusterfs/3.12.9/rpc-transport/rdma.so: cannot
open shared object file: No such file or directory
[2018-05-15 08:54:38.363381] W [rpc-transport.c:287:rpc_transport_load]
0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not
valid or not found on this machine
[2018-05-15 08:54:38.363391] W [rpcsvc.c:1682:rpcsvc_create_listener]
0-rpc-service: cannot create listener, initing the transport
failed
[2018-05-15 08:54:38.363398] E [MSGID: 106243] [glusterd.c:1769:init]
0-management: creation of 1 listeners failed, continuing with succeeded
transport
[2018-05-15 08:54:39.981028] E [MSGID: 106022]
[glusterd-store.c:2234:glusterd_restore_op_version] 0-management: wrong
op-version (31300) retrieved [Invalid argument]
[2018-05-15 08:54:39.981071] E [MSGID: 106244] [glusterd.c:1928:init]
0-management: Failed to restore
op_version

[2018-05-15 08:54:39.981101] E [MSGID: 101019] [xlator.c:503:xlator_init]
0-management: Initialization of volume 'management' failed, review your
volfile again
[2018-05-15 08:54:39.981113] E [MSGID: 101066]
[graph.c:327:glusterfs_graph_init] 0-management: initializing translator
failed
[2018-05-15 08:54:39.981121] E [MSGID: 101176]
[graph.c:698:glusterfs_graph_activate] 0-graph: init
failed

[2018-05-15 08:54:39.981720] W [glusterfsd.c:1375:cleanup_and_exit]
(-->/usr/sbin/glusterd(glusterfs_volumes_init+0xfd) [0x559572dbcf5d]
-->/usr/sbin/glusterd(glusterfs_process_volfp+0x163)
[0x559572dbce03] -->/usr/sbin/glusterd(cleanup_and_exit+0x6b)
[0x559572dbc37b] ) 0-: received signum (-1), shutting
down
*/var/log/glusterfs/glusterd.log*
[2018-05-15 08:54:38.354520] I [MSGID: 100030] [glusterfsd.c:2511:main]
0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.12.9
(args: /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO)
[2018-05-15 08:54:38.359267] I [MSGID: 106478] [glusterd.c:1423:init]
0-management: Maximum allowed open file descriptors set to
65536
[2018-05-15 08:54:38.359305] I [MSGID: 106479] [glusterd.c:1481:init]
0-management: Using /var/lib/glusterd as working
directory
[2018-05-15 08:54:38.359313] I [MSGID: 106479] [glusterd.c:1486:init]
0-management: Using /var/run/gluster as pid file working
directory
[2018-05-15 08:54:38.363363] E [rpc-transport.c:283:rpc_transport_load]
0-rpc-transport: /usr/lib64/glusterfs/3.12.9/rpc-transport/rdma.so: cannot
open shared object file: No such file or directory
[2018-05-15 08:54:38.363381] W [rpc-transport.c:287:rpc_transport_load]
0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not
valid or not found on this machine
[2018-05-15 08:54:38.363391] W [rpcsvc.c:1682:rpcsvc_create_listener]
0-rpc-service: cannot create listener, initing the transport
failed
[2018-05-15 08:54:38.363398] E [MSGID: 106243] [glusterd.c:1769:init]
0-management: creation of 1 listeners failed, continuing with succeeded
transport
[2018-05-15 08:54:39.981028] E [MSGID: 106022]
[glusterd-store.c:2234:glusterd_restore_op_version] 0-management: wrong
op-version (31300) retrieved [Invalid argument]
[2018-05-15 08:54:39.981071] E [MSGID: 106244] [glusterd.c:1928:init]
0-management: Failed to restore
op_version

*[2018-05-15 08:54:39.981101] E [MSGID: 101019] [xlator.c:503:xlator_init]
0-management: Initialization of volume 'management' failed, review your
volfile again [2018-05-15 08:54:39.981113] E
[MSGID: 101066] [graph.c:327:glusterfs_graph_init] 0-management:
initializing translator
failed
[2018-05-15 08:54:39.981121] E [MSGID: 101176]
[graph.c:698:glusterfs_graph_activate] 0-graph: init
failed
*
[2018-05-15 08:54:39.981720] W [glusterfsd.c:1375:cleanup_and_exit]
(-->/usr/sbin/glusterd(glusterfs_volumes_init+0xfd) [0x559572dbcf5d]
-->/usr/sbin/glusterd(glusterfs_process_volfp+0x163)
[0x559572dbce03] -->/usr/sbin/glusterd(cleanup_and_exit+0x6b)
[0x559572dbc37b] ) 0-: received signum (-1), shutting
down

Kind regards
Davide

Kaleb S. KEITHLEY

2018-05-15 12:02:51 UTC

Permalink

You can still get them from
https://buildlogs.centos.org/centos/7/storage/x86_64/gluster-3.13/

(I don't know how much longer they'll be there. I suggest you copy them
if you think you're going to need them in the future.)

Post by Davide Obbi
hi,
http://mirror.centos.org/centos/7/storage/x86_64/
i knew was not going to be long term supported however the downgrade to
*[2018-05-15 08:54:39.981101] E [MSGID: 101019]
[xlator.c:503:xlator_init] 0-management: Initialization of volume
'management' failed, review your volfile again
[2018-05-15 08:54:39.981113] E [MSGID: 101066]
[graph.c:327:glusterfs_graph_init] 0-management: initializing translator
failed
[2018-05-15 08:54:39.981121] E [MSGID: 101176]
[graph.c:698:glusterfs_graph_activate] 0-graph: init
failed
*
Any help appreciated, thanks
*Installed Packages*
glusterfs.x86_64
3.12.9-1.el7
@gluster-3.12
glusterfs-api.x86_64
3.12.9-1.el7
@gluster-3.12
glusterfs-cli.x86_64
3.12.9-1.el7
@gluster-3.12
glusterfs-client-xlators.x86_64
3.12.9-1.el7
@gluster-3.12
glusterfs-events.x86_64
3.12.9-1.el7
@gluster-3.12
glusterfs-fuse.x86_64
3.12.9-1.el7
@gluster-3.12
glusterfs-libs.x86_64
3.12.9-1.el7
@gluster-3.12
glusterfs-server.x86_64
3.12.9-1.el7
@gluster-3.12
python2-gluster.x86_64
3.12.9-1.el7
@gluster-3.12
*# journalctl -u
glusterd    *
-- Logs begin at Wed 2018-03-21 15:06:46 CET, end at Tue 2018-05-15
10:48:01 CEST. --
Mar 21 15:06:58 glusternode-1003 systemd[1]: Starting GlusterFS, a
clustered file-system
server...
Mar 21 15:07:01 glusternode-1 systemd[1]: Started GlusterFS, a clustered
file-system
server.
Mar 21 15:25:07 glusternode-1 systemd[1]: Stopping GlusterFS, a
clustered file-system
server...
Mar 21 15:25:07 glusternode-1 systemd[1]: Starting GlusterFS, a
clustered file-system
server...
Mar 21 15:25:09 glusternode-1 systemd[1]: Started GlusterFS, a clustered
file-system
server.
May 10 12:29:11 glusternode-1 systemd[1]: Stopping GlusterFS, a
clustered file-system
server...
May 10 12:29:11 glusternode-1 systemd[1]: Starting GlusterFS, a
clustered file-system
server...
May 10 12:29:13 glusternode-1 systemd[1]: Started GlusterFS, a clustered
file-system
server.
May 15 10:21:01 glusternode-1 systemd[1]: Starting GlusterFS, a
clustered file-system
server...
May 15 10:21:01 glusternode-1 systemd[1]: glusterd.service: control
process exited, code=exited
status=1
May 15 10:21:01 glusternode-1 systemd[1]: Failed to start GlusterFS, a
clustered file-system
server.
May 15 10:21:01 glusternode-1 systemd[1]: Unit glusterd.service entered
failed
state.
May 15 10:21:01 glusternode-1 systemd[1]: glusterd.service failed.
May 15 10:22:14 glusternode-1 systemd[1]: Starting GlusterFS, a
clustered file-system
server...
May 15 10:22:14 glusternode-1 systemd[1]: glusterd.service: control
process exited, code=exited
status=1
May 15 10:22:14 glusternode-1 systemd[1]: Failed to start GlusterFS, a
clustered file-system
server.
May 15 10:22:14 glusternode-1 systemd[1]: Unit glusterd.service entered
failed
state.
May 15 10:22:14 glusternode-1 systemd[1]: glusterd.service failed.
May 15 10:22:49 glusternode-1 systemd[1]: Starting GlusterFS, a
clustered file-system
server...
May 15 10:22:49 glusternode-1 systemd[1]: glusterd.service: control
process exited, code=exited
status=1
May 15 10:22:49 glusternode-1 systemd[1]: Failed to start GlusterFS, a
clustered file-system
server.
May 15 10:22:49 glusternode-1 systemd[1]: Unit glusterd.service entered
failed
state.
May 15 10:22:49 glusternode-1 systemd[1]: glusterd.service failed.
May 15 10:23:36 glusternode-1 systemd[1]: Starting GlusterFS, a
clustered file-system
server...
May 15 10:23:36 glusternode-1 systemd[1]: glusterd.service: control
process exited, code=exited
status=1
May 15 10:23:36 glusternode-1 systemd[1]: Failed to start GlusterFS, a
clustered file-system
server.
May 15 10:23:36 glusternode-1 systemd[1]: Unit glusterd.service entered
failed
state.
May 15 10:23:36 glusternode-1 systemd[1]: glusterd.service failed.
[2018-05-15 08:54:38.354520] I [MSGID: 100030] [glusterfsd.c:2511:main]
0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.12.9
(args: /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO)
[2018-05-15 08:54:38.359267] I [MSGID: 106478] [glusterd.c:1423:init]
0-management: Maximum allowed open file descriptors set to
65536
[2018-05-15 08:54:38.359305] I [MSGID: 106479] [glusterd.c:1481:init]
0-management: Using /var/lib/glusterd as working
directory
[2018-05-15 08:54:38.359313] I [MSGID: 106479] [glusterd.c:1486:init]
0-management: Using /var/run/gluster as pid file working
directory
[2018-05-15 08:54:38.363363] E [rpc-transport.c:283:rpc_transport_load]
cannot open shared object file: No such file or directory
[2018-05-15 08:54:38.363381] W [rpc-transport.c:287:rpc_transport_load]
0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not
valid or not found on this machine
[2018-05-15 08:54:38.363391] W [rpcsvc.c:1682:rpcsvc_create_listener]
0-rpc-service: cannot create listener, initing the transport
failed
[2018-05-15 08:54:38.363398] E [MSGID: 106243] [glusterd.c:1769:init]
0-management: creation of 1 listeners failed, continuing with succeeded
transport
[2018-05-15 08:54:39.981028] E [MSGID: 106022]
[glusterd-store.c:2234:glusterd_restore_op_version] 0-management: wrong
op-version (31300) retrieved [Invalid argument]
[2018-05-15 08:54:39.981071] E [MSGID: 106244] [glusterd.c:1928:init]
0-management: Failed to restore
op_version
[2018-05-15 08:54:39.981101] E [MSGID: 101019]
[xlator.c:503:xlator_init] 0-management: Initialization of volume
'management' failed, review your volfile again
[2018-05-15 08:54:39.981113] E [MSGID: 101066]
[graph.c:327:glusterfs_graph_init] 0-management: initializing translator
failed
[2018-05-15 08:54:39.981121] E [MSGID: 101176]
[graph.c:698:glusterfs_graph_activate] 0-graph: init
failed
[2018-05-15 08:54:39.981720] W [glusterfsd.c:1375:cleanup_and_exit]
(-->/usr/sbin/glusterd(glusterfs_volumes_init+0xfd) [0x559572dbcf5d]
-->/usr/sbin/glusterd(glusterfs_process_volfp+0x163)
[0x559572dbce03] -->/usr/sbin/glusterd(cleanup_and_exit+0x6b)
[0x559572dbc37b] ) 0-: received signum (-1), shutting
down
*/var/log/glusterfs/glusterd.log*
[2018-05-15 08:54:38.354520] I [MSGID: 100030] [glusterfsd.c:2511:main]
0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.12.9
(args: /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO)
[2018-05-15 08:54:38.359267] I [MSGID: 106478] [glusterd.c:1423:init]
0-management: Maximum allowed open file descriptors set to
65536
[2018-05-15 08:54:38.359305] I [MSGID: 106479] [glusterd.c:1481:init]
0-management: Using /var/lib/glusterd as working
directory
[2018-05-15 08:54:38.359313] I [MSGID: 106479] [glusterd.c:1486:init]
0-management: Using /var/run/gluster as pid file working
directory
[2018-05-15 08:54:38.363363] E [rpc-transport.c:283:rpc_transport_load]
cannot open shared object file: No such file or directory
[2018-05-15 08:54:38.363381] W [rpc-transport.c:287:rpc_transport_load]
0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not
valid or not found on this machine
[2018-05-15 08:54:38.363391] W [rpcsvc.c:1682:rpcsvc_create_listener]
0-rpc-service: cannot create listener, initing the transport
failed
[2018-05-15 08:54:38.363398] E [MSGID: 106243] [glusterd.c:1769:init]
0-management: creation of 1 listeners failed, continuing with succeeded
transport
[2018-05-15 08:54:39.981028] E [MSGID: 106022]
[glusterd-store.c:2234:glusterd_restore_op_version] 0-management: wrong
op-version (31300) retrieved [Invalid argument]
[2018-05-15 08:54:39.981071] E [MSGID: 106244] [glusterd.c:1928:init]
0-management: Failed to restore
op_version
*[2018-05-15 08:54:39.981101] E [MSGID: 101019]
[xlator.c:503:xlator_init] 0-management: Initialization of volume
'management' failed, review your volfile again
[2018-05-15 08:54:39.981113] E [MSGID: 101066]
[graph.c:327:glusterfs_graph_init] 0-management: initializing translator
failed
[2018-05-15 08:54:39.981121] E [MSGID: 101176]
[graph.c:698:glusterfs_graph_activate] 0-graph: init
failed
*
[2018-05-15 08:54:39.981720] W [glusterfsd.c:1375:cleanup_and_exit]
(-->/usr/sbin/glusterd(glusterfs_volumes_init+0xfd) [0x559572dbcf5d]
-->/usr/sbin/glusterd(glusterfs_process_volfp+0x163)
[0x559572dbce03] -->/usr/sbin/glusterd(cleanup_and_exit+0x6b)
[0x559572dbc37b] ) 0-: received signum (-1), shutting
down
Kind regards
Davide
_______________________________________________
Gluster-users mailing list
http://lists.gluster.org/mailman/listinfo/gluster-users

Davide Obbi

2018-05-15 12:08:59 UTC

Permalink

Thanks Kaleb,

any chance i can make the node working after the downgrade?
thanks

Post by Kaleb S. KEITHLEY
You can still get them from
https://buildlogs.centos.org/centos/7/storage/x86_64/gluster-3.13/
(I don't know how much longer they'll be there. I suggest you copy them
if you think you're going to need them in the future.)

Post by Davide Obbi
hi,
http://mirror.centos.org/centos/7/storage/x86_64/
i knew was not going to be long term supported however the downgrade to
*[2018-05-15 08:54:39.981101] E [MSGID: 101019]
[xlator.c:503:xlator_init] 0-management: Initialization of volume
'management' failed, review your volfile again
[2018-05-15 08:54:39.981113] E [MSGID: 101066]
[graph.c:327:glusterfs_graph_init] 0-management: initializing translator
failed
[2018-05-15 08:54:39.981121] E [MSGID: 101176]
[graph.c:698:glusterfs_graph_activate] 0-graph: init
failed
*
Any help appreciated, thanks
*Installed Packages*
glusterfs.x86_64
3.12.9-1.el7
@gluster-3.12
glusterfs-api.x86_64
3.12.9-1.el7
@gluster-3.12
glusterfs-cli.x86_64
3.12.9-1.el7
@gluster-3.12
glusterfs-client-xlators.x86_64
3.12.9-1.el7
@gluster-3.12
glusterfs-events.x86_64
3.12.9-1.el7
@gluster-3.12
glusterfs-fuse.x86_64
3.12.9-1.el7
@gluster-3.12
glusterfs-libs.x86_64
3.12.9-1.el7
@gluster-3.12
glusterfs-server.x86_64
3.12.9-1.el7
@gluster-3.12
python2-gluster.x86_64
3.12.9-1.el7
@gluster-3.12
*# journalctl -u
glusterd *
-- Logs begin at Wed 2018-03-21 15:06:46 CET, end at Tue 2018-05-15
10:48:01 CEST. --
Mar 21 15:06:58 glusternode-1003 systemd[1]: Starting GlusterFS, a
clustered file-system
server...
Mar 21 15:07:01 glusternode-1 systemd[1]: Started GlusterFS, a clustered
file-system
server.
Mar 21 15:25:07 glusternode-1 systemd[1]: Stopping GlusterFS, a
clustered file-system
server...
Mar 21 15:25:07 glusternode-1 systemd[1]: Starting GlusterFS, a
clustered file-system
server...
Mar 21 15:25:09 glusternode-1 systemd[1]: Started GlusterFS, a clustered
file-system
server.
May 10 12:29:11 glusternode-1 systemd[1]: Stopping GlusterFS, a
clustered file-system
server...
May 10 12:29:11 glusternode-1 systemd[1]: Starting GlusterFS, a
clustered file-system
server...
May 10 12:29:13 glusternode-1 systemd[1]: Started GlusterFS, a clustered
file-system
server.
May 15 10:21:01 glusternode-1 systemd[1]: Starting GlusterFS, a
clustered file-system
server...
May 15 10:21:01 glusternode-1 systemd[1]: glusterd.service: control
process exited, code=exited
status=1
May 15 10:21:01 glusternode-1 systemd[1]: Failed to start GlusterFS, a
clustered file-system
server.
May 15 10:21:01 glusternode-1 systemd[1]: Unit glusterd.service entered
failed
state.
May 15 10:21:01 glusternode-1 systemd[1]: glusterd.service failed.
May 15 10:22:14 glusternode-1 systemd[1]: Starting GlusterFS, a
clustered file-system
server...
May 15 10:22:14 glusternode-1 systemd[1]: glusterd.service: control
process exited, code=exited
status=1
May 15 10:22:14 glusternode-1 systemd[1]: Failed to start GlusterFS, a
clustered file-system
server.
May 15 10:22:14 glusternode-1 systemd[1]: Unit glusterd.service entered
failed
state.
May 15 10:22:14 glusternode-1 systemd[1]: glusterd.service failed.
May 15 10:22:49 glusternode-1 systemd[1]: Starting GlusterFS, a
clustered file-system
server...
May 15 10:22:49 glusternode-1 systemd[1]: glusterd.service: control
process exited, code=exited
status=1
May 15 10:22:49 glusternode-1 systemd[1]: Failed to start GlusterFS, a
clustered file-system
server.
May 15 10:22:49 glusternode-1 systemd[1]: Unit glusterd.service entered
failed
state.
May 15 10:22:49 glusternode-1 systemd[1]: glusterd.service failed.
May 15 10:23:36 glusternode-1 systemd[1]: Starting GlusterFS, a
clustered file-system
server...
May 15 10:23:36 glusternode-1 systemd[1]: glusterd.service: control
process exited, code=exited
status=1
May 15 10:23:36 glusternode-1 systemd[1]: Failed to start GlusterFS, a
clustered file-system
server.
May 15 10:23:36 glusternode-1 systemd[1]: Unit glusterd.service entered
failed
state.
May 15 10:23:36 glusternode-1 systemd[1]: glusterd.service failed.
[2018-05-15 08:54:38.354520] I [MSGID: 100030] [glusterfsd.c:2511:main]
0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.12.9
(args: /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO)
[2018-05-15 08:54:38.359267] I [MSGID: 106478] [glusterd.c:1423:init]
0-management: Maximum allowed open file descriptors set to
65536
[2018-05-15 08:54:38.359305] I [MSGID: 106479] [glusterd.c:1481:init]
0-management: Using /var/lib/glusterd as working
directory
[2018-05-15 08:54:38.359313] I [MSGID: 106479] [glusterd.c:1486:init]
0-management: Using /var/run/gluster as pid file working
directory
[2018-05-15 08:54:38.363363] E [rpc-transport.c:283:rpc_transport_load]
cannot open shared object file: No such file or directory
[2018-05-15 08:54:38.363381] W [rpc-transport.c:287:rpc_transport_load]
0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not
valid or not found on this machine
[2018-05-15 08:54:38.363391] W [rpcsvc.c:1682:rpcsvc_create_listener]
0-rpc-service: cannot create listener, initing the transport
failed
[2018-05-15 08:54:38.363398] E [MSGID: 106243] [glusterd.c:1769:init]
0-management: creation of 1 listeners failed, continuing with succeeded
transport
[2018-05-15 08:54:39.981028] E [MSGID: 106022]
[glusterd-store.c:2234:glusterd_restore_op_version] 0-management: wrong
op-version (31300) retrieved [Invalid argument]
[2018-05-15 08:54:39.981071] E [MSGID: 106244] [glusterd.c:1928:init]
0-management: Failed to restore
op_version
[2018-05-15 08:54:39.981101] E [MSGID: 101019]
[xlator.c:503:xlator_init] 0-management: Initialization of volume
'management' failed, review your volfile again
[2018-05-15 08:54:39.981113] E [MSGID: 101066]
[graph.c:327:glusterfs_graph_init] 0-management: initializing translator
failed
[2018-05-15 08:54:39.981121] E [MSGID: 101176]
[graph.c:698:glusterfs_graph_activate] 0-graph: init
failed
[2018-05-15 08:54:39.981720] W [glusterfsd.c:1375:cleanup_and_exit]
(-->/usr/sbin/glusterd(glusterfs_volumes_init+0xfd) [0x559572dbcf5d]
-->/usr/sbin/glusterd(glusterfs_process_volfp+0x163)
[0x559572dbce03] -->/usr/sbin/glusterd(cleanup_and_exit+0x6b)
[0x559572dbc37b] ) 0-: received signum (-1), shutting
down
*/var/log/glusterfs/glusterd.log*
[2018-05-15 08:54:38.354520] I [MSGID: 100030] [glusterfsd.c:2511:main]
0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.12.9
(args: /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO)
[2018-05-15 08:54:38.359267] I [MSGID: 106478] [glusterd.c:1423:init]
0-management: Maximum allowed open file descriptors set to
65536
[2018-05-15 08:54:38.359305] I [MSGID: 106479] [glusterd.c:1481:init]
0-management: Using /var/lib/glusterd as working
directory
[2018-05-15 08:54:38.359313] I [MSGID: 106479] [glusterd.c:1486:init]
0-management: Using /var/run/gluster as pid file working
directory
[2018-05-15 08:54:38.363363] E [rpc-transport.c:283:rpc_transport_load]
cannot open shared object file: No such file or directory
[2018-05-15 08:54:38.363381] W [rpc-transport.c:287:rpc_transport_load]
0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not
valid or not found on this machine
[2018-05-15 08:54:38.363391] W [rpcsvc.c:1682:rpcsvc_create_listener]
0-rpc-service: cannot create listener, initing the transport
failed
[2018-05-15 08:54:38.363398] E [MSGID: 106243] [glusterd.c:1769:init]
0-management: creation of 1 listeners failed, continuing with succeeded
transport
[2018-05-15 08:54:39.981028] E [MSGID: 106022]
[glusterd-store.c:2234:glusterd_restore_op_version] 0-management: wrong
op-version (31300) retrieved [Invalid argument]
[2018-05-15 08:54:39.981071] E [MSGID: 106244] [glusterd.c:1928:init]
0-management: Failed to restore
op_version
*[2018-05-15 08:54:39.981101] E [MSGID: 101019]
[xlator.c:503:xlator_init] 0-management: Initialization of volume
'management' failed, review your volfile again
[2018-05-15 08:54:39.981113] E [MSGID: 101066]
[graph.c:327:glusterfs_graph_init] 0-management: initializing translator
failed
[2018-05-15 08:54:39.981121] E [MSGID: 101176]
[graph.c:698:glusterfs_graph_activate] 0-graph: init
failed
*
[2018-05-15 08:54:39.981720] W [glusterfsd.c:1375:cleanup_and_exit]
(-->/usr/sbin/glusterd(glusterfs_volumes_init+0xfd) [0x559572dbcf5d]
-->/usr/sbin/glusterd(glusterfs_process_volfp+0x163)
[0x559572dbce03] -->/usr/sbin/glusterd(cleanup_and_exit+0x6b)
[0x559572dbc37b] ) 0-: received signum (-1), shutting
down
Kind regards
Davide
_______________________________________________
Gluster-users mailing list
http://lists.gluster.org/mailman/listinfo/gluster-users

--
Davide Obbi
System Administrator

Booking.com B.V.
Vijzelstraat 66-80 Amsterdam 1017HL Netherlands
Direct +31207031558
[image: Booking.com] <http://www.booking.com/>
The world's #1 accommodation site
43 languages, 198+ offices worldwide, 120,000+ global destinations,
1,550,000+ room nights booked every day
No booking fees, best price always guaranteed
Subsidiary of Booking Holdings Inc. (NASDAQ: BKNG)

Kaleb S. KEITHLEY

2018-05-15 12:47:00 UTC

Permalink

Post by Davide Obbi
Thanks Kaleb,
any chance i can make the node working after the downgrade?
thanks

Without knowing what doesn't work, I'll go out on a limb and guess that
it's an op-version problem.

Shut down your 3.13 nodes, change their op-version to one of the valid
3.12 op-versions (e.g. 31203) and restart. Then the 3.12 nodes should
work with 3.13, and/or the formerly 3.13 nodes should work with 3.12.

JoeJulian in #gluster may have other ideas.

Post by Davide Obbi
You can still get them from
https://buildlogs.centos.org/centos/7/storage/x86_64/gluster-3.13/
<https://buildlogs.centos.org/centos/7/storage/x86_64/gluster-3.13/>
(I don't know how much longer they'll be there. I suggest you copy them
if you think you're going to need them in the future.)

Post by Davide Obbi
hi,
http://mirror.centos.org/centos/7/storage/x86_64/

<http://mirror.centos.org/centos/7/storage/x86_64/>

Post by Davide Obbi
i knew was not going to be long term supported however the downgrade to
*[2018-05-15 08:54:39.981101] E [MSGID: 101019]
[xlator.c:503:xlator_init] 0-management: Initialization of volume
'management' failed, review your volfile again
[2018-05-15 08:54:39.981113] E [MSGID: 101066]
[graph.c:327:glusterfs_graph_init] 0-management: initializing translator
failed
[2018-05-15 08:54:39.981121] E [MSGID: 101176]
[graph.c:698:glusterfs_graph_activate] 0-graph: init
failed
*
Any help appreciated, thanks
*Installed Packages*

glusterfs.x86_64
3.12.9-1.el7

Post by Davide Obbi
@gluster-3.12

glusterfs-api.x86_64
3.12.9-1.el7

Post by Davide Obbi
@gluster-3.12

glusterfs-cli.x86_64
3.12.9-1.el7

Post by Davide Obbi
@gluster-3.12

glusterfs-client-xlators.x86_64
3.12.9-1.el7

Post by Davide Obbi
@gluster-3.12

glusterfs-events.x86_64
3.12.9-1.el7

Post by Davide Obbi
@gluster-3.12

glusterfs-fuse.x86_64
3.12.9-1.el7

Post by Davide Obbi
@gluster-3.12

glusterfs-libs.x86_64
3.12.9-1.el7

Post by Davide Obbi
@gluster-3.12

glusterfs-server.x86_64
3.12.9-1.el7

Post by Davide Obbi
@gluster-3.12

python2-gluster.x86_64
3.12.9-1.el7

Post by Davide Obbi
@gluster-3.12
*# journalctl -u

glusterd *

Post by Davide Obbi
-- Logs begin at Wed 2018-03-21 15:06:46 CET, end at Tue 2018-05-15
10:48:01 CEST. --
Mar 21 15:06:58 glusternode-1003 systemd[1]: Starting GlusterFS, a
clustered file-system

server...

Post by Davide Obbi
Mar 21 15:07:01 glusternode-1 systemd[1]: Started GlusterFS, a

clustered

Post by Davide Obbi
file-system

server.

Post by Davide Obbi
Mar 21 15:25:07 glusternode-1 systemd[1]: Stopping GlusterFS, a
clustered file-system

server...

Post by Davide Obbi
Mar 21 15:25:07 glusternode-1 systemd[1]: Starting GlusterFS, a
clustered file-system

server...

Post by Davide Obbi
Mar 21 15:25:09 glusternode-1 systemd[1]: Started GlusterFS, a

clustered

Post by Davide Obbi
file-system

server.

Post by Davide Obbi
May 10 12:29:11 glusternode-1 systemd[1]: Stopping GlusterFS, a
clustered file-system

server...

Post by Davide Obbi
May 10 12:29:11 glusternode-1 systemd[1]: Starting GlusterFS, a
clustered file-system

server...

Post by Davide Obbi
May 10 12:29:13 glusternode-1 systemd[1]: Started GlusterFS, a

clustered

Post by Davide Obbi
file-system

server.

Post by Davide Obbi
May 15 10:21:01 glusternode-1 systemd[1]: Starting GlusterFS, a
clustered file-system

server...

Post by Davide Obbi
May 15 10:21:01 glusternode-1 systemd[1]: glusterd.service: control
process exited, code=exited
status=1
May 15 10:21:01 glusternode-1 systemd[1]: Failed to start GlusterFS, a
clustered file-system
server.
May 15 10:21:01 glusternode-1 systemd[1]: Unit glusterd.service

entered

Post by Davide Obbi
failed

state.

Post by Davide Obbi
May 15 10:21:01 glusternode-1 systemd[1]: glusterd.service failed.
May 15 10:22:14 glusternode-1 systemd[1]: Starting GlusterFS, a
clustered file-system

server...

Post by Davide Obbi
May 15 10:22:14 glusternode-1 systemd[1]: glusterd.service: control
process exited, code=exited
status=1
May 15 10:22:14 glusternode-1 systemd[1]: Failed to start GlusterFS, a
clustered file-system
server.
May 15 10:22:14 glusternode-1 systemd[1]: Unit glusterd.service

entered

Post by Davide Obbi
failed

state.

Post by Davide Obbi
May 15 10:22:14 glusternode-1 systemd[1]: glusterd.service failed.
May 15 10:22:49 glusternode-1 systemd[1]: Starting GlusterFS, a
clustered file-system

server...

Post by Davide Obbi
May 15 10:22:49 glusternode-1 systemd[1]: glusterd.service: control
process exited, code=exited
status=1
May 15 10:22:49 glusternode-1 systemd[1]: Failed to start GlusterFS, a
clustered file-system
server.
May 15 10:22:49 glusternode-1 systemd[1]: Unit glusterd.service

entered

Post by Davide Obbi
failed

state.

Post by Davide Obbi
May 15 10:22:49 glusternode-1 systemd[1]: glusterd.service failed.
May 15 10:23:36 glusternode-1 systemd[1]: Starting GlusterFS, a
clustered file-system

server...

Post by Davide Obbi
May 15 10:23:36 glusternode-1 systemd[1]: glusterd.service: control
process exited, code=exited
status=1
May 15 10:23:36 glusternode-1 systemd[1]: Failed to start GlusterFS, a
clustered file-system
server.
May 15 10:23:36 glusternode-1 systemd[1]: Unit glusterd.service

entered

Post by Davide Obbi
failed

state.

Post by Davide Obbi
May 15 10:23:36 glusternode-1 systemd[1]: glusterd.service failed.
[2018-05-15 08:54:38.354520] I [MSGID: 100030]

[glusterfsd.c:2511:main]

Post by Davide Obbi
0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version

3.12.9

Post by Davide Obbi
(args: /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO)
[2018-05-15 08:54:38.359267] I [MSGID: 106478] [glusterd.c:1423:init]
0-management: Maximum allowed open file descriptors set to
65536
[2018-05-15 08:54:38.359305] I [MSGID: 106479] [glusterd.c:1481:init]
0-management: Using /var/lib/glusterd as working
directory
[2018-05-15 08:54:38.359313] I [MSGID: 106479] [glusterd.c:1486:init]
0-management: Using /var/run/gluster as pid file working
directory
[2018-05-15 08:54:38.363363] E

[rpc-transport.c:283:rpc_transport_load]

Post by Davide Obbi
cannot open shared object file: No such file or directory
[2018-05-15 08:54:38.363381] W

[rpc-transport.c:287:rpc_transport_load]

Post by Davide Obbi
0-rpc-transport: volume 'rdma.management': transport-type 'rdma'

is not

Post by Davide Obbi
valid or not found on this machine
[2018-05-15 08:54:38.363391] W [rpcsvc.c:1682:rpcsvc_create_listener]
0-rpc-service: cannot create listener, initing the transport
failed
[2018-05-15 08:54:38.363398] E [MSGID: 106243] [glusterd.c:1769:init]
0-management: creation of 1 listeners failed, continuing with

succeeded

Post by Davide Obbi
transport
[2018-05-15 08:54:39.981028] E [MSGID: 106022]

wrong

Post by Davide Obbi
op-version (31300) retrieved [Invalid argument]
[2018-05-15 08:54:39.981071] E [MSGID: 106244] [glusterd.c:1928:init]
0-management: Failed to restore

op_version

Post by Davide Obbi
[2018-05-15 08:54:39.981101] E [MSGID: 101019]
[xlator.c:503:xlator_init] 0-management: Initialization of volume
'management' failed, review your volfile

again

Post by Davide Obbi
[2018-05-15 08:54:39.981113] E [MSGID: 101066]
[graph.c:327:glusterfs_graph_init] 0-management: initializing

translator

Post by Davide Obbi
failed
[2018-05-15 08:54:39.981121] E [MSGID: 101176]
[graph.c:698:glusterfs_graph_activate] 0-graph: init

failed

Post by Davide Obbi
[2018-05-15 08:54:39.981720] W [glusterfsd.c:1375:cleanup_and_exit]
(-->/usr/sbin/glusterd(glusterfs_volumes_init+0xfd) [0x559572dbcf5d]
-->/usr/sbin/glusterd(glusterfs_process_volfp+0x163)
[0x559572dbce03] -->/usr/sbin/glusterd(cleanup_and_exit+0x6b)
[0x559572dbc37b] ) 0-: received signum (-1), shutting
down
*/var/log/glusterfs/glusterd.log*
[2018-05-15 08:54:38.354520] I [MSGID: 100030]

[glusterfsd.c:2511:main]

Post by Davide Obbi
0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version

3.12.9

[rpc-transport.c:283:rpc_transport_load]

Post by Davide Obbi
cannot open shared object file: No such file or directory
[2018-05-15 08:54:38.363381] W

[rpc-transport.c:287:rpc_transport_load]

Post by Davide Obbi
0-rpc-transport: volume 'rdma.management': transport-type 'rdma'

is not

succeeded

Post by Davide Obbi
transport
[2018-05-15 08:54:39.981028] E [MSGID: 106022]

wrong

Post by Davide Obbi
op-version (31300) retrieved [Invalid argument]
[2018-05-15 08:54:39.981071] E [MSGID: 106244] [glusterd.c:1928:init]
0-management: Failed to restore

op_version

Post by Davide Obbi
*[2018-05-15 08:54:39.981101] E [MSGID: 101019]
[xlator.c:503:xlator_init] 0-management: Initialization of volume
'management' failed, review your volfile again
[2018-05-15 08:54:39.981113] E [MSGID: 101066]
[graph.c:327:glusterfs_graph_init] 0-management: initializing translator
failed
[2018-05-15 08:54:39.981121] E [MSGID: 101176]
[graph.c:698:glusterfs_graph_activate] 0-graph: init
failed
*
[2018-05-15 08:54:39.981720] W [glusterfsd.c:1375:cleanup_and_exit]
(-->/usr/sbin/glusterd(glusterfs_volumes_init+0xfd) [0x559572dbcf5d]
-->/usr/sbin/glusterd(glusterfs_process_volfp+0x163)
[0x559572dbce03] -->/usr/sbin/glusterd(cleanup_and_exit+0x6b)
[0x559572dbc37b] ) 0-: received signum (-1), shutting
down
Kind regards
Davide
_______________________________________________
Gluster-users mailing list
http://lists.gluster.org/mailman/listinfo/gluster-users

<http://lists.gluster.org/mailman/listinfo/gluster-users>
--
Davide Obbi
System Administrator
Booking.com B.V.
Vijzelstraat 66-80 Amsterdam 1017HL Netherlands
Direct +31207031558
Booking.com <http://www.booking.com/>
The world's #1 accommodation site
43 languages, 198+ offices worldwide, 120,000+ global destinations,
1,550,000+ room nights booked every day
No booking fees, best price always guaranteed
Subsidiary of Booking Holdings Inc. (NASDAQ: BKNG)

Davide Obbi

2018-05-15 13:16:13 UTC

Permalink

i have a 12 nodes cluster with 2 nodes downgraded and currently down. The
procedure of taking down 3.13 nodes means a cluster downtime?

if it matters /var/lib/glusterd/glusterd.info is empty on the 3.12.9 nodes.

Post by Kaleb S. KEITHLEY

Post by Davide Obbi
Thanks Kaleb,
any chance i can make the node working after the downgrade?
thanks

Without knowing what doesn't work, I'll go out on a limb and guess that
it's an op-version problem.
Shut down your 3.13 nodes, change their op-version to one of the valid
3.12 op-versions (e.g. 31203) and restart. Then the 3.12 nodes should
work with 3.13, and/or the formerly 3.13 nodes should work with 3.12.
JoeJulian in #gluster may have other ideas.

them

Post by Davide Obbi
if you think you're going to need them in the future.)

Post by Davide Obbi
hi,
i noticed that this repo for glusterfs 3.13 does not exists
http://mirror.centos.org/centos/7/storage/x86_64/

<http://mirror.centos.org/centos/7/storage/x86_64/>

Post by Davide Obbi
i knew was not going to be long term supported however the

downgrade to