Discussion:
[Gluster-users] Expanding a distributed disperse volume: some questions about the action plan
Ashish Pandey
2018-06-07 14:49:28 UTC
Permalink
Mauro,

Your plan looks fine to me. However, I would like to add few points.

1 - After adding 3 more nodes you will have overall 6 nodes. It would be better if you can have 1 brick on each server for any EC subvolume.
For example: for 4+2, all 6 bricks should be on different nodes. This will help you to have better redundancy. For example even if 2 nodes go down, you will still have 4 good bricks.
In your existing setup, if 2 nodes go down you will loose 4 bricks which will make the volume unusable.
Although it would be tricky to do so, as you have older version on 3 nodes and newer on 3 nodes.

2 - In your plan, if you can use gnfs mount, you could easily copy the data from one volume to other. In this case the client would be running on one of the server which would be same as the server version.

3 - The plan you mentioned will also work. However, you have to have *older* client version on third machine on which you are going to mount both volumes.

4 - The plan which Serkan mentioned is also a feasible plan specially if IO's are not going on. However, just keep the first point in your mind and see if you can reach to that setup.

---
Ashish





----- Original Message -----

From: "Serkan Çoban" <***@gmail.com>
To: "Mauro Tridici" <***@cmcc.it>
Cc: "Gluster Users" <gluster-***@gluster.org>
Sent: Thursday, June 7, 2018 7:19:50 PM
Subject: Re: [Gluster-users] Expanding a distributed disperse volume: some questions about the action plan
in order to copy the data from the old volume to the new one, I need a third machine that can mount both the volumes; it's possible? if yes, which gluster client version should I use/install on the "bridge" machine?
Yes it is possible, you need to install old client version to bridge
server, old clients can talk to new servers

Why not just extend the old cluster & rebalance and then upgrade to
latest version of glusterfs?
Dear Users,
just one year ago, we implemented a tier2 storage based on GlusterFS
v.3.10.5 and we can say that it works like a charm.
The GlusterFS volume has been created using 3 physical servers (each server
contains 12 disks, 1 brick per disk).
Below, you can find some information about the gluster volume type
Volume Name: tier2
Type: Distributed-Disperse
Volume ID: a28d88c5-3295-4e35-98d4-210b3af9358c
Status: Started
Snapshot Count: 0
Number of Bricks: 6 x (4 + 2) = 36
Transport-type: tcp
Now, it's time to expand this volume so we bought 3 new servers similar to
the 3 existing servers. Since we would like to upgrade the Gluster version
to last available one, we would like to know if the following action plan is
1) creation of a new gluster distributed disperse volume (6x(4+2)) using the
3 new servers and the new gluster version;
2) copy of all the data saved on the old gluster volume to the new one;
3) complete erasure of three old servers operating system;
4) installation of the last available version of the operating system and
glusterfs software stack on the three old servers;
5) expansion of the volume (created at point nr.1) adding the new bricks
(created at point nr.4)
- in order to copy the data from the old volume to the new one, I need a
third machine that can mount both the volumes; it's possible? if yes, which
gluster client version should I use/install on the "bridge" machine?
Thank you very much for your attention.
Regards,
Mauro
_______________________________________________
Gluster-users mailing list
http://lists.gluster.org/mailman/listinfo/gluster-users
_______________________________________________
Gluster-users mailing list
Gluster-***@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-users
Mauro Tridici
2018-06-07 19:45:09 UTC
Permalink
Hi Serkan,

thank you very much for your answer.
I really appreciate your suggestion (extension + rebalance + upgrade), but we would like to avoid any errors or risks of data loss during the upgrade procedure (we don’t have an advanced experience and we don’t know what to do if something goes wrong).
Currently, our Gluster volume contains a lot of consolidated and important scientific data and, unfortunately, we didn’t yet activate the Gluster support.

Thank you for sharing with me your opinion.
Regards,
Mauro
Post by Ashish Pandey
in order to copy the data from the old volume to the new one, I need a third machine that can mount both the volumes; it's possible? if yes, which gluster client version should I use/install on the "bridge" machine?
Yes it is possible, you need to install old client version to bridge
server, old clients can talk to new servers
Why not just extend the old cluster & rebalance and then upgrade to
latest version of glusterfs?
Dear Users,
just one year ago, we implemented a tier2 storage based on GlusterFS
v.3.10.5 and we can say that it works like a charm.
The GlusterFS volume has been created using 3 physical servers (each server
contains 12 disks, 1 brick per disk).
Below, you can find some information about the gluster volume type
Volume Name: tier2
Type: Distributed-Disperse
Volume ID: a28d88c5-3295-4e35-98d4-210b3af9358c
Status: Started
Snapshot Count: 0
Number of Bricks: 6 x (4 + 2) = 36
Transport-type: tcp
Now, it's time to expand this volume so we bought 3 new servers similar to
the 3 existing servers. Since we would like to upgrade the Gluster version
to last available one, we would like to know if the following action plan is
1) creation of a new gluster distributed disperse volume (6x(4+2)) using the
3 new servers and the new gluster version;
2) copy of all the data saved on the old gluster volume to the new one;
3) complete erasure of three old servers operating system;
4) installation of the last available version of the operating system and
glusterfs software stack on the three old servers;
5) expansion of the volume (created at point nr.1) adding the new bricks
(created at point nr.4)
- in order to copy the data from the old volume to the new one, I need a
third machine that can mount both the volumes; it's possible? if yes, which
gluster client version should I use/install on the "bridge" machine?
Thank you very much for your attention.
Regards,
Mauro
_______________________________________________
Gluster-users mailing list
http://lists.gluster.org/mailman/listinfo/gluster-users
-------------------------
Mauro Tridici

Fondazione CMCC
CMCC Supercomputing Center
presso Complesso Ecotekne - Università del Salento -
Strada Prov.le Lecce - Monteroni sn
73100 Lecce IT
http://www.cmcc.it

mobile: (+39) 327 5630841
email: ***@cmcc.it
Mauro Tridici
2018-06-07 19:59:00 UTC
Permalink
Dear Ashish,

thank you very much for your support.
I think that I’m not able to change the volume configuration for the reason you mentioned at the end of your n.1 answer.
GNFS mount is a very good idea, thank you!!! I will use it in my action plan.

I’m very happy to have been helped by you (Ashish and Serkan).
Regards,
Mauro
Post by Ashish Pandey
Mauro,
Your plan looks fine to me. However, I would like to add few points.
1 - After adding 3 more nodes you will have overall 6 nodes. It would be better if you can have 1 brick on each server for any EC subvolume.
For example: for 4+2, all 6 bricks should be on different nodes. This will help you to have better redundancy. For example even if 2 nodes go down, you will still have 4 good bricks.
In your existing setup, if 2 nodes go down you will loose 4 bricks which will make the volume unusable.
Although it would be tricky to do so, as you have older version on 3 nodes and newer on 3 nodes.
2 - In your plan, if you can use gnfs mount, you could easily copy the data from one volume to other. In this case the client would be running on one of the server which would be same as the server version.
3 - The plan you mentioned will also work. However, you have to have *older* client version on third machine on which you are going to mount both volumes.
4 - The plan which Serkan mentioned is also a feasible plan specially if IO's are not going on. However, just keep the first point in your mind and see if you can reach to that setup.
---
Ashish
Sent: Thursday, June 7, 2018 7:19:50 PM
Subject: Re: [Gluster-users] Expanding a distributed disperse volume: some questions about the action plan
in order to copy the data from the old volume to the new one, I need a third machine that can mount both the volumes; it's possible? if yes, which gluster client version should I use/install on the "bridge" machine?
Yes it is possible, you need to install old client version to bridge
server, old clients can talk to new servers
Why not just extend the old cluster & rebalance and then upgrade to
latest version of glusterfs?
Dear Users,
just one year ago, we implemented a tier2 storage based on GlusterFS
v.3.10.5 and we can say that it works like a charm.
The GlusterFS volume has been created using 3 physical servers (each server
contains 12 disks, 1 brick per disk).
Below, you can find some information about the gluster volume type
Volume Name: tier2
Type: Distributed-Disperse
Volume ID: a28d88c5-3295-4e35-98d4-210b3af9358c
Status: Started
Snapshot Count: 0
Number of Bricks: 6 x (4 + 2) = 36
Transport-type: tcp
Now, it's time to expand this volume so we bought 3 new servers similar to
the 3 existing servers. Since we would like to upgrade the Gluster version
to last available one, we would like to know if the following action plan is
1) creation of a new gluster distributed disperse volume (6x(4+2)) using the
3 new servers and the new gluster version;
2) copy of all the data saved on the old gluster volume to the new one;
3) complete erasure of three old servers operating system;
4) installation of the last available version of the operating system and
glusterfs software stack on the three old servers;
5) expansion of the volume (created at point nr.1) adding the new bricks
(created at point nr.4)
- in order to copy the data from the old volume to the new one, I need a
third machine that can mount both the volumes; it's possible? if yes, which
gluster client version should I use/install on the "bridge" machine?
Thank you very much for your attention.
Regards,
Mauro
_______________________________________________
Gluster-users mailing list
http://lists.gluster.org/mailman/listinfo/gluster-users
_______________________________________________
Gluster-users mailing list
http://lists.gluster.org/mailman/listinfo/gluster-users
Ashish Pandey
2018-07-27 07:56:44 UTC
Permalink
Hey Mauro,

How did it go? Were you able to expand volumes without any issues?
Just curious to know the approach you took and how smooth was it to
expand this volume.

---
Ashish


----- Original Message -----

From: "Mauro Tridici" <***@cmcc.it>
To: "Ashish Pandey" <***@redhat.com>
Cc: "Gluster Users" <gluster-***@gluster.org>
Sent: Friday, June 8, 2018 1:29:00 AM
Subject: Re: [Gluster-users] Expanding a distributed disperse volume: some questions about the action plan

Dear Ashish,

thank you very much for your support.
I think that I’m not able to change the volume configuration for the reason you mentioned at the end of your n.1 answer.
GNFS mount is a very good idea, thank you!!! I will use it in my action plan.

I’m very happy to have been helped by you (Ashish and Serkan).
Regards,
Mauro




Il giorno 07 giu 2018, alle ore 16:49, Ashish Pandey < ***@redhat.com > ha scritto:

Mauro,

Your plan looks fine to me. However, I would like to add few points.

1 - After adding 3 more nodes you will have overall 6 nodes. It would be better if you can have 1 brick on each server for any EC subvolume.
For example: for 4+2, all 6 bricks should be on different nodes. This will help you to have better redundancy. For example even if 2 nodes go down, you will still have 4 good bricks.
In your existing setup, if 2 nodes go down you will loose 4 bricks which will make the volume unusable.
Although it would be tricky to do so, as you have older version on 3 nodes and newer on 3 nodes.

2 - In your plan, if you can use gnfs mount, you could easily copy the data from one volume to other. In this case the client would be running on one of the server which would be same as the server version.

3 - The plan you mentioned will also work. However, you have to have *older* client version on third machine on which you are going to mount both volumes.

4 - The plan which Serkan mentioned is also a feasible plan specially if IO's are not going on. However, just keep the first point in your mind and see if you can reach to that setup.

---
Ashish





----- Original Message -----

From: "Serkan Çoban" < ***@gmail.com >
To: "Mauro Tridici" < ***@cmcc.it >
Cc: "Gluster Users" < gluster-***@gluster.org >
Sent: Thursday, June 7, 2018 7:19:50 PM
Subject: Re: [Gluster-users] Expanding a distributed disperse volume: some questions about the action plan
in order to copy the data from the old volume to the new one, I need a third machine that can mount both the volumes; it's possible? if yes, which gluster client version should I use/install on the "bridge" machine?
Yes it is possible, you need to install old client version to bridge
server, old clients can talk to new servers

Why not just extend the old cluster & rebalance and then upgrade to
latest version of glusterfs?
Dear Users,
just one year ago, we implemented a tier2 storage based on GlusterFS
v.3.10.5 and we can say that it works like a charm.
The GlusterFS volume has been created using 3 physical servers (each server
contains 12 disks, 1 brick per disk).
Below, you can find some information about the gluster volume type
Volume Name: tier2
Type: Distributed-Disperse
Volume ID: a28d88c5-3295-4e35-98d4-210b3af9358c
Status: Started
Snapshot Count: 0
Number of Bricks: 6 x (4 + 2) = 36
Transport-type: tcp
Now, it's time to expand this volume so we bought 3 new servers similar to
the 3 existing servers. Since we would like to upgrade the Gluster version
to last available one, we would like to know if the following action plan is
1) creation of a new gluster distributed disperse volume (6x(4+2)) using the
3 new servers and the new gluster version;
2) copy of all the data saved on the old gluster volume to the new one;
3) complete erasure of three old servers operating system;
4) installation of the last available version of the operating system and
glusterfs software stack on the three old servers;
5) expansion of the volume (created at point nr.1) adding the new bricks
(created at point nr.4)
- in order to copy the data from the old volume to the new one, I need a
third machine that can mount both the volumes; it's possible? if yes, which
gluster client version should I use/install on the "bridge" machine?
Thank you very much for your attention.
Regards,
Mauro
_______________________________________________
Gluster-users mailing list
http://lists.gluster.org/mailman/listinfo/gluster-users
_______________________________________________
Gluster-users mailing list
Gluster-***@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-users






_______________________________________________
Gluster-users mailing list
Gluster-***@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-users

Loading...