Amar Tumballi
2018-07-20 07:16:47 UTC
We sent an email few days back for proposal to deprecate some features in
glusterfs (
https://lists.gluster.org/pipermail/gluster-devel/2018-July/054997.html).
Shyam recently sent the document as a patch upstream Gluster @
https://review.gluster.org/20538/. (Same is copied below in email here).
Please provide your valuable feedback on the same, so we can make it a
general practice.
This is done for making things more clear about proper expectation from
each of the component/feature, we want to have more classification of each
feature, control them using the code itself, and make sure we keep the list
up-to-date with each release, so our users have proper expectations set.
<content-from-patch>
The purpose of the document is to define a classification for various
xlators and expectations around what each classification means from a
perspective of health and maintenance of the xlator.
The need to do this is to ensure certain classifications are kept in good
health, and helps the community and contributors focus efforts on around
the same.
<https://hackmd.io/q_aEokbxSBSN-LdoaJ6Z6A#Classifications>Classifications
1. Experimental (E)
2. TechPreview (TP)
3. Maintained/Supported (M)
4. Sunset (S)
5. Deprecated (D)
<https://hackmd.io/q_aEokbxSBSN-LdoaJ6Z6A#Experimental-E>Experimental (E)
Developed in the experimental branch, for exploring new features. These are
NEVER released, and MAYBE packaged to help with getting feedback from
interested users.
<https://hackmd.io/q_aEokbxSBSN-LdoaJ6Z6A#Quality-expectations>Quality
expectations
- Compiles
- Does not break nightly experimental regressions
<https://hackmd.io/q_aEokbxSBSN-LdoaJ6Z6A#TechPreview-TP>TechPreview (TP)
Features in master or release branches that are not complete for general
purpose consumption, but are mature enough to invite feedback and host user
data.
These features will receive better attention from maintainers/authors that
are working on maturing the same, than ones in
Experimental/Sunset/Deprecated states.
There is no grantee that these features will move to the Maintained state,
and may just get Deprecated based on feedback, or other project goals or
technical alternatives.
<https://hackmd.io/q_aEokbxSBSN-LdoaJ6Z6A#Quality-expectations1>Quality
expectations
- Same as Maintained, sans
- Performance, Scale, other(?)
<https://hackmd.io/q_aEokbxSBSN-LdoaJ6Z6A#Maintained-M>Maintained (M)
These features are part of the core Gluster functionality and are
maintained actively. These are part of master and release branches and get
high priority attention from maintainers and other interested contributors.
<https://hackmd.io/q_aEokbxSBSN-LdoaJ6Z6A#Quality-expectations2>Quality
expectations
NOTE: A short note on what each of these mean are added here, details to
follow.
- Bug backlog: Actively address bug backlog
- Enhancement backlog: Actively maintain outstanding enhancement backlog
(need not be acted on, but should be visible to all)
- Review backlog: Actively keep this below desired counts and states
- Static code health: Actively meet near-zero issues in this regard
- Coverity, spellcheck and other checks
- Runtime code health: Actively meet defined coverage levels in this
regard
- Coverage, others?
- Per-patch regressions
- Glusto runs
- Performance
- Scalability
- Technical specifications: Implementation details should be documented
and updated at regular cadence (even per patch that change assumptions in
here)
- User documentation: User facing details should be maintained to
current status in the documentation
- Debuggability: Steps, tools, procedures should be documented and
maintained each release/patch as applicable
- Troubleshooting: Steps, tools, procedures should be documented and
maintained each release/patch as applicable
- Steps/guides for self service
- Knowledge base for problems
- Other common criteria that will apply: Required metrics/desired states
to be define per criteria
- Monitoring, usability, statedump, and other such xlator expectations
<https://hackmd.io/q_aEokbxSBSN-LdoaJ6Z6A#Sunset-S>Sunset (S)
Features on master or release branches that would be deprecated and/or
replaced with similar or other functionality in the next major release.
<https://hackmd.io/q_aEokbxSBSN-LdoaJ6Z6A#Quality-expectations3>Quality
expectations
- Retain status-quo when moved to this state, till it is moved to
deprecated
<https://hackmd.io/q_aEokbxSBSN-LdoaJ6Z6A#Deprecated-D>Deprecated (D)
Features/code still in tree, but not packaged or shipped or supported in
any form. This is noted as a category till the code is removed from the
tree.
These feature and their corresponding code and test health will not be
executed.
<https://hackmd.io/q_aEokbxSBSN-LdoaJ6Z6A#Quality-expectations4>Quality
expectations
- None
</content-from-patch>
Regards,
Amar
glusterfs (
https://lists.gluster.org/pipermail/gluster-devel/2018-July/054997.html).
Shyam recently sent the document as a patch upstream Gluster @
https://review.gluster.org/20538/. (Same is copied below in email here).
Please provide your valuable feedback on the same, so we can make it a
general practice.
This is done for making things more clear about proper expectation from
each of the component/feature, we want to have more classification of each
feature, control them using the code itself, and make sure we keep the list
up-to-date with each release, so our users have proper expectations set.
<content-from-patch>
The purpose of the document is to define a classification for various
xlators and expectations around what each classification means from a
perspective of health and maintenance of the xlator.
The need to do this is to ensure certain classifications are kept in good
health, and helps the community and contributors focus efforts on around
the same.
<https://hackmd.io/q_aEokbxSBSN-LdoaJ6Z6A#Classifications>Classifications
1. Experimental (E)
2. TechPreview (TP)
3. Maintained/Supported (M)
4. Sunset (S)
5. Deprecated (D)
<https://hackmd.io/q_aEokbxSBSN-LdoaJ6Z6A#Experimental-E>Experimental (E)
Developed in the experimental branch, for exploring new features. These are
NEVER released, and MAYBE packaged to help with getting feedback from
interested users.
<https://hackmd.io/q_aEokbxSBSN-LdoaJ6Z6A#Quality-expectations>Quality
expectations
- Compiles
- Does not break nightly experimental regressions
<https://hackmd.io/q_aEokbxSBSN-LdoaJ6Z6A#TechPreview-TP>TechPreview (TP)
Features in master or release branches that are not complete for general
purpose consumption, but are mature enough to invite feedback and host user
data.
These features will receive better attention from maintainers/authors that
are working on maturing the same, than ones in
Experimental/Sunset/Deprecated states.
There is no grantee that these features will move to the Maintained state,
and may just get Deprecated based on feedback, or other project goals or
technical alternatives.
<https://hackmd.io/q_aEokbxSBSN-LdoaJ6Z6A#Quality-expectations1>Quality
expectations
- Same as Maintained, sans
- Performance, Scale, other(?)
<https://hackmd.io/q_aEokbxSBSN-LdoaJ6Z6A#Maintained-M>Maintained (M)
These features are part of the core Gluster functionality and are
maintained actively. These are part of master and release branches and get
high priority attention from maintainers and other interested contributors.
<https://hackmd.io/q_aEokbxSBSN-LdoaJ6Z6A#Quality-expectations2>Quality
expectations
NOTE: A short note on what each of these mean are added here, details to
follow.
- Bug backlog: Actively address bug backlog
- Enhancement backlog: Actively maintain outstanding enhancement backlog
(need not be acted on, but should be visible to all)
- Review backlog: Actively keep this below desired counts and states
- Static code health: Actively meet near-zero issues in this regard
- Coverity, spellcheck and other checks
- Runtime code health: Actively meet defined coverage levels in this
regard
- Coverage, others?
- Per-patch regressions
- Glusto runs
- Performance
- Scalability
- Technical specifications: Implementation details should be documented
and updated at regular cadence (even per patch that change assumptions in
here)
- User documentation: User facing details should be maintained to
current status in the documentation
- Debuggability: Steps, tools, procedures should be documented and
maintained each release/patch as applicable
- Troubleshooting: Steps, tools, procedures should be documented and
maintained each release/patch as applicable
- Steps/guides for self service
- Knowledge base for problems
- Other common criteria that will apply: Required metrics/desired states
to be define per criteria
- Monitoring, usability, statedump, and other such xlator expectations
<https://hackmd.io/q_aEokbxSBSN-LdoaJ6Z6A#Sunset-S>Sunset (S)
Features on master or release branches that would be deprecated and/or
replaced with similar or other functionality in the next major release.
<https://hackmd.io/q_aEokbxSBSN-LdoaJ6Z6A#Quality-expectations3>Quality
expectations
- Retain status-quo when moved to this state, till it is moved to
deprecated
<https://hackmd.io/q_aEokbxSBSN-LdoaJ6Z6A#Deprecated-D>Deprecated (D)
Features/code still in tree, but not packaged or shipped or supported in
any form. This is noted as a category till the code is removed from the
tree.
These feature and their corresponding code and test health will not be
executed.
<https://hackmd.io/q_aEokbxSBSN-LdoaJ6Z6A#Quality-expectations4>Quality
expectations
- None
</content-from-patch>
Regards,
Amar
*Hi all,Over last 12 years of Gluster, we have developed many features,
and continue to support most of it till now. But along the way, we have
figured out better methods of doing things. Also we are not actively
maintaining some of these features.We are now thinking of cleaning up some
of these âunsupportedâ features, and mark them as âSunSetâ (i.e., would be
totally taken out of codebase in following releases) in next upcoming
release, v5.0. The release notes will provide options for smoothly
migrating to the supported configurations.If you are using any of these
features, do let us know, so that we can help you with âmigrationâ.. Also,
we are happy to guide new developers to work on those components which are
not actively being maintained by current set of developers.List of features
hitting sunset:âcluster/stripeâ translator:This translator was developed
very early in the evolution of GlusterFS, and addressed one of the very
common question of Distributed FS, which is âWhat happens if one of my file
is bigger than the available brick. Say, I have 2 TB hard drive, exported
in glusterfs, my file is 3 TBâ. While it solved the purpose, it was very
hard to handle failure scenarios, and give a real good experience to our
users with this feature. Over the time, Gluster solved the problem with
itâs âShardâ feature, which solves the problem in much better way, and
provides much better solution with existing well supported stack. Hence the
proposal for Deprecation.If you are using this feature, then do write to
us, as it needs a proper migration from existing volume to a new full
supported volume type before you upgrade.âstorage/bdâ translator:This
feature got into the code base 5 years back with this patch
<http://review.gluster.org/4809>[1]. Plan was to use a block device
directly as a brick, which would help to handle disk-image storage much
easily in glusterfs.As the feature is not getting more contribution, and we
are not seeing any user traction on this, would like to propose for
Deprecation.If you are using the feature, plan to move to a supported
gluster volume configuration, and have your setup âsupportedâ before
upgrading to your new gluster version.âRDMAâ transport support:Gluster
started supporting RDMA while ib-verbs was still new, and very high-end
infra around that time were using Infiniband. Engineers did work with
Mellanox, and got the technology into GlusterFS for better data migration,
data copy. While current day kernels support very good speed with IPoIB
module itself, and there are no more bandwidth for experts in these area to
maintain the feature, we recommend migrating over to TCP (IP based) network
for your volume.If you are successfully using RDMA transport, do get in
touch with us to prioritize the migration plan for your volume. Plan is to
work on this after the release, so by version 6.0, we will have a cleaner
transport code, which just needs to support one type.âTieringâ
featureGlusterâs tiering feature which was planned to be providing an
option to keep your âhotâ data in different location than your cold data,
so one can get better performance. While we saw some users for the feature,
it needs much more attention to be completely bug free. At the time, we are
not having any active maintainers for the feature, and hence suggesting to
take it out of the âsupportedâ tag.If you are willing to take it up, and
maintain it, do let us know, and we are happy to assist you.If you are
already using tiering feature, before upgrading, make sure to do gluster
volume tier detach all the bricks before upgrading to next release. Also,
we recommend you to use features like dmcache on your LVM setup to get best
performance from bricks.âQuotaâThis is a call out for âQuotaâ feature, to
let you all know that it will be âno new developmentâ state. While this
feature is âactivelyâ in use by many people, the challenges we have in
accounting mechanisms involved, has made it hard to achieve good
performance with the feature. Also, the amount of extended attribute
get/set operations while using the feature is not very ideal. Hence we
recommend our users to move towards setting quota on backend bricks
directly (ie, XFS project quota), or to use different volumes for different
directories etc.As the feature wouldnât be deprecated immediately, the
feature doesnât need a migration plan when you upgrade to newer version,
but if you are a new user, we wouldnât recommend setting quota feature. By
the release dates, we will be publishing our best alternatives guide for
glusterâs current quota feature.Note that if you want to contribute to the
feature, we have project quota based issue open
<https://github.com/gluster/glusterfs/issues/184>[2] Happy to get
contributions, and help in getting a newer approach to
Quota.------------------------------These are our set of initial features
which we propose to take out of âfullyâ supported features. While we are in
the process of making the user/developer experience of the project much
better with providing well maintained codebase, we may come up with few
more set of features which we may possibly consider to move out of support,
and hence keep watching this space.[1] - http://review.gluster.org/4809
<http://review.gluster.org/4809>[2] -
https://github.com/gluster/glusterfs/issues/184
<https://github.com/gluster/glusterfs/issues/184>Regards,Vijay, Shyam, Amar*
and continue to support most of it till now. But along the way, we have
figured out better methods of doing things. Also we are not actively
maintaining some of these features.We are now thinking of cleaning up some
of these âunsupportedâ features, and mark them as âSunSetâ (i.e., would be
totally taken out of codebase in following releases) in next upcoming
release, v5.0. The release notes will provide options for smoothly
migrating to the supported configurations.If you are using any of these
features, do let us know, so that we can help you with âmigrationâ.. Also,
we are happy to guide new developers to work on those components which are
not actively being maintained by current set of developers.List of features
hitting sunset:âcluster/stripeâ translator:This translator was developed
very early in the evolution of GlusterFS, and addressed one of the very
common question of Distributed FS, which is âWhat happens if one of my file
is bigger than the available brick. Say, I have 2 TB hard drive, exported
in glusterfs, my file is 3 TBâ. While it solved the purpose, it was very
hard to handle failure scenarios, and give a real good experience to our
users with this feature. Over the time, Gluster solved the problem with
itâs âShardâ feature, which solves the problem in much better way, and
provides much better solution with existing well supported stack. Hence the
proposal for Deprecation.If you are using this feature, then do write to
us, as it needs a proper migration from existing volume to a new full
supported volume type before you upgrade.âstorage/bdâ translator:This
feature got into the code base 5 years back with this patch
<http://review.gluster.org/4809>[1]. Plan was to use a block device
directly as a brick, which would help to handle disk-image storage much
easily in glusterfs.As the feature is not getting more contribution, and we
are not seeing any user traction on this, would like to propose for
Deprecation.If you are using the feature, plan to move to a supported
gluster volume configuration, and have your setup âsupportedâ before
upgrading to your new gluster version.âRDMAâ transport support:Gluster
started supporting RDMA while ib-verbs was still new, and very high-end
infra around that time were using Infiniband. Engineers did work with
Mellanox, and got the technology into GlusterFS for better data migration,
data copy. While current day kernels support very good speed with IPoIB
module itself, and there are no more bandwidth for experts in these area to
maintain the feature, we recommend migrating over to TCP (IP based) network
for your volume.If you are successfully using RDMA transport, do get in
touch with us to prioritize the migration plan for your volume. Plan is to
work on this after the release, so by version 6.0, we will have a cleaner
transport code, which just needs to support one type.âTieringâ
featureGlusterâs tiering feature which was planned to be providing an
option to keep your âhotâ data in different location than your cold data,
so one can get better performance. While we saw some users for the feature,
it needs much more attention to be completely bug free. At the time, we are
not having any active maintainers for the feature, and hence suggesting to
take it out of the âsupportedâ tag.If you are willing to take it up, and
maintain it, do let us know, and we are happy to assist you.If you are
already using tiering feature, before upgrading, make sure to do gluster
volume tier detach all the bricks before upgrading to next release. Also,
we recommend you to use features like dmcache on your LVM setup to get best
performance from bricks.âQuotaâThis is a call out for âQuotaâ feature, to
let you all know that it will be âno new developmentâ state. While this
feature is âactivelyâ in use by many people, the challenges we have in
accounting mechanisms involved, has made it hard to achieve good
performance with the feature. Also, the amount of extended attribute
get/set operations while using the feature is not very ideal. Hence we
recommend our users to move towards setting quota on backend bricks
directly (ie, XFS project quota), or to use different volumes for different
directories etc.As the feature wouldnât be deprecated immediately, the
feature doesnât need a migration plan when you upgrade to newer version,
but if you are a new user, we wouldnât recommend setting quota feature. By
the release dates, we will be publishing our best alternatives guide for
glusterâs current quota feature.Note that if you want to contribute to the
feature, we have project quota based issue open
<https://github.com/gluster/glusterfs/issues/184>[2] Happy to get
contributions, and help in getting a newer approach to
Quota.------------------------------These are our set of initial features
which we propose to take out of âfullyâ supported features. While we are in
the process of making the user/developer experience of the project much
better with providing well maintained codebase, we may come up with few
more set of features which we may possibly consider to move out of support,
and hence keep watching this space.[1] - http://review.gluster.org/4809
<http://review.gluster.org/4809>[2] -
https://github.com/gluster/glusterfs/issues/184
<https://github.com/gluster/glusterfs/issues/184>Regards,Vijay, Shyam, Amar*
--
Amar Tumballi (amarts)
Amar Tumballi (amarts)