[Gluster-users] Usage monitoring per user

[mohammad kashif]

Hi

Is there any easy way to find usage per user in Gluster? We have 300TB
storage with almost 100 million files. Running du take too much time. Are
people aware of any other tool which can be used to break up storage per
user?

Thanks

Kashif

[Alex Chekholko]

Hi,

There are several programs that will basically take the outputs of your
scans and store the results in a database. If you size the database
appropriately, then querying that database will be much quicker than
querying the filesystem. But of course the results will be a little bit
outdated.

One such project is robinhood. https://github.com/cea-hpc/robinhood/wiki

A simpler way might be to just have daily/weekly cron jobs that output text
reports, without maintaining a separate database.

But there is no way to avoid doing a recursive POSIX tree traversal, since
that is how you get your info out of your filesystem.

Regards,
Alex

Post by mohammad kashif
Hi
Is there any easy way to find usage per user in Gluster? We have 300TB
storage with almost 100 million files. Running du take too much time. Are
people aware of any other tool which can be used to break up storage per
user?
Thanks
Kashif
_______________________________________________
Gluster-users mailing list
http://lists.gluster.org/mailman/listinfo/gluster-users

[mohammad kashif]

Hi Alex, John

Thanks for confirming my suspicion that there is no getting away from POSIX
tree traversal . I was aware of age-du but not robinhood.

Cheers

Kashif

I rather like agedu It probably does what you want.
But as Mohammad says you do have to traverse your filesystem.
https://www.chiark.greenend.org.uk/~sgtatham/agedu/
agedu: track down wasted disk space - chiark home page
<https://www.chiark.greenend.org.uk/~sgtatham/agedu/>
www.chiark.greenend.org.uk
agedu. a Unix utility for tracking down wasted disk space Introduction.
Suppose you're running low on disk space. You need to free some up, by
finding something that's a waste of space and deleting it (or moving it to
an archive medium).
------------------------------
*Sent:* 01 May 2018 18:45
*To:* mohammad kashif
*Cc:* gluster-users
*Subject:* Re: [Gluster-users] Usage monitoring per user
Hi,
There are several programs that will basically take the outputs of your
scans and store the results in a database. If you size the database
appropriately, then querying that database will be much quicker than
querying the filesystem. But of course the results will be a little bit
outdated.
One such project is robinhood. https://github.com/cea-hpc/robinhood/wiki
<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fcea-hpc%2Frobinhood%2Fwiki&data=01%7C01%7Cjohe%40novozymes.com%7Cb280d3cf0fde4c28124c08d5af82fa19%7C43d5f49ee03a4d22a2285684196bb001%7C0&sdata=m5s1l7DvoKiXlAhSP8eGlu9lYjtlkIM5k3CTGt6zoH0%3D&reserved=0>
A simpler way might be to just have daily/weekly cron jobs that output
text reports, without maintaining a separate database.
But there is no way to avoid doing a recursive POSIX tree traversal, since
that is how you get your info out of your filesystem.
Regards,
Alex
Hi
Is there any easy way to find usage per user in Gluster? We have 300TB
storage with almost 100 million files. Running du take too much time. Are
people aware of any other tool which can be used to break up storage per
user?
Thanks
Kashif
_______________________________________________
Gluster-users mailing list
http://lists.gluster.org/mailman/listinfo/gluster-users
<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.gluster.org%2Fmailman%2Flistinfo%2Fgluster-users&data=01%7C01%7Cjohe%40novozymes.com%7Cb280d3cf0fde4c28124c08d5af82fa19%7C43d5f49ee03a4d22a2285684196bb001%7C0&sdata=P%2Bkfpz8XaI7j7PM9%2FocnBkf0Wop%2BLZDAbgnvNpIcdd0%3D&reserved=0>