Linux FileSystems and SNMP

From Zenoss Wiki
This is the approved revision of this page, as well as being the most recent.
Jump to: navigation, search

Explanation

When a linux ext2/3/4 filesystem is created a portion of the total blocks will be reserved for the root user. This makes it impossible for user processes to fill up the whole filesystem. This ensures that system logging and other system functions are able to continue functioning even if the disk is "full". If you didn't tune the ext filesystem creation properties it will reserve the default amount for this purpose, which is 5% of the total blocks. A tool like df will report the filesystem size minus the 5%. This is correct since if you reach 100% level you are not allowed to write anymore. The kicker is that when you use SNMP to query the filesystem usage it shows the usage including the 5%. This means that when a filesystem is reported as 100% full by df SNMP will only report it as 95%. This is technically correct, but in this case 95% means that df is likely showing 100% which means that the disk space is completely depleted on the server. To work around this in Zenoss you need to do the following...

Edit the FileSystem Template

Navigate to Advanced -> Monitoring Templates and expand the FileSystem template listing on the left hand side. Select the FileSystem template under /Server. Double click on the high disk usage threshold to edit it. The threshold that's set will be...

here.getTotalBlocks() * .9

You need to modify it so that it looks like this...

(here.getTotalBlocks() * here.zFileSystemSizeOffset) * .9

After you've done that you need to click on the Utilization graph to select it and then select the Manage Graph Points option from the menu. Double click on the usedBlocks graphpoint to edit it. It will have the following RPN expression set on it...

${here/getTotalBlocks},/,100,*

You need to modify it so that it looks like this...

${here/totalBlocks},${here/zFileSystemSizeOffset},*,/,100,*

Adjust the zProperty

Edit the zFileSystemSizeOffset property on the /Server/Linux class or on the device itself. You need to set it corresponding to the amount of reserved space. As explained earlier in most cases the reserved space will be 5% of the total. To account for this you need to change zFileSystemSizeOffset from 1 to .95 which causes the totalBlocks to be multiplied by that value, which results in the calculation being done against 95% of the total blocks instead of the actual total. This will compensate for the reserved space and provide a more accurate value.