Just came into managing a rancid install for the first time in a couple of years. It immediately became apparent that the email notification feature was worthless and likely being ignored by everyone. There was one specific Nexus7000 C7010 (NX-OS 5) that was almost constantly reporting false positives in its “show environment power” output, resulting in a “boy who cried” wolf situation.
Here are the “changes” being reported:
-!Env: fan1 N7K-C7010-FAN-S 720 W Powered-Up -!Env: fan2 N7K-C7010-FAN-S 720 W Powered-Up -!Env: fan3 N7K-C7010-FAN-F 120 W Powered-Up -!Env: fan4 N7K-C7010-FAN-F 120 W Powered-Up +!Env: fan1 N7K-C7010-FAN-S 720 W Powered-Up +!Env: fan2 N7K-C7010-FAN-S 720 W Powered-Up +!Env: fan3 N7K-C7010-FAN-F 120 W Powered-Up +!Env: fan4 N7K-C7010-FAN-F 120 W Powered-Up
After upgrading to the latest 2.3.8, we were still getting these problems. It turns out that rancid is stripping an extra column of data when formatting this output, and it is not doing a good job of handling variations in that width. Specifically, here is an example of raw output from “show env power” on the device in question:
fan1 N7K-C7010-FAN-S 116 W 720 W Powered-Up fan2 N7K-C7010-FAN-S 116 W 720 W Powered-Up fan3 N7K-C7010-FAN-F 11 W 120 W Powered-Up fan4 N7K-C7010-FAN-F 11 W 120 W Powered-Up
And here’re the same lines from the same command run a few minutes later:
fan1 N7K-C7010-FAN-S 88 W 720 W Powered-Up fan2 N7K-C7010-FAN-S 88 W 720 W Powered-Up fan3 N7K-C7010-FAN-F 9 W 120 W Powered-Up fan4 N7K-C7010-FAN-F 9 W 120 W Powered-Up
Here’s the patch I applied to cause nxrancid to remove the same amount of space (13 characters) as the lines that remove other values from that column (headers, etc):
*** rancid-2.3.8.dist/bin/nxrancid.in Fri Jul 29 15:40:40 2011
--- rancid-2.3.8.local/bin/nxrancid.in Thu Jun 14 12:09:22 2012
***************
*** 380,386 ****
s/ Draw / /;
s/ ----------- / /;
s/ N\/A / / ||
! s/ \d+ W / /; # Does not chop enough to line up.
/actual draw/ && next; # Drop changing total power output.
--- 380,386 ----
s/ Draw / /;
s/ ----------- / /;
s/ N\/A / / ||
! s/ [ \d]{9} W / /; # Does not chop enough to line up.
/actual draw/ && next; # Drop changing total power output.
This same problem was reported on 6/6/2012, with a suggested patch, on the rancid-discuss mailing list by Zenon Mousmoulas. Unfortunately I wasn’t able to locate his post until I had already diagnosed the problem (in retrospect I was using the wrong keywords, but I didn’t know what the right ones were until I knew more details of the problem). I’m not 100% comfortable with Zenon’s assumption that 30 characters in will always be the correct place to start removing the unwanted column, causing me to create my alternate patch above. I also sent the cause of my concern and my alternate patch to the mailing list on 6/14/2012.
Update 2012-06-15:
John Heasley proposed a different version of the patch which had a slightly more robust regular expression, but still had the (non-critical) side-effect of displaying the lines with “actual wattage” out-of-line with the columns from the surrounding lines (this issue is in the stock 2.3.8 code also). I proposed a slight variation in his patch, which keeps the more-robust regular expression but also aligns the columns correctly. I’m not sure if the new patch will be accepted, but I like it, so here it is:
--- rancid-2.3.8.dist/bin/nxrancid.in 2011-07-29 15:40:40.000000000 -0500
+++ rancid-2.3.8.local2/bin/nxrancid.in 2012-06-15 09:01:22.000000000 -0500
@@ -379,8 +379,10 @@
s/ \(Watts \) / /;
s/ Draw / /;
s/ ----------- / /;
- s/ N\/A / / ||
- s/ \d+ W / /; # Does not chop enough to line up.
+ s/ N\/A / /;
+ if (/(.*?)(\s+\d+ W)( +\d+ W.*)/) {
+ $_ = sprintf("%s%-".(length($2)-11)."s%s\n", $1, "", $3);
+ }
/actual draw/ && next; # Drop changing total power output.