SQL performance graphs stopped working after collector upgrade to 31.002

13 minutes ago, Jeff8682 said:

I am curious...how are you polling dashboard widgets to find the failed polls?

This is done via check functions within our backup script bound to REST API paths.  We also added a mechanism to suppress warnings for known issues via a text widget in the dashboard (either permanently or with an expiration date).  I don't have expanded checks for every widget type, but the big ones do, like cgraph.  The main check for widgets that mirrors what you see on the screen is below.  The reference to $warning_suffix relates to the suppression logic mainly.  We get the widget_data for additional analysis (e.g., check for gappy graphs -- "at least 20% missing data and spread out over at least 10 samples").  Since it can be a bit intensive, we only run the check logic once per day.  We also check for netscan policy problems (since there are a few cases where integrity constraints are not applied in the UI, like when a collector is removed) and user groups (to validate all users are in one, for example).

        if ($widgettype !~ /^(?:text|html|flash|alert)$/) {
            # get recent data from supported widget types, bail out on exceptions
            if ($widgettype eq "gmap") {
                # must fetch all results for gmap
                eval { $widget_data = $LMAPI->get_all(path => "/dashboard/widgets/$widgetid/data", start => 1000*(time-1800)); };
            }
            else {
                eval { $widget_data = $LMAPI->get_one(path => "/dashboard/widgets/$widgetid/data", start => 1000*(time-1800)); };
            }
            if ($@) {
                my $error = $@;
                $error =~ s/^raw request:.*//sm;
                $error =~ s/.*:\s+\d+\s+//sm;
                $error =~ s/\n+/ /gsm;
                warn "$COMPANY: $dashboardname: $widgetname\[$widgettype\]: widget exception: $error$warning_suffix\n";
                return;
            }
            elsif (not defined $widget_data) {
                warn "$COMPANY: $dashboardname: $widgetname\[$widgettype\]: unable to load widget data$warning_suffix\n";
                return;
            }
        }