Forum Discussion

Richard_Walter1's avatar
8 years ago

Redis- DataSource - Occasionally 'No Data'

Occasionally the Redis- DataSource returned no data for some or all of the DataPoints.  Under test, it was noted that the key value store returned to cli.stdout was being truncated.  It was further noted that the cli.expect string of 'r\n\r\n' occured at multiple points in stdout, not just at the end of the response.  Under certain stress conditions (when the device response was slow) stdout was truncated by the groovy script. 

It was noted that (in a successful return) the last key value returned was under the heading of "# Keyspace", followed by the key values, followed by a final 'r\n\r\n'.

To resolve this issue, the groovy collection script was modified with an additional cli.expect("# Keyspace"), prior to cli.expect('\r\n\r\n').  On the monitored devices this modification fully resolved the issue.  The modified script is shown below:

import com.santaba.agent.groovyapi.expect.Expect;
cli = Expect.open("##HOSTNAME##", ##WILDVALUE##, 5)
if ( "##REDIS.PASS##" != "") {
    cli.send("AUTH " + "##REDIS.PASS##" + "\r\n")
}
cli.send( "INFO\r\n" )
cli.expect("# Keyspace")
cli.expect('\r\n\r\n')
println cli.stdout()
cli.send( "QUIT\r\n" )

It is not known if this is caused by a recent change to the way that the INFO output is returned by Redis.  All available devices exhibited the same behaviour and were at Redis version 3.2.8 on Linux.

  • Nice fix Richard.  This one should be updated in the LogicMonitor repository, really.

  • Kudos!  I actually noticed the same thing a few months ago - and like the lazy community member I am - I fixed it without actually submitting it in to the forums or sending it to LogicMonitor to have the repository updated.  If I remember right, there was also something wrong with one of the datapoints as well - although that could have been the RabbitMQ datasource.  I'm going to try to be better about getting these uploaded.

  • Thanks Brandon - You are correct - one of the datapoints 'bgrewriteaof_in_progress' reports No Data.  The datapoint uses a regex of 'bgrewriteaof_in_progress:(\d+)', and certainly in the environment I was working on, this regex does not match any value returned by the INFO command (or even an INFO ALL command) on Redis version 3.2.8 .  From the Redis documentation, the nearest similar return value I can locate is 'aof_rewrite_in_progress', but I do not have a high confidence level that this is the value that the datapoint should be reporting.

  •  

    On 5/19/2017 at 8:00 AM, Richard Walters - Panoramic Data said:

    Occasionally the Redis- DataSource returned no data for some or all of the DataPoints.  Under test, it was noted that the key value store returned to cli.stdout was being truncated.  It was further noted that the cli.expect string of 'r\n\r\n' occured at multiple points in stdout, not just at the end of the response.  Under certain stress conditions (when the device response was slow) stdout was truncated by the groovy script. 

    It was noted that (in a successful return) the last key value returned was under the heading of "# Keyspace", followed by the key values, followed by a final 'r\n\r\n'.

    To resolve this issue, the groovy collection script was modified with an additional cli.expect("# Keyspace"), prior to cli.expect('\r\n\r\n').  On the monitored devices this modification fully resolved the issue.  The modified script is shown below:

    import com.santaba.agent.groovyapi.expect.Expect;
    cli = Expect.open("##HOSTNAME##", ##WILDVALUE##, 5)
    if ( "##REDIS.PASS##" != "") {
        cli.send("AUTH " + "##REDIS.PASS##" + "\r\n")
    }
    cli.send( "INFO\r\n" )
    cli.expect("# Keyspace")
    cli.expect('\r\n\r\n')
    println cli.stdout()
    cli.send( "QUIT\r\n" )

    It is not known if this is caused by a recent change to the way that the INFO output is returned by Redis.  All available devices exhibited the same behaviour and were at Redis version 3.2.8 on Linux.

     

    We've actually got a fix in the pipeline for this datasources, with additional improvements to datapoints and graphs.

    Thank you for your great contributions.