Forum Discussion

AustinC's avatar
AustinC
Icon for Neophyte rankNeophyte
4 years ago

Groovy script Multi-Threading?

Has anyone tried to use any type of code-level multi-threading within a DataSource, by chance? Is this even possible?

My use case is for querying 16 different hosts for SNMP information and then aggregating the data based upon certain properties - Think "Combine traffic amongst all TenGigE0/0/0/10 interfaces that have the description 'P2P connection', then all TenGigE0/0/0/10 interfaces with description 'P2P interfaces for Zayo'" -- This is not my _actual_ use case, but an approximation.

At present, collection times are hovering around 110 seconds to run through all the operations we need, which is getting a little close to the 2 minute mark for my tastes -- I'd prefer not to have to develop a whole suite of tooling to take advantage of Push metrics if possible, so I'm trying to see what I can do to optimize my groovy script, and it seems like threading is the next logical answer for this; that being said, I've not done anything multithreading before, so there's that :)/emoticons/smile@2x.png 2x" title=":)" width="20" /> 

I've tried using GPars, but it doesn't seem like it's installed by default (so I installed it as a custom library), and I'm having a little trouble getting it to work (which might be because this might not be a thing that I can do, for myriad reasons?).

  • Mike_Rodrigues's avatar
    Mike_Rodrigues
    4 years ago

    @Austin Culbertson one of the module engineers mentioned that VMware_vCenter_VMPerformance is multi-threaded, and maybe a bit cleaner than than SO link.

    ctrl-f for "Executors" in the collection script.

  • AustinC's avatar
    AustinC
    Icon for Neophyte rankNeophyte
    4 years ago
    14 hours ago, Michael Rodrigues said:

    @Austin Culbertson one of the module engineers mentioned that VMware_vCenter_VMPerformance is multi-threaded, and maybe a bit cleaner than than SO link.

    ctrl-f for "Executors" in the collection script.

    Greatly appreciated! I'll take a look at this today and pick it apart.

  • Mike_Rodrigues's avatar
    Mike_Rodrigues
    Icon for Product Manager rankProduct Manager
    4 years ago

    @Austin Culbertson one of the module engineers mentioned that VMware_vCenter_VMPerformance is multi-threaded, and maybe a bit cleaner than than SO link.

    ctrl-f for "Executors" in the collection script.

  • AustinC's avatar
    AustinC
    Icon for Neophyte rankNeophyte
    4 years ago
    2 minutes ago, Michael Rodrigues said:

    @Austin Culbertson it's possible, but I don't have any examples handy. You need to make sure you clean up any threads you spawn.

    Might be able to get started with this:
    https://stackoverflow.com/questions/26004337/java-multithreaded-port-scanner

    If you have Service Insights, you could aggregate that data there across your 16 hosts.

    Unfortunately, Service Insights is not an option for us, as we would need to configure on the order of 1000+ Services, each with maybe only 1-4 instances for data being aggregated. Not really an option for us, unfortunately (I've spent a lot of time banging my head against this particular wall :)/emoticons/smile@2x.png 2x" title=":)" width="20" /> )

    Do you know what library plays nicely in this case? Is GPars the right one? I'm just trying to find something that will work -- would be best if it's something that works natively with existing libraries (though it shouldn't really be a problem if not).

     

    [edit] Herp, I forgot to read the Stack Overflow -- sounds like maybe going straight Java might be best. I'll see if I can work with this -- Thank you!