Forum Discussion

Cole_McDonald's avatar
Cole_McDonald
Icon for Professor rankProfessor
30 days ago
Solved

Deep Dive troubleshooting question

I'm having difficulties tracking down the source of a Service Account Lockout occuring in a fairly complex domain structure. I've found it necessary to set WMI creds at the top of the heirarchy, wi...
  • Dave_Lee's avatar
    Dave_Lee
    27 days ago

    We struggled with something similar recently... In our case, we had a single WMI account across all hosts in a group and we knew it was correct, but we would see accounts logouts happening almost constantly.

    We thought the same as you - that there is a credential cached somewhere, or a credential set wrong on a host or a credential hard coded somewhere.  We didn't come across a way of troubleshooting it, one of my colleagues did find the root cause though.

    In our case, the issue was due to currency chars in the password - specifically the $ sign.   This was causing some of our PowerShell based property/data sources to do handle parts of the password as if we wanted it to do string interpolation.  Of course, the variable it was replacing portions of our password with didn't exist, so it had the effect of removing a portion of the password at runtime.  We were using double quotes rather than single quotes in a PowerShell script.

    Dave

Dave_Lee's avatar
Dave_Lee
Icon for Advisor rankAdvisor
27 days ago

We struggled with something similar recently... In our case, we had a single WMI account across all hosts in a group and we knew it was correct, but we would see accounts logouts happening almost constantly.

We thought the same as you - that there is a credential cached somewhere, or a credential set wrong on a host or a credential hard coded somewhere.  We didn't come across a way of troubleshooting it, one of my colleagues did find the root cause though.

In our case, the issue was due to currency chars in the password - specifically the $ sign.   This was causing some of our PowerShell based property/data sources to do handle parts of the password as if we wanted it to do string interpolation.  Of course, the variable it was replacing portions of our password with didn't exist, so it had the effect of removing a portion of the password at runtime.  We were using double quotes rather than single quotes in a PowerShell script.

Dave

  • Cole_McDonald's avatar
    Cole_McDonald
    Icon for Professor rankProfessor
    24 days ago

    There was a $... I changed all my built-in var grabs to String Literal for the 'Sources I've written.  Hopefully, this is the beast.  Thank you!

  • Cole_McDonald's avatar
    Cole_McDonald
    Icon for Professor rankProfessor
    24 days ago

    BTW... what a fun, obscure problem to have.

    • Dave_Lee's avatar
      Dave_Lee
      Icon for Advisor rankAdvisor
      24 days ago

      fun?  At least three of us here had a go at trying to figure it out and failed to spot it.  I dread to think about the number of hours we wasted trying to track down a potential stale password.  At one point I thought it might be specific to a PowerShell module - so I was close!  I didn't spot that we had a module that wasn't using a literal for the password, nor did I spot the password had a $.

      Anyway, you live and learn - and can share the experience so perhaps others don't have to suffer quite as much! 🤣

      • Cole_McDonald's avatar
        Cole_McDonald
        Icon for Professor rankProfessor
        20 days ago

        I enjoy weird, obscure things like this.  Double interpreter interplay obscures the real cause of the issue.  Had one a while back with passwords being stored and URL Encoded on the way in, but not decoded on the way out (Not LM... different package).  Also found an early azure powershell foible where a get cmd was de-serializing objects before delivery, so the set command paired with it couldn't do anything as it required a serialized object.