Timothy

House Cleaning!

The forums have been long neglected and were overwhelmed with spam.  I cleaned it all up tonight.  I also did some software updates for the website.  There's still some work left to be done.  But, hopefully at the very least, the forums will be usable again if you need support on RD Tabs.

Timothy

Enumerating Nested Group Members in System Center Operations Manager Groups with PowerShell

I finally found a way to reliably return all members of a SCOM group, including members of subgroups (nested groups).  I don't know why Microsoft made this so difficult.  Anyway, other online solutions suggest using the GetRelatedMonitoringObjects() method on the group, but it was unreliable for and didn't work for all object types.  The method below seems to work for everything.  The main difference is that this PowerShell function recursively enumerates groups, but the trick is how do you reliably tell if a class is a group?  Well, thankfully, you can just pipe the class instance into Get-SCOMGroup and if it returns $null, it's not a group!  This function lets you enumerate groups by DisplayName or Class Instance object (from Get-SCOMGroup).

 Function Get-SCOMGroupMembers($group) {
    if ($group.GetType() -eq "".GetType()) {
        $group = Get-SCOMGroup -DisplayName $group
    }

    $group | Get-SCOMClassInstance | % {
        if (($_ | Get-SCOMGroup) -ne $null) {
            Get-SCOMGroupMembers $_
        } else {
            $_
        }
    }
}

Timothy

DPM 2016 + SQL 2016 and "An unexpected error occurred during the installation" ID: 4387

Microsoft System Center Data Protection Manager (DPM) officially supports SQL 2016 with UR2.  But, like previous System Center products, the only installer is pre-UR2, so SQL 2016 is not "officially" supported for an initial install, only as an upgrade install.  This is particularly annoying because it means you have to do a SQL 2014 installation, DPM installation, DPM upgrade to UR2, then a SQL upgrade to 2016.

The good news is that despite it not being officially supported, you can install DPM 2016 RTM with SQL 2016 already installed, then apply UR2. 

However, there is a big gotcha.

One I discovered is that if you have SQL 2016 SP1 installed, the DPM installer crashes with the error "An unexpected error occurred during the installation.  For more details, check the DPM Setup error logs. ID: 4387"

The logs aren't helpful at all.  There are numerous errors in the logs, that are actually safe to ignore, which causes a lot of false positives in your investigation.  At the very end there is a line that says "*** Error : CurrentDomain_UnhandledException" followed by an exception that appears after the installer is closed.

[2/8/2017 6:42:16 PM] * Exception : Invoking Watson with Exception:  => System.ArgumentNullException: Value cannot be null.
Parameter name: input
   at System.Version.Parse(String input)
   at System.Version..ctor(String version)
   at Microsoft.Internal.EnterpriseStorage.Dls.Setup.Inspect.InspectPrerequisites.CheckSqlServerTools(InspectContext context)
   at Microsoft.Internal.EnterpriseStorage.Dls.Setup.Inspect.Inspect.InitializeContext(String sqlMachineName, String sqlInstanceName, String reportingMachineName, String reportingInstanceName, ConnectionOptions wmiSqlConnectionOptions, ConnectionOptions wmiReportingConnectionOptions, Boolean isRemoteDb, Boolean isSqlClustered, List`1 sqlClusterNodes, Boolean isRemoteReporting, String oldSqlMachineName, String oldSqlInstanceName, ProductNameEnum productName, InspectModeEnum inspectMode, Boolean remoteTriggerJob)
   at Microsoft.Internal.EnterpriseStorage.Dls.Setup.Inspect.Inspect..ctor(String reportFilePath, String sqlMachineName, String sqlInstanceName, String reportingMachineName, String reportingInstanceName, ConnectionOptions wmiSqlConnectionOptions, ConnectionOptions wmiReportingConnectionOptions, Boolean isRemoteDb, Boolean isSqlClustered, List`1 sqlClusterNodes, Boolean isRemoteReporting, String oldSqlMachineName, String oldSqlInstanceName, InspectModeEnum inspectMode, InspectSkuEnum inspectSku, ProductNameEnum productName, InspectCCModeEnum ccMode, Boolean remoteTriggerJob)
   at Microsoft.Internal.EnterpriseStorage.Dls.Setup.Wizard.BackEnd.InstantiateInspect(String inspectFile, String sqlMachineName, String sqlInstanceName, String reportingMachineName, String reportingInstanceName, ConnectionOptions wmiSqlConnectionOptions, ConnectionOptions wmiReportingConnectionOptions)
   at Microsoft.Internal.EnterpriseStorage.Dls.Setup.Wizard.InspectPage.RunInspect()
   at Microsoft.Internal.EnterpriseStorage.Dls.Setup.Wizard.InspectPage.InspectThreadEntry()
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)

   at System.Threading.ThreadHelper.ThreadStart()

So I tried a lot of different things like different versions of SSMS (management tools), manually installing the Hyper-V PowerShell components, making sure all the WMI classes are registered (mofcomp all the .mof files manually), and the tips are this link.

What's happening as far as I can tell is that the installer's prerequisite checks fail due to an unexpected response from SP1.  The only way to get around this is to install SQL Server 2016 RTM, then install DPM 2016, then install DPM's UR2, then install SQL's SP1.  Done in that order, it works in my tests.

It's not as bad as having to install SQL 2014 first, then upgrade to 2016, but still a pain.  I wish Microsoft would released System Center products with updated installers for major milestones, like when a new version of SQL is supported.

Timothy

Seed Based Discovery for Exchange 2013-2016 OpsMgr Management Pack

One of the bad things about the Exchange MP it is uses a broadly scoped script based discovery (and it's a complex script, at that) that targets Windows Server.  So it runs on every server every four hours.  Kevin Holman made a little MP that does a simple registry discovery into a seed class that, via override, limits the servers the script runs on.  It's great, but I wanted one that was sealed, so it doesn't show up for overrides and nobody mucks it up.  I tried running fastseal on his XML, but it didn't work because of the way he coded the overrides.  So I just wrote my own that's sealed.  Enjoy!

Download link: http://www.avianwaves.com/Tech/Tools/ManagementPacks/Exchange2013SeedDiscovery.zip

Timothy

SharePoint 2013 Slowing to a Crawl Due to Quick Launch

This morning we ran into a really weird issue with SharePoint 2013.  Out of nowhere, our intranet SharePoint portal would not load.  It just timed out.  After two minutes, you would see the "Sorry, something went wrong" text and a correlation ID.  When I looked up that ID in the logs folder, I saw that /default.aspx was timing out. 

My first thought is that it was a misbehaving web part, so I tried to load the web part editing page with the ?contents=1 in the URL trick.  That also timed out.  I did some research and found you can extend the timeout by modifying executionTimeout on the httpRuntime tag in the web.config.  I set it to 24 hours.  Nothing was going to time out now!

Well, after reloading the page and waiting about five minutes, the page did finally finish loading.  But the quick launch just said "error."  I couldn't find any information on the internet or in the logs what was going on, but at least now I had it narrowed down to (probably) the Quick Launch.  We also were able to access all the sub-sites that had different Quick Launches.  The evidence was growing.

I tried to go into the site settings navigation section.  It took nearly an hour and I just gave up.  I wondered if I could peek into the Quick Launch with a SQL query.  It was at this time that my colleague informed me that the other day he noticed entries duplicating in the Quick Launch any time one user published a particular document.  He didn't know why and since it was just a visual issue and not pressing, he left it alone.  Could this be a clue?

Well, I poked around in the SharePoint contentdb for the site.  Surprisingly, the Quick Launch is actually somewhat simple to query.  You can SELECT * from the NavNodes table and there you go.  When I did this, there were over 3500 rows.  That's a lot for a navigation bar, but if you think about, it's not an obscenely high number.  But who knows what sort of joins this table has with other tables as a matter of course in SharePoint operation.  I scrolled through the results and one entry kept repeating.  Yup, the same one my colleague had seen the other day.  Only instead of repeating twice or four times, it was repeating thousands of times.

This sure seems suspect.  An entry repeated over and over in the Quick Launch and the Quick Launch generating an error after timing out the page?  That could be the culprit.  But how do we get rid of these bad entries?  Deleting directly from the database is not a Microsoft supported scenario (at least not without their technicians performing the deletion).  Time was of the essence as the site was down.  So I took a backup of the contentdb and threw caution to the wind and deleted all the rows that matched the repeated entry's criteria.

It was time to load the website.  About five to ten tense seconds later, the site loaded.  Even better, every reload after that took no time at all!  We fixed it!

One strange issue after the page loaded was that some of the repeated entries were still in the Quick Launch, but only about 30 this time.  They were easy to remove from the Quick Launch using the UI.  I suspect they were left over from either a cache or some other table is involved in constructing the Quick Launch (maybe the most recently visited pages list?).

Anyway, now everything seems to be back to normal.  Carry on, SharePoint, you crazy beast.

Timothy

Enrolling an Out-of-Date Lync Phone Edition Phone With SHA-2 Signed SSL Certificates

Recently there has been an industry-wide push to phase out SHA-1 signed SSL certificates in favor of SHA-2.  This is a good thing from a security perspective, but presents an interesting problem for Microsoft Lync deployments that use Lync Phone Edition (Aries) phones as common area phones.  In order for Lync Phone Edition phones (henceforth, "Lync phones") to connect to Lync servers after transitioning to SHA-2, the firmware must be up-to-date.  Older firmwares did not support SHA-2 and the phone would simply not be able to log into Lync if you tried.  I had trouble finding the exact firmware version where SHA-2 support was added, but I do know that 4.0.7577.4444 and newer work.

Well, this is fine for all your phones already connected to your Lync deployment.  You simply use one of the numerous guides on the internet to push out firmware updates before you switch to a SHA-2 certificate.  But what about if you buy a phone with an older firmware after transitioning to SHA-2?  The phone has to connect to Lync to get a firmware update and it can't connect to Lync until it has the firmware update.  This is a classic chicken/egg problem.  Unfortunately, Microsoft, in their infinite wisdom, provides no means to sideload new firmware to a Lync phone.

Fortunately, there is an easy solution if your phone has a USB connector.  Connect the phone via USB to any computer with the Lync desktop application installed.  Log into your Lync account when prompted.  The Lync desktop application does all the heavy lifting here, so the phone gets all the configuration information it needs without having to connect to the pesky SHA-2 SSL web services on its own.  Now, you just have to wait for the phone (and computer too, I suppose) to be idle for a while.  I suggest doing this towards the end of the day or before a long lunch break.  It will eventually install the firmware update.  You can check that it's installed by looking at the System Information section in the phone's menu.  Now sign out of the phone.  Your phone is ready to be deployed!

I have tested this method successfully with PolyCom CX3000 phones, but the method should work on any Aries-series phone with USB connectivity.  There may be some really old firmwares (such as the phones from the MD5-era certificate devices) where this won't work.  I don't know as I haven't tested.  And, sadly, if you are trying to enroll a PolyCom CX500 or other device without a USB connection, you are still out of luck.  :-(

Timothy

RD Tabs: Increased Memory Consumption with 2012+ Servers and Error 3334

I've noticed increased reports of error 3334, which RD Tabs shows as an unknown disconnection code due to protocol failure.  I did some research and testing and found that it's actually due to memory pressure.  It's most noticeable on 32-bit version of RD Tabs, because you only have about 1.2GB of memory to work with before you run into .Net Framework overhead limits (actual process limit is 2GB).  Future RD Tabs versions will have a better error code description explaining what is happening.  This is not limited to RD Tabs, but all remote desktop clients.  You can read more about this error code here.

So what's going on here?

I ran some tests and when you connect to a 2012 or newer server, the per-tab memory consumption is about 150MB.  For a 2008 server, it's about 20MB.  That's a significant increase!  Even more interesting, if you let a tab stay idle for a while, the memory consumption eventually starts to decrease.  So, it seems that although when you first connect memory usage may be very high, it also may settle down with time.  If you work actively in the tab, the consumption may not go down, so it's dependent upon usage patterns.

This works out so that if you open about seven 2012+ tabs within a fairly short period using the 32-bit RD Tabs, you hit that limit and get error 3334.  I don't know about you, but for me that's not a lot of tabs!  Based on this alone, if you have a 64-bit computer, use the 64-bit version of RD Tabs.  The 32-bit version is just crippling yourself.

Messing around with the various experience settings I found that it's due to bitmap caching (see this link).  It's pretty easy to see that the graphical system of the Remote Desktop Protocol has changed a lot since 2012.  The most noticeable difference being the compression artifacts that are visible when there is a lot of on-screen animation.  My guess is that the bitmap cache feature has changed, using a lot more RAM presumably to work better over slow network connections.  The side effect is that if you open a lot of simultaneous connections, you quickly run out of RAM on your system.

It is therefore my recommendation to disable bitmap caching for all connections over a speedy LAN, and use it selectively over slow WAN links depending on your situation.  You can use Batch Favorite Editing to quickly do this in RD Tabs for all your saved connections.  :-)  Future versions of RD Tabs will have bitmap caching disabled by default.  If you are using the 32-bit version of RD Tabs, you should disable bitmap caching for all connections, regardless, unless there is a very good reason to turn it on.
Timothy

And I'm Back

Sorry the site has been offline for most of the last week.  I'm honestly surprised at how many contacted me to ask if Avian Waves was no more.  I know it's been a while since the last RD Tabs release, but it's still on my radar.  :-)  And now the website is back.  So what happened?  Well, I was late to pay my hosting provider (whoops).  Email from them went deep into a sorted folder in my email and, well, let's just say I didn't notice the bill emails until somebody emailed me that my site was down!  I was looking at moving to a new host anyway because my old server was getting long in the tooth.  I could have resolved this quicker, but the bill was for a whole year, so I ended up going to a new host and I'm getting twice the RAM for a little bit less than I was paying before.  Awesome!

For some reason, some of the CSS and images in the website theme aren't working unless you log in first.  I don't know why yet, but this is why you test your backups before you experience an outage.  I should practice what I preach on my personal projects that I do in my day job.

Timothy

Find All Users in Active Directory in a Specific OU Created on or after a Specific Date with PowerShell

Long title, short post!  Just replace the highlighted portions with your own values.

(get-aduser -SearchBase "ou=Domain Users,dc=mydomain,dc=com" -filter * -properties whencreated) | where {$_.whencreated -ge [datetime]"1/1/2014"} | sort-object whencreated | ft sAMAccountName,Name,WhenCreated -autosize

Timothy

It's Not That Complicated: PowerShell and Task Scheduler

There is no end to complicated methods on the internet regarding the proper way to schedule a PowerShell task.  You see all manner of command line switches and escape sequences.  It's not that complicated!  Your task is powershell.exe and your argument is the path to the script.  If you need to quote the path because it contains spaces, use a combination of single and double quotes to parameterize the argument and then quote the path within the parameter.  When you do this, you also need to remember to precede the inner quote with an ampersand so PowerShell treats it as a script to execute and not just an output string.  It sounds more complicated than it actually is.

Examples:

powershell.exe c:\tasks\mycoolscript.ps1

powershell.exe "& 'c:\my path\with spaces\mycoolscript.ps1'"

Search Posts

Recent Comments

  1. Re: DPM 2016 + SQL 2016 and "An unexpected error occurred during the installation" ID: 4387
    Terry: Edward, you are the Man!!!! Looked for a solution for hours, then found your post and BAM!!! it worked...
  2. Re: DPM 2016 + SQL 2016 and "An unexpected error occurred during the installation" ID: 4387
    Mr. JoeM: Edward! Thank you, saved me hours of work.
  3. Re: Enumerating Nested Group Members in System Center Operations Manager Groups with PowerShell
    Virginia Luther: This is enormously informatics, original and obvious. I experience that all has been depict in systematic...
  4. Re: House Cleaning!
    Virginia Luther: Thanks
  5. Re: DPM 2016 + SQL 2016 and "An unexpected error occurred during the installation" ID: 4387
    Brian: Thank you so much Edward! :-)
  6. Re: DPM 2016 + SQL 2016 and "An unexpected error occurred during the installation" ID: 4387
    Tom: Thank you Edward! After beating my head against a wall for days, tried your suggestion out and lo and...
  7. Re: DPM 2016 + SQL 2016 and "An unexpected error occurred during the installation" ID: 4387
    Mike: DPM 2016 setup will fail if you have SQL Server Management Studio (SSMS) V17.x installed. Re-Install...
  8. Re: DPM 2016 + SQL 2016 and "An unexpected error occurred during the installation" ID: 4387
    Rob: Edward, thanks man! you were a lifesaver. My scenario was Win Server 2016 from scratch, SQL 2016 (N...
  9. Re: DPM 2016 + SQL 2016 and "An unexpected error occurred during the installation" ID: 4387
    Edward: It also crashes with the 4387 error if you have the SQL Management Studio 17 tools installed. Installing...
  10. Re: DPM 2016 + SQL 2016 and "An unexpected error occurred during the installation" ID: 4387
    Ram: Hi - I followed richsmif instruction and was able to successfully install DPM 2016 on SQL 2016. Completed...