It took a while, but Microsoft has finally released a preview instance of SCOM in the cloud: Azure Monitor SCOM Managed Instance.  Like all Microsoft products, it's a mouthful, but it's clear where they are going with this and I'm quite pleased with the approach.  As I mentioned in my previous article on this subject, OpsMgr (SCOM) in the Cloud? Finally, Azure Monitor was a fundamentally new monitoring tool and despite Microsoft themselves completely replacing their internal OpsMgr instances with it, this wasn't something that appealed to a lot of enterprises. 

Azure Monitor is very focused on log aggregation and custom alerting and monitoring.  While SCOM has a steep learning curve, once you learned it, new resources get added to your health model automatically and the amount of customization you can do to large groups of monitored objects is just unprecedented in the monitoring space.  To me, It didn't (and still doesn't) make sense to throw all that out, rewrite everything from scratch, and then roll that into what are basically templates that have to be redeployed through custom automation you have to create yourself, which is what Log Analytics and the monitoring in Azure Monitor based on it provide to admins.  Don't get me wrong, this approach has it's place.  It's very tuned into a specific workflow: CI/CD.  And for that it makes sense.  But the world of enterprise infrastructure is so much more than that.  If you are developing an application, this approach works great.  If you are back office and supporting infrastructure, I just don't think that works well. 

And the market has seemingly spoken as Microsoft was very surprised by how slowly (if at all) OpsMgr users actually migrated to Azure Monitor, continuing even to pay the very high amount of licensing for cloud-based monitoring (you can generally get better VM density on-premises which lowers the effective "per VM" fee for SCOM).  In addition, there are dozens of RMM products that essentially use similar monitoring models as what SCOM has perfected.

So this is great news this is public preview.  Right now it looks to be very IaaS-oriented.  You spin up a managed management server instance, managed SQL instance, and you are good to go.  I hope that as time goes on, the model shifts much more to PaaS, where I do not have to concern myself with knowing what the backend even looks like.  I also hope the licensing follows this model, giving a choice between using legacy licenses or simply doing a per-resource or per-VM licensing fee, removing the current compute infrastructure fee you pay for the managed instance.  If the per-VM model doesn't cover all the cloud hosting costs, they could do what they did with Log Analytics and charge for total storage, retention, and other metrics that aren't based around the underlying compute.  A fully cloud PaaS should not concern its customers with the implementation details, just charge a reasonable monthly fee for platform usage based on easy-to-understand and easy-to-predict fee structure.  But this is Microsoft we are talking about, so I fully expect the licensing to be ridiculous, but hopefully they listen.

I don't want to end this negative!  This is truly a good thing.  The best on-prem monitoring platform is now also the best cloud monitoring platform and the future is bright.  Moving more of the GUI into Azure's management portal (which they are working on), tighter integration with Log Analytics, so you can have both log-based and probe-based monitoring, and new true innovation on the platform is good for everybody in infrastructure.  I'm excited to see where this goes!

Here's a video put out by the great folks at SquaredUp in conjunction with Microsoft about this new preview:

Drop (formerly Mass Drop) makes some pretty nice looking, customizable, reasonably priced (for mechanical) keyboards.  Myself, I bought the CTRL, which is the "tenkeyless" style.  I got Kailh Box White switches.

There's just one really big problem: when you press a key, it sometimes registers two or more presses.  A lot.  Sometimes this is referred to as a "chatter problem."

There's a lot of discussion on this and how to remediate it and a lot of disappointed customers.  Some people look at it as a software problem -- there are suggestions to manipulate the debounce and wait times inside QMK (the embedded OS that these keyboards run).  I had limited success with this.  There are also suggestions to pop the switch out and "wipe off dust" or replace the switches entirely.  This didn't work for me and frankly it doesn't pass the sniff test.  Are really that many people getting dozens to hundreds of faulty switches or dusty connections?

So I tried a different approach.  Still thinking on the second suggestion that the problem might be the electrical connection between the switches and board,  I tried using DeoxIT Gold contact cleaner/lubricant.  This worked.  It completely solved the issue.  No more chatter!

I'm a huge fan of DeoxIT products.  It is expensive, but it absolutely works as advertised.  I've used this for years to help with scratchy faders or bad analog audio connections, it's no wonder it also works on mechanical keyboard switch connections.

So how do you use it?  It's easy, but tedious.  Buy the brush-on kind, pop out every single switch, and apply it to each metal connection pin.  Then stick it back in.  It is 100% fixed.  No double presses.

I used DeoxIT G100L (aka "Gold" or "Pro Gold") but the D100L solution probably works just as well, maybe better since it's designed for non-gold connectors.

After removing all the switches, ready to apply DeoxIT!

DeoxIT Gold:

Before application:

After application:

Microsoft publishes a page which contains all the keyboard shortcuts for Outlook (and other Office products, but I needed Outlook in this instance, so we'll stick with that). It's located here: https://support.microsoft.com/en-us/office/keyboard-shortcuts-for-outlook-3cdeb221-7ae5-4c1d-8c1d-9e63216c1efd.  The problem is that it's just HTML, so while it's good for a quick reference, it's not something you can easily copy and paste into another format, like say Excel, to whip up a nice handy table you can print out or format some other way.

PowerShell to the rescue!

Taking a quick look at the page source shows that all the shortcuts are in HTML tables.  So all we really need to do is some simple HTML parsing and we can extract all the data we need into a hashtable, then convert it to CSV or whatever else we want.

Long story short, here's the basic script.

$outlookShortcutsHelpDocURL = 'https://support.microsoft.com/en-us/office/keyboard-shortcuts-for-outlook-3cdeb221-7ae5-4c1d-8c1d-9e63216c1efd?ui=en-US&rs=en-US&ad=US'
$windowsShortcutsElementRegEx = '<div.*?id="PickTab-supTabControlContent-1".*?>.*?</div>.*?<div.*?id="PickTab-supTabControlContent'
$shortcutTableRegEx = '<table.*?>.*?</table>'
$shortcutItemRegEx = '<tr>.*?<td>(.*?)</td>.*?<td>(.*?)</td>.*?</tr>'
$regexMatchOptions = [System.Text.RegularExpressions.RegexOptions]::Singleline + [System.Text.RegularExpressions.RegexOptions]::IgnoreCase

# ---------------------------------------------------------------------------------

$page = Invoke-WebRequest -Uri $outlookShortcutsHelpDocURL -UseBasicParsing
$windowsShortcuts = [Regex]::Matches($page.Content, $windowsShortcutsElementRegEx, $regexMatchOptions)
$shortcutTables = [Regex]::Matches($windowsShortcuts[0], $shortcutTableRegEx, $regexMatchOptions)

$shortcuts = $shortcutTables | % {
  [Regex]::Matches($_, $shortcutItemRegEx, $regexMatchOptions) | % {
     if ($_.Captures -and $_.Captures.Groups -and $_.Captures.Groups.Count -ge 3) {
        $shortcut = (($_.Captures.Groups[2].Value -ireplace '<.+?>','') -ireplace '\s+',' ').Trim()
        $description = (($_.Captures.Groups[1].Value -ireplace '<.+?>','') -ireplace '\s+',' ').Trim()
        @{$shortcut=$description}
     }
  }
}
$shortcuts

So what's going on above?  Well, above the line (the #--------- part) are some variables I'm defining to hold the URL and Regular Expressions (RegEx) that are extracting the data from the HTML.  I decided to do it this way so it's easier to edit in the future when Microsoft changes the page formatting.  The next big part is the Invoke-WebRequest.  That fetches the live version from Microsoft.  Notice the "UseBasicParsing" flag.  That tells PowerShell NOT to use the Internet Explorer parser.  IE parsing is a bit of a mess nowadays and we don't need a full parser anyway since regular expressions will be doing all the heavy lifting. 

There are three sequences of RegEx happening on the page output.  First, using $shortcutTableRegEx, we're looking for the section that contains Windows shortcuts (Outlook for Windows Desktop).  Second, we're extracting all the tables inside that section with $shortcutTableRegEx.  Then finally we are getting individual shortcut items with $shortcutItemRegEx.  Then we do a replace to remove any extra HTML or whitespace and then Trim() to remove leading or trailing whitespace.  Finally, the item pair is output into a hashtable with the @{} section.  When applied over the "for-each" loop, you get one big hashtable array, like below.

Now, while that's easy to read on the screen, it's not a great format for trying to export elsewhere, such as to CSV.  PowerShell doesn't really handle converting arrays of hashtables very well for exports.  While it unrolls nicely to the screen, it's actually a pretty complex object.  So to export it, we need to help it along a bit by extracting the data from the embedded arrays in the array we created, cast that to a PSCustomObject, then send it to the export cmdlet. 

$shortcuts | % { [PSCustomObject]@{"Shortcut"=[String]$_.Keys;"Description"=[String]$_.Values} } | Export-Csv shortcuts.csv -NoTypeInformation

You'll notice I'm actually still using a hashtable which is what is being cast to PSCustomObject.  This hashtable doesn't format to the screen as nicely (by default), but does work well for exporting.  The reason it doesn't look nice is because of how Format-Table works.  If you sent this to Format-Table, it will try to show the first column's entire width.  The columns are very wide, so it ends up just showing one column.  It does this even with the AutoSize parameter set.  We'd have to customize Format-Table's output to get it to look nice, which is too much work.  ;-) 

So anyway, since we generated a CSV now, this is what it looks like if we open the CSV in Excel.

According to unnamed sources in some reliable publications, Microsoft Systems Center Operations Manager (OpsMgr aka SCOM) is going to the cloud.  It's called Project Aquila (at least internally).  Finally.  I've been wanting this for years.  Of course Microsoft's official stance has been that System Center remains their "on-prem" (aka private cloud, aka public cloud but not managed) solution, but it was clear they were heavily pushing Azure Monitor (previously known as Log Analytics and OMS).  However, I don't think adoption has been great as a forklift replacement.  There are parts of Azure Monitor I like, but it doesn't replace OpsMgr.  They've also reimagined the product multiple times, changed its name multiple times, refactored the UX multiple times, etc.  You just don't do that with a successful new product.  And if you've used both Azure Monitor and OpsMgr, it's pretty clear why.

Both have steep learning curves, so let's get that out of the way.  One may seem easier to you than the other, but it's only because you've dedicated yourself to learning it.  That isn't a problem.  So I won't comment on Azure Monitor's learning curve except that they didn't make anything easier to help ease transition.

The problem, to me, is more the approach to monitoring.  Azure Monitor is DevOps focused, which is a good thing, but in my opinion it neglects some basic features as a result of this focus.  For example, there is very little in the way of deploying a baseline of monitors without custom scripts and lots of upkeep.  There is no concept of automatic discovery which is the best part of OpsMgr.  If you have a ton of DevOps labor resources, this is less of a problem, but it's also something OpsMgr does automatically and natively.  If you want to create a monitor that touches everything of a defined class across all entities, you can do it easily -- that's a core functionality in OpsMgr.  You can then set all the parameters and thresholds across the management group through overrides.  You do it once, and it is continuously pushed out to new resources automatically.   You can't easily do this in Azure Monitor without a lot of upkeep and auditing.  And if your monitoring parameters change, you have to manually (or by script) copy it across the organization.

On the other hand, Azure Monitor makes it insanely easy for anybody on any team or any scope to create custom monitoring to exactly suit their needs.  OpsMgr severely lacks this.  Even with the wizard driven monitors (which give you only a fraction of what is possible to monitor), scoping it to subgroups of classes is not trivial.  And creating new classes inside the management console is impossible.  The only way to get the full functionality out of it is to create a management pack, using Visual Studio Authoring Extensions or a third party like Silect.  It's not a great experience at all, debugging is very difficult, and documentation is lacking.

Visualizations and analytic intelligence is vastly superior in Azure Monitor.  Not a surprise since it's using the vast log collection to get insights.  And any new intelligence MS creates is immediately available and can retroactively provide insight.  It's the most impressive part of Azure Monitor.  You can see how this was/is a desperate need for OpsMgr because SquaredUp built their business on top of building a better OpsMgr UI on the web which MS has neglected.  They've since moved onto adding visualizations to other products, but for years their key focus was only OpsMgr.

The key difference in approach is that Azure Monitor works exclusively on log feeds.  There is no probe-based monitoring.  You can still simulate probes in some ways, such as heartbeating, by alerting on a lack of a periodic log entry being treated like a missed heartbeat, but fundamentally, it's not the same.  If you actually need a probe, you have to make it some other way and feed the output of that into the log stream.  In contrast, OpsMgr natively supports probe based monitoring, both from the agent or agentless.  Of course, on the flip side, while OpsMgr can react to logs, it doesn't aggregate logs like how Azure Monitor does with Log Analytics (we'll just ignore ACS since it's not that great and was always separate from OpsMgr anyway).  So root cause analysis of issues can be more difficult with OpsMgr.   

As you can see, they each do something different well.  My personal preference, tainted by years of working with SCOM, is that I prefer SCOM's shortcomings to Azure Monitor's shortcomings.   I feel what Azure Monitor lacks is easily filled in by other products, but the reverse isn't necessarily true.  But, all that said, why should we even have to choose?  MS makes both products, they can give us the features of both!

Aquila could be what changes this.  First of all, the rumors are that they are refactoring SCOM into containers.  This gives really terrific portability and seamless updating, making the product ready to run on the modern cloud.  Powered by Azure SQL (or replicated to/from MS-SQL) and with multiple regions and datacenters, SCOM will have HA options like it's never had before, both in the cloud and on-prem (or self-managed VMs).  Because of how easy it is to update containers, whether the container runs inside MS managed instances or if you run it on-prem or in your own cloud VM, there will be feature parity and the product will stay up-to-date.  Secondly, this brings Azure Monitor and OpsMgr closer together for integration and rapid feature enhancement.  The best of both worlds would be having the ability to, from a single pane of glass, have all your SCOM management packs, recommended and tweaked baseline monitoring, organization standards, and so forth deployed, but then on top of that have the log consumption, analytics, visualizations, and ad hoc log alerting of Azure Monitoring.  Further, since we are again talking about containers now, OpsMgr could run in any cloud, anywhere.  You can get the same monitoring experience in AWS, GCS, and so on.  So you now have a bridge connecting your cloud monitoring.  All of it rolled up to SCOM + Azure Monitor.  That would truly set the entire MS monitoring picture above and beyond the competition by leaps and bounds.  And of course finally we can't ignore it would give a better pathway for ACS migration to Log Analytics, which actually is a part of SCOM that I think is pretty terrible but it seems there are a lot of organizations that haven't made the migration yet.

So, in conclusion, I'm hoping Aquila has great success, this could be a game changer.

I installed Exchange 2016 tools on my desktop machine back in 2017. Since then numerous CUs have come out and we migrated to Office 365. Years later we upgraded the last couple Exchange servers we have to 2019 for ongoing hybrid management. This old 2016 installation is just legacy baggage at this point.

When I tried to uninstall, however, I found that I couldn't. I suppose because the management tools were on an old CU, they weren't compatible with the current state of my desktop. The error I was getting was pretty non-descript, it was the old "An Incomplete Installation Was Detected" error. Well, this was not, in fact, an incomplete installation. It was just management tools and old.

The recommended guidance was to basically do a fresh install to the latest CU and try again but that just seemed silly, so I executed the uninstall from the command line to try to get more info. This is the cached setup.exe that exists in the Exchange installation, not the one off the installation ISO (which I no longer had lying around).

MpsSvc is the Windows Firewall. Not a surprise here that it can't muck around with that, but we can just temporarily adjust the ACL using sc sdshow/sdset, like below. Make sure to record the original ACL! It might change over time if you are viewing this article years later, so do a fresh sdshow to confirm.

I copied the ACL from a more permissive service picked at random. In this case, atashost. Using sdset, the permissions were applied.

Now I ran the installer again. I got further but still failed at a step.

This time it was AppIDSvc. The ACL looked fine, but maybe it's being protected somehow in a way I don't understand. I did find an article while searching that mentioned you can remove it (and others) from the uninstall script. So I did that.

https://social.technet.microsoft.com/Forums/en-US/e3c3174f-b2f8-4dfa-9db0-f34b0453bf03/exchange-2016-installation-failed-unable-to-uninstall-or-install-again-mailbox-role-and-others

I edited ServiceControl.ps1 and removed both the signature (since we were breaking that) and the AppIDSvc from it.

Delete both:

And here we go again!

Success!

Now return the firewall service to it's original state.

Exchange still showed up for me in Programs and Features, but clicking uninstall again indicates it did get removed and this is just Windows Installer leftover stuff it will clean up on its own.

The last bit of cleanup was just removing the Exchange Server program folders. Unfortunately, this didn't work right away as files were locked.

Guessing these might be locked until reboot, I rebooted.

And that did the trick!

I'm a big user of both RD Tabs (of course) and ConEmu, a great terminal app for Windows. I am not sure when this issue started, but it was not always present. I think in later Windows 10 builds something may have changed to cause this, but I don't have any evidence and I can't trace the behavior to anything in RD Tabs' source code (certainly the source code has not changed materially in a couple years).

So here's the issue. If you are connected to a remote desktop session where in that session you are running ConEmu, when you switch out of RD Tabs to another Window then back in, you'll see the "@" symbol appear in your ConEmu window one or more times. This happens consistently every time the focus changes back to that window.

Using a keylogger, I was able to see that every time the focus changes on the RDP control, it appears that it resets the keyboard state, sending 50 or more "key up" events for all modifier keys as well as a key-down key-up cycle for keycode 255. I don't have any code like this in RD Tabs. And just to be sure, I disabled all the input focus fix-ups in a debug build and it still happened. I think it's just a change in the underlying RDP control, at least the hosted control that RD Tabs uses. I don't see this behavior in the native app.

In a way it makes sense, there certainly was a problem in older versions of the control where keyboard modifiers could get stuck if you switched out of remote focus to another app then clicked back in. To the remote session, it would, for example, see a SHIFT KEY DOWN event, but not always a SHIFT KEY UP event. By just blanketing the remote session when focused is resumed with KEY UP for all modifiers, it prevents this issue from happening.

So that's all well and good, why does it impact ConEmu? I'm not entirely sure, but I think that keycode 255 may be the culprit since it's the only KEY DOWN event that appears in the input stream. But, again, I'm not sure.

In any case, it manifests as a "@" in the input stream. And there's nothing in RD Tabs I can do to fix this, I can't stop those key codes from going to the remote desktop.

But you can fix this with a little workaround in ConEmu! Basically you just need to assign an empty macro to the keycode it is receiving and it won't output anything anymore.

Go to the Keys & Macro section, select "Macros" in the drop-down. Find an empty macro. I just used Macro 32. Give the "Choose hotkey" field input focus, then click out of RD Tabs to another window and click back in. It will change from to just an empty field. Even though you don't see anything, it's been assigned to that weird rogue keypress. Click save and you are done.

UPDATE November 3, 2021: the double click problem returned this week in the left mouse button.  I took apart the mouse and replaced the button again.  I'm skeptical now that the Japanese Omron switches are any better than the Chinese ones.  It might just be an issue with all Omron switches.

---

There was a Black Friday deal in 2019 for the Logitech G903 mouse.  In theory, it's a great mouse: wired and wireless, high precision, comfortable, tons of button configuration.  But it has one really common issue: the dreaded double click problem.  I was coming from a G502, so it seemed to be a perfect mouse to transition to.  They are really similar, although the 903 doesn't have a wing -- which I'm glad.  I got about six months of use before the problem popped up consistently.

So I decided I'd go the route of replacing the switches.  The problem, supposedly, is more cheaply manufactured Chinese Omron switches as opposed to (again supposedly) higher quality Japanese Omron switches.  There's information all over about this, including how to fix it, but here's a good link I found with pictures: https://blog.thirdechelon.org/2019/10/logitech-g900-g903-double-click-fix-replace-the-switches/

So below is my go at it, in gallery form, for others to look at.

My biggest struggles with this were peeling off the sticker and feet without damaging the plastic chassis or damaging the sticker and removing the battery without breaking it.  As for the sticker, I did okay.  The feet I didn't really care about because I planned to replace them.  They are easy to come by.  And, surprisingly, the new feet are slicker than the stock kind.  The battery was a major pain.  It's very strongly glued and there isn't a good way to slide something in to separate it.  I basically had to pry it with a plastic pry tool (dear god, don't use anything metal) and hope it didn't break.  It didn't.

By default, Exchange Online (like on-premises Exchange before it) allows all users in an organization to view only the free/busy status (blue bars) on other users' calendar appointments.  Some organizations require that the appointment summary or all calendar details be visible by all users.  This is fairly simple for a single user to change on their own, but enforcing it across the organization takes a bit more effort.  PowerShell comes in handy here!  And there are plenty of articles out there on how to do that.  But if you want to continuously enforce this configuration for all new mailboxes, or fix permissions when users change it, we need to schedule a recurring task.  Azure Automation works well for this purpose in the Office 365 / Azure Cloud universe.

Start by creating a "service account" that is an Exchange Organization administrator.  This can just be a standard Azure Active Directory account.  It doesn't need to have an Office 365 license assigned to it.  If you enforce MFA, make sure to generate an application password to use later on.

Go to the Exchange Administrator from the Office 365 admin panel.  Then select Permissions and then Admin Roles.

Edit the Organization Management role and add the user you created in Azure AD.

In the Azure admin portal, create an Azure Automation Account (or user an existing one).

Go to the Credentials section of the automation and add the username and password of the Azure AD credentials you created.  Note: if you have MFA enabled, the password must be the application password.

Go to the Runbooks section.  Create a new runbook.  Edit the runbook.

Exchange Online PowerShell cmdlets are available as a remote session.  You use the stored credentials to establish the session, then import it into the running session.  The number of cmdlets are large.  You should filter which cmdlets you want to import.  This saves a lot of time and prevents the odd crashes (the automation runspaces have a pretty limited amount of resources to work with).

$cred = Get-AutomationPSCredential -Name 'ExchangeOnlineAutomation'
$session = New-PSSession –ConfigurationName Microsoft.Exchange -ConnectionUri https://outlook.office365.com/powershell-liveid/ -Credential $cred -Authentication Basic -AllowRedirection -ThrottleLimit 1 
$commands = @("Get-MailboxFolderPermission","Set-MailboxFolderPermission","Get-Mailbox") 
$null = Import-PSSession -Session $session -DisableNameChecking:$true -CommandName $commands -AllowClobber:$true

Note: Azure Automation doesn't support piping to Out-Null, so the $null= is a trick to eat the output of a command you don't want to show up in the final script output.

From here, the rest is pretty simple, we just loop through every mailbox, look at the calendar's default ACE, and adjust it if necessary.  

$mailboxes = Get-Mailbox -ResultSize Unlimited
$mailboxes | % {
    $mb = $_
    $calendarDefaultACE = Get-MailboxFolderPermission "$($mb.PrimarySmtpAddress):\Calendar" -User Default
    if ($calendarDefaultACE -and $calendarDefaultACE.AccessRights -inotcontains "reviewer") {
        Set-MailboxFolderPermission "$($mb.PrimarySmtpAddress):\Calendar" -User Default -AccessRights Reviewer
    }
}

The above is assigning the ability of all users to see full calendar appointment details.  If you just want to assign the ability to see the appointment summary, use LimitedDetails instead of Reviewer.

We don't want to leave any sessions dangling as you only get three to work with for Exchange Online PowerShell.  So we should wrap all the above code in try..catch..finally and close the session at the end.  Add some logging and we have the following.

$cred = Get-AutomationPSCredential -Name 'ExchangeOnlineAutomation'
try {
    # Import an Exchange Online session
    Write-Output "Connecting to Exchange Online."
    $session = New-PSSession –ConfigurationName Microsoft.Exchange -ConnectionUri https://outlook.office365.com/powershell-liveid/ -Credential $cred -Authentication Basic -AllowRedirection -ThrottleLimit 1
    $commands = @("Get-MailboxFolderPermission","Set-MailboxFolderPermission","Get-Mailbox")
    $null = Import-PSSession -Session $session -DisableNameChecking:$true -CommandName $commands -AllowClobber:$true

    $mailboxes = Get-Mailbox -ResultSize Unlimited
    Write-Output "Found $(@($mailboxes.count)) mailboxes."

    Write-Output "`r`nAssigning default permissions for calendar to REVIEWER."
    $mailboxes | % {
        $mb = $_
        try {
            $calendarDefaultACE = Get-MailboxFolderPermission "$($mb.PrimarySmtpAddress):\Calendar" -User Default
            if ($calendarDefaultACE -and $calendarDefaultACE.AccessRights -inotcontains "reviewer") {
                Set-MailboxFolderPermission "$($mb.PrimarySmtpAddress):\Calendar" -User Default -AccessRights Reviewer
                Write-Output "$($mb.PrimarySmtpAddress): Setting permissions."
            }
        } catch {
            Write-Output "$($mb.PrimarySmtpAddress): ERROR - $($_.Message)"
        }
    }
} finally {
    # Close all open Exchange sessions
    Get-PSSession | Remove-PSSession
    Write-Output "`r`nExchange Online session closed."
}

You can test your runbook now.  If it works as expected, publish it.

Now exit out of the editor and set a schedule.  Every run costs a few cents, but really you probably only need to run this once per day so the cost should be extremely small.

I was setting up Azure File Sync tonight and ran into a weird issue that I found absolutely nothing online about.  Whenever I tried to add this one server as a server endpoint, I kept getting an error stating the provisioning was canceled and in the progress area it said "Provisioning failed" with the Enabling Sync step saying "Server job with command name NewServerEndpointWithDatePolicy and id <some guid> failed with -2134347507."

Not very descriptive!  And searching yielded no results, so it's apparently not a common error.

Well thankfully the Azure File Sync agent has very good logging.  In the Event Viewer, you look under Applications and Services Logs, Microsoft, FileSync, Management, Operational for these types of errors.

The very last entry was the error, event ID 5103.

This is basically the same text that showed up in the portal, but with one vital extra clue: Sync is not supported on a volume using compression.

Oh, well that explains it!  Except... The only problem is that this volume isn't compressed.  I double checked this.

Yup, not compressed.  But maybe some deep subfolder is compressed.  So I reset compression by recursively disabling compression on the entire volume again.  So now I knew none of the files were compressed. 

Trying again led to the same error.  This is where I hit a brick wall and decided to take a break.

I found later that the entire volume had been compressed at some point in the past.  But this was years ago, and as shown above, compression is now disabled.

So it appears that disabling compression isn't enough for the validation step of Azure File Sync.  Something from the past is still lurking around in the system.  What could the agent possibly be looking at that made it think the volume was still compressed?

At this point, I put on my programmer's hat and tried to think how they developed the detection logic.  Since nobody else I could find is having this issue, it must be something pretty unique or rare happening here.  I checked WMI properties on the volume (there's a flag for compressed volume), but that also showed that the volume was not compressed.  I dug a bit in the registry, but I couldn't find anything interesting.  I looked up the Win32 API call to obtain volume properties and executed that, but it also the volume was uncompressed.  So what is the agent looking at?

Then it dawned on me.  NTFS metadata is stored in the system hidden "System Volume Information" folder that is at the root of every drive.  Maybe there is a clue there.  I went into the folder properties of File Explorer and disabled "Hide protected operating system files (recommended)."

Then I went back to the root and it all became very clear.

Compressed folders show up in blue because I have "Show encrypted or compressed NTFS files in color" enabled (this is disabled in Server 2016+ by default).  But if you look at the properties of the folder, you can also clearly see that the System Volume Information folder itself is compressed.

So all I needed to do was uncheck that, apply recursively, and be on my way, right?  Nope.  System Volume Information is highly protected by the operating system due to the critical nature of the data stored in there.  So to remove compression, we have to go through a lot of hoops first.

The easiest way to do this is to run a few commands from an elevated command prompt.  The GUI just gets in the way.

First I took ownership of System Volume Information.  (If you are going to repeat these steps, obviously change the commands to your admin username and file path where applicable.)

TAKEOWN /F "D:\System Volume Information" /R

A bunch of errors flashed by, but this is okay.  This will get us far enough.  Now I ran the following two commands.

ICACLS "D:\System Volume Information" /GRANT Administrator:F

ICACLS "D:\System Volume Information" /GRANT Administrator:F /T /C

More errors flashed by, also okay.

Now back in the GUI, I went to the properties and unchecked the compression option.  I chose apply recursively and watched it go.  It ran into a lot of errors (of course) and I just had to keep clicking through them until it finished.  Curiously, when I was done, all the subfolders were no longer comrpessed, but the System Volume Information root folder still showed up as compressed.  I repeated the steps but chose "this folder only" this time and that took care of that.  Weird, but okay.

At this point, I rebooted the server.  On startup, Windows will reset the permissions to System Volume Information, so we don't have to worry about undoing the permissions we added above.

Back in Azure, before I could try to add the server endpoint again, I had to delete the previous attempt.  It's not intuitive (no surprise for Azure) but in this case, you right click and select delete.

Now I attempted to add the server endpoint back and waited a few minutes for provisioning to start.

Success!

From this we can deduce that the developers looked at the compression attribute on System Volume Information for the detection logic instead of the drive root or via the volume properties API call.  That's different from the other ways I saw how to do it when I googled, but maybe it's the correct way for whatever reason compression has to be disabled for Azure File Sync?  I really don't know.  It may also be a bug.

In any case, the error message needs to be written a lot more clearly, this issue was not obvious at all!

Several minor bugs were discovered after the general release of RD Tabs 3.0.11.  The update is available now right here.  More information is available in the forums.