Errors when Configuring Skype for Business Server 2015/2019 Pool Pairing & Resolution
I recently was setting up a Skype for Business Server 2019 Pool Pair between two 2019 Front-Ends and came across a weird at first error when attempting to run the Invoke-CsBackupService cmdlet on the second FE.
To start, I used the standard Microsoft guide which involved defnining the pair relationship in Topology Builder, Bootstrapping both FE's, running Start-CsWindowsService and then came the errors. When running the below two commands, the one on pool 2 failed with the errors of:
Invoke-CsBackupServiceSync -PoolFqdn <Pool1 FQDN>
Invoke-CsBackupServiceSync -PoolFqdn <Pool2 FQDN>
Error 1: Invoke-CsBackupServiceSync : Server error occurred in backup service on machine "FE.domain.com". Server error: "Object reference not set to an instance of an object.".
Error 2: Invoke-CsBackupServiceSync : Failed to perform a Windows Communication Foundation (WCF) call to backup service instance on computer 'FE.domain.com"
Event Viewer and a Bing search didn't give much hope either as these were very generic errors.
After doing some digging and ripping out of any remaining hair that I have, I found this Microsoft Doc on the issue. LyncBackupService.exe stops responding after pool pairing enabled on a Skype for Business 2019 server - Skype for Business | Microsoft Docs
It is stated that when the backup service is installed during Bootstrap, it installs the RTM version on the server and not that of the current CU. BINGO. So the question now came, why did it work on FEPool1 and not FEPool2. Well FEPool1 is the CMS so it already had the latest CU (CU5HF1) applied. After running the SFB Server Update Executable, we can see that the Core Management Server was on the RTM release as expected.
After installing the update and running Start-CsWindowsService, we came to the moment of truth. I re-ran the below two commands and found sweet success! Or so I thought...
Unfortunately, running the below commands after 5 minutes of the last two, I unfortunately was presented with another issue where the Import on the second pool was showing an error state. Event Viewer stated it was having issues with the XDS Database.
Get-CsBackupServiceStatus -PoolFqdn <Pool1 FQDN>
Get-CsBackupServiceStatus -PoolFqdn <Pool2 FQDN>
Error Text: Skype for Business Server 2019, Backup Service central management backup module failed to complete import operation.
After doing some more digging, it was recommended to check that the CMS Database was fully installed with the following command (It was already ran prior to this)
Install-CsDatabase -CentralManagementDatabase -SqlServerFqdn FE.domain.com -SqlInstanceName rtc -Clean
Unfortunately, this also did not resolve that issue. After doing yet more digging, I found that re-running step 1 in the Deployment Wizard got things working again. I re-ran the invoke-CsBackupServiceSync cmdlets and checked the status to find sweet sweet success with the states all correct!
Thank you for reading this far into my journey. I hope this helps you out if you have issues with setting up the Lync/SFB Backup Service!