Resolving “RDS Licensing Mode Not Configured” Error on Windows Server 2012 Despite Proper Settings


4 views

When dealing with Windows Server 2012 RDS deployments, one particularly stubborn issue keeps resurfacing: the licensing mode appears unconfigured in the RD Licensing Diagnoser despite repeated attempts to set it. This creates a chicken-and-egg situation where your RDS farm remains non-functional due to licensing validation failures.

Through extensive troubleshooting, I've discovered this behavior typically occurs when:

  • The licensing server wasn't properly added to the RDS deployment
  • Group Policy settings override local configurations
  • The RD Session Host role was installed after configuring licensing
  • Registry permissions prevent changes from persisting

Here's the step-by-step approach I've used to permanently resolve this:

# First verify current licensing configuration
Get-WmiObject -Namespace "Root/CIMv2/TerminalServices" -Class Win32_TerminalServiceSetting | 
Select-Object LicenseServer, LicensingMode

# If blank or incorrect, force the configuration via PowerShell
$tsgs = Get-WmiObject -Namespace "Root/CIMv2/TerminalServices" -Class Win32_TerminalServiceSetting
$tsgs.SetMode(2, "FQDN_OF_YOUR_LICENSE_SERVER") | Out-Null

# For Per User licensing (mode 4)
$tsgs.SetMode(4, "FQDN_OF_YOUR_LICENSE_SERVER") | Out-Null

When PowerShell commands fail, directly modifying these registry values often succeeds:

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Terminal Server\RCM\Licensing Core]
"LicensingMode"=dword:00000002
"X509Certificate"=""
"LicenseServers"=hex(7):46,00,51,00,44,00,4e,00,00,00,00,00

Important notes about the hex value:

  • Replace with your license server FQDN converted to UTF-16LE hex
  • Terminate with two null bytes (00,00)
  • Each character takes two bytes (e.g., 'A' becomes 41,00)

After making changes, verify with this diagnostic script:

$tsAdmin = New-Object -ComObject Microsoft.TS.Admin.1
$tsAdmin.GetLicenseServerList() | ForEach-Object {
    Write-Output "License Server: $_"
    $status = $tsAdmin.IsLicenseServerValid($_)
    Write-Output "Validation Status: $status"
}

For particularly stubborn cases:

  1. Remove all RDS roles via Server Manager
  2. Delete these registry keys:
    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Terminal Server\RCM
    HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSLicensing
  3. Reboot and reinstall RDS roles
  4. Configure licensing before adding session hosts

When deploying a Windows Server 2012 R2 Remote Desktop Services (RDS) environment, many administrators encounter a persistent issue where the RD Licensing Diagnoser reports "The licensing mode for Remote Desktop Session Host server is not configured" even after repeated configuration attempts.

This issue typically involves three key components:

1. The RD Session Host server

2. The RD Licensing server

3. The group policy settings affecting RDS

Before diving into solutions, let's verify the current state with PowerShell:

# Check current licensing mode
Get-WmiObject -Namespace "Root/CIMv2/TerminalServices" 
              -Class Win32_TerminalServiceSetting 
              | Select-Object LicensingMode

# Check license server configuration
Get-WmiObject -Namespace "Root/CIMv2/TerminalServices" 
              -Class Win32_TerminalServiceSetting 
              | Select-Object SpecifiedLicenseServerList

The configuration not sticking often occurs due to:

- Group Policy conflicts

- Incorrect permissions

- Cached policies

- Missing registry updates

1. Manual Configuration via GUI

First, try setting the licensing mode manually:

1. Open Server Manager

2. Navigate to Remote Desktop Services → Collections

3. Right-click your collection → Properties

4. Under RD Licensing, select "Per Device" or "Per User"

5. Add your license server

2. PowerShell Configuration

If GUI configuration doesn't persist, try PowerShell:

# Set licensing mode to Per Device
$tsEnv = Get-WmiObject -Namespace "Root/CIMv2/TerminalServices" 
                       -Class Win32_TerminalServiceSetting
$tsEnv.SetLicensingMode(2, "YourLicenseServer")

# For Per User mode
$tsEnv.SetLicensingMode(4, "YourLicenseServer")

3. Group Policy Investigation

Check for conflicting policies with:

gpresult /h rds_report.html

Look specifically for:

- Computer Configuration → Policies → Administrative Templates → Windows Components → Remote Desktop Services → Remote Desktop Session Host → Licensing

- User Configuration policies that might override settings

4. Registry Verification

Manually check the registry settings:

# Check the licensing mode value
Get-ItemProperty -Path "HKLM:\SYSTEM\CurrentControlSet\Control\Terminal Server\RCM\Licensing Core" 
                 -Name "LicensingMode"

# Verify license server
Get-ItemProperty -Path "HKLM:\SYSTEM\CurrentControlSet\Control\Terminal Server\RCM\Licensing Core" 
                 -Name "LicenseServers"

If the issue persists, try these advanced steps:

# Force policy update
gpupdate /force

# Reset RDS components
Get-RDSessionCollection | Set-RDSessionCollection -UserGroup "Domain\RDS Users"

# Check for WMI corruption
winmgmt /verifyrepository

After making changes, always:

1. Restart the server

2. Verify changes with RD Licensing Diagnoser

3. Test actual RDS connections