-->

Sunday, November 5, 2017

Exchange/SFB UM Cross Forest with Selective Trust - Part 3

In Part 2 of this series, I showed you how set up our UM Certificate so SFB will trust Exchange.

This post will walk you through setting up the UM Dial Plans, UM Mailbox Policies, and Auto-Attendants in Exchange and SFB.

**Note** Further down you will see two domain names, exchangeitup.com (which is the Exchange Forest) and exchitup.com (which is the SFB Forest).

**Note** You do not need to set up a Partner Application between Exchange SFB for UM Integration.
In my environment, SFB was already set as a Partner with our Europe Exchange Forest and there can only be one because it uses the autodiscover settings for that Exchange forest.
This threw me for a loop for while trying to figure out how to add my Exchange, but after trying different ways, I finally found UM Integration works without it :)

Create Unified Messaging Dial Plan

On an Exchange server, fire up the Exchange Management Shell (EMS) and run the following cmdlet:

New-UMDialPlan -Name "Headquarters" -VoIPSecurity "Secured" -NumberOfDigitsInExtension 4 -URIType "SipName" -CountryOrRegionCode 1

**Note** Change "Headquarters" to whatever you want to call the Dial Plan...its best to be location specific if you have multiple sites.

**Note** We're using 4 digit extensions, but you can set it to your needs. If you currently run 3 digit, then set  -NumberOfDigitsInExtension 3

Next, run the following cmdlet:

Set-UMDialPlan "Headquarters" -ConfiguredInCountryOrRegionGroups "Anywhere,*,*,*" -AllowedInCountryOrRegionGroups "Anywhere"

Just to make sure the settings take, I always restart the UM Services...it won't cause any harm to do in production hours because you're not using it yet ;)

Restart-Service MSExchangeUM
Restart-Service MSExchangeUMCR

Configure the Dial Plan

Creating a dial plan automagically creates a UM Mailbox Policy, with the naming convention using the Dial Plan name e.g. Headquarters Default Policy.

We need to modify that new policy with the following cmdlet:

Get-UMMailboxPolicy | Set-UMMailboxPolicy -AllowedInCountryOrRegionGroups "Anywhere" -MinPINLength "6" -AllowCommonPatterns $true

Since we only have one UM Mailbox Policy, we don't need to specify the name in the above command.

**Note** You can change the -MinPINLength to something lower than 6 but you'll want it secure.

Configure Auto-Attendants

**Note** We're not going to set up a Subscriber Access, because to be honest, who even uses those any more? I haven't in years, because users check their voicemail directly from Outlook or their phone.

In the EAC, go to Unified Messaging > UM Dial Plans

EAC UM Dial Plan


Select your Headquarters dial plan and hit the Edit (Pencil) button, or double-click the dial plan.

Scroll down to the Auto-Attendant section, and hit the +

UM Auto-Attendant

Enter a name like Headquarters-AA
***Do not include spaces in the name, Exchange will problems with that***.

Check both boxes for "create this auto attendant as enabled" and "set the auto attendant to respond to voice commands".

And enter an access number and hit the + to add it.

Then, click Save:

UM Auto-Attendant Config


**Note** The access number can be internal like 555-555-1234, or if you need external parties to be able to call it, you will need associate one of your DID's provided by your SIP Trunk provider.

Enable Users for Unified Messaging

Using the EMS, run the following cmdlet:

Enable-UMMailbox -Extensions 1234 -SIPResourceIdentifier "stacey@exchangeitup.com" -Identity "exchitup\stacey" -UMMailboxPolicy "Headquarters Default Policy"

**Note** Enter a unique -Extension or else Exchange will tell you it's already in use. Also, we didn't set a PIN, as we'll let Exchange randomly assign that.

**Note** -SIPResourceIdentifier stacey@exchangeitup.com is the Exchange forest; -Identity "exchitup\Stacey" is the Accounts/SFB Forest

Sync the EUM Proxy Addresses

In order for SFB to pick up the UM settings, a couple proxy address need to be copied over from the Exchange Forest to the SFB Forest.

The SIP Address also needs to be synced from the Accounts Forest to Resource Forest (but that should be done already, else there would be no integration at all between SFB/Exchange)

**Note** To automate this, it should be done with MIM or whatever DirSync tool you use

To manually set it, open Active Directory Users and Computers in the Exchange Resource Forest.

Navigate to the OU where your user resides and open the Properties.

Click the Attribute Editor Tab and scroll down to proxyAddresses and double-click it.

Copy both addresses beginning with eum and EUM:

Proxies
 
Or you can use the EMS by running the cmdlet below and copy both EUM/eum lines:
 
Get-Recipient "stacey branham" | fl *email*

EmailAddresses            : {eum:1538;phone-context=Headquarters.exchangeitup.com,
                                         EUM:stacey.branham@exchangeitup.com;phone-context=Headquarters.exchangeitup.com,



Now switch to your SFB Forest and navigate to the OU of your user.

Follow the above steps and paste in the two eum/EUM addresses in the proxyAddress field.

This will result in:

Get-ADUser staceybranham -Properties proxyaddresses

proxyaddresses    : {eum:1538;phone-context=headquarters.exchangeitup.com,                                   EUM:stacey.branham@exchangeitup.com;phone-context=Headquarters-exchitup.com

Configure UM IP Gateway

In the EMS, run the following:

cd $exscripts

.\ExchUCUtil.ps1 -forest "yourSFBForestname.com"

**Note** You must include the -forest switch for Exchange to see SFB in another forest.

**Note** You will need to give it about 10-15 minutes after creating the dial plans before running the above script, or else the output will come up empty. You can however run the script as many times as you want.

At the end of the output you should see:

Poolfqdn                             UMIPGateway                                  DialPlans
-----------                             ------------------                                  ------------
sfb.exchitup.com                sfb.exchitup.com                              Headquarters

Now go to the EAC, and navigate to Unified Messaging > UM IP Gateways and you'll see your SFB frontend server listed:

EAC UM IP Gateways

Create Auto-Attendant Contacts

On your SFB server, navigate to C:\Program Files\Common Files\Skype for Business Server 2015\Support

Run the OCSUmUtil.exe as administrator and click Load Data:

OCSUmUtil

This will take quite a while depending on your replication speed...

Loading

In the Exchange UM Dial Plan Forest, drop-down to your Exchange Resource Forest

Click the Add button:

Click Add OCSUmUtil


In the next window, select the Organizational Unit where you want the contact created.

Give it a name like Headquarters-AA

Assign a SIP Address like sip:Headquarters-AA (remove the trailing domain name in the address field. It should only be sip:Headquarters-AA

Set your desired server or pool if you have more than one.

Select "use this pilot number from Exchange UM" and enter the number you set for the AA in Exchange.

Select Auto-Attendant and enter the Headquarters-AA name.

And click OK:

OCSUmUtil AA


Your AA contact will be created in the OU that you specified, and it will be listed in the OCSUmUtil:

OCSUmUtil Done


Now test it out!

In your SFB client and search for the AA contact like Headquarters-AA, and perform a Skype Call.

It should say something like "thank you for calling the Exchange auto-attendant..."

You can also direct dial the 504-555-1111 number.

Now test your voicemail, by calling the number you assigned to your user account and let it go to voicemail. It should send you over to the UM prompt to leave a message.

Saturday, November 4, 2017

Exchange/SFB UM Cross Forest with Selective Trust - Part 2

In Part 1 of this series I showed you how to allow the SFB servers and groups to authenticate with the Exchange servers over a Selective Trust.

In this post I'll show you how to create a certificate to use for Unified Messaging to allow SFB/UM integration.

Create A Certificate For UM

We'll need to create a certificate in Exchange in order for SFB/UM traffic to use TLS.

The main requirement is that the Exchange Server FQDN's need to be present on the cert.

**Note** If you have multiple Exchange servers, include each server name on a single cert.

We're going to use a Private CA (Certificate Authority) because this is internal only, so you don't need to spend money on a public cert.

Also, we won't be using our SAN cert since it's not good practice to use server FQDNs because it can cause problems.

Certificate Request

On an Exchange server, open the EAC and navigate to Servers > Certificates.

Click the + and leave the first option "create a request for a cert from a certificate authority" checked, and click Next

Request UM Cert 2

Name the cert something like Exchange UM, and click Next:

Request UM Cert 2

Just click Next, do not choose the wildcard option:

Request UM Cert 3

Click and select the Exchange server you're currently on, and click OK, then Next:

Request UM Cert 4

You don't need to touch anything here, just click Next:

Request UM Cert 5

Here, you will remove all entries except your namespace (eg. mail.exchangeitup.com) and add your Exchange servers.

Select each domain name and click - button, leaving your namespace.

Click the + and type in the FQDN of each Mailbox Server, then click Next:

Request UM Cert 6

Enter your company details and click Next:

Request UM Cert 7

Enter a UNC path to store the request file...it's easiest to use the exchange server you're on like \\mbx1.exchangeitup.com\c$\temp\cert\umcert.req:

Request UM Cert 8

Browse to the path where you saved the .req and submit that file to your Internal CA.

Fire up your browser and go to your Internal CA's enrollment webpage.

Click Request a certificate:

Request UM CA 1

Click submit an advanced cert request:

Request UM CA 2


Click submit a certificate request by using...blah blah...file:

Request UM CA 3

Open your umcert.req file in a text editor like Notepad and copy the whole block of text.

Then, paste it into the Saved Request form.

Change the cert type dropdown to Web Server.

And click Submit:

Request UM CA 4

When it's done, click the Download certificate:

Request UM CA 5

Save the .cer file on your Exchange server to somewhere like C:\Temp\Cert\ExchangeUM.cer

Complete Cert Request

Back in the EAC, you'll see your pending cert request:

Complete UM Cert Req 1

Select the request and click Complete in the right-pane:

Complete UM Cert Req 2

Provide the UNC of the .cer file and click OK:

Complete UM Cert Req 3
Once it's finished, you'll see the cert is installed and valid:

Complete UM Cert Req 4

Assign the UM Cert

Double-click your new UM cert to bring up the editor, and click services.

Put a check in the boxes for Microsoft Exchange Unified Messaging and Unified Messaging Call Router. Then click Save:

Assign UM Cert 1

If you are overwriting another cert that is assigned to those services, you will get a prompting asking to confirm. Click Yes.

Export/Import UM Cert To Other Exchange Servers

Select the UM cert and click the ..., then click Export exchange certificate:

Export UM Cert 1

Input a UNC path to save the .pfx file like \\mbx1.exchangeitup.com\C$\temp\cert\exchangeum.pfx and set a password click OK:

Export UM Cert 2

Click the ... again and select Import exchange certificate:

Import UM Cert 1

Enter your UNC path and password for the .pfx file:

Import UM Cert 2

Click the + and select all of your other Exchange servers and click OK, then Finish:

Import UM Cert 3
 
On each of your Exchange servers restart the UM Service and the UM Call Router Service by running the following cmdlets in the Exchange Management Shell (EMS):
 
Restart-Service MSExchangeUM
 
Restart-Service MSExchangeUMCR 
 
Export/Import UM Certificate on the SFB Server
 
On your Exchange server, browse where you stored the exchangeum.cer file (in our case C:\Temp\Cert\exchangeum.cer) and copy that file.
 
Now switch over to your SFB server and paste it somewhere like C:\Temp\Cert
 
Go to start and search for certlm.msc and open the certificate manager.
 
Navigate to Trusted Root Certification Authorities > Certificates
 
SFB Cert import 1
 
Right-click anywhere in the right-pane, select All Tasks > Import
 
Follow the wizard to import the Exchange UM cert:
 
SFB Cert import 2
 
SFB Cert import 3
 
SFB Cert import 4
 
SFB Cert import 5
 
 
 
Now we have allowed the Skype For Business server to authenticate with the Exchange forest, and set SFB to trust the Exchange UM certificate.
 
In the next post we will set up the actual UM integration.


Sunday, October 8, 2017

Exchange/SFB UM Cross Forest with Selective Trust - Part 1

This will be a series on integrating Exchange Unified Messaging and Skype For Business in a cross-forest scenario, with a Selective Trust configured.
My Exchange 2016 environment resides in a Resource Forest with a Selective Trust configured to ensure security between our US and Rest-of-World (ROW) forests.

Our Skype For Business resides in the Accounts Forest, because management didn't want multiple SFB environments in multiple Resources Forests. I was overruled on that (not-so-smart) decision.

This causes some problems when configuring Dial Plans with Unified Messaging because the SFB servers need to be able to authenticate against Exchange in order to configure Auto Attendants.

When running the OCSUmUtil in the Accounts Forest with a Selective Trust, the Dial Plans will come up empty. This is because the SFB servers are blocked from authenticating on the Resource Forest and can't read the config.


To allow authentication, the following groups from the Accounts Forest need to be set as "Allowed to Authenticate" on the Exchange servers and the Domain Controllers in the Resource Forest:

RTCComponentUniversalServices

RTCUniversalServerAdmins

RTCUniversalUserAdmins


To easily add those groups and keep a clean Active Directory I suggest following my
previous post on creating a Selective Trust Security Group in the Accounts Forest.

And then follow
my other post to set the Selective Trust auth permissions.

If you just want to hurry and add those to allowed to auth, do the following:

1. Log onto a domain controller in your Resource Forest

2. Open Active Directory Users and Computers (ADUC)

3. Click View

4. Select Advanced Features

5. Browse to the OU where the Exchange Server(s) you are trying to authenticate to

6. Right-click the Exchange server objects, then select Properties, then the Security tab

7. Add the RTCComponentUniversalServices; RTCUniversalServerAdmins; RTCUniversalUserAdmins groups

8. Grant Allowed to authenticate rights

9. Click Apply, then OK.

10. Browse to the Domain Controllers OU.

11. Browse to the OU where the DC's you are trying to authenticate to

12. Right-click the DC objects, then select Properties, then the Security tab

13. Add the RTCComponentUniversalServices; RTCUniversalServerAdmins; RTCUniversalUserAdmins groups

14. Grant Allowed to authenticate rights

15. Click Apply, then OK

Now give it some time for replication.

In the next post I'll show you to configure the UM certs to allow SFB and Exchange to communicate over TLS.

Sunday, October 1, 2017

Exchange Get All Duplicate Recipient SMTP Addresses

In my organization we have two Exchange 2016 Resource Forests (eu.domain.com and us.domain.com) and an Accounts Forest (domain.org) that both Exchange forests share.

We use MIM (Microsoft Identity Manager) to control our DirSync between all three forests.

We have about 15,000 users total, and a few thousand more service/admin/functional accounts.

The issue is, tons of those non-user accounts are set with the same proxy addresses and email addresses as the user accounts.

For instance if Joe Schmo is an IT admin and he has joe.schmo@domain.com for his UserMailbox Primary SMTP, his admin.schmo account also has joe.schmo@domain.com, which syncs over to the Exchange forest, causing tons of Event ID 9217 warnings and also causes email bounces because Exchange can't find the correct recipient.

In Exchange there's no one-liner to find duplicate SMTP addresses, nor is there in Active Directory...you can only do a query to find them one by one. This is a giant problem if you have an environment like me, where there's a couple hundred duplicate recipients!

So, I wrote a quick script that will run through your Exchange environment and spit out a TXT file with the displaynames and SMTP addresses that are duplicated.

Copy the following into Notepad and save it as Get-Dupe-Recipients.ps1 to somewhere like C:\Scripts:

$dupes = @{}
Get-recipient -resultsize unlimited |
  Select PrimarySmtpAddress,displayName |
   foreach {$dupes[$_.PrimarySmtpAddress] += @($_.DisplayName)
   }

$dupes.keys |
 foreach {
         if (($dupes[$_]).count -gt 1) {
          $_
          $dupes[$_]
          "`n"
          }
        }


What the script does is uses a hash-table to find any SMTP address that is greater than than 1 for any recipient.

Once you have the .ps1 saved, fire up the Exchange Management Shell (EMS) and cd to the location of the .ps1 like so:

cd C:\Scripts

Then, run the following:

Get-Dupe-Recpients.ps1 | Out-File C:\Temp\RecipDupes.txt

**Note** You can change the Out-File location to where ever you want.

Now, you have list of all duplicate recipients...go clean 'em up!

Saturday, September 30, 2017

Exchange Veeam Backup Error Code 1935 Cannot Connect to Administrative Share

I use Veeam to backup my Exchange 2016 environment because (most importantly) it works with an IP-less DAG, and it's relatively quick and easy to do restores.

The issue is: it doesn't tie into Windows as well as something like DPM, so there's a bit of permission tuning you have to do; especially if you have a Resource Forest with a Selective Trust like I do.

We recently created a new dedicated backup account, but this account is located in the Accounts Forest, which means it has to authenticate over to the Resource Forest where Exchange lives.

Because of this, Veeam started throwing the following error:

Failed to prepare guest for hot backup. Error: Failed to connect to guest agent. Errors: 'Cannot connect to the host's administrative share. Host: [EXCH1.exchangeitup.com]. Account: [exchangeitup\exchbackup2]. Win32 error: The computer you are signing into is protected by an authentication firewall. The specified account is not allowed to authenticate to the computer. Code: 1935 Cannot connect to the host's administrative share. Host: [10.10.10.1]. Account: [exchangeitup\exchbackup2].

Deciphering the error:

The backup account doesn't have "Allowed to authenticate" rights on the actual Exchange server(s).

The Fix:

Follow my previous post to create a security group to allow Selective Trust auth:

http://exchangeitup.blogspot.com/2017/04/exchange-resource-forest-creating.html

After you have that set, add that backup account to your new group.

Or a messier way, messier because I don't like adding single users for permissions (use groups, your fellow IT admins will thank you later) you can just add auth rights directly:

1. Log onto a domain controller in the forest where your Exchange servers are homed
2. Open Active Directory Users and Computers (ADUC)
3. Click View 
4. Select Advanced Features
5. Browse to the OU where the Exchange Server(s) you are trying to authenticate to
6. Right-click then select Properties, then the Security tab
7. Add the backup user
8. Grant Allowed to authenticate rights
9. Click Apply, then OK.

Now your backups should run without auth errors
 

Saturday, September 9, 2017

Exchange Database Failovers and MSMQ Event ID 2250

My databases had been automatically switching over from one of my three Exchange 2016 CU4 DAG nodes, several times a day. I would move them back and they would fail back over almost instantly.

I started checking the Event Logs, and there were tons of Event ID 2250 and Event ID 2252 in the Application Log that coincided with the failover timestamps in the Exchange/High Availability/Operational Logs. This only happened on the one Mailbox Server though...the other two servers were running like champs.

The full Event IDs are:

MSMQ Event ID: 2250

Message Queuing will not be able to accept messages temporarily because system paged pool is low. During this period, machine quota will be set to 0. No manual intervention is required at this stage. Once memory utilization has normalized, Message Queuing will automatically resume accepting messages.

MSMQ Event ID: 2252

Message Queuing is resuming accepting messages because the system memory usage has normalized.

**Note** The 2252 event would happen directly after the databases moved off of this server.

Googling those events turned up nothing that really helped. As you can see the 2250 states that the page pool was exhausted, which was not the case, and my pagefile is set to preferred architecture (RAM + 10MB in my case 32,778MB).

Since it was only this one server, it had to be some sort of communication error.

So I did the normal Test-ReplicationHealth, Get-MailboxDatabaseCopyStatus and I noticed that PowerShell was throwing tons of errors like "MBX1 does not exist on DC1" and WinRM errors every time I would run a cmdlet.

That led me check the Domain Controllers and sure enough DCDIAG from Exchange-to-DC and DC-to-DC came back nasty with errors for the DC holding the FSMO roles.

The Cause:

Another domain admin was monkeying around with the firewall and GPOs on the DC's causing all kinds of network issues.

DO NOT turn off the Windows Firewall on production servers, and check your GPO scopes before enabling them!!!!!

The Fix:

1. Take away admin rights from everyone else :)

2. Reboot the DC and check DCDIAG to ensure it was clean...it is now.

3. Reboot the crippled Mailbox server
    - The errors were gone before the reboot, but just to make sure, I bounced it anyway.

**Note** Rebooting the FMSO DC will result in mailbox database dismounts. This is because the FMSO DC holds the AD configuration container for the databases.They should remount cleanly after the DC is back online, but you should do this during off hours if you can, since it will affect client connections to Exchange.

Now your databases should stay put according to their activation preference and those MSMQ events will be gone.

Saturday, August 19, 2017

Exchange External Forward 550 5.7.54 Unable To Relay In Non-Accepted Domain

I recently had this problem where an Accepted Domain in my Exchange environment would get a bounce when mailing a Shared Mailbox that was set with an external forward. In case you have the same setup as me, I've found a fix.

Consider this scenario: you have two Exchange Resource Forest us.domain.com and eu.domain.com.
The eu.domain.com is set as an Internal Relay Accepted Domain in the us.domain.com Exchange environment.
 
The us.domain.com has a Shared Mailbox, which forwards to an external email address with the ForwardingSmtpAddress switch like so:
 
Get-Mailbox "Shared Mailbox Name" |fl *forw*
DeliverToMailboxAndForward : True
ForwardingAddress          :
ForwardingSmtpAddress      : smtp:externalemailaddress@externaldomain.net

 
When users from the eu.domain.com Resource Forest send a message to the Shared Mailbox in the US, they get the following error:
 
The email to the following recipient(s) could not be delivered:
externalemailaddress@externaldomain.net
The remote mail server told: 550 5.7.54 SMTP; Unable to relay recipient in non-accepted domain

All other external senders (people from outside the company) can send and messages get forwarded successfully.
 
Setting the forward by using the ForwardingAddress switch and creating a MailContact causes all senders from both the other forest and external to fail; so you must keep the ForwardingSmtpAddress set.
 
A brief overview of ForwardingSmtpAddress vs ForwardingAddress:
 
ForwardingAddress:
 
This is a RecipientIdParameter value which has a higher priority than ForwardingSmtpAddress; meaning if you set the parameter ForwardingAddress, other forwarding settings will be overritten. This setting does not require the Set-RemoteDomain -AutoForwardEnabled, it does require an external MailContact in Exchange.
 
ForwardingSmtpAddress :
 
This is a msExchGenericForwardingAddress AD attribute. It has lower a priority than ForwardingAddress and is not accessible in in the EAC; you must use PowerShell to set it. You must also use the Set-RemoteDomain -AutoForwardEnabled $True cmdlet to allow forwarding, but it does not require a MailContact in Exchange.

The Fix:

We'll need to create a dedicated Send Connector to the domain for our external forward.

In the EAC, navigate to Mail Flow, Send Connectors, +

In the New Send Connector window, give it a name like "External Forward" and click Next

 
Create Send Connector


Leave MX record selected and click Next

Send Connector to MX


Click the + and under the FQDN type the domain name for the external contact, click Save, and Next

Send Connector Domain


For the Source Server, click the + and select your Edge server if you have one, or your Mailbox servers (all of them) and click Ok, then Finish

Send Connector Source Server

Enable Verbose Logging on the Connector:

You'll want full logging on the connector so you can check the SMTPSend protocol logs later to verify successful sending.

In the Exchange Management Shell (EMS), run the following:

Set-SendConnector "External Forward" -ProtocolLoggingLevel Verbose

Optional:

If the external domain requires it, you'll need to enable forced TLS, else messages will be dropped.

In the EMS run the following:

Get-SendConnector "External Forward" | Set-SendConnector -RequireTLS $true

Now test! Have someone from the other Resource Forest send to the Shared Mailbox and have an external sender send a message the Shared Mailbox and verify that it forwards by using the protocol logs.

**Note** Depending on what source server you used (Edge or Mailbox) that's where you'd check the SMTPSend protocol logs.