Exchange 2013/2016 – Move request to specific server/database is failing with “Database doesn’t satisfy the constraint SecondCopy “

I had one very strange issue with multi DAG environment where move requests to only once specific server were failing. No matter of the Source Database, if the target database is on the affected server the move was is failing with :

Error: Mailbox changes failed to replicate.
Database doesn’t satisfy the constraint
SecondCopy because the commit time 19.09.2017 13:11:15 isn’t guaranteed by
replication time 31.12.9999 23:59:59.

In this article are you can wind more information regarding the Database contains.

There is alos tome reuirements there :

If the DataMoveReplicationConstraint is set to SecondCopy, then for a given replicated database at least one passive database copy must: 1.Be healthy.
2.Have a replay queue within 10 minutes of replay lag time.
3.Have a copy queue length less than 10 logs.
4.Have an average copy queue length less than 10 logs. The average copy queue length is computed based on the number of times the application has queried the database status.

All this were of course OK im case.

I checked replication health with Test-ReplicationHealth and found that the HighAvailability component was trowing errors that it is inactive :

RunspaceId : ac1e9231381-123-132-123123123
Server : server
Check : DatabaseAvailability
CheckDescription : Verifies that databases have sufficient availability. If this check fails, it means that some
databases are at risk of losing service.
Result : *FAILED*
Error : Failures:

servername:
Server ‘server.domain.com’component (HighAvailability) state is offline. If you need
to activate databases copies on this server, you can use Set-ServerComponentState -Component
‘HighAvailability’ -State ‘Active’ and retry Move-ActiveMailboxDatabase.

But get-ServerComponentsState is showing everything active for all servers

Also there was and error in Aplication log:

Log Name: Application
Source: MSExchange Mid-Tier Storage
Date: 28.12.2017 10:27:19
Event ID: 10011
Task Category: (10)
Level: Error
Keywords: Classic
User: N/A
Computer: server.domain.com
Description:
Replication for database MDB1 is not flushed yet. Constraint: SecondCopy, number of copies: 2, minimum replay time: 31.12.9999 23:59:59, commit time: 28.12.2017 10:27:19. Failure reason: Database 828ffdad-2323-4e18-9e5e-123123123 doesn’t satisfy the constraint SecondCopy because the commit time 28.12.2017 10:27: isn’t guaranteed by replication time 31.12.9999 23:59:59.

Accounting to this article there is two places where the Component states are saved:

Local – in the server Registry and remote-in Active directory. configuration name context.

when I get the remote ServerComponent states i see that the serverwideoffilnecomponent is inactive!

$components =Get-ServerComponentState servername -Component serverwideoffline

[PS] C:\Windows\system32>$components.Localstates

Requester State Component
——— —– ———
Functional Active serverwideoffline
Maintenance Active serverwideoffline

[PS] C:\Windows\system32>$components.Remotestates

Requester State Component
——— —– ——— ———
Functional Active ServerWideOffline
Maintenance Inactive ServerWideOffline

After setting the components to active again the issue in the test-ReplicationHealth report got resolved and with this the move request are also not failing any more.

Also the date in the event 10011 is no more 31.12.9999 23:59:59. but something in the near future 🙂

 

 

Exchange 2010 – Public Folder database replication not working “550 5.2.0 RESOLVER.ADR.BadPrimary”

Public folder database replication is not working. Message taking log is showing error:

‘550 5.2.0 RESOLVER.ADR.BadPrimary; recipient primary SMTP address is missing or invalid’;’550 5.2.0 RESOLVER.ADR.BadPrimary; recipient primary SMTP address is missing or invalid’

Issue is caused by wrong proxyAddresses for the  Public folder databases. There is eather two Primary SMTP addresses or wrong Primary SMTP address.

They can be found in ADSI edit.pf

 

Disable Out of Office Autoreplays for all external Recipients in Exchange Server

I spend some time trying to acheave this goal so I decided to share it with you.

Fists I tried to create a Transport rule following this article.

It does not helped.

So i made some research and found that I can disable Autoreply to recipients outside the organisation in Outlook and OWA with Set-mailbox command:

Set-Mailbox  username -ExternalOofOptions InternalOnly

What this does is that it make grayed out the option to set up autoreplay to external organisation in OWA or ECP. But if the Autoreplay is set by the time of this change, it will continue to send Autoreplay messages to external receipents. Autoreplay needs to eb turned off and one again, then the Field for external organisation will be greyed out.

This can be solved by setting the “AllowedOOFType None” for the  Default remote domain in EMS

Set-Remotedomain default –AllowedOOFType None

This is disabling the OOF to all external domains.

If you want to disable it only for specific domains, you have to add every domain to remote domains and then put the value to None for each of them

 

 

 

The source data is corrupted or not properly Base64 encoded when Importing Cetificate in exchange.

You can get error when importing wildcard certificate or certificate from Public certification Authority.

Import-ExchangeCertificate : The source data is corrupted or not properly Base64 encoded.
At line:1 char:27
+ Import-ExchangeCertificate <<<<  -Path “C:\cert\certs\www.mydomain.crt.csr”
+ CategoryInfo          : ReadError: (:) [Import-ExchangeCertificate], InvalidOperationException
+ FullyQualifiedErrorId : 76D5CB03,Microsoft.Exchange.Management.SystemConfigurationTasks.ImportExchangeCertificate

This is happening because the certificate is missing private key.

Import the certificate in Personal store and export it in cer format form mmc. Delete the certificate from mmc.

In Exchange Power shell run :

Import-ExchangeCertificate -FileData ([Byte[]]$(Get-Content -Path “<Path of the certificate>.cer” -Encoding Byte -ReadCount 0))

This imports the certificate in mmc but it will not be visible in ECP or IIS.

Run :

Certutil -repairstore my “xx xx xx 02 03 1b c9 fd c5 40 xx a6 55 0a 91 xx”

Where “xx xx xx 02 03 1b c9 fd c5 40 xx a6 55 0a 91 xx” is the Serial number of the newly certificate.

Now the certificate will be visible in ECP and IIS and you can assign services to it.

 

Exchange 2010/2013 – MSExchange Management Application Error 5000 Events

Error 5000 in application log indicates error saving the Admin audit log. There is many variations depending on the reason for the error. It looks like this :

 

Failed to save admin audit log for this cmdlet invocation.
Organization: First Organization
Log content:
Cmdlet Name: Set-Mailbox
Object Modified:
Parameter: SingleItemRecoveryEnabled = True
Parameter: Identity =domains/xxxxx/Users/xxxxxxxx/xxxxx
Caller: NT AUTHORITY\SYSTEM (powershell)
ExternalAccess: True
Succeeded: True
Run Date: 2016-07-19T10:23:50
OriginatingServer: RZ1GRP01 (15.00.1178.000)

Error:
Microsoft.Exchange.Data.ApplicationLogic.AuditLogServiceException: The Exchange Web Service returned an error while trying to access the audit log. Reason: ‘Error’,’ErrorQuotaExceeded’,’Mailbox has exceeded maximum mailbox size.’.
at Microsoft.Exchange.Data.ApplicationLogic.EwsAuditClient.CallEwsWithRetries(LID lid, Func`1 delegateEwsCall, Func`3 responseMessageProcessor, Func`3 responseErrorProcessor)
at Microsoft.Exchange.Data.ApplicationLogic.EwsAuditLog.WriteAuditRecord(IAuditLogRecord auditRecord)
at Microsoft.Exchange.ProvisioningAgent.EwsAuditLogger.WriteAuditRecord(IAuditLogRecord auditRecord)
at Microsoft.Exchange.ProvisioningAgent.AdminLogProvisioningHandler.WriteAuditRecord(Stopwatch stopwatch)

The Error part in red can be different. Important is to know that for Admin audit logging is used one system mailbox – SystemMailbox{e0dc1c29-89c3-4034-b678-e6c29d823ed9} 

If the Database were this mailbox is located is dismounted, we get error “MapiExeptionMailboxOffline” if hte mailbox is too big you get “‘Error’,’ErrorQuotaExceeded’,’Mailbox has exceeded maximum mailbox size.'” and you have to increase this quotas for this mailbox.

get-mailbox -arbitration -identity “SystemMailbox{e0dc1c29-89c3-4034-b678-e6c29d823ed9}” | get-mailboxstatistics | fl *size*

checked the quota limits on this mailbox:
get-mailbox -arbitration -identity “SystemMailbox{e0dc1c29-89c3-4034-b678-e6c29d823ed9}” | fl *quota*
ProhibitSendQuota : Unlimited
ProhibitSendReceiveQuota : Unlimited
RecoverableItemsQuota : 30 GB (32,212,254,720 bytes)
RecoverableItemsWarningQuota : 20 GB (21,474,836,480 bytes)
CalendarLoggingQuota : 6 GB (6,442,450,944 bytes)
UseDatabaseQuotaDefaults : False
IssueWarningQuota : Unlimited
RulesQuota : 64 KB (65,536 bytes)
ArchiveQuota : 100 GB (107,374,182,400 bytes)
ArchiveWarningQuota : 90 GB (96,636,764,160 bytes)

Use set-mailbox -arbitration -identity “SystemMailbox{e0dc1c29-89c3-4034-b678-e6c29d823ed9}” -RecoverableItemsQuota unlimited -RecoverableItemsWarningQuota unlimited  -CalendarLoggingQuota unlimited  to set the quotas to unlimied.

 

 

Exchange 2013 – getting lots of EventID 1040 – Warnings

Issue – Event log is flooded with getting lots of EventID 1040 – Warnings –

The average of the most recent heartbeat intervals [470] for request [Sync] used by clients is less than or equal to [540].

Make sure that your firewall configuration is set to work correctly with Exchange ActiveSync and direct push technology. Specifically, make sure that your firewall is configured so that requests to Exchange ActiveSync do not expire before they have the opportunity to be processed.

Since the avarage is 470, changed the value of HeartbeatAlertThreshold from 540 to 400. in C:\program files\Microsoft\Exchange Server\v15/Client access\Sync\web.config file

recycled active sync apppool.

Exchange 2010 – OWA and ECP not opening “”

Recently we had an issue with OWA. Page cannot be displayed with Hollowing error:

Error browsing OWA:
<!– Web.Config Configuration File –>

<configuration>
<system.web>
<customErrors mode=”RemoteOnly”/>
</system.web>
</configuration>

Hinweise: Die aktuelle Seite kann durch eine benutzerdefinierte Fehlerseite ersetzt werden, indem Sie das defaultRedirect-Attribut des <customErrors>-Konfigurationstags dieser Anwendung so setzen, das es auf einen benutzerdefinierten Fehlerseiten-URL zeigt.

<!– Web.Config Configuration File –>

<configuration>
<system.web>
<customErrors mode=”On” defaultRedirect=”mycustompage.htm”/>
</system.web>
</configuration>

This issue can be caused by corrupted web.config file in clinetaccess\owa .

Usually in this folder there is some web.bak file which is form the previous version of exchange server. This file can be renamed to web.config for troubleshooting purposes. After replacing this file the issue exist still. We event tooke a config file form working server.

The next step / possible solution was:

Navigated to C:\Windows\Microsoft.NET\Framework64\v2.0.50727\CONFIG

Renamed web.config to .old and renamed web.config.default to web.config.

Ran iisreset in elevated command prompt

The issue still exists.

changed the %temp% and %tmp% variable path to the default Temp folder .

There can be also issues if the installation path for exchange is given with variables %exchangeinstallpath%. but this was not the case.

Since nothing from the steps above do not helped we decided to copy the clientaccess\owa folder from working server to the affected one.

After that OWA virtual Directory was recreated and the OWA can be open again.

Unfortunately it appeared that onli the Administrator can open OWA. We have errors in application log:

error 2280 with the description “The Module DLL C:\Program Files\Microsoft\Exchange Server\V14\ClientAccess\Owa\auth\exppw.dll failed to load. The data is the error”

Found that the authenticated users have only read permissions on the DLL. Granted Read, Read&execute permissions to the dll and on the cline access folder.

After that all users are able to access OWA.