My husband changed his Active Directory password. Routine enough – we’ve got 15k accounts at the office and require a password change every 90 days. That’s\u00a0150-200 people changing their password every day. They get themselves locked out a lot (mobile devices, cached workstation credentials, and a host of other unique places people manage to store their creds), but it’s trivial to unlock an individual user.<\/p>\n
*Except* — after the account was unlocked, his Windows 10 mail client updated properly and was interacting with the Exchange server.\u00a0Android, however, still wouldn’t accept his new password. If he typed the\u00a0wrong<\/em> thing, it would say invalid password. But whenever he typed the\u00a0right<\/em> thing, he got an error indicating the phone and tablet were unable to communicate with the server. Which was bogus — I could\u00a0see<\/em> the communication coming across the reverse proxy server. With 200 codes — although you can have a very successful HTTP call deliver an application error message. But it wasn’t like he couldn’t COMMUNICATE with the server. He turned sync off on the phones to avoid getting locked out again, and in the process of troubleshooting ended up deleting all of his accounts hosted on our Exchange 2013 server.<\/p>\n I looked through all of the\u00a0event logs,\u00a0Exchange logs … nothing interesting. In desperation, I enabled the individual user ActiveSync logging:<\/p>\n Set-CASMailbox mailNickName\u00a0<\/em>-ActiveSyncDebugLogging:$true<\/p>\n Had him attempt to add the mailbox profile again, and dropped the log myself:<\/p>\n Get-ActiveSyncDeviceStatistics -Mailbox mailNickName\u00a0<\/em>-GetMailboxLog:$true -NotificationEmailAddress\u00a0mysmtp@mydomain.ccTLD<\/em><\/p>\n Bingo! An exception in the provisioning (Microsoft-Server-ActiveSync?Cmd=Provision) call — I see the phone information come across, the mobile device gets partially added to his account (no OS, phone number, carrier type information … but\u00a0if you go into OWA and remove the mobile device, an Android device gets added). Error:<\/p>\n Command_WorkerThread_Exception : Now\u00a0that<\/strong> is an error I’ve never seen before. As a programmer, I know what it means … you’ve basically got some components that don’t match another. But … huh? He changed his\u00a0password<\/em>. Connected to the Exchange server directly (instead of remotely viewing logs & files) and saw Windows Update had dropped files and a reboot was pending. Which … some files replaced, others staged for replacement pending a reboot. *That* is some components not matching others. Rebooted our box, and voila … registration goes through, mailbox sync started.<\/p>\n I don’t know how many people allow auto-updates with a manual reboot on a production enterprise server (we manually patch and reboot during a scheduled maintenance window) where this\u00a0could\u00a0<\/em>happen … but evidently Windows Update can get your Exchange server into a state where already configured clients are able to send and receive mail. But clients are unable to update passwords, and new clients cannot be configured.<\/p>\n","protected":false},"excerpt":{"rendered":" My husband changed his Active Directory password. Routine enough – we’ve got 15k accounts at the office and require a password change every 90 days. That’s\u00a0150-200 people changing their password every day. They get themselves locked out a lot (mobile devices, cached workstation credentials, and a host of other unique places people manage to store …<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[30],"tags":[64,65,32,66],"class_list":["post-611","post","type-post","status-publish","format-standard","hentry","category-system-administration","tag-exchange","tag-exchange-2013","tag-technology","tag-windows-update"],"_links":{"self":[{"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=\/wp\/v2\/posts\/611","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=611"}],"version-history":[{"count":2,"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=\/wp\/v2\/posts\/611\/revisions"}],"predecessor-version":[{"id":613,"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=\/wp\/v2\/posts\/611\/revisions\/613"}],"wp:attachment":[{"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=611"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=611"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.rushworth.us\/lisa\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=611"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
\n— Exception start —
\nException type: System.IO.FileLoadException
\nException message: Could not load file or assembly ‘Microsoft.Exchange.Configuration.ObjectModel, Version=15.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35’ or one of its dependencies. The located assembly’s manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
\nException level: 0
\nException stack trace: at Microsoft.Exchange.AirSync.DeviceInformationSetting.ProcessSet(XmlNode setNode)
\nat Microsoft.Exchange.AirSync.DeviceInformationSetting.Execute()
\nat Microsoft.Exchange.AirSync.ProvisionCommand.Microsoft.Exchange.AirSync.IProvisionCommandHost.ProcessDeviceInformationSettings(XmlNode inboundDeviceInformationNode, XmlNode provisionResponseNode)
\nat Microsoft.Exchange.AirSync.ProvisionCommandPhaseOne.Process(XmlNode provisionResponseNode)
\nat Microsoft.Exchange.AirSync.ProvisionCommand.ExecuteCommand()
\nat Microsoft.Exchange.AirSync.Command.WorkerThread()
\nInner exception follows…
\nException type: System.IO.FileLoadException
\nException message: The located assembly’s manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
\nException level: 1
\nException stack trace:
\n— Exception end —<\/p>\n