We had a strange problem today with our main Exchange Server. The server is running Exchange 2003, Windows Server 2003 and on decent HP hardware. We noticed the problem when we had a call from Administration to say that emails weren’t leaving their Outbox in Outlook. After experiencing the same problem from our machines we restarted the SMTP service on the Exchange Server. This had no effect on the problem so we looked at the Information Stores in the Exchange System Manager, and received a popup with something to the effect that the IS wasn’t running and the error code c1041721.
After checking that all auto-start services were running we scoured the Event logs and couldn’t find anything relating to Exchange and the problems we had. We thought that the issue began between 13:12 and 13:15 by talking to a few staff members and checking when email stopped flowing through the server.
After some quick Googling the consensus was to try restarting the Information Store service, even though it was running in the task manager and occupying RAM and CPU. The service timed out during the shutdown and went into an uncontrollable state so we killed the process for store.exe. We were able to start the Information Store service again, without issue and without a reboot. Our Outlook Exchange connections appeared happier, with some machines stating that the connection had been lost and restored, and email flowed immediately.
After the Information Store server started there were some interesting messages in the Event Viewer:
ExchangeSA EventID 9175 where the MAPI call OpenMsgStore failed with the following error: Microsoft Exchange Server computer is not available. Either there are network problems or the Exchange Server is down for maintenance. MAPI Provider failed. Microsoft Exchange Server Information Store ID no: 8004011d-0506-00000000
VSS EventID 8194 Volume Shadow Copy Service error: Unexpected error querying for the IVssWriterCallback interface. hr = 0x800706ba.
MSExchangeIS EventID 9665 The memory settings for this server are not optimal for Exchange.
ESE Backup EventID 905 Information Store (247004) Server registered: Microsoft Exchange Server / Microsoft Information Store (callback DLL mdbrest.dll, flags 0x103).
ESE EventID 300 Information Store (247004) BG Staff Mail: The database engine is initiating recovery steps.
ESE EventID 301 Information Store (247004) BG Staff Mail: The database engine has begun replaying logfile e:\Program Files\Exchsrvr\Admin Staff\E01.log.
ESE EventID 302 Information Store (247004) BG Staff Mail: The database engine has successfully completed recovery steps.
The VSS Event 8194 had been logged 5-6 times in a row before the ESE recovery events. The Exchange Server runs ShadowProtect for server snapshots and takes an incremental backup every 15 minutes during the day. The last snapshot had been taken just before midday and a snapshot should have initiated around the time the Exchange IS went, effectively, offline.
Since starting the Information Store service Exchange seems to be running as normal. Although I am presently running the Data Protector backup for Exchange before resuming ShadowProtect backups. While I don’t think ShadowProtect was directly responsible for the problem, I think there was something happening between ShadowProtect/VSS/Exchange that went pear shaped and hung the Information Store? Googling “c1041721” and ShadowProtect or VSS brought up basically no results, I would be interested to hear from anyone else who has had this error, or something similar, come up.
Update: The Data Protector Exchange backup completed successfully and I restarted the ShadowProtect backups and it picked up from the last incremental backup without any problems.