Exchange c1041721 Information Store error

We had a strange problem today with our main Exchange Server. The server is running Exchange 2003, Windows Server 2003 and on decent HP hardware. We noticed the problem when we had a call from Administration to say that emails weren’t leaving their Outbox in Outlook. After experiencing the same problem from our machines we restarted the SMTP service on the Exchange Server. This had no effect on the problem so we looked at the Information Stores in the Exchange System Manager, and received  a popup with something to the effect that the IS wasn’t running and the error code c1041721.

After checking that all auto-start services were running we scoured the Event logs and couldn’t find anything relating to Exchange and the problems we had. We thought that the issue began between 13:12 and 13:15 by talking to a few staff members and checking when email stopped flowing through the server.

After some quick Googling the consensus was to try restarting the Information Store service, even though it was running in the task manager and occupying RAM and CPU. The service timed out during the shutdown and went into an uncontrollable state so we killed the process for store.exe. We were able to start the Information Store service again, without issue and without a reboot. Our Outlook Exchange connections appeared happier, with some machines stating that the connection had been lost and restored, and email flowed immediately.

After the Information Store server started there were some interesting messages in the Event Viewer:

ExchangeSA EventID 9175 where the MAPI call OpenMsgStore failed with the following error: Microsoft Exchange Server computer is not available. Either there are network problems or the Exchange Server is down for maintenance. MAPI Provider failed. Microsoft Exchange Server Information Store ID no: 8004011d-0506-00000000

VSS EventID 8194 Volume Shadow Copy Service error: Unexpected error querying for the IVssWriterCallback interface. hr = 0x800706ba.

MSExchangeIS EventID 9665 The memory settings for this server are not optimal for Exchange.

ESE Backup EventID 905 Information Store (247004) Server registered: Microsoft Exchange Server / Microsoft Information Store (callback DLL mdbrest.dll, flags 0x103).

ESE EventID 300 Information Store (247004) BG Staff Mail: The database engine is initiating recovery steps.

ESE EventID 301 Information Store (247004) BG Staff Mail: The database engine has begun replaying logfile e:\Program Files\Exchsrvr\Admin Staff\E01.log.

ESE EventID 302 Information Store (247004) BG Staff Mail: The database engine has successfully completed recovery steps.

The VSS Event 8194 had been logged 5-6 times in a row before the ESE recovery events.  The Exchange Server runs ShadowProtect for server snapshots and takes an incremental backup every 15 minutes during the day. The last snapshot had been taken just before midday and a snapshot should have initiated around the time the Exchange IS went, effectively, offline.

Since starting the Information Store service Exchange seems to be running as normal. Although I am presently running the Data Protector backup for Exchange before resuming ShadowProtect backups. While I don’t think ShadowProtect was directly responsible for the problem, I think there was something happening between ShadowProtect/VSS/Exchange that went pear shaped and hung the Information Store? Googling “c1041721” and ShadowProtect or VSS brought up basically no results, I would be interested to hear from anyone else who has had this error, or something similar, come up.

Update: The Data Protector Exchange backup completed successfully and I restarted the ShadowProtect backups and it picked up from the last incremental backup without any problems.

Daylight Saving, ProCurve and Time Synchronisation

Setting the new start and end dates for day light saving on your switches before daylight saving started would have been excellent planning and forward thinking. However, if your like me and didn’t remember until you rolled into work on Monday morning, three weeks ago, then here are the commands to get your switches back under control.

If you have a time server on your network you can set your switches to sync with it, otherwise you can set the switches to sync with an external time server if your firewall allows.

switch# conf

switch# sntp server ntp.cs.mu.OZ.AU

switch# timesync sntp

switch# sntp unicast

switch# time daylight-time user-defined begin-date 5/10 end-date 5/4

switch# wr mem

If you have a time server on your network, substitute ntp.cs.mu.OZ.AU from line 2 with the name or IP of your time server

Error tracking can be tedious at the best of times, but if the time on your network and switches is correct and consistent across devices then the job is easier for you and your team.

“…. I got a fever”

Well, after finding all sorts of great information from technical blogs everywhere to help me with work, research or whatever, the time has come to cowbell2contribute and try and give something back.  I hope that in the coming months this blog will contain useful information and How-to’s on ProCurve, Blade and Proliant HP equipment, Vista, Exchange, Active Directory, VMware and other network or server equipment that might be useful. 

We have just finished the budgeting process for 2009 and, pending approval on some items, have an idea of the projects that I will be part of over the next year or so. This means I have the opportunity to document and share our migration to Exchange 2007 and Office Communication Server, the  rollout of HP’s ProCurve Identity Management (IDM), Network Access Control and any other interesting projects I stumble across.

Over the next few days I’ll update the About page with my contact details and a blurb about what I do and the kind of things I’m interested in.

 

I have to give props to the man responsible for inspiring/motivating me to produce this blog, the great man, Lachlan Hardy.

If you’ve seen the video from SNL, you’ll understand what I mean when I say,

“Guess what? I got a fever! And the only prescription.. is more cowbell!”

 

In the meantime I’ll get busy and finish some of the half written posts that I’ve started…

– Greg