Domino server crashing frequently

During the last two weeks, my Domino server (v7.0.2) has been crashing. Sometimes daily, sometimes every other day. Receive the following error message when restarting the Domino Server:
"(servername) has faulted and is now back up and running".
I checked the nsd log but I am not sure what I am looking for; as to the cause of the crash. I ran the nfixup.exe and it seemed to fix things for a couple days, but crashed again. I have several users with mail databases over 10GB. Could this be the cause of the crashes?

Any help will be appreciated!
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Sjef BosmanGroupware ConsultantCommented:
There are a few potenial causes for this behaviour.

1/ most likely it is one of the many databases on the system, with a corruption somewhere. The crash info can be very interesting, it might supply the necessary clues as to which task was active and with what database.
2/ it might also be a system database, e.g. the router uses one that's called, or These databases are often used and are a known source of trouble, albeit rare. You could stop the Domino server, rename all files mail*.box, and restart the server. The necessary databases will be re-created by the Router task.
3/ last but not least: upgrade!
deelew41Author Commented:
Thank you for your reply @sjef!
1. I will try and check the crash info and see if I can determine the cause of the crash.
2. If not, I will try and re-create the necessary databases.
3. I am in the process of getting information on how to complete the upgrade. I know it needs to be done as I cannot get support for IBM for v7.0.2!!! Are you familiar or could you provide instructions on how to upgrade from 7.0.2 to 8.5.3?
Sjef BosmanGroupware ConsultantCommented:
About #3: I just read your other question :-)
Acronis True Image 2019 just released!

Create a reliable backup. Make sure you always have dependable copies of your data so you can restore your entire system or individual files.

Francois KoutchoukCTOCommented:
We had the same problem with full-text index on mail files larger than 10G on 7.0.2..  
Deleted the FT and the server stayed up.  Of course the users are not happy.
Upgrading to 8.5.2 did not fix the problem, by the way, still crashes.
So users now have a replica copy of their mail files locally, encrypted and full-text indexed.
deelew41Author Commented:
My server is still crashing! I have renamed the file to bad and had the server re-create another, I renamed the log.nsf file to bad and had the server re-create the log.nsf file, as well as the ddm.nsf. It ran fine for a couple days and began to crash frequently again. I looked at the nsd log file but could not find anywhere that said the cause of the crash. I am at a loss and need to get this fixed ASAP. I have attached the nsd log file; could someone please take a look and see if you can let me know the cause of these crashes??? I know I need to upgrade the server but I need to get the clients upgraded first. If I can at least have time to upgrade the clients and still have the server run, that would be great. Thanks!!!
Sjef BosmanGroupware ConsultantCommented:
Difficult to tell... As far as I can tell, the SMTP task crashes, while executing some external script:

 [ 1] 0x7c8285ec ntdll+165356 (1f4,927c0,0,4e0dbf4)
 [ 2] 0x77e61c8d KERNEL32+138381 (1f4,927c0,4e0de0c,3)
@[ 3] 0x6018fe17 nnotes._OSRunExternalScript@4+1111 (4e0de0c)
@[ 4] 0x601909c4 nnotes._FRTerminateWindowsResources+980 (1,0,0,4e0e904)
@[ 5] 0x60190d78 nnotes._OSFaultCleanupExt@20+872 (b74a34,0,0,0,0)
@[ 6] 0x60190dd8 nnotes._OSFaultCleanup@12+24 (0,0,0)
@[ 7] 0x6019c822 nnotes._OSNTUnhandledExceptionFilter@4+178 (4e0e904)
 [ 8] 0x77e761b7 KERNEL32+221623 (4e0e904,77e61ac1,4e0e90c,0)
 [ 9] 0x77e792a3 KERNEL32+234147 (0,0,0,0)

That's all I can see. What the external script is I don't know:

IMHO it's a lot better to upgrade the server first, and the clients afterwards. Why do you prefer to do the clients first?
deelew41Author Commented:
I thought I had read somewhere online that it was suggested to upgrade the clients before the server! I guess I may have mis-read the post!!! I am unsure what the external script would be either as the server was running before I started working here. The weird thing is that it ran fine up until the last couple of weeks and then it has been crashing regularly. I was asked to run a command to allow winmail.dat files to open in Lotus on this server (and our other Lotus server) recently. Could that maybe be the problem? Good starting point????
Sjef BosmanGroupware ConsultantCommented:
Oh yes, absolutely, that's a good idea, to see what happens when you undo the modifications for winmail.dat files. You did install something on the server at the time?

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
deelew41Author Commented:
I had to add the following commands:

Set config TNEFKeepAttachment=1
Set config TNEFEnableConversion=1
tell router update config

Would this config be in the notes.ini file? Otherwise, where and how do I go about undoing this modification? I did not install anything on the server otherwise. It is the last thing I remember changing/adding on the server before the crashes started least that is what I remember!!!
Sjef BosmanGroupware ConsultantCommented:
Ah... I checked, and found this document.

See the Restrictions...
deelew41Author Commented:
I found the TNEF commands I added in the notes.ini file. I am going to remove them and see how long the server runs!!! Hopefully I can close this question!!! Thanks for your help!
Sjef BosmanGroupware ConsultantCommented:
Or upgrade to 7.0.2FP9 ;-) ...  or maybe even newer...
deelew41Author Commented:
Yes, I plan on upgrading to at least 8.5.x once I have this issue resolved and receive the hardware I need (it is on order). I need to upgrade since I have no IBM support for this version!!!
Sjef BosmanGroupware ConsultantCommented:
Really, check all the documentation you can find on the subject, and then upgrade the server first. Quote: "IBM Lotus recommends upgrading servers before clients".

Have a nice weekend!
Francois KoutchoukCTOCommented:
Server first for sure.  
Never tried to run the Winmail.dat conversion on the server because it crashes often on the client... and I'd rather crash clients than servers.
deelew41Author Commented:
Yes, I have unfortunately found that out!!! The article I found said to do it on the server but obviously not!!!

Have a great weekend!
Look out for server console whenever servers starts. check on which nsf it crashes.
it might be possible it is crashing while performing consistency check on file.

hope this will solve your issue.
deelew41Author Commented:
The server has ran without crashing for the last three days!! I believe it had to do with the command I ran to allow winmail.dat files through the server. Once removed, the server has ran fine. Lesson learned!!!
Sjef BosmanGroupware ConsultantCommented:
What lesson exactly? ;-))

Good news, by the way!
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Lotus IBM

From novice to tech pro — start learning today.