GFI Software Aurea SMB Solutions


Home » GFI User Forums » Kerio Connect » Suddenly Failing Backups
Suddenly Failing Backups [message #143895] Mon, 17 September 2018 02:45 Go to next message
El-Man is currently offline  El-Man
Messages: 11
Registered: October 2011
Location: New Zealand
Hello, all.

A client's Kerio Connect server is suddenly failing to run its backup consistently. The Differentials work now and again, but there hasn't been a successful Full backup for a long time now. Here are the relevant bits from the logs:

Error Log:

[16/Sep/2018 16:10:04] backup.cpp: Cannot write file C:\Program Files\Kerio\MailServer\store/mail/<redacted>/#msgs/0000009e.eml (file size 13.40 MB) to backup ZIP file \\BoundaryNAS\Public\Keriobackup/F20180915T181003Z

[16/Sep/2018 16:10:06] backup.cpp: BACKUP: Backup failed in folder <redacted>, 3632865 MB (3632865 total) of 8501032 MB free on destination disk

Debug Log:

[16/Sep/2018 16:10:05][3448] Compression of message store backup finished. 231478 files were added to the archive \\BoundaryNAS\Public\Keriobackup/F20180915T181003Z in 46 volumes.

[16/Sep/2018 16:10:05][3448] {backup} Connection to folder \\BoundaryNAS\Public\Keriobackup is OK

[16/Sep/2018 16:10:06][3448] {backup} Estimate amount of folders were 3691 and amount of really visited folders is 2524.

As you can see, the backups happen to a NAS, which is connected on a gigabit network. You might think that this connection is unreliable, but Connect says it's okay, and there have been no other connection or disk errors. Nor is it running out of space, there's over 3.5Tb free. It just gets up to that stage saying compression is finished, then that the backup has failed, and deletes all the backup files it just wrote out.

Also, the folder it fails on is always different.

Any ideas?
Re: Suddenly Failing Backups [message #143905 is a reply to message #143895] Mon, 17 September 2018 16:20 Go to previous messageGo to next message
Maerad is currently offline  Maerad
Messages: 273
Registered: August 2013
Do you run an antivirus or intrusion detection program on the NAS or Kerio host? If yes, the kerio folder should be whitelisted, because if you open (copy) a file, the realtime scanner might block the mail OR needs more time to scan it and the copy command gets a time out.

Also what does the Windows Event Log and NAS Log say?

Try to copy a part of the store folder with the error mail to the NAS manually, also try to copy a big file and see, if there is a connection issue.

I would suspect something blocking the network transfer (locks the file) or the connection itself is bad (that why copy a big file over).

Another ID would be, that the NAS has some kind of timeout added for shares. Look tru the options.

[Updated on: Mon, 17 September 2018 16:21]

Report message to a moderator

Re: Suddenly Failing Backups [message #143910 is a reply to message #143905] Tue, 18 September 2018 04:47 Go to previous messageGo to next message
El-Man is currently offline  El-Man
Messages: 11
Registered: October 2011
Location: New Zealand
Thanks, I'll look into that and report back.
Re: Suddenly Failing Backups [message #143924 is a reply to message #143910] Wed, 19 September 2018 04:58 Go to previous messageGo to next message
El-Man is currently offline  El-Man
Messages: 11
Registered: October 2011
Location: New Zealand
Okay. In order:

Yes, the mail server runs Eset for Kerio Connect, and there is a built-in antivirus on the NAS. But the store is definitely excluded on the server, and the destination is excluded on the NAS.

Nothing in the Windows Event Logs regarding connection or privilege at the time of the backups.

I copied 5x 2Gb files to the NAS, and it worked perfectly. However, when I copied one of the mail store folders (70k files, 45Gb) to the NAS, there was an alert box saying 'Error 0x8007003B: An unexpected network error occurred'. On an .eml file that was 120Kb. Pressed Try Again and it carried on fine.

Oh, lord, I hate these Windows code errors - there's hardly ever a definitive fix.

I need to narrow this down further, so I'll change the destination to somewhere else. That will at least tell us if the problem is with the NAS or the network. If it still doesn't work, I'll test a USB disk in the mail server.
Re: Suddenly Failing Backups [message #143930 is a reply to message #143924] Wed, 19 September 2018 11:22 Go to previous messageGo to next message
Maerad is currently offline  Maerad
Messages: 273
Registered: August 2013
Before you try too much, disable the antivir on your NAS and on the server (if you have eset for kerio, there shouldn't even be an option to disable a special folder...) and try coping the folder again with the load of small files.

Also I would check, if you can disable additional services on the NAS, do a firmware update and reboot it. It could be, that there are too many small files and the NAS gets overloaded.

But first - try with disabled antivirus. This is in most cases the main culprit. Also you could copy the files to a usual windows share - that could give you an idea, if the NAS has problems.
Re: Suddenly Failing Backups [message #143962 is a reply to message #143930] Sun, 23 September 2018 17:03 Go to previous messageGo to next message
El-Man is currently offline  El-Man
Messages: 11
Registered: October 2011
Location: New Zealand
Turned off the Antivirus on the NAS, but the full backup still failed. Will try updating its firmware next.
Re: Suddenly Failing Backups [message #145923 is a reply to message #143962] Fri, 07 June 2019 09:29 Go to previous messageGo to next message
dzjowk is currently offline  dzjowk
Messages: 6
Registered: August 2008
Hi El-Man,

We have been experiencing the same issue for several weeks now. Rebooting, updating, ... on both ends did not solve the issue.
Are you still experiencing failing backups, or what did you do to fix ?
Re: Suddenly Failing Backups [message #145943 is a reply to message #145923] Thu, 13 June 2019 10:44 Go to previous messageGo to next message
zebby is currently offline  zebby
Messages: 118
Registered: March 2009
We also saw this where it was struggling to save a 2.5GB file on a drive with 137GB free. In our case it was only when doing full backups.
We reported but no solution was found so in the end I just created VB script to delete the contents of the backup directory and scheduled it to run before the KC full backup ran.
Since we implemted this we've had no issues.
Re: Suddenly Failing Backups [message #145950 is a reply to message #143962] Thu, 13 June 2019 16:02 Go to previous message
Maerad is currently offline  Maerad
Messages: 273
Registered: August 2013
El-Man wrote on Sun, 23 September 2018 17:03
Turned off the Antivirus on the NAS, but the full backup still failed. Will try updating its firmware next.
Another thing that came to mind - copy the folder again, this time disable windows defender and any kind of antivirus on the client system or server.
Also disable indexing for the network share, if it's enabled.

How is the NAS itself integrated into the domain? COnnected to the AD?

I run the backup on a windows server 2012 r2, backing up to a server 2012. I only gave rights to the machine itself to open the shares, no user is allowed (in case someone has a encrypt trojan). If you allow access, maybe public reading, there might be some tool or whatever also accessing the folder, thus creating a problem.

Also - what NAS do you have? How's the RAM/CPU load on it, if you copy the folder itself (many small files etc.)? Did you disable every service on the NAS aside from what you really need?

The windowserror you had tells us there's a problem with the network, share or nas. Not with the server or kerio. I guess it could be, that the NAS can't do all the files so fast, so the share goes on timeout for a bit. Also search the NAS options for anything like timeout, energy save or whatever, that could explain this behavior.
Previous Topic: estimated stable date 9.2.10
Next Topic: Filter Autoreply Signature Setting
Goto Forum:
  


Current Time: Fri Jul 19 23:02:24 CEST 2019

Total time taken to generate the page: 0.03805 seconds