| Backups for Large servers? [message #110817] |
Sun, 16 February 2014 23:05  |
mcholdings
Messages: 15
Registered: July 2012
|
|
|
|
Hi all,
We have a LOT of mail. 285 mailboxes, 846G of mail (as of this morning) over 2.7 million files.
Kerio backup can't deal with it - takes over 2 days to complete a backup, and mostly fails.
Rsync is very slow - I can make it work (takes me 5 nights to get a baseline...), but, very slow.
I tried a few custom systems and not succeeding. Anyone else in this position? What are you using? i'm looking to see what others are doing as well as looking in the market.
I've recently moved to a physical machine with the mail stored on raid 10 disks (virtual had too many disk bottlenecks). I am wondering if we have grown too far for the solution.... We are still single server (we only have a single head office\data centre with 20 small branches), but it's pretty fast now (dual Xeon Dell with raid 1 main disk, raid 10 data store, 24G ram which is more than enough...)
|
|
|
|
|
|
| Re: Backups for Large servers? [message #110884 is a reply to message #110875] |
Mon, 17 February 2014 22:19  |
mcholdings
Messages: 15
Registered: July 2012
|
|
|
|
|
I backup to a dedicated NAS. It doesn't seem to be the actual volume of data, but the number of individual files. I have an identical NAS that does the backups for my other servers, and it takes 100's of gigs a day no problem. The issue seems to be writing 2.7 million files....
|
|
|
|
|
|
| Re: Backups for Large servers? [message #110913 is a reply to message #110911] |
Tue, 18 February 2014 18:02 |
Machete
Messages: 187
Registered: February 2012
Location: United States
|
|
|
|
I've seen the problem mcholdings is describing - transferring/saving large amounts individual files is much slower than fewer large files. I've never found a good solution when I have the problem and just attribute it to read/write the start/end of each file. But I'm no expert to know why it's slower.
Taking HoosierMac's idea - can you connect your NAS direct to the server? I'm pretty sure the most recent Xeon box I got from Dell had a SATA channel. Some NAS devices have USB3.0 as well on them for direct connectivity.
|
|
|
|
| Re: Backups for Large servers? [message #110916 is a reply to message #110817] |
Tue, 18 February 2014 19:17 |
 |
sfpete
Messages: 102
Registered: June 2007
Location: San Francisco, CA
|
|
|
|
Agreed x3.
You'll want your local backup going to fast direct attached storage to maximize speed and minimize the window.
And then of course back _that_ built-in kerio backup data to your preferred backup system.
|
|
|
|
|
|
| Re: Backups for Large servers? [message #111025 is a reply to message #110919] |
Fri, 21 February 2014 01:12 |
dpswatson
Messages: 3
Registered: October 2013
Location: United States
|
|
|
|
I'm kind of in the same boat with a similarly-sized message store. I gave up on the built-in Kerio backup method because it took too long and caused too much of a performance hit (plus the only way to interrupt it is to restart the Kerio server)
I'm currently using an SSH-based backup method from another linux VM over a private LAN link, which seems to get good backups most of the time (and it doesn't kill performance), but it still takes far too long, IMHO.
So, after you get your rsync baseline, does it stay in sync OK? I'm thinking of switching to using rsync between two servers running on different boxes, and backing up the second box only. That would also make the second box viable as an offline spare, eliminating the need to restore if the primary server failed.
|
|
|
|
| Re: Backups for Large servers? [message #111026 is a reply to message #111025] |
Fri, 21 February 2014 01:24 |
mcholdings
Messages: 15
Registered: July 2012
|
|
|
|
Now I have my baseline, it is staying in sync (almost gigabit link to a remote site via 24Ghz Wireless). It did take 3 days to get the baseline right though.
I think the issue is 'files', rather than space. I can transfer a terrabyte over the link no problems in a short time - it is the simple volume of files being stored.
I'm looking also at a disk based imaging backup - which in theory removes the files barrier. I don't think local storage would be much faster than rsync - I'm not touching my bandwidth limits, rather disk throughput.
A note though - I was only raid 1 before - now I am raid 10 it is much better for this size store.
|
|
|
|
| Re: Backups for Large servers? [message #111744 is a reply to message #110817] |
Mon, 17 March 2014 20:39 |
RoadKingRick
Messages: 3
Registered: October 2008
|
|
|
|
I would like to weigh in here. If it helps one person.......
When our store was just about 1 TERABYTE we were backing up via KMS to a single external eSATA 3GB/s drive and the full backup was taking well over two days.
We switched to a 6GB/s internal eSATA card with an external fast eSATA RAID5 and the first full backup took 23 hours! Less than HALF the time.
Our store has crept up to 1.5 TB and the backup is taking longer, well over a day, but the fast eSATA might be a very viable alternative to those with a smaller store.
I can't stress the difference the 6GB/s controller card and fast external RAID5 made!
I have not tried USB3 yet, but I (think) I read somewhere that eSATA better handles the particular type of data that KMS backup produces (repeated,2 GB chunks).
Has anyone timed backing up the store to an external SAS LTO tape drive?
|
|
|
|
| Re: Backups for Large servers? [message #111799 is a reply to message #110817] |
Wed, 19 March 2014 09:30 |
giobbi
Messages: 29
Registered: October 2005
Location: Sweden
|
|
|
|
RoadKingRick; Could you post your setup?
I'm in the same position, same amount of files. Since mail is the single most important in any company maybe i should look at more proper hardware with blockbased backup like Netapp (snapshots). This kind of backup only backup the new blocks witch is lightspeed fast. If im not mistaking - new software (DSM 5) from synology has this now (with iscsi?) for a fraction of cost..
|
|
|
|
| Re: Backups for Large servers? [message #111996 is a reply to message #110817] |
Tue, 25 March 2014 18:57 |
MarkK
Messages: 342
Registered: April 2007
|
|
|
|
Are you running a complete backup every time? Or are you doing a base plus incremental?
You may want to take a look at Retrospect's backup product. It is easy to use, the latest version does block level backups, does file level deduplication, can be set to run at any interval (nightly, weekly, every X minutes, etc), and will backup to just about anything. There is a Windows and Mac version, both can backup clients running Windows, Mac, or Linux. Cost is actually very reasonable too.
As others have said, the time factor is complicated by the fact that you are backing up thousands of small files that add up to a lot of data, rather than a few large files. So the fastest drive write connection is going to be of great benefit, whether that be a SATA or eSATA or USB3 connected storage.
|
|
|
|
Members
Search
Help
Register
Login
Home
