Mass export from exchange to archiver
|
Logged in as: Guest
|
|
Users viewing this topic:
none
|
|
Login  | |
|
Mass export from exchange to archiver - 13.Dec.2006 4:50:01 AM
|
|
|
nicow
Posts: 34
Joined: 7.Mar.2004
From: Groningen
Status: offline
|
Hi,
I want to implement MA to reduce my exchange DB size. Therefore I need to export all my old e-mail to MA. I understand I need to use eewiz to do this. However, we have over 2.000.000 e-mails in a 70GB database. I see a lot of problems coming my way if I want to export this to, of all possibilities, text files and importing them back into a sql database again.
Sounds to me like one hell of an operation. Isn't there another way to accomplish this?
Thanks!
|
|
|
|
RE: Mass export from exchange to archiver - 13.Dec.2006 6:54:51 AM
|
|
|
Mark Busuttil
Posts: 4836
Joined: 16.Oct.2005
Status: offline
|
Hi Nicow,
The process of Exporting from Exchange and Importing into the GFI MailArchiver is very simple. The basic setup is as follows:
1) You configure the GFI Import Service with Databases which will store your exported Emails from EEWIZ.
2) Let us say you have 3 databases, 2004, 2005, 2006.
3) You run the EEWIZ, and it will populate the desired Mailboxes and folders within Exchange that you wish to Export.
4) EEWIZ will extract those emails from Exchange and archived to the appropriate databases based on the Date of when the email was sent.
Currently, this is the only way that you are able to Import Emails from Exchange into GFI MailArchiver.
However, can you please clarify what problems you might encounter?
Thank You!
_____________________________
Regards,
Mark Busuttil
GFI Software Ltd - www.gfi.com
Messaging, Content Security & Network Security Software
GFI: MailEssentials - MailSecurity - MailArchiver - FAXmaker - LANguard – WebMonitor
|
|
|
|
|
RE: Mass export from exchange to archiver - 13.Dec.2006 9:40:29 AM
|
|
|
nicow
Posts: 34
Joined: 7.Mar.2004
From: Groningen
Status: offline
|
Hi Mark,
Thank you for your answer. I didn't notice the pickup folder of the importer. This simplifies most of the process.
However, import goes very slow. I am using this setup:
1 mailserver
1 sqlserver, with mailarchiver service, importing mail from the mailserver and from the shared pickup folder
1 client
I am running the eewiz on the client, importing from the mailserver, exporting to the shared pickup folder. This goes at a rate of 2 emails per second, so in my situation it would take about 1100000 seconds to import all e-mail (if nothing goes wrong), about 12 days...
What would be the fastes setup? I prefer not to run eewiz on the sqlserver, since it is not "mail enabled".
Thank you,
Nico
|
|
|
|
|
RE: Mass export from exchange to archiver - 13.Dec.2006 10:53:51 AM
|
|
|
Mark Busuttil
Posts: 4836
Joined: 16.Oct.2005
Status: offline
|
In order to reach maximum performance, we would recommend you to ensure that you have configured your Autogrowth Settings correctly.
Please ensure that you have the following AutoGrowth Settings:
i) Database File - 100 MegaBytes
ii) Transaction Logs - 10 MegaBytes.
Please check if there is an overall increase in performance once applied the settings above.
Thank You!
_____________________________
Regards,
Mark Busuttil
GFI Software Ltd - www.gfi.com
Messaging, Content Security & Network Security Software
GFI: MailEssentials - MailSecurity - MailArchiver - FAXmaker - LANguard – WebMonitor
|
|
|
|
|
RE: Mass export from exchange to archiver - 17.Dec.2006 3:48:43 PM
|
|
|
nicow
Posts: 34
Joined: 7.Mar.2004
From: Groningen
Status: offline
|
Hi,
Changing the autogrowth settings didn't make a difference. I think it is because of mailsecurity in vsapi mode. I set it in gateway mode now, hope that helps.
Anyway, the MA store crashed last night so the importer grew a nice queue:
2006-12-17,21:41:38,521,1,"#000015F4","#0000000A","info ","Enveloper","Process: starting base processing"
2006-12-17,21:41:38,802,1,"#000015F4","#0000000A","info ","Enveloper","Process: max queue size exceeded, skipping processing"
2006-12-17,21:42:23,928,1,"#000015F4","#0000000A","info ","Enveloper","Process: max queue size exceeded, skipping processing"
2006-12-17,21:42:57,351,1,"#000015F4","#0000000A","info ","Enveloper","Process: max queue size exceeded, skipping processing"
The queue folder contains 181.885 files.. Now what?
Thanks
< Message edited by nicow -- 17.Dec.2006 3:50:29 PM >
|
|
|
|
|
RE: Mass export from exchange to archiver - 17.Dec.2006 4:48:12 PM
|
|
|
nicow
Posts: 34
Joined: 7.Mar.2004
From: Groningen
Status: offline
|
I also get these:
2006-12-17,22:39:47,260,1,"#00000BCC","#00000010","warning","Transfer","warning: Transfer has been interrupted. - UploadError"
2006-12-17,22:40:47,511,1,"#00000BCC","#00000010","info ","Transfer",">>Transfer messages to store [3e0be862-2989-4b8c-8772-e3d550a43490]"
2006-12-17,22:41:52,934,1,"#00000BCC","#00000010","warning","Transfer","warning: Transfer has been interrupted. - UploadError"
2006-12-17,22:42:53,185,1,"#00000BCC","#00000010","info ","Transfer",">>Transfer messages to store [3e0be862-2989-4b8c-8772-e3d550a43490]"
2006-12-17,22:43:26,249,1,"#00000BCC","#00000010","warning","Transfer","warning: Transfer has been interrupted. - UploadError"
2006-12-17,22:44:26,500,1,"#00000BCC","#00000010","info ","Transfer",">>Transfer messages to store [3e0be862-2989-4b8c-8772-e3d550a43490]"
2006-12-17,22:44:58,500,1,"#00000BCC","#00000010","warning","Transfer","warning: Transfer has been interrupted. - UploadError"
2006-12-17,22:45:58,533,1,"#00000BCC","#00000010","info ","Transfer",">>Transfer messages to store [3e0be862-2989-4b8c-8772-e3d550a43490]"
and quite a lot of these:
17,22:47:02,628,1,"#00000BCC","#0000000F","info ","Worker","Message DC1_[1141593468][24][5393].envelope is assigned queue 0[3e0be862-2989-4b8c-8772-e3d550a43490]"
2006-12-17,22:47:02,675,1,"#00000BCC","#0000000F","info ","Worker","Message DC1_[1141593609][24][5394].envelope is assigned queue 0[3e0be862-2989-4b8c-8772-e3d550a43490]"
2006-12-17,22:47:02,675,1,"#00000BCC","#0000000F","info ","Worker","Message DC1_[1141593750][24][5395].envelope is assigned queue 0[3e0be862-2989-4b8c-8772-e3d550a43490]"
|
|
|
|
|
RE: Mass export from exchange to archiver - 18.Dec.2006 3:22:11 AM
|
|
|
josephd
Posts: 151
Joined: 3.Nov.2005
Status: offline
|
Hello,
You can check the ErrorsDal.log in Store\DebugLogs folder for more detailed upload error description? You can also check http://forums.gfi.com/m_900741555/mpage_1/key_/tm.htm#900741627 to see if it applies to your problem.
Regards,
Joseph D'Emanuele
GFI Software Ltd.
|
|
|
|
|
RE: Mass export from exchange to archiver - 18.Dec.2006 5:16:17 AM
|
|
|
nicow
Posts: 34
Joined: 7.Mar.2004
From: Groningen
Status: offline
|
Had the same entries in my errordal.log as in the above post. I think this fixed my problem, at least MAIS is importing.
I'll keep you informed on the progress
|
|
|
|
|
RE: Mass export from exchange to archiver - 18.Dec.2006 11:25:35 AM
|
|
|
nicow
Posts: 34
Joined: 7.Mar.2004
From: Groningen
Status: offline
|
quote:
I'll keep you informed on the progress
It aint working out well. the importer crashed, so my harddrive ran full again and im missing a lot of e-mail now. So a few questions and some suggestions:
Suggestion 1: I noticed a lot of people would just like to import their complete mailboxstore. Please create an all in tool to do this. without selecting all e-mailfolders and mailboxes by hand one by one. This sounds like a minor adjustment to the current tool.
Suggestion 2: Also, it would be nice if the importer could just insert mail into the datebase by itself, so it will be shure it archived the e-mail, or better, after adding an email resource in the webinterface, just ask to import all e-mails.
question 1: A lot of e-mail was exported to the pickup folder as 0 byte files since the hd was full. Are they marked as being imported or are the marked as being imported after the messageID from exchange is registered in the database after the real import? Or for short: can I just run the exporter again to add all the e-mail which is now missing?
question 2: Is there a way to move the queue and pickup folder, since the current disk where their on is just 4GB..
Question 3: It already took me more than 10 hours just for importing some old mail while I assumed this would be an automated proces. If someone could give some procedure, other then selecting 80 mailboxes by hand and monitoring the involved server all day because of my space issues, I would be quite content.
Bug(?) 1: Also, the importer crashed after the hd ran full. Or that is what the eventlog told me (
Unknown runtime execution error: There is not enough space on the disk.). It may be by design but I think it should be marked as a bug. I dont see why the importer needs space anyway to keep running...
Conclusion: This all added: it seems to me it is quite a pain in the *ss to import my current emailstore. I think this import tool deservers a big update, since this is one of the first things new customers get confronted with. It is that the day to day archiving works very good and the webinterface is near perfect, otherwise i would have left this package after i found out about how to import e-mail.
Thanks for any feedback
|
|
|
|
|
RE: Mass export from exchange to archiver - 21.Dec.2006 8:01:45 AM
|
|
|
nicow
Posts: 34
Joined: 7.Mar.2004
From: Groningen
Status: offline
|
quote:
ORIGINAL: nicow
question 1: A lot of e-mail was exported to the pickup folder as 0 byte files since the hd was full. Are they marked as being imported or are the marked as being imported after the messageID from exchange is registered in the database after the real import? Or for short: can I just run the exporter again to add all the e-mail which is now missing?
I just restarted the whole process now. But maybe the eewiz should verify the written messages or something... I just started so I could delete my databases, other may not be so lucky.
quote:
question 2: Is there a way to move the queue and pickup folder, since the current disk where their on is just 4GB..
I should have searched the forum. Found it my anwser here http://forums.gfi.com/fb.aspx?m=900732899 Moving those folders kinda solved all my problems.
quote:
Question 3: It already took me more than 10 hours just for importing some old mail while I assumed this would be an automated proces. If someone could give some procedure, other then selecting 80 mailboxes by hand and monitoring the involved server all day because of my space issues, I would be quite content.
Tried some other things, just using the eewiz still seems to be the best option.
|
|
|
|
|
RE: Mass export from exchange to archiver - 21.Dec.2006 6:50:08 PM
|
|
|
milanod
Posts: 10
Joined: 5.Dec.2005
From: Delaware
Status: offline
|
nicow,
I feel for ya.
I do not agree with Mark. The import is not simple.
It took me 7 days and multiple crashes / errors / lost emails / duplicates before it finally finished my 40GB of mails, not to mention dozens of complaints from my users of mail server coming to a halt. I spent 2 weeks on and off at night (every night) starting / stopping the importer, restarting services, etc..
I started the project on Dec 6. and am still not finished tidying things up.
I agree, the importer needs a HUGE upgrade.
Here are my tips:
Dont do it during peak times.
Shut off all services that arent needed.
shut down any A/V programs
Make sure you have plenty of free space for the binaries.
GFI: please look into upgrading this import tool to account for the various issues above.
One thing that should really be needed, is to make the importer a complete seperate utility, designed to run on a sep machine.
|
|
|
|
|
RE: Mass export from exchange to archiver - 28.Dec.2006 1:51:26 PM
|
|
|
nicow
Posts: 34
Joined: 7.Mar.2004
From: Groningen
Status: offline
|
quote:
Here are my tips:
Dont do it during peak times.
Shut off all services that arent needed.
shut down any A/V programs
Make sure you have plenty of free space for the binaries.
Thanks. Here are some of mine:
- put the pickup folder, the queue folder and the archive stores on seperate disks. In my case I hadn't enough disks so I used a networkdrive. This speeded things up with about 500%
- if you run mailsecurity, put it in gateway mode, or else it wil scan all emails you export
- set logging mode to simple while exporting. Saves a lot of space. Transaction logs tend grow faster than the database...
- I already spoiled one try while exporting, so a lot of email was marked as being exported already. After that: extract mail in chuncks of about 3 a 4 GB, the exported tends to crash every now and then, so you probably have no clue where you left, so if you do small part (I did "all usernames starting with A to C" for example) you can resume more easily. Also export to a seperate folder, so you don't start importing before the export is finished
|
|
|
|
|
RE: Mass export from exchange to archiver - 30.Dec.2006 3:59:54 PM
|
|
|
iatros56
Posts: 2
Joined: 30.Dec.2006
Status: offline
|
Hi guys,
I'm brand new at this. When I used the PST - Exchange Export Tool it dumped a bunch of txt files into my eewiz folder.
How do I import them into a store that I have already created, or did I already mess up...LOL?
Thanks.
Mike
|
|
|
|
|
RE: Mass export from exchange to archiver - 2.Jan.2007 4:41:34 AM
|
|
|
Mark Busuttil
Posts: 4836
Joined: 16.Oct.2005
Status: offline
|
What version of GFI MailArchiver are you currently using?
_____________________________
Regards,
Mark Busuttil
GFI Software Ltd - www.gfi.com
Messaging, Content Security & Network Security Software
GFI: MailEssentials - MailSecurity - MailArchiver - FAXmaker - LANguard – WebMonitor
|
|
|
|
|
RE: Mass export from exchange to archiver - 2.Jan.2007 12:11:04 PM
|
|
|
RGagne
Posts: 129
Joined: 21.Jun.2005
From: Memphis, TN
Status: offline
|
Hey all,
Just a reminder, you can run EEWIZ with the /iepf switch and this will ignore any flags set by previous attempts at importing so that all emails get imported regardless of whether or not the emails had been attempted to be imported before.
|
|
|
|
 New Messages |
 No New Messages |
 Hot Topic w/ New Messages |
 Hot Topic w/o New Messages |
 Locked w/ New Messages |
 Locked w/o New Messages |
|
 Post New Thread
Reply to Message
Post New Poll
Submit Vote
Delete My Own Post
Delete My Own Thread
Rate Posts |
|
|
Printable Version
