Bulk Exporting (Full Version)

All Forums >> [Archiving & Fax] >> GFI MailArchiver



Message

richgill -> Bulk Exporting (17.Jun.2008 10:35:07 AM)

I am trying to Bulk Export a lot of email. When I click on the Download button the Bulk Export Tools counts to 1min 40 seconds and then displays the error "Error: Data Access Layer: Query failed......The operation has timed out"

Can the timeout for the Bulk Export Tool be increased?  (I would like to export hundreds of thousands of emails in one go)

Thanks!



jgonzalez -> RE: Bulk Exporting (19.Jun.2008 9:03:41 PM)

I hope you get an answer as I have a similar problem with the bulk export tool.  I am trying to do a mass export and it seems that the bulkexport tool chokes if the number of messages, or total message size, is too great.  I've taken to exporting in small chunks - imagine what a pain that is - because if I try to export more than 4,000-5,000 messages (depending on message sizes) I get a "Bulk Export has encountered an error and must close, you may lose any work you have done" (or something to that effect) and my options are to DEBUG or CANCEL.  Debug does nothing and cancel closes the whole thing.

This is absolutely frustrating and i feel for you since I too will be exporting ~250,000 messages.

BTW, I'm using MailArchiver 5, build 20071109.  Is there a newer version of the Bulk Export?



richgill -> RE: Bulk Exporting (20.Jun.2008 7:19:05 AM)

I have managed to download 145,000 messages in one go.  This is about 20 days worth of email for us. Given that I want to export years worth of data this is rather annoying!!  I don't know why you can only export 5,000 in one go! What time does the counter get to before giving you the error? Mine craps out at 100secs. Do you have a complex search query?



richgill -> RE: Bulk Exporting (20.Jun.2008 7:20:40 AM)

fwiw I am running MA 5 build 20071109



jgonzalez -> RE: Bulk Exporting (20.Jun.2008 7:48:04 AM)

quote:

ORIGINAL: richgill

I have managed to download 145,000 messages in one go.  This is about 20 days worth of email for us. Given that I want to export years worth of data this is rather annoying!!  I don't know why you can only export 5,000 in one go! What time does the counter get to before giving you the error? Mine craps out at 100secs. Do you have a complex search query?

Wow!  I don't see how you do it.  It seems I can't export more than 5,000 messages (on average) without it crashing.  Granted, we do have some messages that contain 20MB attachments, so I think it's the combined size that kills it.  There is no particular time as to when it crashes, it's gone as far as 60% then crashed, but usually it will crash around 5% or so.
We probably average 15-20,000 emails per day, so I'd say we're close to what you're getting.  If I could do 145k in one go I'd be happy.

For the record, our MA is sitting on an HP Proliant with dual dual-core XEON processors and 4GB of RAM.  We are using SQL server and the DB are sitting on their own drives with their own controllers.

Oh, and my complex search query is;

SENT --> AFTER --> 05/01/2008  AND
SENT --> BEFORE--> 05/06/2008

I have to break it up into 4 or 5 days chunks or the likelihood of it crashing is great.



richgill -> RE: Bulk Exporting (20.Jun.2008 9:30:32 AM)

I guess you have a different problem!  I have a similar query running on a similar spec'ed server. I set the dates, click "Find Now" and the results are returned.  When I click "Download" a timer starts in the bottom right corner.  If the downloading starts before the counter hits 1min 40 secs then all is OK and all the emails are saved to disk. If it does hit to 1min40, I get an "Error: Data Access Layer: Query Failed" error.  I was hoping there was a timeout setting in the registry or a file. I will have a look!



richgill -> RE: Bulk Exporting (20.Jun.2008 10:04:41 AM)

I just increased the IIS connection timeout and it seems to have worked.  Needs a bit more testing though to be sure that's it.



John Letourneau -> RE: Bulk Exporting (23.Jun.2008 11:41:19 AM)

Another option for this would be to do the following:

  • Control the SQL query timeout by following this procedure:



1. Open the web.config file from .\MailArchiver\ASPNET in a text editor.
2. Add the following line to <appSettings> section (default value is 2 minutes)

<add key='CommandTimeout' value='value_in_seconds'/>

As 'value_in_seconds' you can put a value of 180 which is equivalent to 3 minutes.

3. Save and close the web.config.
4. Restart the GFI MailArchiver service.



richgill -> RE: Bulk Exporting (23.Jun.2008 11:48:04 AM)

I had already tried that, but the download still times out at 1minute 40 seconds.



John Letourneau -> RE: Bulk Exporting (24.Jun.2008 11:50:33 AM)

richgill,

Did the IIS connection timeout hold up for you?



richgill -> RE: Bulk Exporting (24.Jun.2008 12:14:30 PM)

It seemed to at first, but now I get a timeout at 1min 40s. I'll try a reboot - see if that helps!



meschmidt -> RE: Bulk Exporting (15.Jul.2008 11:47:37 PM)

I am also having this problem with the timeout after 1 minute and 40 seconds.

Is there a fix for this?



richgill -> RE: Bulk Exporting (17.Jul.2008 11:41:40 AM)

No - I just tried again and the program crashed out at 1min 40s.  Any ideas welcome!!



spike711 -> RE: Bulk Exporting (28.Jul.2008 5:37:25 PM)

How are you getting over 5000 emails to export?



spike711 -> RE: Bulk Exporting (29.Jul.2008 12:18:50 AM)

Found over 5000 answer here.
http://forums.gfi.com/m_900763325/mpage_1/key_export%2c5000/tm.htm#900763325



Page: [1]