Forums  Register  Login  My Profile  Inbox  Address Book  My Subscription  My Forums 

Member List  Search  FAQ  Ticket List  Log Out

 

Performance problem

  		Logged in as: Guest
Users viewing this topic: none
  Printable Version
All Forums >> [Archiving & Fax] >> GFI MailArchiver >> Performance problem Page: [1] 2   next >   >>
Login
Message << Older Topic   Newer Topic >>
Performance problem - 12.Nov.2009 3:58:12 AM   
maxdevaine

 

Posts: 9
Joined: 27.Oct.2009
Status: offline 		Hi, i have :

Server : DL-380G5
HDD : 1TB RAID5 - 8xSAS 10K 146GB
Proc : Intel Quad-Core Xeon E5420 2,5GHz

VMWare ESXi 3.5 U4

Windows Server 2003 (use 2CPU Core, 3GiB RAM)
GFIMailArchiver 6.1 SR1 build 2009052
Firebird database (per Month)

GFI is connectid to Exchange 2007 SP1 via IMAP, sync mailbox items via MAPI. We have 250 email mailboxes (280GiB size exchange database).
We are archiving 12000 emails per day (in October 120 000 per Month).

GFI is lagging.
Emails collected in journal mailbox (now cca 20 000 emails) - GFI unequal.
GFI indexing only 60% emails.
In "C:\windows\temp\" collect lot 200-330MiB files (fb_sort_p09pj7, fb_sort_6t9pj7, .... = sum 20-30GiB).

CPU usage is very high.
I have disable attachment indexing, but help only little.

VMWare ESXi Disk statistic :
---------------------------------------------
Disk Read Request (max/average) : 68000 (34000)
Disk Write Request (max/average) : 19000 (3000)

Disk Read Rate (max/average): 28MiB/s (8MiB/s)
Disk Write Rate (max/average): 52MiB/s (12MiB/s)

Physical disk write latency : 3 (0,5) ms
Physical disk read latency : 11 (4,6) ms

Disk Commands Issued : 69000 (21000)
Queue Read/Write/Command Latency : 0/0/0


Please, what is weak point? The Storage? CPU? RAM? Firebird database?

Its possible to disable indexing emails body? Index only header? No body, no attachments?

Thank you for help.
Best Regards
Max Devaine

< Message edited by maxdevaine -- 12.Nov.2009 3:59:38 AM >
Post #: 1
RE: Performance problem - 12.Nov.2009 4:04:55 AM   
ianb

 

Posts: 297
Status: offline 		1) I suggest you clean up your windows temp
2) If you have indexes on the same physical disk as the archive stores (firebird databases) I suggest you put them on another physical disk.

What process is using a log of CPU usage?

It is not possible to disable email body indexing.

_____________________________

Ian Bugeja
GFI Software
Blog-Twitter-YouTube-Facebook

(in reply to maxdevaine)
Post #: 2
RE: Performance problem - 12.Nov.2009 4:37:36 AM   
maxdevaine

 

Posts: 9
Joined: 27.Oct.2009
Status: offline 		1) Files in temp directory cant delete (It is possible only after restart GFI services). The files is temporarily firebird files.
2) "same physical disk" is :
HP Smart Array P400/256 MB SDRAM
8xSAS 10000rpm RAID5 disk ARRAY - This is not sufficient performare ?

This process using High CPU (at some time 0% CPU usage) :
MArc.Core.exe 50% (at some time under)
MArc.Store.exe 50% (at some time more)


This process read / write Bytes in 3 hour after startup machine:

MArc.Core.exe :
436 000 000 000 V/V Read Bytes
22 000 000 000 V/V write Bytes

MArc.Store.exe :
186 000 000 000 V/V Read Bytes
116 000 000 000 V/V Write Bytes

Thanks
Max Devaine

< Message edited by maxdevaine -- 12.Nov.2009 5:22:15 AM >

(in reply to ianb)
Post #: 3
RE: Performance problem - 12.Nov.2009 5:40:58 AM   
ianb

 

Posts: 297
Status: offline 		Do you have any categorization policies?

I would also suggest you set indexing from "Immediately" to "Half an hour"

< Message edited by ianb -- 12.Nov.2009 5:49:46 AM >

_____________________________

Ian Bugeja
GFI Software
Blog-Twitter-YouTube-Facebook

(in reply to maxdevaine)
Post #: 4
RE: Performance problem - 12.Nov.2009 6:33:05 AM   
maxdevaine

 

Posts: 9
Joined: 27.Oct.2009
Status: offline 		We have not use any categorization policies.
We have only settings : delete few emails by subject and sender

Now indexing "Immediately".
I try to use "Half an hour".

Thanks
Max Devaine

(in reply to ianb)
Post #: 5
RE: Performance problem - 16.Nov.2009 4:31:59 AM   
Encode

 

Posts: 120
Joined: 29.Jul.2008
Status: offline 		Are there any other vmware machines using the same disk?

What is the size of your Firebird database?

_____________________________

GFI Mail Archiver 6 Customer.
Running V6.2 (20091214) on Firebird.

Patches installed:
MARC62_PATCH_20091223_01

(in reply to maxdevaine)
Post #: 6
RE: Performance problem - 16.Nov.2009 8:04:45 AM   
maxdevaine

 

Posts: 9
Joined: 27.Oct.2009
Status: offline 		Yes, i use on the same storage 3 virtual maschines (GFI, 2x Debian GNU/Linux). But other virtual maschines (2xLinux) minimal use the storage : cca 200-1000KiB/s and IO is small. This maschines is undemanding performace (DNS, Mysql+Apache2-test, only 1-5 connections).

Now is database statistic :
-------------------------------------
Archive store contains : 131 519
Database size : 1146MiB
Search indexe size : 144MiB
Indexed emails : 64266
-------------------------------------
We have use indexing ( fixed time ) : "01:00"
Journal mailbox now contains 17444 emails.

I turned on debug mode.
After it may send to support ftp.

Thanks
Max Devaine

(in reply to Encode)
Post #: 7
RE: Performance problem - 16.Nov.2009 8:09:35 AM   
Encode

 

Posts: 120
Joined: 29.Jul.2008
Status: offline 		Do you have mailbox folder structure retrieval switched on? That can slow things down initially whilst it processes existing email, and get better after a while.

Is Journal mailbox going down any, or is it always going up?

How many Archive databases do you have?

_____________________________

GFI Mail Archiver 6 Customer.
Running V6.2 (20091214) on Firebird.

Patches installed:
MARC62_PATCH_20091223_01

(in reply to maxdevaine)
Post #: 8
RE: Performance problem - 16.Nov.2009 8:47:30 AM   
maxdevaine

 

Posts: 9
Joined: 27.Oct.2009
Status: offline 		Yes, I have mailbox folder structure retrieval switched on, but i have by mistake false password entered (so this function up to now not work). I submit now true password.


"Is Journal mailbox going down any, or is it always going up?"
-> Where? I don't understand :-/. We have journal mailbox define on the Mailbox Database.


We have now on GFI Mail Archiver :
2008/05 - 2009/06 - 30 000 emails in database (per Month) (imported emails via eewiz)
2009/07 - 113 000 emails (113 000 indexed)
2009/08 - 130 463 emails (129 026 indexed) - 1146MiB database size
2009/09 - 143 394 emails (143 394 indexed) - 1217MiB database size
2009/10 - 83 741 emails (83 741 indexed) - 705MiB


Users have unlimited mailbox size in exchange (some users have cca 20 000 emails with cca 30 items and more).

Thanks
Max Devaine

(in reply to Encode)
Post #: 9
RE: Performance problem - 16.Nov.2009 9:00:38 AM   
Encode

 

Posts: 120
Joined: 29.Jul.2008
Status: offline 		"Is Journal mailbox going down any, or is it always going up?
-> Where? I don't understand :-/. We have journal mailbox define on the Mailbox Database."

Is the number of emails in the journal mailbox going up, or is it coming down?

Did the problem just start recently?

_____________________________

GFI Mail Archiver 6 Customer.
Running V6.2 (20091214) on Firebird.

Patches installed:
MARC62_PATCH_20091223_01

(in reply to maxdevaine)
Post #: 10
RE: Performance problem - 16.Nov.2009 3:25:28 PM   
maxdevaine

 

Posts: 9
Joined: 27.Oct.2009
Status: offline 		Oh, yes.

The number of emails oscillate. Going up (+2000 emails) and down (-500, -800 emails ...).

I monitore the number of emails via exchange cmdlet :

Get-MailboxStatistics -Identity "mailarchiver"

(now 17848 emails, so now get up)



No, we have buy GFI in October 2008.
Exchange+GFI on same physical computer and on same storage - same problem (after month)
Exchange+GFI on same physical computer (other virtual maschine), but other storage - same problem (after month)
GFI on other physical maschine and on brutal SAS storage - same problem (after a few Months)

Of all instances : We have problem with very slow web interface and outlook connector (comunication with server)

I dont know why GFI demanding too many performace :-/.

GFI say : Firebird demand max 15000 (maybe 16000) emails per day. This conditions fulfil (cca 6000 - 12000 emails per day).
But, GFI now say : Firebird max 25 users. We have 250 users :-/. Is this problem?

Thanks
Max Devaine

PS: Im sorry for my bad english :-/

(in reply to Encode)
Post #: 11
RE: Performance problem - 17.Nov.2009 5:17:45 AM   
Encode

 

Posts: 120
Joined: 29.Jul.2008
Status: offline 		Using SQL Server instead of Firebird would be recommended if you have that many users and have so much volume you are having to have monthly databases. We are on Firebird but will be moving to SQL Server+binary on NTFS soon for better performance.

How many people do you have using the Outlook connector - that puts extra load on the server also.

_____________________________

GFI Mail Archiver 6 Customer.
Running V6.2 (20091214) on Firebird.

Patches installed:
MARC62_PATCH_20091223_01

(in reply to maxdevaine)
Post #: 12
RE: Performance problem - 17.Nov.2009 7:58:57 AM   
edozier

 

Posts: 3
Joined: 24.Jul.2009
Status: offline 		I would recommend using SQL.  We started out with firebird, but system slowed to a crawl and the outlook connector would fail to sync.  Our storage is on a Equallogic PS5000e 16TB SAN. Once we moved to sql everything was so much better.

We currently have 120 users. Approx 400MB/mailbox average.

You didn't sepcify if you were using x32 or x64 os.  x64 os on a VM is not supported.

(in reply to maxdevaine)
Post #: 13
RE: Performance problem - 17.Nov.2009 8:18:18 AM   
maxdevaine

 

Posts: 9
Joined: 27.Oct.2009
Status: offline 		to Encode : Outlook connector and web interface use only 5 test users ...

to edozier:
64bit VM on ESXi 3.5 is supported.
Exchange 2007 SP1 64bit (of course) on ESXi 3.5
GFI 32bit on ESXi 3.5

280GiB Exchange database / 250 users => Approx 1,2GiB/mailbox, but lot users have minimal mailbox and few users have big mailbox

Edozier please, tell my, how many use CPU and memory your GFI maschine?

Encode : Please, tell my, how many use CPU and memory your GFI maschine? And what is your GFI maschine and datastore + number of users / mailbox?

Thanks.
Max Devaine

< Message edited by maxdevaine -- 17.Nov.2009 8:26:55 AM >

(in reply to edozier)
Post #: 14
RE: Performance problem - 17.Nov.2009 11:06:05 AM   
Encode

 

Posts: 120
Joined: 29.Jul.2008
Status: offline 		Hi Max, Our throughput is similar to yours - we have it on a physical machine (not vmware) with 2 x fast dual core processors, on a newish server (1yr old) so is quite fast, 4GB ram, C drive 2 x 15k RPM Raid 1 (temporary files by GFI here), GFI indexes on 4 x 15k RPM RAID 10, GFI Firebird Databases and binary on 2 x 10k RPM RAID 1.

We don't have any problems with it not emptying the journal mailbox, MArc.Store.exe and MArc.Core2.exe frequently use about 20-25% CPU each spread out over the 4 cores so it is not 100%.

We have UMPolling but not Outlook connector.

We are still upgrading to SQL to improve folder browse and hopefully search speed too.

Maybe connection to SQL via IMAP is slowing it down, can you connect it to exchange via another method? Or you could try changing the number of VMware cores or split the hard drive to minimise risk of drive contention.

_____________________________

GFI Mail Archiver 6 Customer.
Running V6.2 (20091214) on Firebird.

Patches installed:
MARC62_PATCH_20091223_01

(in reply to maxdevaine)
Post #: 15
Page:   [1] 2   next >   >>
All Forums >> [Archiving & Fax] >> GFI MailArchiver >> Performance problem Page: [1] 2   next >   >>
Jump to:





New Messages No New Messages
Hot Topic w/ New Messages Hot Topic w/o New Messages
Locked w/ New Messages Locked w/o New Messages
 Post New Thread
 Reply to Message
 Post New Poll
 Submit Vote
 Delete My Own Post
 Delete My Own Thread
 Rate Posts