Forums & Blog

Hi,

The Spam Assassin process spamd.exe is using a huge amount of memory on our server. I have noticed it consuming up to 450mb at one point. We run a small, low volume mail server, processing +-1000 a day. Our server configuration is well above the recommended minimum system requirements.

Typically spamd consumes around 100mb on our server, and then every 12 to 24 hours I need to end the spamd process as its memory consumption climbs to < 200mb. This is not ideal, since I have to remember to monitor spamd, and on the occasions where I have forgotten, there is the risk the server becomes non-responsive requiring a forced reboot.

1) Why is spamd consuming so much memory?

2) Why is spamd not releasing memory?

3) Is there a way to limit the amount of memory that spamd consumes?

I run about the same volume of mail with the SPAMD.exe process using about 140MB.  A spike of 450MB would kill my server.  I wonder if this is an issue with the Windows version of SPAM ASSASSIN?  Maybe on top of only being single threaded it's not very efficient (just a guess).

Smartertools People: How much RAM should the SPAMD.exe process be using when mail volume is averaging 1000 messages a day?

Thanks, 

Mike 

That is the problem, the spike to 450mb did kill our server! It has now become part of my routine to log into the server and check the memory usage of spamd, ending the process once I see it is around 150mb+. This morning it was at 212mb.

We run SmarterMail on the same server we run our web sites on. Considering our mail volume is so low, if we have to get a separate dedicated server for SmarterMail, it certainly makes the economics less appealing. Of course part of the reason our mail volume is so low is because of the great new spam filtering features in 4.0, which weeds out about 2/3 of the junk we use to get.

Now the delemma is: if those great new features use so much memory that the server crashes, or they require a dedicated server unto themselves, how great are they?

James, Tim, other SmarterTools folks,

I was hoping to get some feedback, information, or just a response to this issue that I detailed above. Bueller... Bueller... Bueller...

Thanks

The version of SpamAssassin that is bundled with SmarterMail 4 is really just that; bundled software developed by third parties not related to SmarterTools. So while SmarterTools people could probably give you some answer, it would not actually be 'authoritative' on the subject, but simply an informed opinion.

The simple fact is that the windows build of SpamAssassin that is bundled with SmarterMail is really only fit for SOHO use. If stability and reliability are expected, then one ought to look at a native *NIX SpamAssassin solution, and this for a number of reasons.

SpamAssassin was designed to run on UNIX platforms, taking advantage of some advantages of those systems (the Perl preforking mechanism being an example). A Windows port will inevitably have to provide workarounds at really low levels, and this is the main reason why any Windows version of SpamAssassin is a lot less... productive than a UNIX one. It's more than a 'native affinity' which SpamAssassin has for UNIX, but rather an inherent design issue.

The UNIX versions of SpamAssassin are stable and reliable. Windows ports, however, are not. Besides the aforementioned design issues that need to be overcome in a port, one must also note the fact that while the main SpamAssassin application is developed by a dedicated team, the original Windows port bundled in SM4 only really has one developer.

I think any expectations one will have regarding the bundled version of SpamAssassin should take these factors into consideration. I think it's nice of SmarterTools to have bundled a version of SpamAssassin in SM4, but I also think that they should have stressed the status of this 'bundle'; people really don't seem to appreciate it for what it is.

 

Informed opinions are welcome, authoritative or not, so I certainly appreciate your response.

I don't wish to get off topic, but saying that SpamAssassin is unrelated to SmarterMail, really seems off the mark. As I see it, inclusion of SpamAssassin in the 4.0 release of SmarterMail implies endorsement of use. In addition, including "Out of the box SpamAssassin" in the list of product features certainly is a selling point for this new release - I know it was for me.

Given the above, I think there is a certain responsibility if not liability that comes with the inclusion of SpamAssassin in SM 4. Really I would just like to find out if SmarterTools or others have had issues with spamd eating up big nasty blocks of memory, and, what if anything can be done about it, with the exception of disabling the "Out of the box SpamAssassin" feature.

SpamD does like to take up memory, although I've never seen it as high as yours. Ours hovers around 100mb pretty much all the time.  We do endorse it's use, but unfortunately, it doesn't work well for everyone.

NotMyRealName:

Informed opinions are welcome, authoritative or not, so I certainly appreciate your response.

I don't wish to get off topic, but saying that SpamAssassin is unrelated to SmarterMail, really seems off the mark. As I see it, inclusion of SpamAssassin in the 4.0 release of SmarterMail implies endorsement of use. In addition, including "Out of the box SpamAssassin" in the list of product features certainly is a selling point for this new release - I know it was for me.

Given the above, I think there is a certain responsibility if not liability that comes with the inclusion of SpamAssassin in SM 4. Really I would just like to find out if SmarterTools or others have had issues with spamd eating up big nasty blocks of memory, and, what if anything can be done about it, with the exception of disabling the "Out of the box SpamAssassin" feature.

SmarterTools have tried to make it easy for people by bundling SpamAssassin and ClamAV with Sm4, but this does seem to ahve backfired a bit as the Windows port of SA just doesn't perform as expected.

I think it maye have been better to simply provide the facility to work with SA, just liek the Declude option. It would have been down to system admins to make a decision on which was the best route to go for tehir SpamAssassin install. SmarterTools wouldn't have been taking the flak for a flaky Windows SA that isn't really their fault.

True, but I am sure there are plenty of small installations that use SM for simply company email who can run the windows SA without performance issues and can really benefit from it.

I just killed the spamd process on our server... it was using 260mb. I know that this is a windows port, but I am curious if others who are running the Out of the Box SpamAssassin implementation are seeing similar memory usage?

It seems that after the SpamAssassin process has run for about 12 to 24 hours is when its memory usage starts to really grow.

I read this on http://sawin32.sourceforge.net/faq.html

 Why does spamd.exe take so much RAM?

The daemon spamd.exe is a multithreading application. Each thread accepts connections from clients such as spamc.exe. The number of threads is determined by options --max-children, --min-children, --max-spare and --min-spare. By default spamd.exe lunches one main thread and 1-2 spare child processes that are initially idle. If the mail load increases, spamd.exe may spawn more child processes. To save memory, you can disable all idle child processes by running spamd.exe with options --max-spare=0 --min-spare=0.

Is there a way within SmarterMail to edit the command line that launches spamd.exe?

It would be nice if there was some way to keep spamd.exe in check. Mine is constantly taking up 240+MB in RAM. And I'm on on a VPS web server with only 1GB dedicted to me (not bad for a VPS) but still. It's taking up valuable RAM that I don't think it really needs. My email volume is extremely light, only 5k messages per day!!!! Something isn't right.

Let me just state my situation. I run a smtartermail 4 gateway server in a virtual machine. According to stats it handled 919 emails in the last 24 hours, and 68 in the last hour.

Now the surprises :-)

 The machine has 256mb RAM allocared. Used RAM is currently 401mb, and the spamd process is at 86mb at the moment.

 

Performance is within acceptable parameters. So to say - I just realized now that some more ram may be good. THe server runs without being watched for - hm - months, the occasional reset (ms patchday) nonwithstanding.

 But your memory looks too high.

You guys are all low. I just checked after running 4.x for a week with SpamAssassin enabled, and on a 2GB RAM Machine, spamd was taking up 1.2 GB of RAM.  There has got to be a way to limit this.  I am afraid to disable it totally, because I have received several emails from customers asking whether there was something wrong with their accounts, because all the spam went away :)  That is a pig, though.

I run SM4 on my dedicate web server with sites as well. I have only 1GB of RAM, but have more on the way. I am having issues with the w3wp web process taking most of my memory, sometimes up to 550MB. So I monitor it daily while my new memory is on the way. I am having to do IIS Resets a couple of times a day.

Not I just noticed that all of the sudden my spamd, which is normally less than 150MB is near 400MB. My SM report is showing 1582 messages delivered in the last 24 hours. So not a heavy load.

This is the first time I have noticed that it spiked like this, but it did cause my websites to basically stop responding.

Any suggestions would be helpful?