Announcing the SmarterMail Beta

The upcoming release of SmarterMail has hundreds of amazing new features and improvements that will appeal to users, domain administrators, and system administrators alike.

Although the next SmarterMail is essentially feature complete, we are still working to finalize code throughout and review and inspect every language and visual element for absolute consistency. With your help beta testing, we hope to have the most feature packed and stable SmarterMail ever released.

Thanks for taking the journey with us.

Two-Step Authentication for System Administrators

Keeping your SmarterMail installation secured is crucial. We introduced the ability to create "permission levels" for system administrators in previous versions of SmarterMail, and now we're adding in a way to protect system administrator logins.

Similar to the implementation for users, when Two-Step Authentication is enabled for a system administrator login, they will have the ability to set up an authenticator app like Google Authenticator or Twilio's Authy app. We're also exploring the implementation of other forms of secondary authentication for users and system administrators in an upcoming release.

New and Modified Management Tools Throughout

We took feedback from system administrators about how they use SmarterMail and how they monitor what is happening on the server. As a result, we have modified and introduced pages throughout SmarterMail that allow you to look at data differently and act on it more consistently.

Better Handle on Users

Oftentimes, when troubleshooting issues, system administrators are drawn to a particular user causing a problem, or some outside influence. Inbox slowness, missing messages, etc. are common things administrators deal with daily. Well, we're giving system administrators a bit more insight into potential causes for these types of issues, making them easier to diagnose and resolve.

Previously, administrators could view information on "User Activity", which showed online and offline users, and gave some information about what users were connecting to, from a protocol perspective, but never what they were doing.

Now, administrators have a more concise view of what users are connecting to by looking at User Connections. This page breaks down each protocol into its own tab, and displays every user on the server so administrators can see who is using what protocol.

In addition, administrators can see what those users are doing by reviewing User Statuses. This page shows all users on the server and also provides some insight into any possible issues with those users, issues that could cause things like delays in mail delivery, problems logging in, and more. Administrators can see if a user is currently involved in a migration, if their folders are being indexed, if their password is expired, and more.

Providing system administrators with tools to help ease the pain points when managing a mail server has long been a focus for SmarterTools. Breaking out this information into more detailed, and more digestible, ways is just one more way we're helping SmarterMail administrators.

Antispam

SmarterTools continues to evaluate how we can provide more and more functionality to help block spam for FREE. Below we'll explain some of what's coming in the next version of SmarterMail.

Consolidated and Updated RBLs/URIBLs

We looked at all RBLs and URIBLs that are included with SmarterMail, updated their hostnames and lookup values, and, in some cases, consolidated values. For example, we made it much easier to get more complex results from higher-end RBLs by allowing custom weights per required lookup value.

Internal SpamAssassin

We reviewed how we've implemented internal SpamAssassin and made some adjustments to improve its performance and reliability. Instead of attaching weights, like other checks, we're using a "scoring value" to normalize the value used when weighing SpamAssassin results. This normalization takes the SpamAssassin raw score and multiples it by a Scoring Factor (that is fully customizable) to come up with a final spam score.

Rspamd

Whether you want to use a docker, a virtual machine, or a bare metal server in a remote location, SmarterMail now includes integration with Rspamd, just like we have with remote SpamAssassin. Rspamd is an advanced spam filtering system that, as a Linux distribution, requires installation on a remote system. However, it can tie in nicely with SmarterMail. As such, it's now available as an optional spam check in addition to the SpamAssassin installation included with SmarterMail. Users can assign a scoring factor for it as well as manage the other settings Rspamd requires for its checks.

Improved List Performance

We also changed how spam checks run, which greatly improved the overall performance. Now, all RBL and URIBL tasks run concurrently instead of one after the other while handling poorly-performing lists much better.

SPF, DKIM, and DMARC Updates

Verifying that a sender can be trusted is crucial. Oftentimes, spammers "spoof" senders, using one From address but actually sending messages from a completely different account. SPF, DKIM, and ultimately DMARC were all implemented to combat this practice.

To help improve sender trust, we're rolling out a new feature that will be seen on the user side – a Sender Verification Shield, which we'll detail in an upcoming blog post – and part of implementing that feature entailed a complete review of SPF, DKIM, and DMARC.

DMARC, if it's set up, will pass if either SPF or DKIM checks pass. So, we did a complete review of DKIM, SPF, and DMARC implementations in SmarterMail. This review led us to improving DKIM and SPF checks, and DMARC results, which resulted in better performance of all three. We also added a new header denoting DMARC status. (Along with a few other new spam detail headers.)

HAM/SPAM Training

System administrators also have a couple of tools at their disposal to improve antispam performance of any third-party add-ons. For example, Cyren or other services. On the Options tab, we've added 2 new options:

  • Send user spam feedback to antispam providers
  • Send user spam feedback to training folder
These settings can be used to send samples to antispam providers, letting them know that a particular message not already flagged is potential spam, or in the case of local training folders, allow any locally installed system know a message is suspect. There's also a built-in failsafe to prevent false positives from being reported.

Intrusion Detection and Security Improvements

Over the years customers brought to our attention new methods of attack they see against their servers, and they've mentioned how previous methods of intrusion detection may not have been sufficient. Therefore, we completely re-evaluated and improved upon our industry leading security mechanisms for a mail server. For example, one minor but impactful change is that it's now possible to see how many times a particular IP address has been blacklisted in the past 30 days. Addresses with seemingly constant, successive bans may require further exploration, or even firewall entries so they're completely blocked from the network.

Additionally, instead of requiring separate rules, one for each protocol, when setting up DoS protection, you now just need ONE rule as it will cover all protocols. In addition, we've consolidated the various brute force rules into just two: one for brute force by email, and one for brute force by IP. Both cover each protocol as well, even brute force attempts via webmail.

We understand this is a major change, but we've made it very easy when upgrading from older Builds to this new structure. SmarterMail will handle the conversion of any existing rules into this new rule structure and leave unaffected rules alone. Administrators shouldn't need to do any prior preparation; it will just work.

Finally, we added the ability to automatically block entire Class C ranges for IDS violations. At times, you'll see one or two IPs from the same Class C hitting your server. So, you block them. Then, another IP from the same Class C appears and it, too, runs afoul of IDS blocks. Moving forward, we'll offer a setting, based on some variables, that will automatically block the entire Class C so you don't have to do it manually, one or two IPs at a time.

Enhanced Message Archiving

While there are third-party products and services available for archiving messages, SmarterTools has always felt it's an integral part of running your own mail server. As with other features, we are always looking for ways to improve it. So, we looked at the requests we've had from customers and made some significant changes to the functionality and feature-set of message archiving.

First off, we changed how archiving rules are set up for domains and for the system as a whole. Domain-specific rules can be created when the domain is set up, as part of its original configuration, or added on a domain-by-domain basis for existing domains. If administrators want to archive all domains, that's set up in the system's General settings.

Next, we added an auto-clean feature. Administrators can have archived messages automatically purged in yearly increments up to 10 years, or not at all. This added flexibility means every domain/business can have its own rules and its own settings, tailor-made for the business.

We also added the ability for administrators to perform multiple, concurrent searches of the archive, and each set of results is stored for 30 days. This makes it very easy for administrators to go back and review search results when additional information is requested, rather than re-doing a search.

Finally, how message archiving runs was completely overhauled, making it more efficient and less impactful from a performance standpoint. So, we added features but it's faster and uses less system resources when performing searches, when moving messages, etc.

MAPI Improvements

Although millions already use MAPI and SmarterMail, some customers require more advanced and/or unique features within MAPI. With this release, SmarterMail now covers it all. For example:

  • Delegation
  • Contact Groups
  • Categories/tags
  • Attachment support for all features
  • Zoom rooms
  • Public Folders
  • Much more
As of now, the only remaining item, which is currently being worked on is server-side rules. Currently we allow for the creation and saving of server-side rules. What we're working on is the transition to processing server-side rules in place of content filtering rules. We need to convert content filtering into the rule-based system that Microsoft created. That conversion is extremely complicated when you consider millions of rules across millions of users. That complexity is why this is still being worked on. We need to spend extra time making sure we continue to provide the level of functionality available today, then make sure that any conversion occurs seamlessly and completely when upgrading. As you can imagine, it's a very large undertaking and we want to ensure it's handled properly.

EWS Improvements

Although EWS in previous versions of SmarterMail was 100% complete and provided every feature available in the version we supported (14.2.390.3), we upgraded EWS to the most stable version available, which is 15.1.1779.4. Then, we tested our implementation with every EWS client, service, and product customers have brought to our attention. This helped reinforce our commitment to bringing every single feature and function Microsoft makes available in this protocol to users.

More XMPP RFCs Included

We have added even more XMPP functionality to SmarterMail so that it can be used with a wider number of clients. In addition, XMPP can be auto-discovered, which means it can be automatically deployed across a large number of users. Then, when those users connect to a client that includes XMPP chat, or to any chat client, the account is set up immediately. We also added functionality for history, attachments, and more.

Distributed Functionality

One of the principles SmarterMail is built on is allowing system administrators to delegate responsibility to domain administrators. In fact, we wrote a blog post about how SmarterMail administrators see a huge time savings compared to Exchange administrators. This next release builds upon those principles and even allows domain administrators to extend functionality to users.

For example, much of what's discussed in this post carries over to domain administrators and users alike. Things like the antispam changes that are coming will help system administrators, but also help domain administrators and end users. Changes we've made to DKIM, SPF, and DMARC roll into a user being able to see a trust level for people who email them, giving them the confidence that a person truly is who they say they are. In addition, domain administrators will also have access to new pages that give them details on user connections and user statuses. Having access to this information takes some of the burden off of system administrators for issues like slow mail delivery and other common problems users face. Domain administrators will have more insight into user behavior, and what's going on with the mail server in general, and then be able to act on that information. They'll be better informed, and also have the ability to take action before having to contact a system administrator.

eM Client Relationship

We've worked with eM Client for almost a decade. Over the years we've had a great relationship with them, and that relationship is just getting better. Our previous integration only scratched the surface of what eM Client and SmarterMail can do, and we have incredible news to share now, and will have even more to announce in the future.

Starting with this release, SmarterMail and eM Client have proprietary methods to autodiscover and auto-deploy configurations to users. When you setup eM Client you can have your email, contacts, tasks, notes, and chat running within seconds.

Another feature being introduced is the ability to create calendar appointments and use SmarterMail for your online meetings. Creating the online meeting as part of the appointment is automatic, making for an incredible solution for small and medium businesses. More importantly, this partnership benefits enterprise organizations who want to move away from Microsoft Exchange and Microsoft Outlook, and simplify the overall deployment of email, video conferencing, and live chat like never before.

Working with a provider like eM Client helps us move closer to the goal of making SmarterMail a complete communications platform for customers.

SSO Integration for Services

To continue supporting things like mailbox migrations and message retrieval, we've implemented SSO as required by providers such as Microsoft, Google, and others who are discontinuing various methods of basic authentication. Ultimately this is a good move as it does improve the overall security of crucial accounts. However, it complicates matters as both Google and Microsoft are using this as a way to create walled gardens and prevent users from accessing their accounts. Google, for one, is putting in rather ridiculous stipulations on service providers who simply try to create tools that make it easy for customers to access their own data. Regardless, wherever possible, SSO is supported.

Integration with Third Parties and API's

We've improved and added a variety of API calls to make it easier to integrate SmarterMail into any environment. For example, SmarterMail now exposes APIs for Online Meetings. That means email clients and/or organizations can now build the ability to dynamically add, edit, or delete Online Meetings into whatever business flow they are using.

For example, a common request from our customers, and our customers' customers, is the ability to use third-party scheduling services, like Harmonizely, with SmarterMail. The changes we've made in this version make integration with these services much easier and much more efficient.

Updated WHMCS Integration

The new SmarterMail for WHMCS consists of two complementary open-source modules, developed in PHP, that integrate SmarterMail as an add-on into WHMCS. They include the ability to create packages based off specific SmarterMail features such as mailbox counts, disk space allocations, etc. They also offer domain administrators and end users some management capabilities. For example, domain administrators can add users, add aliases, and more while end users can see some basic information about their accounts as well as log directly in to SmarterMail. The new integration also offers the ability to create packed add-ons for things like EAS accounts, MAPI Á EWS, and more. This greatly expands the capabilities and what WHMCS users can actually do with SmarterMail.

This is a brand new set of modules developed by SmarterTools that replace the two previous WHMCS modules SmarterTools developed. As a result, anyone transitioning to the new modules will need to create new products within WHMCS. Customers who use products configured using the older module can coexist with customers created using the new modules, but for the sake of consistency we recommend those customers be moved over to new products created using this newly developed solution.

Overall Performance Gains

Our goal is, and always has been, to offer more features and functionality without sacrificing performance. It's a constant struggle to provide more without tangentially impacting systems more. Nevertheless, we've done it. In many cases we've reduced resource usage, and in some areas we stayed the same.