Set up Sophos Firewall Mail Protection in MTA mode

Sophos Firewall can accept, inspect and forward SMTP traffic itself in MTA mode to the internal mail server or the next mail hop. The firewall is then not just a port release for TCP 25, but an active Mail Transfer Agent with routing, spam and malware scanning, quarantine, spool and mail logs.

Today, we recommend Mail Protection on the firewall mainly for deliberately planned on-premises or hybrid scenarios, for example when a local Exchange server or another internal mail server should be protected directly through the firewall. For Microsoft 365, Google Workspace and many modern cloud mail environments, Sophos Central Email or another cloud mail gateway is usually the cleaner architecture because MX, quarantine, headers, TLS, SPF/DKIM/DMARC and support are closer to the actual mail service.

When Mail Protection is used in MTA mode, the article needs to do more than explain a feature: MX record, automatic MTA rule, SMTP route and scan policy, relay permissions, TLS, recipient verification, spool, quarantine and logs must fit together. Otherwise, the firewall can reject legitimate email, delay mail or unintentionally become reachable as a relay.

When Mail Protection on the firewall makes sense

Mail Protection on Sophos Firewall is mainly useful when SMTP traffic should deliberately pass through the firewall and the firewall should do more than simply forward port 25.

Typical scenarios:

• Incoming email should first be accepted and scanned on the firewall.
• An internal mail server should not be reachable directly from the internet.
• Spam, malware, file type or content scanning should apply before delivery.
• Outgoing email should be sent in a controlled way through the firewall or a smarthost.
• Quarantine, mail spool and SMTP logs should be traceable on the firewall.

Not every mail setup should run through the firewall. If Sophos Central Email, Microsoft Defender for Office 365 or another cloud mail gateway already handles the complete mailflow, Mail Protection on the firewall should not be inserted additionally without documenting the mailflow precisely. Duplicate gateways quickly lead to unclear responsibilities for quarantine, headers, SPF/DKIM/DMARC, TLS and troubleshooting.

Distinguish MTA mode, Legacy mode and SMTP relay

With Sophos Firewall, three things must be clearly separated:

• MTA mode: Firewall accepts email, scans it and delivers it onward. Incoming or outgoing SMTP mailflow with policies.
• Legacy mode: older proxy-based mail processing. Existing environments that are being migrated or deliberately kept running.
• SMTP Relay as a local service: internal systems send through the firewall. Printers, scanners, applications or monitoring systems.

MTA mode is the normal target mode for modern firewall Mail Protection scenarios. The local SMTP Relay service, by contrast, is a Device Access topic. It should be reachable only from defined internal networks. Too broad an allowance can encourage relay abuse. Hardening local services is described in Secure Sophos Firewall access: configure Device Access correctly.

Requirements

Before setup, these points should be clarified:

• Sophos Firewall with valid Email Protection or a suitable bundle.
• Not every model supports MTA mode. XGS 87/87w and XGS 88/88w are appliances without MTA mode support.
• Public DNS zone and MX record are known.
• Internal mail server, destination port and delivery path are documented.
• Public IP address or WAN address for incoming SMTP traffic is defined.
• Firewall can route to and reach the internal mail server.
• Outgoing DNS access from the firewall works.
• Desired TLS and certificate strategy is clarified.
• Quarantine and release process is organisationally defined.

A maintenance window should be planned before changing the mailflow. Testing productive MX records without a fallback plan is risky because incoming email may quickly be delayed or rejected depending on the sender.

Define the target architecture

First, decide which direction the firewall should process.

Incoming mailflow

For incoming mailflow, external MX records point to the public address on which Sophos Firewall accepts SMTP. The firewall scans the message and forwards it to the internal mail server.

Typical sequence:

1. External sender connects by SMTP to the public MX address.
2. Sophos Firewall accepts the connection in MTA mode.
3. Mail Protection checks sender, recipient, spam, malware, attachments and policy.
4. The firewall delivers the email to the internal mail server.
5. The internal mail server delivers it to the mailbox or processes the message further.

It is important that the internal mail server does not also remain reachable unfiltered from the internet. If a DNAT rule still points directly to the mail server in parallel, part of the traffic may bypass Mail Protection. For normal server publishing, Publish a server with DNAT is the right foundation, but in MTA mode the firewall itself is the SMTP acceptance point.

Outgoing mailflow

For outgoing mailflow, the internal mail server sends through Sophos Firewall. The firewall can scan messages, forward them to a smarthost or deliver them directly, depending on the configuration.

Clarify beforehand:

• Is the public firewall IP allowed to send email directly?
• Are SPF, DKIM and DMARC correct for the selected sending path?
• Is a provider smarthost required?
• Must outgoing SMTP traffic be limited to specific internal systems?
• Where are rejected or delayed messages monitored?

For outgoing mail traffic, use a separate, traceable rule and policy structure. A general LAN to WAN rule without clear restriction is usually too broad for mail servers. The basics of rule order and security profiles are covered in Understand and build Sophos Firewall rules cleanly.

Prepare MX change and external tests

A Mail Protection change becomes critical only when external senders really use the new path. Therefore, MX record, DNS TTL, external reachability and rollback should be checked before the production switch.

Before the change:

• Document current MX record, priority and TTL.
• Reduce DNS TTL early if a quick fallback might be needed.
• Clearly distinguish the old mail path and the new Sophos Firewall address.
• Identify old direct DNAT rules to the mail server.
• Define test recipients and test senders.
• Define fallback plan: old MX, old DNAT rule or temporary smarthost.
• Prepare monitoring for spool, quarantine and mail server queue.

Useful external checks from a system outside your own network:

dig MX example.com
dig A mail.example.com
nc -vz mail.example.com 25
openssl s_client -starttls smtp -connect mail.example.com:25 -servername mail.example.com

The commands do not replace a complete mailflow test, but they quickly show whether DNS, port 25 and STARTTLS are basically reachable. example.com and mail.example.com must be replaced with the real domain and the real mail host.

After switching, send an incoming test email immediately and document the full path:

• Connection visible in Log Viewer?
• Entry present in smtpd_main.log?
• Recipient verification successful?
• Delivery to internal mail server completed?
• Message arrived in the mailbox?
• No parallel direct delivery bypassing the MTA?

If the firewall accepts email but does not deliver it, the MX should not be reverted immediately. First clarify whether it is an internal routing, DNS, TLS or mail server problem. But if external senders cannot connect to the firewall or legitimate email is broadly rejected, a quick fallback to the previous mail path is often more sensible than prolonged experimentation in the production MX path.

Enable MTA mode

The basic settings are located under:

Email > General settings

For this guide, MTA mode must be active. If the firewall is still running in Legacy mode, switch with Switch to MTA mode.

After activation, Sophos Firewall automatically creates a firewall rule for SMTP/SMTPS named Auto added firewall policy for MTA. This rule should remain visible and high up in the rule base. An MTA rule must not be treated like a normal LAN-to-WAN rule or casually moved down, otherwise incoming SMTP traffic will no longer land in the expected MTA path.

Then check the basic settings:

1. Under SMTP settings, set the SMTP hostname. This is the hostname the firewall uses in HELO and banner contexts for system-generated messages. Do not blindly enter the internal mail server name if the public mail name is different.
2. Enable Reject based on IP reputation if incoming SMTP connections should be rejected based on poor sender reputation.
3. Under SMTP TLS configuration, choose a suitable publicly trusted certificate if SMTP TLS should be presented cleanly through the firewall.
4. Enable Disable legacy TLS protocols unless deliberately documented legacy systems speak against it.
5. Enable Scan outgoing mails if outgoing messages from the mail server should also be checked by the firewall.

In existing environments, do not simply switch between Legacy mode and MTA mode without testing the mailflow. Processing, logs and policy logic differ. Before a migration, document the current firewall configuration, MX records, mail server connectors and relay settings.

Configure SMTP routing and domains

For incoming email, the firewall must know which domains it should accept and where it should deliver them.

Create address group for mail domains

First, create an address group for the protected mail domains:

1. Open Email > Address group.
2. Click Add.
3. Check that Group type is set to Email address/domain.
4. Leave Type set to Manual if the domains are maintained manually.
5. Under Email address/domain, enter and add the domain, for example example.com.
6. Click Save.

This is about domains, not individual recipient addresses. Individual existing or migrated email addresses may still apply depending on the configuration, but in current SMTP route and scan policies they can no longer be planned as new entries. For a clean target state, work with domains and suitable recipient verification.

Create SMTP route and scan policy

The actual MTA policy is created under this path:

Email > Policies and exceptions > Add a policy > SMTP route and scan

A typical sequence for incoming mail to an internal mail server:

1. Set a clear Name, for example Inbound example.com to Exchange.
2. Under Protected domain, select the address group created earlier.
3. Define Route by:
   • Static host: For fixed internal mail server IP addresses.
   • DNS host: For a DNS name such as mailserver.example.com.
   • MX: If the firewall should deliver based on MX records.
4. With Static host, select the internal mail server under Host list. IP hosts are created under Hosts and services > IP host if needed.
5. Set Global action to Accept if the domain should be accepted and checked by policy.
6. Enable Spam protection and deliberately decide whether spam should be warned, quarantined, dropped or delivered without action.
7. Enable Malware protection. For Zero-Day Protection with single antivirus scan, Sophos must be used as the primary engine.
8. Enable File protection and Data protection only if the effects on attachments, large messages, SPX and DKIM are understood.
9. Click Save.

For multiple internal mail servers, the routing type matters. With Static host, delivery switches to the next host if the first host is unreachable. With DNS host and multiple A records, delivery can be distributed. That can be useful, but it must match the mail server design, TLS certificates and troubleshooting.

DNS is particularly important for internal mail servers. The firewall must be able to resolve internal destinations correctly, and external senders must reach the public MX entry. If internal DNS resolution plays a role, Set up DNS Request Routes on Sophos Firewall helps.

Plan policies for spam, malware and attachments

Mail Protection is only as good as the policies that actually apply. A policy should not only be created, but also named with a clear purpose.

Important policy questions:

• Which domains or recipient groups are affected?
• Is incoming, outgoing or bidirectional mail traffic processed?
• What happens with spam, malware, suspicious attachments or unwanted file types?
• Are emails blocked, quarantined, delivered or marked with headers?
• Should recipient verification be used via Callout or Active Directory?
• Who may check quarantine and release emails?
• Which false positive processes exist?

For Spam Protection, SPF, RBL, Greylisting, BATV and recipient verification should not be treated as mere checkboxes. RBL or SPF hits are not processed like normal spam actions, but can reject messages directly. Recipient verification reduces backscatter and invalid recipients, but can itself become a source of errors if there are AD, TLS or mail server problems.

DKIM is especially sensitive for outgoing messages. SPX encryption, subject prefixes, blocked file types, Data Protection or an outgoing banner can change headers or body. This can break a DKIM hash at the recipient MTA. If outgoing signatures are important, decide whether signing should happen on the mail server, on Sophos Firewall or at a later gateway.

If Zero-Day Protection is used, suspicious email attachments can be analysed additionally. The limits, reports and release decision are explained in Understand and operate Sophos Firewall Zero-Day Protection.

Secure relay and Device Access

A common mistake is confusing MTA mailflow with an open SMTP relay. The firewall must not be usable as a relay from arbitrary networks.

For incoming email, SMTP must generally be accepted on the WAN side so external mail servers can deliver the domain. For outgoing email, by contrast, it must be clear which internal hosts may relay through the firewall.

Check:

• Under Administration > Device access, SMTP Relay is enabled only in the zones that are really required.
• If ACL Exception Rules are used, sources are tightly defined.
• Under Email > Relay settings, only defined mail servers, scanners or application servers are entered as allowed sources under Host-based relay.
• Unneeded zones, guest networks and untrusted networks must not have broad relay access.
• If cloud services such as Exchange Online Protection should deliver or relay through the firewall, the allowed source ranges must be maintained very precisely. Broad Any permissions are an open-relay risk.
• Logging is active so that abuse or misconfigurations are visible.

If printers, scanners or applications need to send email, a dedicated internal relay path should be documented for them. Such systems should not speak directly to arbitrary external SMTP destinations if the environment can avoid it.

Test mailflow

After setup, one successful send is not enough. Several tests should be performed and the results documented.

Test incoming

Check at least:

• External MX record points to the expected address.
• Port 25 is reachable externally.
• Firewall accepts the connection in MTA mode.
• Email is forwarded to the internal mail server.
• Recipient exists and receives the message.
• Spam or malware test is handled as expected.
• Quarantine or log entry is traceable.

Test outgoing

Check at least:

• Internal mail server sends through the expected route.
• Firewall rule and mail policy apply.
• SPF, DKIM and DMARC match the sending path.
• Destination server accepts the message.
• Bounces or deferred messages are monitored.
• No other internal systems send directly to external destinations unexpectedly.

Check logs

Log Viewer helps with quick visual checks. For deeper analysis, the mail log files are important. The mapping is listed in Sophos Firewall troubleshooting: services and logs.

Relevant log files:

• SMTP MTA: smtpd_main.log.
• SMTP errors: smtpd_error.log, smtpd_panic.log, smtpd_reject.log.
• Anti-Spam: sasi.log.
• Legacy SMTP/MTA: awarrensmtp.log, awarrenmta.log, awarrenmta_debug.log.
• POP/IMAP Proxy: warren.log.

During urgent troubleshooting, note the test time, collect sender, recipient, subject, source IP and Message-ID, and then correlate Log Viewer and log files by time.

Quarantine, spool and storage

Mail Protection creates local data. Depending on volume, messages end up in quarantine, spool or temporary areas. This makes storage space, SSD health and the recovery plan more relevant than with a pure firewall rule.

Practical operating questions:

• Who checks quarantine and how often?
• How are false positives released?
• When is a message deleted instead of released?
• How is a growing mail spool detected?
• Is there monitoring for storage space and System Health?
• Is a short mailflow test planned after firmware updates?

For storage and reporting topics, Clean up Sophos Firewall storage and reports and Check Sophos Firewall SSD Health fit. In HA environments, also note that mail quarantine and processed mail data can be node-specific operational data. The HA basics are covered in Sophos Firewall HA cluster variants.

Check quarantine directly in WebAdmin

If a firewall is managed through Sophos Central, access through Central is convenient. For Mail Protection, admins should still know where the quarantine is located locally and how to check it directly on the firewall.

Email > SMTP quarantine

There, quarantined messages can be filtered by time period, sender, recipient, subject and quarantine reason. For operations, three actions are especially relevant:

• Delete: Delete the message from quarantine.
• Release: Release the message to the recipient.
• Release and report: Release the message and report it as incorrectly classified.

Important: Virus-infected messages and messages classified as malicious by Zero-Day Protection cannot simply be released. Zero-Day Protection entries also need suitable permissions if they are to be deleted. If the quarantine fills up, older emails are cleaned up. This is another reason not to check quarantine and storage only after complaints.

There is an important special case with SFOS 22.0 MR1: If the firewall is managed through Sophos Central, quarantine actions such as Release or Delete can fail with Invalid API request. This does not automatically mean that mailflow, quarantine itself or the MTA policy is broken. The practical workaround is to sign in directly to the local WebAdmin of the firewall and release or delete the message under Email > SMTP quarantine.

After updating to SFOS 22.0 MR2 or later, the workflow should be tested again: quarantine a harmless test message, perform the action through the planned administration path, and then check Log Viewer, quarantine and the recipient mailbox. This makes it clear whether the issue is actually fixed or whether roles, Central access or local permissions are also involved.

Troubleshooting

External email does not arrive

First check DNS and reachability: MX record, public IP, port 25, NAT leftovers, MTA mode and responsible mail domain. Then check in Log Viewer and smtpd_main.log whether the connection reaches the firewall. If no connection is visible, the problem is probably before Mail Protection.

Firewall accepts email but does not deliver it

Then internal mail server, routing, DNS, destination port, TLS, recipient verification or policy are more likely. Check whether the firewall can reach the mail server and whether the mail server accepts the connection. Reject logs and mail server logs should be evaluated together by time.

Many emails remain in the spool

A growing spool often points to delivery problems: internal mail server not reachable, TLS requirement does not match, DNS resolution fails or destination server rejects the message. In this case, do not only redeliver individual messages, but look for the cause in the routing and SMTP path.

An important rule order case is easy to miss: if an automatically or manually created firewall rule is above the MTA rule and matches SMTP traffic, the actual MTA rule is no longer evaluated. Emails can then remain stuck in the mail spool even though DNS, port 25 and mail server basically look correct.

Check:

1. Open Rules and policies > Firewall rules.
2. Check rules above the MTA or SMTP rule.
3. Check newly created automatic rules, IPsec rules, hotspot rules or manually Top-positioned rules.
4. Run the SMTP test again and compare Log Viewer, mail spool and smtpd_main.log.

The rule must not be moved down blindly if it serves other production purposes. The decisive question is whether it unexpectedly intercepts SMTP traffic before the MTA rule.

Quarantine digest missing for alias addresses

If users use alias addresses, quarantine settings should not be checked only for the primary email address. According to Sophos, quarantine settings are not applied automatically to alias addresses by default. If digest emails or releases are missing for alias recipients, alias addresses must be considered together with the primary address in the quarantine or user context.

Legitimate sender is detected as spam

False positives should not immediately be answered with broad exceptions. First check sender domain, SPF/DKIM/DMARC, headers, reputation, policy match and affected recipients. If an exception is necessary, it should be tightly limited and documented with a review date.

Internal systems cannot relay

Check whether SMTP Relay under Administration > Device access is allowed from the correct zone and whether the source matches the ACL. Then check mail logs. If a scanner or application should relay, the source should be documented as a host object and not an entire network enabled unnecessarily.

Mail behaves differently after firmware update

After firmware updates, MTA mode, policies, certificates, mail spool, quarantine and relevant logs should be checked. For larger updates, Check Sophos Firewall before SFOS 22 upgrade is also suitable.

Operating checklist

• Mail Protection licence and appliance support checked.
• MTA mode deliberately selected and documented.
• MX record, public IP and internal destination server are correct.
• MX change, external tests and rollback are prepared.
• No parallel unfiltered DNAT rule bypasses the MTA.
• Incoming and outgoing policies are clearly named.
• Address group for the protected domains is maintained cleanly.
• Firewall rule order does not prevent the MTA rule from applying.
• TLS, DKIM, banner, SPX and Data Protection have been checked for side effects.
• SMTP Relay is allowed only from defined internal sources.
• Quarantine and false positive process are defined.
• Alias addresses are considered in the quarantine and digest process.
• Direct WebAdmin access for quarantine releases is known if Central actions fail.
• Mail spool, storage space and System Health are monitored.
• Logs are retained locally, in Sophos Central or via Syslog for long enough.
• A mailflow test is performed after firmware updates.

For longer retention and correlation with other security events, check Central Firewall Reporting or Send Sophos Firewall Syslog to SIEM.

FAQ

Quarantine action reports Invalid API request

If Release or Delete fails with Invalid API request on a firewall opened through Sophos Central, check the SFOS version first. With SFOS 22.0 MR1, this exact workflow can be affected.

The next step is not to rebuild the mail policy. First switch directly to the local WebAdmin of the firewall and edit the message under Email > SMTP quarantine. Then document:

1. SFOS version and build.
2. Whether the action was attempted through Sophos Central or directly in WebAdmin.
3. Sender, recipient, subject and quarantine reason.
4. Result after testing directly in WebAdmin.
5. Whether an update to SFOS 22.0 MR2 or later is planned or already installed.