Set up Sophos Firewall Mail Protection in MTA mode

In MTA mode, Sophos Firewall accepts email itself, inspects it, and delivers it to the internal mail server or the next mail hop. Mail flow only works when the MX record, automatic MTA rule, SMTP route and scan policy, relay, TLS, and recipient verification work together.

The setup has six steps: define the mail flow and fallback path, enable MTA mode, add the mail domain, create the route and scan policy, secure relay access, and change the MX record. Then validate inbound and outbound mail flow with external commands, quarantine, spool, and logs.

When MTA mode is a good fit

Mail Protection on the firewall is most useful in deliberately designed on-premises and hybrid environments where a local Exchange or another mail server must be protected. For Microsoft 365, Google Workspace, and many cloud-only environments, Sophos Central Email or another cloud email gateway is usually the clearer architecture. Placing two mail gateways in sequence complicates quarantine, headers, TLS, SPF/DKIM/DMARC, and troubleshooting.

Three Sophos Firewall functions must be distinguished:

• MTA mode: The firewall accepts, inspects, and routes email.
• Legacy mode: Older transparent proxy processing for existing deployments.
• SMTP Relay in Device Access: Controls the zones from which the MTA is reachable. Inbound internet email requires WAN; outbound relay is additionally restricted to specific internal mail servers.

Prerequisites include a valid Email Protection license, working routing and DNS, a known public IP address, the internal mail server and target port, and planned TLS, quarantine, and fallback strategies. According to the current Sophos overview, MTA mode isn’t available on XGS 87/87w and XGS 88/88w.

Anti-spam, RDNS, SPF, RBL, IP Reputation, and SXL2 Live Protection require internet access. Email routing, malware scanning, MIME filtering, and SPX can continue with the appropriate license in an air-gapped environment, but an active MTA doesn’t mean every protection check is available there.

Set up MTA mode end to end

1. Define the mail flow and fallback path

For inbound mail flow, the public MX record points to the address on which Sophos Firewall accepts TCP 25. The firewall inspects the message and delivers it to the internal mail server. An old DNAT rule must not leave that server directly reachable from the internet without inspection, or the MTA can be bypassed. Publish a server with DNAT explains the general DNAT logic.

For outbound mail flow, only the intended mail server sends through the firewall. The delivery path, smarthost, PTR/rDNS, HELO, SPF, DKIM, and DMARC must match the public sender identity. A general LAN to WAN rule is too broad; the rule base should clearly identify the permitted SMTP sender. Understanding Sophos Firewall rules covers rule order.

Document the following before migration:

• current MX record, priority, and TTL;
• new public firewall address and internal destination server;
• existing DNAT and SMTP rules and mail server connectors;
• inbound and outbound test senders;
• old MX record or mail path as a fallback;
• monitoring for the mail server queue, firewall spool, and quarantine.

Reduce the TTL early if a fast rollback may be necessary. Changes to the production MX record belong in a maintenance window.

2. Configure MTA mode and basic SMTP settings

Go to Email > General settings and select Switch to MTA mode if necessary. Sophos Firewall then creates the Any-to-Any rule Auto added firewall policy for MTA for SMTP and SMTPS. Don’t edit this rule, and keep it at the top of the firewall rule list. Switching to Legacy mode deletes it; switching back creates it again.

Configure the basic settings:

1. For SMTP hostname, enter the domain name, such as example.com, not the internal mail server’s hostname. The value appears in HELO and the SMTP banner for system-generated notifications.
2. Turn on Reject based on IP reputation to reject connections from senders with a bad reputation.
3. Under SMTP TLS configuration, select a publicly trusted certificate and only allow Allow invalid certificate for documented exceptions.
4. Turn on Disable legacy TLS protocols unless documented legacy systems require otherwise. This setting disables protocols earlier than TLS 1.1; it doesn’t automatically enforce only current TLS versions.
5. Turn on Scan outgoing mails if outbound messages must also be inspected.

Use Require TLS negotiation carefully: if SFOS can’t establish the required TLS connection, it discards email to the affected destination or from the configured sender domain. The firewall also validates SMTP TLS using the domain’s IP address rather than its domain name. Multiple domains on one IP address can therefore cause certificate validation errors.

3. Add the mail domain as an Address Group

1. Open Email > Address group > Add.
2. Set Group type to Email address/domain and Type to Manual.
3. Add the protected domain, such as example.com.
4. Save the group.

New SMTP route and scan policies are designed around domains, not individual recipient addresses. Migrated individual addresses can remain effective, but they can’t be newly added or edited in current policies.

4. Create an SMTP route and scan policy

Go to Email > Policies and exceptions > Add a policy > SMTP route and scan:

1. Enter a clear name, such as Inbound example.com to Exchange.
2. Under Protected domain, select the Address Group.
3. Select Route by:
   • Static host: Fixed internal mail server IP. If it fails, the firewall tries the next host in the list.
   • DNS host: A DNS name such as mailserver.example.com. Multiple A records are used across deliveries and failed servers are skipped.
   • MX: Delivery based on an MX record.
4. For Static host, select the mail server under Host list. Create its IP host under Hosts and services > IP host if required.
5. Set Global action to Accept.
6. Enable Spam protection and Malware protection according to the operating policy.
7. Enable File protection and Data protection only after understanding their impact on attachments, large messages, SPX, and DKIM.
8. Save the policy.

With Route by MX, the MX resolved by the firewall must not point back to the same firewall, or a routing loop occurs. The firewall must resolve internal destinations correctly; DNS request routes help with split DNS.

The Route inbound mail through gateway option is only needed for special designs, such as destination servers in the WAN zone, applying the original firewall rule to LAN/DMZ destinations, or selecting a specific gateway with multiple internet links. Don’t enable it without a concrete routing requirement.

5. Secure inbound access and outbound relay

Under Administration > Device access, allow SMTP Relay from every zone that must reach the MTA. Inbound internet email needs WAN; outbound email additionally needs the internal mail server’s zone, usually LAN or DMZ. Then use Email > Relay settings > Host-based relay to restrict access to the specific mail servers, scanners, or applications.

Broad host or network permissions create an open-relay risk. If printers or applications must send email, document them as specific host objects. Secure Sophos Firewall access explains the Device Access fundamentals.

The SMTP route and scan policy doesn’t support SMTP AUTH. Host-based relay is therefore the reliable option for devices. Separate Authenticated relay settings exist for users and groups, but Sophos states that they don’t support an RFC-compliant SMTP authentication standard, so client compatibility must be tested. This differs from the firewall authenticating to an upstream smarthost, for which SFOS supports PLAIN and LOGIN. Never enter an interface IP of the same firewall as a smarthost, because that creates a routing loop.

6. Change the MX record

Only change the public MX record to the firewall address after the MTA rule, domain, policy, relay, and internal delivery path are ready. Immediately send an external test message and verify that it appears in Log Viewer or Mail logs, is delivered to the internal server, and reaches the mailbox.

If external senders can’t reach the firewall or legitimate messages are broadly rejected, restore the documented previous mail path. If the firewall accepts messages but can’t deliver them, first check routing, DNS, TLS, recipient verification, and mail server logs.

Choose protection settings deliberately

Spam protection is more than the action applied to spam. SPF and RBL failures are rejected directly and don’t follow the normal actions for spam or probable spam. Greylisting intentionally rejects a message temporarily and requires the sending server to retry.

Recipient verification prevents messages to unknown recipients:

• With callout: Queries the destination mail server. If it is temporarily unavailable, SFOS accepts recipients after a defined period instead of permanently blocking all mail flow.
• In Active Directory: Checks over Simple, SSL, or STARTTLS and has a 30-second timeout.

Malware Protection can use single or dual antivirus scanning. To use Zero-Day Protection with single-antivirus scanning, set Sophos as the primary engine. Sophos Firewall Zero-Day Protection explains its limits and release decisions.

For outbound messages, processing order matters. SPX encryption, subject prefixes, File or Data Protection, and outbound banners can modify the header or body after an existing DKIM signature was added. DKIM validation then fails at the recipient. Decide whether the internal mail server, Sophos Firewall, or a later gateway signs messages.

Set up and test SPX email encryption explains how the template, trigger priority, password model, and Reply Portal work together.

Validate mail flow

Run these commands from a system outside your own network:

dig MX example.com
dig A mail.example.com
dig AAAA mail.example.com
nc -vz mail.example.com 25
openssl s_client -starttls smtp -connect mail.example.com:25 -servername mail.example.com

Replace example.com and mail.example.com with the real values. These commands test DNS, TCP 25, and STARTTLS, but not relay authorization or complete delivery. openssl s_client remains interactive after the handshake; stop it with Ctrl+C.

Test at least these cases:

• external message to a valid recipient;
• external message to an invalid recipient;
• outbound message through the intended mail server;
• spam or malware test appropriate for the policy;
• STARTTLS and the presented certificate chain;
• quarantine action and delivery after release;
• blocked relay attempt from an unauthorized source;
• no direct delivery bypassing the MTA through an old DNAT rule.

For the first review, use Email > Mail logs, Email > Mail spool, Email > SMTP quarantine, and Log Viewer. For deeper analysis, record the test time, sender, recipient, subject, source IP, and Message-ID, and correlate them with these files:

• MTA: smtpd_main.log
• rejects: smtpd_reject.log
• scan errors: smtpd_error.log
• internal MTA errors: smtpd_panic.log
• anti-spam: sasi.log
• legacy SMTP proxy: awarrensmtp.log
• POP/IMAP proxy: warren.log

Sophos Firewall services and logs explains the mapping and Advanced Shell access.

Operate quarantine and the mail spool

Under Email > SMTP quarantine, filter by time, sender, recipient, subject, and quarantine reason:

• Release: Deliver the message.
• Delete: Delete the message.
• Release and report: Release and report only false positives classified as Spam or Probable spam to SophosLabs.

Virus-infected messages and messages classified as malicious by Zero-Day Protection can’t be released. Deleting Zero-Day Protection entries requires write permission for that function. When quarantine is full, older messages are purged.

Only users who have authenticated with the firewall at least once receive quarantine digests. Check alias addresses separately in the quarantine design; messages to aliases don’t appear in the User Portal.

Email > Mail spool contains messages that haven’t yet been delivered or have failed. SFOS retries delivery for three days and discards messages after another four days; discarded messages remain visible in Mail logs. A growing spool is therefore an alert for routing, DNS, TLS, policy, or mail server problems, not a reason to repeatedly select Retry without diagnosis.

Quarantine and spool use local storage. Monitor free space, SSD status, and System Health; see Clean up storage and reports and Check SSD Health. In HA, logs and reports are separate for each node and aren’t synchronized, so check both nodes. Sophos Firewall HA clusters covers the fundamentals.

Troubleshooting

External email doesn’t arrive

Check MX, A/AAAA, public IP, TCP 25, and SMTP Relay from WAN. Then verify that MTA mode is active, the protected domain matches the policy, and no old DNAT or higher-priority firewall rule changes the expected path. If smtpd_main.log shows no connection, the problem is probably before Mail Protection.

The firewall accepts email but doesn’t deliver it

Check the internal mail server, route, DNS, target port, TLS, and recipient verification. Static host, DNS host, and MX use different resolution and failover paths. Correlate the firewall’s reject and error logs with the mail server logs.

Many messages remain in the spool

First inspect Email > Mail spool and the MTA logs. A common cause is a rule above the automatically generated MTA rule that already matches SMTP. Under Rules and policies > Firewall rules, check new rules positioned at Top, automatically generated IPsec or hotspot rules, and other overlaps. Don’t move a rule blindly; its actual SMTP match is what matters.

SFOS 22.0 MR2 also fixes NC-177930, where messages remained in the spool after a mailpoller crash. Include the firmware version in the diagnosis when the issue occurs on an older release.

Central reports Invalid API request

On SFOS 22.0 MR1, Release and Delete could fail when the firewall was opened through Sophos Central. The safe workaround is to sign in directly to the local WebAdmin and perform the action under Email > SMTP quarantine. SFOS 22.0 MR2 Build 546 fixed the issue. If it still fails there, check the admin profile, Central access, and local permissions separately.

Reject based on RBL can’t be enabled

Sophos lists a version-specific case under NC-144563 for SFOS 20.0.2 MR2 Build 378: If the default RBL groups under Email > Address group have been renamed, Reject based on RBL can’t be enabled when creating an SMTP route and scan policy. In an existing policy, the option can’t be turned on again after it has been disabled. The current Known Issues List doesn’t specify a fixed version.

The default groups are named Premium RBL services and Standard RBL services. First document the exact build and current group names. If both conditions match, restore the original names of the default RBLs, reopen the policy, and check the option. Don’t confuse custom RBLs with these two system groups.

On other builds, or when the default names are unchanged, a grayed-out option doesn’t prove NC-144563. Check the policy type, Spam protection, Email Protection license, and the remaining configuration separately.

A legitimate sender is classified as spam

Check the sender domain, SPF/DKIM/DMARC, headers, reputation, policy match, and affected recipients. Only then create a narrow exception with a review date.

Internal systems can’t relay

Check the source zone under Administration > Device access, the host object under Email > Relay settings > Host-based relay, and the MTA logs. If a scanner expects standard SMTP AUTH, Host-based relay is usually more reliable. Test the separate, non-RFC-compliant Authenticated relay with the specific client.

Operations checklist

• License, model support, DNS, and required internet services are verified.
• MX, TTL, public and internal mail paths, and rollback are documented.
• The automatic MTA rule is unchanged and remains at the top.
• No old DNAT rule bypasses Mail Protection.
• Address Group, routing destination, and policy match are tested.
• Inbound SMTP Relay from WAN and outbound host permissions are separated correctly.
• The effects of SPF/RBL, recipient verification, TLS, DKIM, SPX, and banners are understood.
• External positive and negative tests were completed.
• Quarantine, spool, storage, and logs are monitored.
• Mail flow is retested after firmware updates.

For longer retention and correlation, use Central Firewall Reporting or send Sophos Firewall logs to a SIEM.

FAQ