Skip to content
Avanet

Configure Sophos Firewall Mail Protection in legacy mode

In Legacy mode, Sophos Firewall operates as a transparent mail proxy. The internal mail server remains the actual SMTP endpoint; the firewall forwards traffic through the existing firewall and NAT rules while checking it for spam, malware, file types, or data control matches.

This is fundamentally different from MTA mode. The firewall does not become a mail transfer agent, does not take over mail delivery for protected domains, and does not provide an MTA mail spool for this path. A successful SMTP port test therefore proves neither proxy scanning nor the correct policy.

⚠️ Changing the SMTP deployment mode is a global change to the mail protection path. Before switching, document a backup, existing policies, firewall and NAT rules, and a tested fallback path. An MTA issue is not a reason to switch to legacy mode without planning.

Legacy mode in nine steps

  1. Document the existing inbound and outbound SMTP path, including IP addresses, ports, NAT, and Firewall Rule IDs.
  2. Check whether transparent proxy operation really fits better than MTA mode.
  3. Secure a configuration backup and independent management access.
  4. Under Email > General settings, select Switch to legacy mode.
  5. Define the SMTP hostname, size limits, oversize action, IP reputation, DoS limits, ports, and TLS behavior.
  6. Create only the required SMTP malware and SMTP spam policies, or verify their order.
  7. Restrict inbound DNAT and outbound SNAT paths to the actual mail server.
  8. In the firewall rules that actually match, turn on Scan SMTP or Scan SMTPS.
  9. Validate inbound and outbound test emails against the Rule ID, policy result, certificate, and legacy proxy logs.

Choose legacy mode or MTA mode

Legacy mode mainly suits existing environments where the internal mail server is already published directly through NAT and that path must remain in place. SFOS sits transparently between the remote peer and the server. MX target, SMTP acceptance, and delivery logic remain part of the existing mail server design.

MTA mode is the better choice when the firewall must accept emails itself, route them by protected domain, relay them, and retain them in a spool during temporary delivery failures. Mail logs and mail spool explicitly belong to this operating model. The complete setup is described in Configure Mail Protection in MTA mode.

According to the SFOS 22 help, MTA mode is not available on XGS 87/87w and XGS 88/88w. However, this does not automatically make legacy mode a suitable cloud email architecture. Microsoft 365, Google Workspace, and modern hosted services use their own TLS, authentication, and anti-abuse requirements. Their support for a transparent proxy must be confirmed first.

In short: MTA mode has its own mail flow. Legacy mode protects an already working SMTP path. Mixing these models leads to troubleshooting in the wrong log, at the wrong NAT destination, or in a spool that does not exist.

Example topology and fallback path

The following example uses documentation values and must be adapted to the actual environment before implementation:

  • internal mail server 10.20.30.25 in the DMZ zone
  • public SMTP address 192.0.2.25 on the WAN path
  • inbound SMTP service TCP 25
  • optional SMTPS on TCP 465, only if the peers and server actually use this variant
  • firewall rules SMTP_In_Legacy and SMTP_Out_Legacy

192.0.2.25 belongs to the TEST-NET range and is not a production value. Before the change, run an external inbound test and an outbound test with timestamps. Save the currently matched Rule IDs, the public source address used by the outbound server, and the certificate chain.

The fallback path involves more than switching the mode back. New scan options, policy order, DNAT, reflexive or manual SNAT rules, and temporary tests must also be restorable to the documented previous state.

Define global email settings

Where MTA mode is available, the help identifies it as the default mode. Under Email > General settings, deliberately select Switch to legacy mode after securing the documented fallback path. Switching modes alone does not create a validated legacy mail path.

Under Email > General settings, SMTP hostname only applies to system-generated notification emails in HELO and SMTP banner strings; its default is SMTP. It replaces neither the production mail server FQDN nor its public DNS. If you change it, validate an SFOS notification rather than only a normal server-generated message.

XML API check: The SFOS 23 documentation lists operation status 506 for Email Configuration, paired with Message.AVGeneralConfInvalidSmtpNtfyHostname. This is an unresolved message key, not an English error message or an HTTP status code. If this status appears in the actual XML API response, check the submitted notification hostname, the operation response, and the saved configuration before treating the change as successful. The key does not establish specific validation rules. The additional documentation entry proves neither that the behavior was introduced in SFOS 23 nor that it applies to REST.

Size limit, oversize action, and DoS protection

Under Email > General settings > SMTP settings, Don’t scan emails greater than defines the maximum message size for scanning. In the SMTP path, the value 0 means 51,200 KB according to the SFOS 22 help, not unlimited. For larger messages, the options are Accept, Reject, and Drop.

Accept delivers an oversized message without scanning it. Reject refuses it and informs the sender, while Drop discards it without notification. This choice is a deliberate risk and operational decision. An untested Drop complicates troubleshooting; an untested Accept creates a scan gap that must be documented.

Verify sender’s IP reputation checks the sender IP before the spam criteria in the SMTP policy and rejects emails from IP addresses with a bad reputation. The six SMTP DoS fields accept these ranges in SFOS 22:

  • Maximum connections: 1 to 20000 connections to the mail server
  • Maximum connections/host: 1 to 10000 connections from one host
  • Maximum emails/connection: 1 to 1000 messages per connection
  • Maximum recipients/email: 1 to 256 recipients per message
  • Emails rate: 1 to 20000 messages from one host per minute
  • Connections rate: 1 to 20000 connections from one host per second

An accepted value is not automatically a suitable value. Derive production limits from actual mail volume and a baseline so that legitimate sending peaks are not treated as a DoS.

Bypass spam check for SMTP/S authenticated connections globally skips spam checks for connections the mail server reports as authenticated. This is only acceptable after validating authentication, allowed sources, and abuse protection on this path. A successful sign-in replaces neither malware scanning nor a negative test using a connection that isn’t authenticated. Domains under Spam check exceptions are also a global bypass and aren’t used as a quick replacement for a tightly scoped exception.

The global email banner offers Inline, no conversion, MIME part, and Off. It only appears when SMTP or SMTPS scanning is active in the matching firewall rule. Because the body change can break an existing DKIM signature, validate the actual outbound path by checking the headers at the recipient.

Plan POP/IMAP and ports separately

POP/IMAP has its own TLS certificate selection for the CA under POP and IMAP TLS configuration. Keep Allow invalid certificate off to reject invalid server certificates; Disable legacy TLS protocols also blocks only protocols earlier than TLS 1.1 here. SMTP settings do not replace this configuration. The complete retrieval path is covered in POP/IMAP email scanning; CA and certificate administration in Certificate management.

Legacy mode also covers transparent POP3/S and IMAP/S scanning. Under POP/S and IMAP/S settings, 0 for Don’t scan emails greater than means 10,240 KB. The Recipient headers setting determines which message headers SFOS searches for the recipients used by POP/IMAP policies; the defaults are Delivered-To, Received, and X-RCPT-TO. Change this setting only when a captured message header from the mail server in use justifies it.

The documented standard ports are SMTP 25, SMTPS 465, POP3 with STARTTLS 110, POP3S 995, IMAP with STARTTLS 143, and IMAPS 993. SFOS can also scan email on other ports; the help explicitly mentions port 587. An alternative port must be present as a service in the matching firewall rule and associated with the scan protocol that is actually enabled. Changing a port alone does not turn implicit TLS into STARTTLS or vice versa.

A POP-IMAP scan policy can combine sender and recipient groups with criteria such as inbound classification, source IP address, message size, or message header. The only actions available here are Accept and Prefix subject. The policy does not activate the proxy by itself: Under Scan email content in the matching firewall rule, you must also select the POP3, POP3S, IMAP, or IMAPS protocol actually in use.

Do not overestimate TLS because of a checkbox

Under SMTP TLS configuration, select the CA or server certificate intended for scanning. Keep Allow invalid certificate turned off. According to the help, Disable legacy TLS protocols only disables protocols earlier than TLS 1.1 and does not prove a specific TLS 1.2 or TLS 1.3 session.

Sophos also points out an important legacy limitation: the firewall establishes the TLS connection using the domain IP address instead of the domain name. If multiple domains share one IP address, certificate validation can fail. In this case, Sophos recommends another protection path such as Sophos Email Security. Do not bypass the check with Allow invalid certificate.

Require TLS negotiation enforces TLS for selected remote hosts or networks; Require sender email domains enforces it for sender domains. If the TLS connection cannot be established, SFOS discards the affected emails. Skip TLS negotiation intentionally allows unencrypted SMTP connections to selected peers and belongs only in documented exceptions. According to the help, each field accepts up to 512 host entries; rule out overlap between enforcement and bypass lists before using broad selections.

Use scan policies deliberately

Without a valid Email Protection subscription, mail traffic may still be allowed but receives no Email Protection. Reachability is therefore not proof of licensing or scanning. Global email settings are processed before SMTP and POP-IMAP policies. Under Email > General settings > SMTP settings, enable IP reputation in legacy mode with Verify sender’s IP reputation and save with Apply; the MTA control is instead called Reject based on IP reputation. The global outbound email banner supports text only.

After the Email Protection subscription is activated, Sophos Firewall automatically applies the default policy default-smtp-av to SMTP traffic in legacy mode. Custom policies are created under Email > Policies and processed in list order. First verify which existing policy matches the specific sender and recipient.

Create new legacy SMTP policies under Email > Policies > Add policy, using SMTP malware scan or SMTP spam scan. To update an existing policy, go to Email > Policies, select Edit for the relevant policy, change the required settings, and save the policy. Then recheck the policy order and repeat the positive and negative tests.

Under Email > General settings > Malware protection, select the primary antivirus engine globally as Sophos or Avira. If you select Avira, SFOS turns off Zero-day Protection in SMTP policies that use Single antivirus. Inventory all affected single-antivirus policies before changing the engine and validate them again afterward.

SMTP malware scan

When creating the policy, specify a clear name and the sender and recipient email address or domain groups. Under Block file types, select the attachments to block; use Ctrl+Shift to select multiple types. MIME whitelist allows the selected MIME headers, while the remaining file types are blocked according to the help. This file-type allowance is not an antivirus bypass. Disable, by contrast, explicitly means that emails are not scanned.

The Delivery option for recipient applies to infected, protected and suspicious attachments:

  • Don’t deliver: neither the message nor a notification is sent to the recipient.
  • Deliver original: sends the original message to the recipient; this does not clean it.
  • Remove and deliver: removes the infected attachment, sends a removal notification and delivers the message. This action does not apply to file types specified under Block file types; this does not establish another delivery action for blocked types.

Protected attachments remain unscanned; the recipient is notified by default unless configured otherwise. Save the configuration with Save and check the actual delivery and notification results with harmless positive and negative cases.

An SMTP malware scan policy controls blocked file types, MIME exceptions, antivirus scanning, and delivery actions. With Single antivirus, the selected engine applies only to inbound messages according to the help; outbound messages are scanned by both engines. Dual antivirus runs the primary and secondary engines sequentially.

The Quarantine action is combined with the recipient and administrator actions. Don’t deliver, Deliver original, and Remove and deliver have very different consequences. A protected or unscannable attachment must not automatically be equated with malware. Each action therefore needs a test message, an expected recipient state, and a documented release path.

Quarantine doesn’t automatically mean that the recipient receives no message; the Delivery option for recipient remains decisive. According to Sophos, Notify sender only works together with Don’t deliver. Protected attachments aren’t scanned but can still trigger a notification. The separate administrator action determines whether no copy, the original, or a message without the attachment is sent to administrators. Don’t infer all four outcomes from one successful test message.

⚠️ The delivery mapping for the administrator option Remove attachment is not conclusively resolved. The summary above is not an approved delivery instruction for this option. Resolve the recipient destination and administrator notification independently before production use; do not infer the administrator action from recipient actions.

SMTP spam scan

Specify a name and the sender and recipient email address or domain groups. Deliberately choose exact or keyword matching for these groups: a keyword search must not unintentionally broaden the scope. Do not assume undocumented regex, wildcard or AND/OR syntax.

  • Inbound email is and Outbound email is: choose the direction separately; criteria are spam, probable spam, virus outbreak or probable virus outbreak.
  • Source IP/network address and Destination IP/network address: specify the required sender or recipient IP address or network.
  • Sender remote blacklist: select the appropriate sender RBL group.
  • Message size: set an upper or lower size limit; do not confuse this with the global oversize limit.
  • Message header: select the header; for Other, enter the details, choose exact or keyword matching and specify the keyword value.

Reject refuses the message and informs the sender. Accept delivers it. Prefix subject adds to the subject and also delivers it; for example, a Probable spam prefix labels a controlled test. Drop discards without notifying the sender; Quarantine retains the message in quarantine. SPX template selection is documented for Accept and Prefix subject, exclusively for outbound emails. This establishes neither SPX support for every other action nor automatic encryption of every accepted message. Save with Save and check matching and action separately.

An SMTP spam scan policy can match spam classification, source or destination, RBL, message size, headers, or a data control list. Depending on the path, the actions include Reject, Accept, Change recipient, Prefix subject, Drop, and Quarantine.

The Data control list criterion and SPX assignment in this policy only apply to outbound messages. None, however, applies the selected action to all messages between the specified sender and recipient groups. Change recipient doesn’t additionally deliver to the original recipient, but replaces it with the configured destination. Test these three scopes with one positive and one negative recipient case before placing the policy in the production order.

Prepare custom file types and Data Control

Give the custom file type a name and save the object with Save before changing the policy. The separate MTA creation path is Email > Policies and exceptions > File type > Add; it does not replace the legacy path. File types are also used in web policies; their configuration belongs in Web Protection. Also give the Data Control List a clear name and select Save after choosing the CCLs.

In legacy mode, create custom file types under Email > Policies > File type > Add from a template or from file extensions or MIME types. Enter extensions without a leading dot; only custom types can be edited. A new type isn’t added automatically to existing policies. Open the affected scan policy, add the file type, and save the policy again. A positive attachment test and a similar negative test show whether the intended policy action really applies.

Build a Data Control List under Email > Data control list > Add from the required Content Control Lists. The Type and Region filters help select only relevant financial, identity, or other sensitive data patterns. A list match doesn’t define an action; the linked scan policy does. A small pilot list with a positive and a negative content test is safer than a broad collection of unverified CCLs.

SPX can be selected in this policy for outbound messages in legacy mode. However, the password model, portal, and validation form a separate security workflow; see Configure SPX email encryption. Do not add a data control list or SPX assignment to the first basic proxy test.

Don’t resolve a confirmed misclassification by disabling a broadly applied policy. In legacy mode, make targeted changes to the appropriate SMTP malware scan or SMTP spam scan policy; a positive test and a negative test outside the intended scope must confirm the effect. Create and test email exceptions safely instead covers only the exception object in MTA mode: source, sender, or recipient are alternative triggers (OR), not an exact AND condition combining all three attributes. This MTA exception object is not a legacy control.

Use optional email journaling with data protection in mind

Give the journal entry a clear name and save the recipient selection and journal destination with Save before starting the following tests.

Under Email > General settings > Email journaling > Add, SFOS can send copies of inbound SMTP/S messages for selected recipients or address groups to a separate journal address. Selecting Any captures all inbound messages. The function applies only to SMTP/S; it does not journal POP or IMAP traffic.

Journaling creates an additional email copy. It is not automatically a tamper-proof archive and does not prove that legal retention obligations are met. Before enabling it, define the purpose, authorized recipients, access to the journal mailbox, encryption, retention period, storage requirements, and responsible owner.

For the first test, select a single test mailbox instead of Any. An inbound message to this mailbox must appear at the normal destination and in the journal mailbox; a message to a recipient outside the selection must not create a journal copy. The journal address must not trigger a mail flow that sends the copy back to SFOS and creates a loop.

Only extend the recipient selection after the positive and negative tests. For rollback, remove the journal entry or restore the documented previous state. Copies already delivered remain in the journal mailbox and must be handled according to its own retention rules.

Combine NAT and firewall rules

Publish the inbound SMTP path

For inbound email, a DNAT rule translates the public WAN address to the internal mail server. Restrict Original Source as tightly as the mail design permits; Original Destination is the intended public address; Translated Destination is 10.20.30.25 or the actual mail server. Original and translated service remain limited to the SMTP ports actually offered.

A reflexive rule also creates SNAT for the reverse direction. Select it only if this exact public source identity is intended for the outbound path. Multiple WAN links, smarthosts, or different provider routes require their own routing and SNAT design. General ordering and the destination zone after NAT are explained in Publish a server with DNAT.

Use two tightly scoped firewall rules

To create a rule under Rules and policies > Firewall rules, select the actual protocol family IPv4 or IPv6 and open Add firewall rule > New firewall rule. Specify the name and required fields, then save with Save. Do not reuse this article’s IPv4 example addresses in an IPv6 rule.

⚠️ The Destination networks mapping in the inbound legacy mail-proxy example has not been conclusively reconciled with the general DNAT workflow. Do not blindly use the internal destination host below as a universal field value or replace it with the public address on speculation. Resolve this mapping for the specific proxy path before production use; the outbound SNAT variant does not settle it.

For validation, separate rules are easier to understand than one bidirectional rule containing several zones and Any objects:

  • Inbound: WAN to the internal mail server zone, destination host 10.20.30.25, only required SMTP services, logging enabled
  • Outbound: internal mail server zone and host to WAN or the specific smarthost, only required SMTP services, logging enabled

Under Scan email content, turn on Scan SMTP in both required directions and Scan SMTPS only when SMTPS is actually used. If the standard port for the selected protocol is missing from Services, SFOS offers Add ports. For a nonstandard port, define the matching service deliberately instead. Service, NAT, server listener, and scan option must describe the same port path; Add ports does not replace that check.

Place the rules above more general rules that already match the same traffic. After saving, the logged Firewall Rule ID is decisive. Rule structure, NAT zone, and order are covered in Configure Sophos Firewall rules safely.

Optional: use a different public source address outbound

This setup applies to forwarded SMTP traffic from the internal server, not system-generated MTA messages. It is useful when a different assigned public identity is required instead of the WAN MASQ address. Provider allocation, routing, SPF, RDNS and provider authorisation must fit that identity; SNAT does not change these prerequisites.

  1. Under Network > Interfaces > Add interface > Add alias, select the actual physical WAN interface, IP version IPv4 and the assigned address with the appropriate IPv4/Netmask; Save. Provider path, Device Access and alias checks are covered in Configure an alias IP.
  2. Under Hosts and services > IP host > Add, create a named host for the same public address: IP family IPv4, Type IP, IP address the assigned address; Save. A host object alone does not bind an address to the interface.
  3. Under Rules and policies > NAT rules > New NAT rule, restrict Original source to the actual internal mail server and set Translated source (SNAT) to the public IP host. Restrict Original destination to the required mail destinations or smarthost and Original service to the required SMTP services. Translated destination (DNAT) and Translated service (PAT) remain Original. Inbound interface and Outbound interface must match the actual server-to-WAN path. Save with Save and place it above other SNAT rules that also match; check NAT and firewall order separately. NAT basics explains first matching rule behaviour. Any, Top, Port1 and Port2 in examples are not mandatory values.
  4. Under Rules and policies > Firewall rules > Add firewall rule > New firewall rule, create a suitable outbound IPv4 rule or check the existing narrow rule: Action Accept, actual server zone and server host as source, WAN and required destinations, only required SMTP services, Log firewall traffic enabled. Select Scan SMTP and, where applicable, Scan SMTPS as above; Save and check firewall order independently of NAT.
  5. Create a new SMTP connection to a controlled external recipient. Check Firewall Rule ID, NAT Rule ID, translated source IP in capture or at the external recipient and actual delivery together. An existing session or SMTP port test is insufficient. Also negatively test a source outside the scope or a service that is not allowed.

For reversal, first restore the documented rules and their order, then check the previous source IP and delivery using a new connection. Remove the alias and IP host only when no other service depends on them.

Validate mail flow and proxy scanning

First send a small external message to a test mailbox. Then have the internal mail server send a second message to a controlled external recipient. Give both tests unique subjects and UTC timestamps.

For STARTTLS on port 25 and a direct TLS connection on port 465, the following read-only checks from an authorized test system can help:

openssl s_client -starttls smtp -connect mail.example.net:25 -servername mail.example.net
openssl s_client -connect mail.example.net:465 -servername mail.example.net

Replace mail.example.net with the actual FQDN and test only ports that are really offered. OpenSSL confirms reachability, certificate chain, and negotiated TLS parameters. It proves neither successful mail delivery nor malware, spam, or data control scanning.

In Log Viewer, source, destination, service, action, and Firewall Rule ID must match the new rules. For the legacy SMTP/S proxy, correlate awarrensmtp.log and awarrenmta.log with the same timestamp. The log files are explained in Sophos Firewall services and logs.

Then perform a negative test. An unintended source, an unapproved port, or a test message without the policy criterion must not accidentally receive the same protection path. Do not use real malware in production testing; use established harmless test patterns and a controlled mailbox for scan validation.

Troubleshoot by symptom

SMTP works, but the policy does not apply

Check the Firewall Rule ID first. If another rule matches, correct the order, source, destination, NAT target, and service. If the expected rule matches, Scan SMTP or Scan SMTPS, policy order, and sender and recipient groups must fit the test. A policy alone does not activate the transparent proxy.

Inbound mail does not reach the server

Check the public destination address, DNAT hit, translated destination, destination zone, server listener, and return path separately. An open TCP connection to the firewall does not prove that DNAT and the firewall rule reach the internal server. Packet capture and Rule ID must show ingress and forwarding.

Outbound mail uses the wrong public IP address

Check SNAT, the reflexive rule, WAN gateway, SD-WAN route, and reply packet behavior. The legacy proxy does not automatically select the public source address required for SPF, RDNS, or provider authorization. Prove the specific path before making a global route precedence change.

TLS fails after scanning is enabled

Record the FQDN, destination IP, certificate, issuer, chain, and negotiated version. With multiple domains on one IP address, the documented IP-based certificate check can be the cause. Do not enable Allow invalid certificate as a quick fix.

A message is missing and nothing appears in the MTA spool

This is not a useful success criterion in legacy mode because mail spool and MTA-specific mail logs belong to MTA mode. The relevant chain here consists of the firewall rule, NAT, SMTP server logs, Log Viewer, awarrensmtp.log, and awarrenmta.log. Check quarantined messages separately under Email > SMTP quarantine.

Roll back safely

For rollback, first restore the pilot rules and scan options to their documented previous state. Then remove new policy assignments or restore their order. Remove temporary DNAT, SNAT, or certificate changes only when no other service depends on them.

Only then restore the SMTP deployment mode if the change included that switch. The original inbound and outbound mail flow must work again with the expected Rule IDs, public addresses, and server logs. Do not delete messages, quarantine contents, or proxy logs as a standard rollback action.

Operations checklist

  • Transparent proxy operation is a deliberate choice and MTA requirements have been excluded.
  • Backup, management access, and original Rule IDs are documented.
  • SMTP hostname, SMTP and POP/IMAP size limits, oversize action, IP reputation, and DoS limits are justified.
  • Standard or alternative ports in use and POP/IMAP recipient headers match the actual server path.
  • The primary antivirus engine and its effect on Zero-day Protection in single-antivirus policies have been checked.
  • Certificate, TLS exceptions, and affected domains have been checked.
  • DNAT, SNAT, destination zone, and actual server ports match.
  • Inbound and outbound firewall rules are tight, logged, and proven to match.
  • Default and custom policies have a traceable order.
  • Optional journaling is limited to the required recipients and has a documented data-protection and retention purpose.
  • Positive, negative, TLS, and delivery tests have passed.
  • Legacy proxy logs and mail server logs can be correlated by time.
  • Owner, review date, and complete fallback path are recorded.

FAQ

Is legacy mode simpler and therefore better than MTA mode?

Not generally. Legacy mode protects an existing SMTP path as a transparent proxy. MTA mode accepts emails itself, routes them by protected domain, and provides mail logs and mail spool. The right choice depends on the intended mail flow.

Is an SMTP malware or spam policy sufficient for scanning?

No. Under Scan email content, the firewall rule that actually matches must enable the SMTP protocol in use. Policy, service, NAT, and rule match must all align.

Why can't I find the message in mail spool in legacy mode?

Because mail spool and MTA-specific mail logs belong to MTA mode. In legacy mode, check the firewall and NAT path, mail server logs, Log Viewer, awarrensmtp.log, and awarrenmta.log.