Skip to content
Avanet

Configure Sophos Firewall Mail Protection in legacy mode

In Legacy mode, Sophos Firewall operates as a transparent mail proxy. The internal mail server remains the actual SMTP endpoint; the firewall forwards traffic through the existing firewall and NAT rules while checking it for spam, malware, file types, or data control matches.

This is fundamentally different from MTA mode. The firewall does not become a mail transfer agent, does not take over mail delivery for protected domains, and does not provide an MTA mail spool for this path. A successful SMTP port test therefore proves neither proxy scanning nor the correct policy.

⚠️ Changing the SMTP deployment mode is a global change to the mail protection path. Before switching, document a backup, existing policies, firewall and NAT rules, and a tested fallback path. An MTA issue is not a reason to switch to legacy mode without planning.

Legacy mode in nine steps

  1. Document the existing inbound and outbound SMTP path, including IP addresses, ports, NAT, and Firewall Rule IDs.
  2. Check whether transparent proxy operation really fits better than MTA mode.
  3. Secure a configuration backup and independent management access.
  4. Under Email > General settings, select Switch to legacy mode.
  5. Define the SMTP size limit, oversize action, IP reputation, DoS limits, and TLS behavior.
  6. Create only the required SMTP malware and SMTP spam policies, or verify their order.
  7. Restrict inbound DNAT and outbound SNAT paths to the actual mail server.
  8. In the firewall rules that actually match, turn on Scan SMTP or Scan SMTPS.
  9. Validate inbound and outbound test emails against the Rule ID, policy result, certificate, and legacy proxy logs.

Choose legacy mode or MTA mode

Legacy mode mainly suits existing environments where the internal mail server is already published directly through NAT and that path must remain in place. SFOS sits transparently between the remote peer and the server. MX target, SMTP acceptance, and delivery logic remain part of the existing mail server design.

MTA mode is the better choice when the firewall must accept emails itself, route them by protected domain, relay them, and retain them in a spool during temporary delivery failures. Mail logs and mail spool explicitly belong to this operating model. The complete setup is described in Configure Mail Protection in MTA mode.

According to the SFOS 22 help, MTA mode is not available on XGS 87/87w and XGS 88/88w. However, this does not automatically make legacy mode a suitable cloud email architecture. Microsoft 365, Google Workspace, and modern hosted services use their own TLS, authentication, and anti-abuse requirements. Their support for a transparent proxy must be confirmed first.

In short: MTA mode has its own mail flow. Legacy mode protects an already working SMTP path. Mixing these models leads to troubleshooting in the wrong log, at the wrong NAT destination, or in a spool that does not exist.

Example topology and fallback path

The following example uses documentation values and must be adapted to the actual environment before implementation:

  • internal mail server 10.20.30.25 in the DMZ zone
  • public SMTP address 192.0.2.25 on the WAN path
  • inbound SMTP service TCP 25
  • optional SMTPS on TCP 465, only if the peers and server actually use this variant
  • firewall rules SMTP_In_Legacy and SMTP_Out_Legacy

192.0.2.25 belongs to the TEST-NET range and is not a production value. Before the change, run an external inbound test and an outbound test with timestamps. Save the currently matched Rule IDs, the public source address used by the outbound server, and the certificate chain.

The fallback path involves more than switching the mode back. New scan options, policy order, DNAT, reflexive or manual SNAT rules, and temporary tests must also be restorable to the documented previous state.

Define global SMTP settings

Size limit, oversize action, and DoS protection

Under Email > General settings > SMTP settings, Don’t scan emails greater than defines the maximum message size for scanning. In the SMTP path, the value 0 means 51,200 KB according to the SFOS 22 help, not unlimited. For larger messages, the options are Accept, Reject, and Drop.

Accept delivers an oversized message without scanning it. Reject refuses it and informs the sender, while Drop discards it without notification. This choice is a deliberate risk and operational decision. An untested Drop complicates troubleshooting; an untested Accept creates a scan gap that must be documented.

Verify sender’s IP reputation checks the sender IP before the spam criteria in the SMTP policy. The SMTP DoS values limit connections, messages, and recipients. Production limits are derived from actual mail volume and a baseline, not copied from a generic internet example.

Bypass spam check for SMTP/S authenticated connections globally skips spam checks for connections the mail server reports as authenticated. This is only acceptable after validating authentication, allowed sources, and abuse protection on this path. A successful sign-in replaces neither malware scanning nor a negative test using a connection that isn’t authenticated. Domains under Spam check exceptions are also a global bypass and aren’t used as a quick replacement for a tightly scoped exception.

The global email banner offers Inline, no conversion, MIME part, and Off. It only appears when SMTP or SMTPS scanning is active in the matching firewall rule. Because the body change can break an existing DKIM signature, validate the actual outbound path by checking the headers at the recipient.

Do not overestimate TLS because of a checkbox

Under SMTP TLS configuration, select the CA or server certificate intended for scanning. Keep Allow invalid certificate turned off. According to the help, Disable legacy TLS protocols only disables protocols earlier than TLS 1.1 and does not prove a specific TLS 1.2 or TLS 1.3 session.

Sophos also points out an important legacy limitation: the firewall establishes the TLS connection using the domain IP address instead of the domain name. If multiple domains share one IP address, certificate validation can fail. In this case, Sophos recommends another protection path such as Sophos Email Security. Do not bypass the check with Allow invalid certificate.

Require TLS negotiation enforces TLS for selected remote hosts or networks; Require sender email domains enforces it for sender domains. If the TLS connection cannot be established, SFOS discards the affected emails. Skip TLS negotiation intentionally allows unencrypted SMTP connections to selected peers and belongs only in documented exceptions.

Use scan policies deliberately

After the Email Protection subscription is activated, Sophos Firewall automatically applies the default policy default-smtp-av to SMTP traffic in legacy mode. Custom policies are created under Email > Policies and processed in list order. First verify which existing policy matches the specific sender and recipient.

SMTP malware scan

An SMTP malware scan policy controls blocked file types, MIME exceptions, antivirus scanning, and delivery actions. With Single antivirus, the selected engine applies only to inbound messages according to the help; outbound messages are scanned by both engines. Dual antivirus runs the primary and secondary engines sequentially.

The Quarantine action is combined with the recipient and administrator actions. Don’t deliver, Deliver original, and Remove and deliver have very different consequences. A protected or unscannable attachment must not automatically be equated with malware. Each action therefore needs a test message, an expected recipient state, and a documented release path.

Quarantine doesn’t automatically mean that the recipient receives no message; the Delivery option for recipient remains decisive. According to Sophos, Notify sender only works together with Don’t deliver. Protected attachments aren’t scanned but can still trigger a notification. The separate administrator action determines whether no copy, the original, or a message without the attachment is sent to administrators. Don’t infer all four outcomes from one successful test message.

SMTP spam scan

An SMTP spam scan policy can match spam classification, source or destination, RBL, message size, headers, or a data control list. Depending on the path, the actions include Reject, Accept, Change recipient, Prefix subject, Drop, and Quarantine.

The Data control list criterion and SPX assignment in this policy only apply to outbound messages. None, however, applies the selected action to all messages between the specified sender and recipient groups. Change recipient doesn’t additionally deliver to the original recipient, but replaces it with the configured destination. Test these three scopes with one positive and one negative recipient case before placing the policy in the production order.

Prepare custom file types and Data Control

In legacy mode, create custom file types under Email > Policies > File type > Add from a template or from file extensions or MIME types. Enter extensions without a leading dot; only custom types can be edited. A new type isn’t added automatically to existing policies. Open the affected scan policy, add the file type, and save the policy again. A positive attachment test and a similar negative test show whether the intended policy action really applies.

Build a Data Control List under Email > Data control list > Add from the required Content Control Lists. The Type and Region filters help select only relevant financial, identity, or other sensitive data patterns. A list match doesn’t define an action; the linked scan policy does. A small pilot list with a positive and a negative content test is safer than a broad collection of unverified CCLs.

SPX can be selected in this policy for outbound messages in legacy mode. However, the password model, portal, and validation form a separate security workflow; see Configure SPX email encryption. Do not add a data control list or SPX assignment to the first basic proxy test.

Don’t disable a broad policy because of a confirmed misclassification. Create and test email exceptions safely explains how to skip individual checks for a precise source, sender, and recipient combination and then test non-matching traffic.

Use optional email journaling with data protection in mind

Under Email > General settings > Email journaling > Add, SFOS can send copies of inbound SMTP/S messages for selected recipients or address groups to a separate journal address. Selecting Any captures all inbound messages. The function applies only to SMTP/S; it does not journal POP or IMAP traffic.

Journaling creates an additional email copy. It is not automatically a tamper-proof archive and does not prove that legal retention obligations are met. Before enabling it, define the purpose, authorized recipients, access to the journal mailbox, encryption, retention period, storage requirements, and responsible owner.

For the first test, select a single test mailbox instead of Any. An inbound message to this mailbox must appear at the normal destination and in the journal mailbox; a message to a recipient outside the selection must not create a journal copy. The journal address must not trigger a mail flow that sends the copy back to SFOS and creates a loop.

Only extend the recipient selection after the positive and negative tests. For rollback, remove the journal entry or restore the documented previous state. Copies already delivered remain in the journal mailbox and must be handled according to its own retention rules.

Combine NAT and firewall rules

Publish the inbound SMTP path

For inbound email, a DNAT rule translates the public WAN address to the internal mail server. Restrict Original Source as tightly as the mail design permits; Original Destination is the intended public address; Translated Destination is 10.20.30.25 or the actual mail server. Original and translated service remain limited to the SMTP ports actually offered.

A reflexive rule also creates SNAT for the reverse direction. Select it only if this exact public source identity is intended for the outbound path. Multiple WAN links, smarthosts, or different provider routes require their own routing and SNAT design. General ordering and the destination zone after NAT are explained in Publish a server with DNAT.

Use two tightly scoped firewall rules

For validation, separate rules are easier to understand than one bidirectional rule containing several zones and Any objects:

  • Inbound: WAN to the internal mail server zone, destination host 10.20.30.25, only required SMTP services, logging enabled
  • Outbound: internal mail server zone and host to WAN or the specific smarthost, only required SMTP services, logging enabled

Under Scan email content, turn on Scan SMTP in both required directions and Scan SMTPS only when SMTPS is actually used. An enabled checkbox does not automatically add a missing service to a correct security design. Service, NAT, server listener, and scan option must describe the same port path.

Place the rules above more general rules that already match the same traffic. After saving, the logged Firewall Rule ID is decisive. Rule structure, NAT zone, and order are covered in Configure Sophos Firewall rules safely.

Validate mail flow and proxy scanning

First send a small external message to a test mailbox. Then have the internal mail server send a second message to a controlled external recipient. Give both tests unique subjects and UTC timestamps.

For STARTTLS on port 25 and a direct TLS connection on port 465, the following read-only checks from an authorized test system can help:

openssl s_client -starttls smtp -connect mail.example.net:25 -servername mail.example.net
openssl s_client -connect mail.example.net:465 -servername mail.example.net

Replace mail.example.net with the actual FQDN and test only ports that are really offered. OpenSSL confirms reachability, certificate chain, and negotiated TLS parameters. It proves neither successful mail delivery nor malware, spam, or data control scanning.

In Log Viewer, source, destination, service, action, and Firewall Rule ID must match the new rules. For the legacy SMTP/S proxy, correlate awarrensmtp.log and awarrenmta.log with the same timestamp. The log files are explained in Sophos Firewall services and logs.

Then perform a negative test. An unintended source, an unapproved port, or a test message without the policy criterion must not accidentally receive the same protection path. Do not use real malware in production testing; use established harmless test patterns and a controlled mailbox for scan validation.

Troubleshoot by symptom

SMTP works, but the policy does not apply

Check the Firewall Rule ID first. If another rule matches, correct the order, source, destination, NAT target, and service. If the expected rule matches, Scan SMTP or Scan SMTPS, policy order, and sender and recipient groups must fit the test. A policy alone does not activate the transparent proxy.

Inbound mail does not reach the server

Check the public destination address, DNAT hit, translated destination, destination zone, server listener, and return path separately. An open TCP connection to the firewall does not prove that DNAT and the firewall rule reach the internal server. Packet capture and Rule ID must show ingress and forwarding.

Outbound mail uses the wrong public IP address

Check SNAT, the reflexive rule, WAN gateway, SD-WAN route, and reply packet behavior. The legacy proxy does not automatically select the public source address required for SPF, RDNS, or provider authorization. Prove the specific path before making a global route precedence change.

TLS fails after scanning is enabled

Record the FQDN, destination IP, certificate, issuer, chain, and negotiated version. With multiple domains on one IP address, the documented IP-based certificate check can be the cause. Do not enable Allow invalid certificate as a quick fix.

A message is missing and nothing appears in the MTA spool

This is not a useful success criterion in legacy mode because mail spool and MTA-specific mail logs belong to MTA mode. The relevant chain here consists of the firewall rule, NAT, SMTP server logs, Log Viewer, awarrensmtp.log, and awarrenmta.log. Check quarantined messages separately under Email > SMTP quarantine.

Roll back safely

For rollback, first restore the pilot rules and scan options to their documented previous state. Then remove new policy assignments or restore their order. Remove temporary DNAT, SNAT, or certificate changes only when no other service depends on them.

Only then restore the SMTP deployment mode if the change included that switch. The original inbound and outbound mail flow must work again with the expected Rule IDs, public addresses, and server logs. Do not delete messages, quarantine contents, or proxy logs as a standard rollback action.

Operations checklist

  • Transparent proxy operation is a deliberate choice and MTA requirements have been excluded.
  • Backup, management access, and original Rule IDs are documented.
  • SMTP size limit, oversize action, IP reputation, and DoS limits are justified.
  • Certificate, TLS exceptions, and affected domains have been checked.
  • DNAT, SNAT, destination zone, and actual server ports match.
  • Inbound and outbound firewall rules are tight, logged, and proven to match.
  • Default and custom policies have a traceable order.
  • Optional journaling is limited to the required recipients and has a documented data-protection and retention purpose.
  • Positive, negative, TLS, and delivery tests have passed.
  • Legacy proxy logs and mail server logs can be correlated by time.
  • Owner, review date, and complete fallback path are recorded.

FAQ

Is legacy mode simpler and therefore better than MTA mode?

Not generally. Legacy mode protects an existing SMTP path as a transparent proxy. MTA mode accepts emails itself, routes them by protected domain, and provides mail logs and mail spool. The right choice depends on the intended mail flow.

Is an SMTP malware or spam policy sufficient for scanning?

No. Under Scan email content, the firewall rule that actually matches must enable the SMTP protocol in use. Policy, service, NAT, and rule match must all align.

Why can't I find the message in mail spool in legacy mode?

Because mail spool and MTA-specific mail logs belong to MTA mode. In legacy mode, check the firewall and NAT path, mail server logs, Log Viewer, awarrensmtp.log, and awarrenmta.log.