Evaluate Sophos Phish Threat results and reports

Sophos Phish Threat provides results on three levels: The Dashboard shows organization-wide trends, the campaign view explains a single simulation, and Reports track behavior or training status across campaigns. Only the combination of these levels answers whether messages were delivered, how users responded, and which follow-up action is appropriate.

This procedure is intended for authorized administrators in the correct Sophos Fusion (formerly Sophos Central) tenant. It treats report data as personal security data: a hit is a training and risk signal, not an accusation and not proof of a real compromise.

Prepare the evaluation scope

Before the analysis, document the tenant, campaign, target group, campaign type, start and end dates, question to be answered, and desired basis for comparison. For a trend comparison, the campaign type, difficulty level, and target group must be sufficiently similar. Otherwise, the resulting rates cannot be compared meaningfully.

For a reliable evaluation, this sequence applies:

1. Check the overall situation and ongoing campaigns on the Phish Threat Dashboard.
2. In the campaign, review delivery, events, and the timeline.
3. Evaluate cross-campaign user behavior or training under Reports.
4. Trace anomalies up to the user event and delivery status.
5. Only after that, define target groups for training, a follow-up campaign, or delivery corrections.

Reading dashboard metrics correctly

Entry is via My Products > Phish Threat. Active campaigns shows active and upcoming campaigns. The dashboard shows only the two campaigns closest to completion when there are more than two active campaigns; See all leads to the full view. A missing campaign is therefore not automatically deleted or ended.

Organization summary shows average values from the last five campaigns: the average percentage of caught users and the average percentage of reporting users.

• Caught-to-open ratio indicates how many users were caught relative to the users who opened the phishing email. Caught ratio represents this value graphically.
• Report-to-open ratio indicates how many users reported relative to the users who opened the phishing email. Threat reported ratio represents this value graphically.
• Caught users prioritizes users who were most frequently deceived by simulations.
• Threat reporters shows users who have reported phishing emails.

The denominator of both rates is the opens, not all addressed or delivered messages. A high Caught-to-open ratio is therefore to be evaluated differently than a high absolute number of caught users. Likewise, a low number of opens does not automatically prove good security behavior: delivery errors or image/open tracking can influence the basis. How to distinguish between missing images, tracking pixels and open events added after a click is explained in Check missing images and open tracking.

Awareness factors adds four program indicators:

• Users tested: Proportion of the user base that was exposed to a simulation.
• Last campaign: Indicates how recently the last campaign was run.
• Users caught: aggregated error rate from attack simulations.
• Passed training: Completion rate for assigned awareness training.

Sophos recommends testing all users at least once every three months as a guideline. This does not replace the organization’s own risk, data protection, and training planning. Export to PDF exports dashboard data and campaign reports as PDF. The documented function does not indicate how long Sophos retains generated files or report data.

Evaluate a campaign in detail

Open the campaign under My Products > Phish Threat > Campaigns. The summary contains the start and end dates as well as donut charts for email delivery, user events, and actions in the email.

The following areas answer different questions:

• Campaign response timeline shows for the first hours and days when emails were sent, users were caught, and emails were reported. This allows the speed of response to be assessed.
• Device breakdown shows device types on which emails were opened or users were caught. This is an investigative clue, not proof of an insecure device.
• User Behavior shows the reactions to the campaign emails.
• Email shows attack, training enrollment, and reminder emails depending on the campaign type.
• Training displays the course chosen for this campaign.
• Paused shows pause and resume dates, depending on the status. If a campaign was paused more than once, only the most recent pause and resume dates are visible.

Use Export to export campaign data as PDF or CSV. Generate the export only after reviewing the campaign and the required view. The available documentation specifies neither a general retention period nor automatic delivery or scheduling for these exports; do not assume that such features exist.

Check campaign results and events

The campaign results can be viewed with By User, By Group, or By Attack. By Attack lists all templates that are currently or were previously part of the campaign. In By User, timestamps appear; in By Group, the number of users per event appears. Group values and user timestamps are therefore different forms of presentation and cannot be compared directly.

For Failed or Blocklisted, first resolve the cause of the delivery problem. Then select the affected rows and resend the campaign email. A successful redelivery automatically removes the recipient’s email ID from Bounced Mailboxes. Without this correction, later campaigns may also fail to reach the user.

Important: Pending is not proof of delivery. A low interaction rate must not be evaluated as long as the relevant recipients are Failed, Blocklisted, or still Pending.

Analyze user behavior across campaigns

The following filters are available under My Products > Phish Threat > Reports > User Behavior:

• All users: all users registered for simulated attack campaigns,
• Caught users: users caught at least once,
• Repeat users: repeatedly caught users,
• Reported threats: Users who have reported simulated threats,
• Failed to report threats: Users who opened a simulated attack email but did not report it,
• Entered credentials: users who entered credentials in a simulation,
• Opened attachments: Users who have opened a simulated attachment.

The date field limits the report; the selected period is applied with Apply. For a repeatable review, filters, date range, and evaluation time are documented together. Export to CSV exports the current view. Before sharing it, verify that the file contains only the intended period and users.

Caught users, Repeat users, and Entered credentials serve risk-based prioritization. Meaningful follow-up steps are appropriate training, a controlled follow-up campaign, and in the case of repeated patterns, a conversation within the designated organizational process. Reported threats is positive behavior; Failed to report threats merely means ‘opened but not reported’ and must not be equated with ‘caught’ or ‘credentials entered’.

With Create campaign with these users, a new, pre-filled campaign can be started from the currently filtered report target group. Before proceeding, recipients, purpose, legal basis or internal approval, training, and sending time are checked. A report filter is not an automatic approval for sending.

Track training status

Under My Products > Phish Threat > Reports > Training, the report shows registration and compliance status. Available are:

• Enrolled in Trainings for users with training registration,
• Incomplete Trainings for users who have not completed at least one course.

Here, a date range is also selected and applied with Apply. Export to CSV exports the current view; Create campaign with these users transfers the filtered users into a new campaign.

An incomplete training is not automatically a refusal. Before escalation, the end of the campaign, registration time, reminder email, delivery, and possible absences are checked. For success monitoring, Finished training in the campaign is compared with the cross-campaign training report.

Carry out follow-up actions in a controlled manner

The evaluation produces a traceable action list:

1. Fix delivery problems before the behavior assessment.
2. Prioritize Entered credentials, repeated incidents, and incomplete training according to the internal risk model.
3. Acknowledge positive reporting behavior and monitor the reporting rate as a program goal.
4. Choose comparable target groups and templates for the follow-up campaign.
5. After the start, validate delivery, events, and training again.

The campaign lifecycle affects what can be changed: Active Campaign and Upcoming Campaign can be edited, deleted, and cloned; Past Campaigns cannot be edited. Delete cannot be undone. Clone generates an entry under Campaign Drafts, carries over the details of the original, and adds the creation date to the name. Before reuse, however, recipients, template, training, and schedule must still be rechecked.

Validate results

Before publication or escalation, at least these controls are carried out:

• The dashboard names the same relevant campaign as Campaigns; if there are more than two active campaigns, See all was checked.
• Open-related rates are not described as a proportion of all recipients.
• In the campaign, recipients, template, and period match the evaluation assignment.
• The total or sample in By Group is cross-checked with entries in By User.
• Pending, Failed, and Blocklisted are reported separately from Delivered.
• CSV or PDF is opened after the export; title, campaign, period, filters, and columns are checked.
• A sample of conspicuous users is checked based on event timestamps and not just based on a dashboard list.
• Follow-up measures are documented with the responsible person, deadline, and success criterion.

When investigating discrepancies between the dashboard and the detail view, first account for their different scopes: Organization summary shows averages from the last five campaigns, while the campaign page evaluates one campaign and Reports evaluates a selected date range.

Data protection, deletion, and audit

Reports and exports contain names, email addresses, reactions, training status, and potentially risk-relevant behavioral data. Access and sharing are limited to the necessary personnel. Exports are stored in an approved repository, deleted according to internal retention rules, and not distributed unprotected by email. Roles, purpose limitation, retention, and approvals are defined in the data protection and governance runbook. The functions described here by Sophos do not constitute a general product retention period; this is not derived from the export behavior.

With Clear campaign events, user events can be deleted from the campaign results. Email sent remains, and the action is logged in Audit Log. Beforehand, purpose, approval, and evidence preservation must be clarified. The function is neither a complete user deletion nor a substitute for a regulated data protection process.

Phish Threat reports usage data to Sophos Fusion every 30 days. If a user who was registered in a campaign within the last 30 days is deleted, Sophos Fusion can recreate them because of this activity. Therefore, before a permanent deletion, the period is set to the last 30 days under Phish Threat > Reports > User Behavior > All enrolled > All users and checked via Export to CSV against Last Targeted or Last Enrolled. Permanent deletion should only take place after 30 days without activity. The other identity and directory sources are additionally considered according to the process Safely delete a Sophos Fusion user.

Assign campaign link to a process

If multiple campaigns, trainings, or reminders are active at the same time, the token contained in an example link can help with assignment. An authorized administrator extracts the token from a secure pattern after the first equals sign and decodes its JWT payload. The field campaign_token is then compared with the campaign ID, which is visible when hovering over a campaign link or in its URL.

Strict limits apply: A production link or token must not be transmitted to arbitrary public decoders, tickets, or chat systems. It may contain tracking, campaign, and expiration data. An internally approved local procedure is used, or explicit privacy and security approval is obtained. Decoding only means reading the payload; it does not replace signature validation or trust verification. The complete link is not published in the report or audit comment.

Narrow down common mistakes

• Campaign missing on the dashboard: Check See all or Campaigns; the dashboard only shows the two with the nearest completion date when there are more than two active campaigns.
• Rate seems unexpectedly high or low: Check the denominator. Caught-to-open ratio and Report-to-open ratio refer to openings, not to all recipients.
• No or too few events: Check Delivery Status, target group, template, and time period; handle Pending, Failed, and Blocklisted separately.
• Further campaigns do not reach a user: Fix the error under Settings > Bounced Mailboxes and only then send again.
• Report and campaign differ: Compare date filter with Apply, campaign scope, and the average values shown in Organization summary from the last five campaigns.
• Export contains unexpected persons: Before use, check the current view, filter, and date range; Export to CSV adopts the current view.
• User reappears after deletion: Check activity of the last 30 days as well as Last Targeted and Last Enrolled and verify directory/synchronization sources.
• Campaign link cannot be assigned: Decode the complete token locally without sharing it, then compare campaign_token exactly with the campaign ID.

For a support escalation, the tenant ID, campaign name and ID, campaign type, period with time zone, selected filter, affected delivery status, anonymized example users, event timestamps, and the deviation between expected and visible count are recorded. Tokens, credentials, and complete tracking links are not copied into the normal case documentation.