Skip to content
Avanet
Sign in

Create, test and operate Sophos SIEM custom parsers

Sophos Fusion Admin Sophos SIEM

A custom parser maps fields in a log message to Sophos XDR’s standardized event schemas. It helps when an integrated source is not normalized correctly. Work under Security Operations > Integrations > Custom Parsers: create the parser, test it with different individual messages, check the actual mapping, and only then enable it. A Next-Gen SIEM subscription and the Admin or Super Admin role are required; other roles can only view the list of configured parsers.

The parser does not configure log transport. The source must first send data; the SIEM overview distinguishes ingestion, normalization and detection. For an already supported integration, first establish its expected source format rather than prematurely overriding the global parser logic.

Prepare the source, schema and rollback path

Before starting, record the source product, format, event types, time reference and required analysis. Sophos supports CEF, LEEF, JSON, XML and unstructured messages. Each relevant variation needs a sanitized sample: for example, successful and failed logins, missing optional fields, and different user or address values. Remove secrets while preserving the structure needed for parsing.

The destination schema describes the event type. A login might fit Auth, a network connection Netflow, and a DNS request DNS. Custom parsers support only a subset of all XDR schemas. Under Data Lake Search > Schema Library, check searchable schemas and fields; the normalization reference also lists the corresponding parser fields and data types. A field name alone does not establish that the source supplies its value.

For the change, document the parser name, owner, source format, required destination fields, test messages and previous processing. For an existing parser, save its previous script in a location with restricted access. Plan a rollback that does not require deletion: keep new parsers disabled initially; if problems occur, disable them again or carefully restore the saved script. Sophos does not maintain or update custom parsers; responsibility stays with the operator.

Choose Standalone, Parent or Child

  • Standalone suits a source with exactly one message type and one destination schema.
  • Parent identifies the source and prepares common values. Sophos recommends scwx.generic as its destination schema. Associated child parsers normalize individual event types.
  • Child needs a parent and the appropriate final destination schema. Variables and parser fields from the parent are available to the child.

A firewall might supply logins, network connections and HTTP events. One parent identifies the common source format; three children handle the different contents. Custom child parsers cannot have further child parsers. A single simple source does not automatically require this division.

Create the parser and identify its source precisely

  1. Open Security Operations > Integrations > Custom Parsers and select Add Custom Parser.
  2. Choose a descriptive name, such as EXAMPLE_APP_JSON_AUTH. This is an example name, not a fixed value; product, format and purpose should be recognizable.
  3. Select Parent, Child or Standalone. For a child, select the correct parent; then set source format and destination schema.
  4. Select Create to open the script editor. The generated boilerplate includes the splitter function for the selected format.

The confirm string determines whether a message matches this parser. !CONFIRMWITH=PATTERN selects a Go-compatible regular expression; !CONFIRMWITH=EXPRESSION selects an expression that evaluates to true or false. In each case, the actual expression belongs in !CONFIRMSTRING in the script’s declaration block. It must identify the intended source and, where applicable, event type sufficiently precisely. A general word such as “network” is usually too broad. The complete template in the HTTP/parser practical example linked below shows the mode and matcher together; its narrow lab format is not interchangeable with the following example-auth excerpt.

Even a syntactically valid parser can capture another source’s messages. Sophos does not guarantee a fixed order for all possible parser collisions. Also test at least one similar message that should not match the new parser. Later, the extractor path shows which parent and child were actually used.

Map fields and check timestamps

The script distinguishes local variables from parser fields ending in $. Only parser fields that match the destination schema are normalized; an arbitrary custom field name ending in $ does not create a new schema field. Schema fields often use snake_case, while parser fields use camelCase. Check exact spelling against the selected schema.

This fictional JSON is a small starting point for an Auth mapping:

{"product":"example-auth","user":"pilot-user","src_ip":"192.0.2.25","time":"2026-10-27T10:15:00Z","result":"success"}

192.0.2.25 is a documentation address, not a production target. A JSON script excerpt can extract the user and source address as follows:

json = JSON(originalData$)
sourceUserName$ = json["$.user"]
sourceAddress$ = json["$.src_ip"]

A complete template for a narrowly defined synthetic format, including a timestamp, enum mapping and search acceptance checks, is available in the HTTP/parser practical example.

The excerpt is not a complete parser definition. The confirm string, schema, timestamp, authentication action and outcome are deliberately still missing and must match the actual source format. In particular, do not copy the text success into a numeric enum field without checking it. The Auth schema defines values such as SUCCESS and FAILED for action_result; explicitly verify their parser mapping.

For timestamps, compare the raw value, time zone and normalized event_time_usec. The DATETIME() parser function uses Go time format templates. A timestamp with an unsuitable time zone can produce technically successful but temporally incorrect analysis. Also check common values extracted by the root parser against the source.

Repeated fields such as multiple attachments, indicators or metadata records require lists of objects matching the schema. A comma-separated string is not a substitute. ADDFIELD() can combine arrays into repeated objects; with JSON arrays, corresponding positions must remain aligned. Missing values must not cause, for example, a hash to be assigned to the wrong file type.

Test individual messages and inspect results

If the source is already sending data, search for suitable raw messages under Data Lake Search using FROM generic WHERE sensor_id='[my data source]'. Replace the placeholder with the source identifier verified against actual events. In event details, select Original Data > Copy Original Data and paste the content into Sample Message in the parser editor. Alternatively, use one sanitized log line, not an entire log file.

  1. Use Parse Message to check the initial split. Parsed Source Message shows key/value pairs for structured formats or indexed parts for unstructured messages.
  2. Under Source Destination Mapping, check which source values reach which schema fields. Compare user, source/destination address, time, action and outcome individually with the raw message.
  3. Check the extractor path. It must show the intended parser path. A warning about another parser is not merely cosmetic.
  4. Add further representative messages and deliberately nonmatching samples. For parent/child parsers, test both common source identification and child selection.
  5. Run Run Test for all stored samples and compare schema, field values and parser path. Sophos recommends repeated tests, around 12 to 15 runs, to identify inconsistent matching. A particular number of runs does not guarantee correct processing of all future messages.

The testing tools evaluate the current editor contents but do not save them. A disabled invalid parser can be saved, so saving alone is not functional acceptance. If fields are missing, first check source contents, data type, exact parser field names and, where relevant, message splitting.

Save and enable in a controlled manner

For a new parser, Save and Continue saves the draft. New parsers are disabled by default. After successful testing, leave the editor with Exit, select the Play icon under Actions in the parser list, and confirm with Enable. For parent/child parsers, enable the parent before its children.

After enabling, generate an authorized harmless source event or clearly correlate an event expected in normal operation. Search the normalized results using source, time, schema and required fields. Sophos gives WHERE sensor_type=[your_sensor_type] as one example; the placeholder represents the actual sensor type and is not a complete query ready to copy.

Processing into additional schemas takes place alongside generic events. A search result initially proves only the normalization tested. Automatic detectors are not guaranteed for every custom source. Documented supported paths include DGA and Domain Watchlist through DNS query_name, and IP Watchlist through source/destination addresses in NIDS or Netflow. Validate a required detection separately; an ordinary test event need not generate an alert.

Observe customer-rule and MDR ownership boundaries

is_custom_alert, or vendor_alert_details.is_custom_alert for Email, identifies the origin of an alert rule. According to the schema help, detections marked as custom alerts with true go directly to the tenant for self-service and do not enter the MDR service queue; their severity is unchanged. Do not set this flag in a way that makes a customer alert appear to be an unmodified vendor detection.

This does not mean that every message normalized by a custom parser automatically falls under the same exclusion. Source event, resulting detection and case ownership remain separate questions. The Cases and Rules article explains the existing boundary for self-managed cases created from customer rules.

Troubleshoot and roll back safely

Wrong parser in the extractor path: Compare the confirm strings of affected parsers. For a collision with a global parser, ask Sophos Support to review specificity rather than layering additional broad overrides on top.

Message appears only in generic: Check transport and the raw message, then the confirm string, enabled parser and chosen schema. The absence of a normalized result alone does not prove a transport failure.

Valid script, incorrect values: Compare raw message and mapping field by field, especially time and enum values. For active parsers, carefully restore an incorrect change to the saved version and check again.

Disable a new faulty parser using the Pause icon and Disable. In a hierarchy, all children must be disabled or deleted before the parent can be disabled. Disabling immediately stops additional schema normalization; generic events continue to be created. This rollback does not automatically correct previously misnormalized historical data or resulting detections.

Deletion using the Trash icon and Delete cannot be undone. Disabling is normally sufficient for an initial rollback. Only decide on permanent removal after securing the configuration, clarifying dependencies and completing acceptance.

Global parsers themselves cannot be edited. An override replaces their processing for a specific use case, while an extension adds previously unnormalized event types. Both require the correct global parent association and ongoing maintenance. Without an unambiguous parser name and verified field mapping, preserve the existing global path; do not correct a different time zone or a new source format in production using an untested override.