Create and test custom IPS signatures on Sophos Firewall

A custom IPS signature is useful when Sophos does not provide a suitable signature for a clearly defined network or application pattern. It can, for example, detect a known plaintext string, an unusual protocol characteristic or a precise combination of port, direction and payload.

The signature alone does not protect anything. It must be used in an IPS policy, that policy must be attached to the firewall rule that actually matches the traffic, and the data stream must be visible to IPS. A pattern that is too broad can block legitimate traffic; one that is too narrow may produce no matches.

Test a new signature first in a pilot IPS rule with Action: Allow packet. Even if Recommended action is already set to Allow packet, the action in the IPS policy rule determines the outcome. Drop packet, Drop session, Reset or Bypass session affect production traffic and should only be put into operation after reproducible positive and negative tests.

A controlled pilot in six steps

1. Describe the detection scenario, specifying the protocol, direction, port and an unambiguous pattern.
2. Under Intrusion prevention > Custom IPS signatures, create a signature with a narrowly scoped rule and Recommended action: Allow packet.
3. Add the signature to a dedicated rule in an IPS policy that can be deleted, and select Action: Allow packet there.
4. Assign this IPS policy only to the intended pilot firewall rule and enable Log firewall traffic there.
5. Generate a matching test and a deliberately non-matching test, then compare Log Viewer, ips.log and the rule match.
6. Only change the action in the IPS policy rule after successful, consistent acceptance testing; if unexpected matches occur, restore the previous policy assignment.

A saved entry or a successful syntax check is not yet proof that it works. Success means that the positive test matches exactly the expected custom signature and firewall rule, the negative test produces no match, and the production application behaves as before.

When a custom signature is appropriate

Custom signatures are suitable for a stable characteristic that is visible at packet or stream level. This might be a proprietary protocol value, a clear exploit indicator or temporary protection against an internally known vulnerability. The expected traffic path and desired response must be established before writing the signature.

For changing IP addresses or domains, hosts, services and groups, threat feeds or narrowly scoped firewall rules are usually more suitable. A custom signature is also no substitute for patch management or a well-defined vendor rule. Maintaining a permanent custom implementation is rarely proportionate for a one-off log match.

Encrypted payloads are an important limitation. IPS can only detect a content pattern in an HTTPS payload if the content is actually decrypted in the selected processing path and is visible to the engine. Without suitable TLS inspection, only unencrypted or otherwise visible characteristics are normally available.

Check licensing and lifecycle requirements first

Enabling IPS requires an active Network Protection subscription or a valid trial licence, as well as administrator access for the signature, policy and firewall rule. Before the pilot, record the existing policy assignment and rule ID, and retain a backup for rollback. The 30 days mentioned below start when the licence expires or IPS is manually disabled, not at the next configuration attempt.

Custom signatures cannot be configured if the IPS trial has expired or IPS Protection is disabled under Intrusion prevention > IPS policies. Sophos recommends re-enabling IPS within 30 days if existing custom signatures are to be retained. A backup or export should therefore form part of the rollback plan before licence changes, IPS changes or major policy changes.

IPS must then be globally enabled, and the firewall rule processing the traffic needs an IPS policy. The complete basic setup is described in Set up Sophos Firewall IPS and test it safely. A custom signature complements this traffic path; it is not a parallel protection mechanism.

Clarifying the scope of the rule syntax

Documentation limitations for SFOS 22/23: The syntax help contains inconsistently formatted examples: distance: 4.within:4; is not a reliable template to copy; for byte_jump, the explanatory text mentions comparison operands that are absent from the syntax schema. The labels for reserved TCP bits are also unclear. These passages are not interpreted as additional engine capabilities. Confirm the complete custom rule in the installed engine before enabling blocking. If a rule is rejected or appears unclear, retain the exact build, rule and error message and ask Sophos Support to clarify; do not authorise a production blocking action until then.

The input form separates Protocol and Custom rule. Within the rule, individual keywords are combined with their values and a semicolon. The more independent characteristics that must match, the lower the risk of accidental matches generally becomes. At the same time, the signature must not become so specific that a harmless protocol change renders it ineffective.

For a strictly controlled plaintext test, TCP can be used as the protocol, with the following as an example of a narrowly scoped payload pattern:

content:"AVANET-IPS-PILOT"; nocase;

AVANET-IPS-PILOT is a deliberately conspicuous documentation value. The associated pilot firewall rule is additionally restricted to the test service, for example TCP port 8080. Replace the token, direction and port with values that occur in the actual data stream visible to IPS. This example is not a universal attack signature and should not be applied unchanged to broad production rules.

Payload and search windows

content searches for a character or byte sequence; binary values are written between pipe characters. nocase ignores case in a content match, while rawbytes works on the raw data. depth and offset limit the search using absolute positions in the payload, whereas distance and within work relative to the previous match. uricontent, isdataat and pcre cover more specialised URI, position and regular-expression conditions.

A narrow search range reduces accidental matches and processing overhead. In particular, pcre, large windows or multiple broad content patterns should only be introduced using realistic packets and with load being monitored. If depth is smaller than the content pattern being searched for, the signature can never match.

The modifiers nocase, rawbytes, depth, offset, distance and within belong to content; they are not independent detection rules. Without depth, the search runs from offset to the end of the payload. Without within, the search for a second content pattern also remains open from distance to the end. A starting point alone is therefore not enough to keep the search window narrow. uricontent searches the normalised request URI, not arbitrary payload data. isdataat checks whether data exists at a specified position; with relative, this position is relative to the end of the previous content match.

Headers, streams and structured values

Source, destination and port can be restricted using srcaddr, dstaddr, srcport and dstport. Keywords available for IP headers include ttl, tos, id, ipopts, fragoffset, fragbits, dsize, ip_proto and samip. TCP characteristics are described using flags, flow, seq, ack and window; itype, icode, icmp_id and icmp_seq apply to ICMP. rpc, byte_test and byte_jump are intended for structured or binary protocols.

flow applies only to TCP. Here, to_server and from_client are synonymous, as are to_client and from_server. bi_direction extends detection to both directions; when a port restriction is used, the negative test must therefore also account for return traffic. The choice between no_stream and only_stream must not be confused with a packet-size check: dsize checks the payload size of a packet and does not match packets reassembled from a stream. Combining it with only_stream is therefore unsuitable. If no match occurs, first decide whether the characteristic should be sought in an individual packet or in the reassembled stream, rather than simply changing the size or content pattern.

For complex rules, the custom IPS syntax described above and supported by SFOS 22 applies. Undocumented Snort keywords or rules copied from other engines should not be adopted without verification.

Operands: address, port, URI and data position

The following fragments belong in Custom rule, not in the shell. These fragments apply to SFOS 22/23; they are not complete attack signatures. Combine only the conditions appropriate to the detection scenario; Protocol, the pilot policy and Allow packet remain part of the test described above. Square brackets in syntax patterns indicate optional parts and are not entered.

• srcaddr:192.0.2.10; dstaddr:198.51.100.20; compares the source and destination IPs against individual addresses. Replace the documentation addresses with the observed test addresses; this does not establish support for subnets or address lists.

• srcport:40000; dstport:8080; compares the source and destination ports against numbers. The destination port here represents the test service. A changing client source port should only be fixed if it really forms part of the detection scenario; otherwise, new connections will fail this condition.

• uricontent:"/ips-pilot"; searches for the replaceable path in the normalised request URI. Mixed text/binary patterns are also possible. The comparison must match the normalised URI value, not merely the way it appears in the browser address bar.

• content:"AVANET-IPS-PILOT"; isdataat:50,relative; additionally requires data at position 50 relative to the end of this content match. Without relative, the position is absolute within the payload. Choose the number according to the actual protocol structure; it checks data availability, not the content of that data.

• content:"ABC"; content:"DEF"; distance:1; within:10; uses two replaceable patterns with a relative search window. distance sets the distance from the end of the first match, and within limits the subsequent search. For absolute windows, offset and depth also take numeric values, for example offset:4; depth:20; on the associated content. Choose these numbers from the actual position and length of the pattern; they are not protocol defaults.

Regular expressions with pcre

The basic forms are pcre:"/REGEX/"; and pcre:"m/REGEX/";. A ! before the opening quotation mark negates the expression, for example pcre:!"/AVANET-IPS-PILOT/i";. This is a non-match condition and can have a very broad scope without additional restrictions. Replace REGEX or the pilot token with the required pattern. Options follow the closing /, for example pcre:"/AVANET-IPS-PILOT/i";.

• i ignores case; s allows . to match line breaks as well.
• m allows ^ and $ also to match the starts and ends of lines within the buffer. x ignores unescaped whitespace in the pattern, except within character classes.
• A anchors the match to the start of the buffer. E binds $ only to the actual end, not to the position before a trailing line break.
• G reverses the default greediness of quantifiers; a subsequent ? reverses it again.
• R starts relative to the end of the last pattern match. U uses decoded URI buffers; B does not use decoded buffers.

Before the pilot, therefore, establish which buffer the pattern should apply to and from which position. A correct regex on raw data will not necessarily match a decoded URI.

Reading binary fields: byte_test and byte_jump

byte_test reads a field and checks its numeric value. The schema is byte_test:COUNT,[!]OPERATOR,VALUE,OFFSET[,relative][,big|little][,NUMBER_TYPE,string];. COUNT is the number of bytes read, and OFFSET is the starting position in the payload; with relative, the start is relative to the last pattern match. VALUE is the comparison value. The documented operators are <, >, =, ! and &: less than, greater than, equal to, not equal to and bitwise AND respectively. A ! before a comparison operator negates its result. big or little sets the byte order. With string, text digits are read instead of a binary numeric field; hex, dec and oct apply only to this text mode.

Example of a binary field: byte_test:2,=,16,0,big; reads two bytes from payload position 0 as big-endian and compares them with 16. The count, position, byte order and comparison value must come from the protocol, not from this example. An arbitrary numeric value is not proof of an attack.

byte_jump, by contrast, reads a length and moves the position for subsequent checks. The schema is byte_jump:COUNT,OFFSET[,relative][,multiplier FACTOR][,big|little][,string][,hex|dec|oct][,align][,from_beginning];. The first two operands determine the field width and read position; relative makes the offset relative to the last pattern match. multiplier multiplies the value read to determine the jump distance. align rounds this up to the next 32-bit boundary. With from_beginning, the jump distance is applied from the start of the payload rather than from the current position. string and the numeric bases serve the same purpose as in byte_test.

byte_jump:2,0,big; is therefore a fragment for a two-byte length field at the start of the payload, with an explicitly selected byte order. It is not a size comparison; a comparison operator and comparison value are not operands of byte_jump. For both byte keywords, explicitly select big or little for binary multibyte fields: do not rely on a default byte order inferred from the documentation. First verify the field and jump destination against the actual test payload, then check positive and negative cases for the subsequent condition. An incorrect length can move the search position past the intended pattern.

IP fields, options and fragments

• ttl:64;, ttl:>64; and ttl:<64; compare the IP Time to Live for an exact value, a greater value or a lower value. tos:16; compares the TOS field, and id:1234; compares the IP ID field. These example numbers are not recommended detection values; choose them according to the header characteristic actually being sought.
• ipopts:rr; checks for an IP option that is present. Permitted values are rr (Record Route), eol (End of List), nop (No Operation), ts (Timestamp), sec (Security), lsrr (Loose Source Routing), ssrr (Strict Source Routing), satid (Stream Identifier) and any (any IP option). Thus, any requires an option, not simply any IP packet.
• fragoffset:0; compares the fragment offset in the IP header against a decimal number. This field is not the content search offset in the payload.
• fragbits checks M (More Fragments), D (Don’t Fragment) and R (Reserved). A preceding + allows other bits in addition to those specified, * requires at least one of the specified bits, and ! requires the specified bits not to be set. Thus, fragbits:+M; checks that More Fragments is set without excluding other bits.
• dsize:300;, dsize:>300;, dsize:<300; and dsize:300<>400; describe an exact payload size, a greater-than/less-than comparison or a range. The numbers refer to the packet payload, not the entire connection. The limitations concerning reconstructed streams explained above still apply.
• ip_proto:6;, ip_proto:!6;, ip_proto:>6; and ip_proto:<6; compare the numeric protocol identifier in the IP header, not the TCP/UDP port. samip; takes no value and requires identical source and destination IPs.

TCP fields and stream selection

flags checks TCP flags: S for SYN, A for ACK, F for FIN, R for RST, U for URG and P for PSH. Without a modifier, flags:S;, for example, checks for exactly SYN. A preceding +, * or ! has the meaning explained above for fragbits. flags:+S; therefore allows additional flags. The help shows an optional second flag group after a comma, but does not explain its comparison semantics. Do not therefore treat flags:S,A; as a confirmed instruction to ignore ACK; have this behaviour clarified for the installed build before use. Special cases for reserved TCP bits and ‘no flags set’ are deliberately not approved here as syntax to copy; they require a rule confirmed for the specific version and must not be inferred from an unclear label.

For flow, the directions already explained are available, along with established, no_stream and only_stream. flow:established; requires an established TCP connection. flow:no_stream; excludes reconstructed stream packets, while flow:only_stream; restricts detection to them. These fragments explain each individual selection and are not an instruction to use both stream options together. bi_direction remains the extension to outbound and return traffic described above.

seq:1234;, ack:1234; and window:4096; compare the TCP sequence number, acknowledgement number and window size respectively against a number. Exact sequence or ACK values change between connections and are only useful if that precise value forms part of the detection scenario. The TCP window is not a payload search window.

ICMP and RPC

itype:8; and icode:0; compare the ICMP type and code. Both also support <, > and ranges such as itype:3<>5;; type and code are different fields. icmp_id:1234; and icmp_seq:1; compare the ICMP identifier and sequence value exactly. Choose the values according to the intended ICMP scenario; do not equate them with TCP sequence numbers.

rpc:100000,2,*; checks SUNRPC CALL requests for application number 100000, version 2 and any procedure. The schema is rpc:APPLICATION,VERSION,PROCEDURE;; * is permitted for the version and procedure, but not for the application number. Replace the numbers with the application, version and procedure being sought. A wildcard does not remove the need to analyse the protocol: it deliberately leaves that part unrestricted and thus broadens detection.

Create a signature and assign it to a policy

Under Intrusion prevention > Custom IPS signatures > Add, set Name, Protocol, Custom rule, Severity and Recommended action. A name such as PILOT-TCP-8080-AVANET-TOKEN makes the purpose and test scope clear. Severity describes your own risk assessment; it does not prove that the pattern is malicious.

For the first run, leave Recommended action set to Allow packet. Important: the action in the IPS policy rule overrides this recommendation. The pilot rule must therefore also use Allow packet. The other actions have substantially greater consequences:

• Drop packet discards only the matching packet.
• Drop session terminates the session after a match.
• Reset terminates a TCP session and sends a reset to the originator.
• Bypass session allows the traffic and performs no further inspection of the rest of the session.

With session-based actions, SFOS inspects only up to the first matching packet. Before changing from Allow packet to such an action, you must therefore confirm that this exact first match concerns the intended session.

When saving, SFOS reconfigures the IPS engine. If sufficient free RAM is available, this happens without interruption. If free RAM is low, the engine may restart and cause a brief interruption. Changes should therefore be made within a monitored time window, even for an apparently small signature. After saving, check the IPS status and ips.log for a restart or errors before continuing the pilot.

After this preliminary check, save the signature with Save and reopen its entry: Protocol, Custom rule, Severity and Recommended action must match the planned values. Leave Firewall Acceleration and PKI Acceleration unchanged during the pilot; enabling or disabling either restarts IPS and distorts the comparison. Zone checking, Trusted MAC, IP-MAC pairs and DoS exceptions are separate mechanisms covered under DoS and spoof protection, not properties of a custom signature.

Next comes the often-overlooked second part: under Intrusion prevention > IPS policies, open a policy intended for the pilot that can be deleted, add a rule, select Custom signature, select the signature and place the specific rule above broader rules. SFOS evaluates IPS policy rules from top to bottom. Save the policy rule with Save, then also save the enclosing IPS policy with Save. Reopen the policy and check the saved signature selection and action. Then attach this policy to the pilot rule that actually matches the traffic under Rules and policies > Firewall rules > [pilot rule] > Detect and prevent exploits (IPS); save the firewall rule as well and test using a new session.

Manage the same signature object via the XML API

When automating the pilot, you work with the XML object IPSCustomSignature, not an IPS policy rule. The SFOS 22/23 reference names the operations Add Custom Signature, Update custom signature and Delete Custom Signature. Access, explicit write operations, backups, response evaluation and stopping when the payload is unclear are explained centrally in Use the XML API safely. Before writing, the object definition and operation must be confirmed in API help for the installed build; no additional executable write request is specified here.

In the object example, Name, Protocol, CustomRule, Severity and RecommendedAction appear within IPSCustomSignature. Name identifies the signature, not the policy or firewall rule. The parameter table requires a scalar string of no more than 15 characters for this; Delete also requires this exact name. For API automation, therefore, choose a short, unique name such as IPS-PILOT-8080. The longer WebAdmin example name above is not a template to copy for the API. Before Update or Delete, read the actual saved identity and all policy uses, rather than inferring the target from a SID or a similarly named entry.

The table marks all five values as mandatory and scalar for Add/Update. Protocol permits TCP, UDP, ICMP or ALL; Severity permits Critical, Major, Moderate, Minor or Warning. CustomRule contains the signature definition as a string. The table label Custom Rule, with a space, is not the XML spelling CustomRule. Its additional instruction to separate letters with a full stop explains neither general rule conversion nor XML escaping. Do not therefore mechanically alter the syntax described above by adding full stops; instead, verify the rule content and correct XML encoding for the build. The vendor example uses only SignatureDefinition as a placeholder; this does not produce a ready-to-use signature or one proven to detect malicious traffic.

Do not confuse recommendation and action: The XML example uses RecommendedAction, whereas the parameter table labels the same subject area Action. It lists the text values Allow Packet, Drop Packet, Drop Session, Reset and Bypass Session, as well as the additional value 3. This is not confirmed authorisation to replace RecommendedAction with an XML tag named Action. No meaning or mapping to a text action is explained for 3; neither guess this value’s meaning nor use it as shorthand for a particular action. Read back the recommendation actually saved and continue to check the IPS policy rule’s action separately. Creating an API object replaces neither policy assignment nor positive and negative testing.

Cross-check API responses against the operation and saved state

The following rows are documented in the SFOS 22/23 object references. The Message.* names remain unresolved documentation symbols: they are neither guaranteed runtime messages nor translated error text or conclusively confirmed causes. The numbers are also XML status codes, not HTTP status codes. Retain the operation, code and actual response together; do not replace the object-specific rows with a general interpretation of codes from the central API article.

For Add Custom Signature, the reference lists:

• 200 — Message.CustomSigAddSuccess
• 500 — Message.CustomSigAddFailed
• 502 — Message.CustomSigRecordExists
• 504 — Message.CustomSigInvalidSignatureRule
• 505 — Message.IPSpolicyEngineRestartFail
• 506 — Message.CustomSignatureIpsOff

For Update custom signature, it lists:

• 200 — Message.CustomSigUpdateSuccess
• 500 — Message.CustomSigUpdateFailed
• 502 — Message.CustomSigRecordExists
• 504 — Message.CustomSigCustomSignatureCheck1
• 505 — Message.IPSpolicyEngineRestartFail
• 506 — Message.CustomSignatureIpsOff

In particular, 504 distinguishes Add from Update through the symbol; Message.CustomSigCustomSignatureCheck1 must not be reinterpreted as a supposedly unambiguous syntax error message. Before another Add/Update attempt, reopen the entry in WebAdmin or check it using a read query confirmed for the build: do Name, Protocol, Custom rule, Severity and Recommended action match the desired state? For 502, compare the existing and desired identities; do not automatically overwrite the entry or create it under a second name. For 504, additionally cross-check the exact rule against the build documentation. For 505, check the saved state, IPS status and ips.log together; the symbol implies neither a safe rollback nor permission to restart the engine. For 506, check the IPS and licence state as described in the lifecycle section, but do not toggle IPS speculatively. Even after 200, the saved values, policy linkage and traffic path still need to be checked; 500 does not prove that no partial change has occurred.

For Delete Custom Signature, these separate rows are documented:

• 200 — Message.CustomSigDeleteSuccess
• 500 — Message.CustomSigDeleteFailed
• 503 — Message.CustomSignatureInUse
• 506 — Message.CustomSignatureIpsOff

Only authorise Delete after the rollback described below and a complete usage check. For 503, investigate the policy references to the actual object; do not indiscriminately delete a production policy. For 506, clarify the IPS and licence state rather than globally disabling IPS for a deletion attempt. After the response, read back the configuration to confirm the absence of the exact name and the expected state of dependent policies; for 500 or an unclear response, do not blindly retry deletion. If the saved state, response or responsibility remains unclear, stop automation and clarify the sanitised response, rule version, build and logs with Sophos Support.

Carry out positive and negative tests

The positive test sends the agreed pattern over the intended port and in the expected direction. In Log Viewer, at the top right of WebAdmin, select the IPS module and filter by test time, Source and Destination. The signature or SID, action and timestamp must correspond to the test. ips.log provides additional engine details. Packet Capture under Diagnostics > Packet capture also shows the Firewall Rule ID and IPS Policy ID. The relationship between the rule, Packet Capture and security module is explained in Test a firewall rule systematically.

After the pilot phase, the effect can also be evaluated under Reports > Network & threats > Intrusion attacks. The report is useful for comparisons across periods, but does not replace timely positive and negative testing in Log Viewer. XGS 87/87w and XGS 88/88w do not support on-appliance reporting; use the immediate Log Viewer checks described here on these models.

Next, carry out at least one negative test: the same service without the token, a different port or a different direction. The signature must not match. For a content pattern, normal application requests are also important, because short or generic strings can occur in entirely legitimate payloads.

Only once both tests are consistent should the planned blocking action be set and tested again. A blocking signature is only considered accepted if precisely the positive case is stopped, the negative case continues to work, and no other firewall or IPS rule is causing the effect.

Check the number of signatures

The number can be determined in WebAdmin without using the shell. Under Intrusion prevention > IPS policies, open a policy that can be deleted and create a new policy rule. With Select all, SFOS shows the total above Action. The list is only visible when adding a rule to a policy that can be deleted; the dialogue can then be closed without saving.

Sophos additionally documents two read-only queries under 5. Device management > 3. Advanced shell:

psql -U nobody -d signature -p 5434 -c "select count (*) from tblidprules;"
psql -U nobody -d corporate -c "select count (*) from tblidpcustomsignature;"

The first command counts the standard signatures, and the second counts the custom signatures. These database queries do not change anything, but should be run as part of a documented support or diagnostic session. The count is not proof of quality or functionality and can change with pattern updates.

Each query is expected to return a single result row under count, followed by (1 row). The Sophos example values 28623 and 1 come from an SFOS 19.0.1 example and are not target values for SFOS 22/23. If database, connection or permission errors occur, do not alter tables or services: use the WebAdmin check and retain the exact output and build for support.

If the signature does not work as expected

There are no matches

First, check whether IPS is enabled, whether the traffic matches the expected firewall rule with the correct IPS policy, and whether the custom signature is actually included in a policy rule being evaluated. Then check the protocol, direction, port, encryption and actual payload. A string displayed in a browser will not necessarily appear unchanged in the network packet.

A narrowly filtered Packet Capture showing the direction, port and content visible at that capture point is helpful. However, the absence of a plaintext pattern in an encrypted capture does not prove that it is absent from the decrypted IPS buffer. Therefore, check the capture point, decryption and stream reconstruction together. For a controlled plaintext test, the pattern actually transmitted must be verifiable; then check offset, depth, distance, within, stream state and the order of the policy rules.

Too many connections match

Leave the signature set to Allow packet until Source, Destination, port, direction or search window have been narrowed down. Generic words, short binary sequences and unrestricted regular expressions are typical causes. A broad firewall rule makes the effect even harder to control.

If the firewall shows increased resource usage or an IPS restart after saving, retain the timestamp, model, firmware, available resources and ips.log. Repeatedly saving different variants is not a sound test in this situation; first establish the cause and arrange a maintenance window.

Roll back safely

If unexpected matches occur, first remove the custom rule from the pilot policy or restore the previous IPS policy on the firewall rule. Then check new sessions using positive and negative tests. Only delete the custom signature once it has no other use.

Before deletion, document which policy, firewall rule and application used it. Logs, the tested rule version and the reason for the rollback belong in the change record. Globally disabling IPS or removing the entire production policy is not an appropriate rollback for a single faulty signature.

FAQ