Create and test custom IPS signatures on Sophos Firewall

A custom IPS signature is useful when Sophos doesn’t provide a suitable signature for a clearly defined network or application pattern. It can detect a known plain-text string, an unusual protocol characteristic, or a precise combination of port, direction, and payload.

The signature alone doesn’t protect anything. It must be used in an IPS policy, that policy must be assigned to the firewall rule that actually matches, and the data stream must be visible to IPS. A pattern that is too broad can block legitimate traffic; one that is too narrow never matches.

Test a new signature first in a pilot IPS rule with Action: Allow packet. Even if Recommended action is already Allow packet, the IPS policy rule action is decisive. Drop packet, Drop session, Reset, and Bypass session change production traffic and should only be used after a reproducible positive and negative test.

A controlled pilot in six steps

1. Describe the detection case with protocol, direction, port, and a unique pattern.
2. Under Intrusion prevention > Custom IPS signatures, create a narrow signature with Recommended action: Allow packet.
3. Add the signature to a dedicated rule in a deletable IPS policy and select Action: Allow packet there.
4. Assign this IPS policy only to the intended pilot firewall rule and enable Log firewall traffic on that rule.
5. Generate one matching and one deliberately non-matching test, then compare Log Viewer, ips.log, and the rule match.
6. Change the IPS policy rule action only after stable acceptance; restore the previous policy assignment if unexpected matches occur.

A saved entry or a successful syntax check is not proof that it works. Success means that the positive test matches exactly the expected custom signature and firewall rule, the negative test doesn’t match, and the production application behaves as before.

When a custom signature is appropriate

Custom signatures suit a stable characteristic that is visible at packet or stream level. This can be a proprietary protocol value, a clear exploit indicator, or temporary protection for an internally known vulnerability. The expected data path and intended response must be defined before writing the signature.

For changing IP addresses or domains, hosts, services, and groups, threat feeds, or narrow firewall rules are usually more appropriate. A custom signature also doesn’t replace patch management or a well-maintained vendor rule. Developing a permanent custom signature for a single log event is rarely proportionate.

Encrypted payload is an important boundary. IPS can only detect a content pattern in an HTTPS payload if the content is actually decrypted and visible to the engine on the selected processing path. Without suitable TLS inspection, only unencrypted or otherwise visible characteristics are normally available.

Check the license and lifecycle first

Custom signatures can’t be configured if the IPS trial has expired or IPS Protection is turned off under Intrusion prevention > IPS policies. Sophos recommends turning IPS back on within 30 days if existing custom signatures must be retained. A backup or export therefore belongs in the rollback plan before licensing, IPS, or major policy changes.

IPS must then be active globally and the processing firewall rule needs an IPS policy. The complete baseline is covered in Set up and safely test Sophos Firewall IPS. A custom signature extends this data path; it isn’t a parallel protection mechanism.

Narrow the rule syntax deliberately

The form separates Protocol and Custom rule. The rule combines individual keywords, their values, and semicolons. Requiring several independent characteristics to match generally reduces accidental hits. At the same time, the signature must not become so specific that a harmless protocol change makes it ineffective.

For an exclusively controlled plain-text test, select TCP as the protocol and use a narrow payload pattern such as:

content:"AVANET-IPS-PILOT"; nocase;

AVANET-IPS-PILOT is a deliberately conspicuous documentation value. The corresponding pilot firewall rule is additionally limited to the test service, for example TCP port 8080. Replace the token, direction, and port with values that occur in the real data stream visible to IPS. This isn’t a universal attack signature and must not be placed unchanged on broad production rules.

Payload and search window

content searches for a character or byte sequence; binary values are enclosed in pipe characters. nocase ignores case for a content match, and rawbytes works on raw data. depth and offset limit the search absolutely in the payload, while distance and within work relative to the previous match. uricontent, isdataat, and pcre cover more specialized URI, position, and regular-expression cases.

A narrow search range reduces accidental matches and processing overhead. In particular, pcre, large windows, and several broad content patterns should only be introduced with realistic packets and observed load. If depth is shorter than the searched content pattern, the signature can never match.

The modifiers nocase, rawbytes, depth, offset, distance, and within belong to content; they aren’t independent detection rules. Without depth, the search continues from offset to the end of the payload. Without within, the search for a second content pattern also remains open from distance to the end. A starting position alone therefore doesn’t keep the search window narrow. uricontent searches the normalized request URI, not arbitrary payload. isdataat checks whether data exists at a specified position; with relative, that position is measured from the end of the previous content match.

Headers, streams, and structured values

Source, destination, and port can be narrowed with srcaddr, dstaddr, srcport, and dstport. IP header options include ttl, tos, id, ipopts, fragoffset, fragbits, dsize, ip_proto, and samip. TCP characteristics use flags, flow, seq, ack, and window; itype, icode, icmp_id, and icmp_seq apply to ICMP. rpc, byte_test, and byte_jump are intended for structured or binary protocols.

flow applies only to TCP. to_server and from_client mean the same thing, as do to_client and from_server. bi_direction extends detection to both directions; when a port condition is used, the negative test must therefore also consider return traffic. Don’t confuse the choice between no_stream and only_stream with a packet-size check: dsize checks a packet’s payload size and doesn’t match packets rebuilt from a stream. Combining it with only_stream is therefore inappropriate. If a match is missing, first decide whether the characteristic should be sought in an individual packet or in the rebuilt stream, rather than merely changing the size or content pattern.

For complex rules, use the SFOS 22 custom IPS syntax described above. Don’t adopt undocumented Snort keywords or rules copied from other engines without verification.

Operands: address, port, URI, and data position

The following fragments belong in Custom rule, not in the shell. They are a reference for SFOS 22/23, not tested attack signatures. Combine only conditions appropriate to the detection case; Protocol, the pilot policy, and Allow packet remain part of the test described above. Square brackets in syntax patterns indicate optional parts and aren’t entered literally.

• srcaddr:192.0.2.10; dstaddr:198.51.100.20; compares source and destination IPs with individual addresses. Replace the documentation addresses with the observed test addresses; this doesn’t establish support for subnets or address lists.

• srcport:40000; dstport:8080; compares source and destination ports with numbers. The destination port represents the test service here. Fix a changing client source port only when it really belongs to the detection case; otherwise new connections fail this condition.

• uricontent:"/ips-pilot"; searches for the replaceable path in the normalized request URI. Mixed text/binary patterns are also possible. The comparison must suit the normalized URI value, not merely the spelling in the browser address bar.

• content:"AVANET-IPS-PILOT"; isdataat:50,relative; additionally requires data at position 50 relative to the end of this content match. Without relative, the position is absolute in the payload. Choose the number from the actual protocol layout; it checks data availability, not the contents of that data.

• content:"ABC"; content:"DEF"; distance:1; within:10; uses two replaceable patterns with a relative search window. distance sets the gap from the end of the first match; within limits the following search. For absolute windows, offset and depth also take numeric values, for example offset:4; depth:20; attached to the relevant content. Choose these numbers from the actual pattern position and length; they are not protocol defaults.

Regular expressions with pcre

The basic forms are pcre:"/REGEX/"; and pcre:"m/REGEX/";. A ! before the opening quotation mark negates the expression, for example pcre:!"/AVANET-IPS-PILOT/i";. This is a non-match condition and can be very broad without additional limits. Replace REGEX or the pilot token with the required pattern. Options follow the closing /, as in pcre:"/AVANET-IPS-PILOT/i";.

• i ignores case; s lets . also match newlines.
• m lets ^ and $ also match line starts and ends within the buffer. x ignores unescaped whitespace in the pattern, except within character classes.
• A anchors the match at the start of the buffer. E binds $ only to the actual end, not the position before a final newline.
• G reverses the default greediness of quantifiers; a following ? reverses it again.
• R starts relative to the end of the last pattern match. U uses decoded URI buffers; B doesn’t use decoded buffers.

Before the pilot, therefore, decide which buffer and starting position the pattern should use. A correct regex on raw data need not match a decoded URI.

Read binary fields: byte_test and byte_jump

byte_test reads a field and tests its numeric value. The pattern is byte_test:COUNT,[!]OPERATOR,VALUE,OFFSET[,relative][,big|little][,NUMBER_TYPE,string];. COUNT is the number of bytes read and OFFSET the starting position in the payload; with relative, the start is relative to the last pattern match. VALUE is the comparison value. The documented operators are <, >, =, !, and &: less than, greater than, equal, unequal, and bitwise AND. A ! before a comparison operator negates its result. big or little sets the byte order. With string, text digits are read instead of a binary numeric field; hex, dec, and oct apply only to this text mode.

Binary-field example: byte_test:2,=,16,0,big; reads two bytes from payload position 0 as big endian and compares them with 16. Count, position, byte order, and comparison value must come from the protocol, not this example. An arbitrary numeric value isn’t evidence of an attack.

byte_jump instead reads a length and moves the position for subsequent checks. The pattern is byte_jump:COUNT,OFFSET[,relative][,multiplier FACTOR][,big|little][,string][,hex|dec|oct][,align][,from_beginning];. The first two operands determine field width and read position; relative measures the offset from the last pattern match. multiplier multiplies the value read to determine the jump distance. align rounds this up to the next 32-bit boundary. With from_beginning, the jump distance is applied from the start of the payload instead of the current position. string and the number bases have the same purpose as with byte_test.

byte_jump:2,0,big; is thus a fragment for a two-byte length field at the start of the payload with explicitly selected byte order. It isn’t a size comparison; a comparison operator and comparison value aren’t operands of byte_jump. For both byte keywords, explicitly select big or little for binary multibyte fields: don’t rely on a default byte order inferred from the documentation. First trace the field and jump target in the actual test payload, then check positive and negative cases for the following condition. An incorrect length can move the search position past the intended pattern.

IP fields, options, and fragments

• ttl:64;, ttl:>64;, and ttl:<64; compare the IP time to live exactly, above, or below a value. tos:16; compares the TOS field; id:1234; compares the IP ID field. These example numbers aren’t recommended detection values; choose them from the header characteristic actually sought.
• ipopts:rr; checks for an IP option. Allowed values are rr (Record Route), eol (End of List), nop (No Operation), ts (Timestamp), sec (Security), lsrr (Loose Source Routing), ssrr (Strict Source Routing), satid (Stream Identifier), and any (any IP option). Thus any requires an option, not just any IP packet.
• fragoffset:0; compares the fragment offset in the IP header with a decimal number. This field isn’t the content search offset in the payload.
• fragbits checks M (More Fragments), D (Don’t Fragment), and R (Reserved). A leading + permits other bits in addition to those specified, * requires at least one specified bit, and ! requires that the specified bits aren’t set. Thus fragbits:+M; checks for More Fragments without excluding other bits.
• dsize:300;, dsize:>300;, dsize:<300;, and dsize:300<>400; specify exact payload size, an upper/lower comparison, or a range. The numbers refer to packet payload, not the entire connection. The limitations regarding rebuilt streams explained above still apply.
• ip_proto:6;, ip_proto:!6;, ip_proto:>6;, and ip_proto:<6; compare the numeric protocol identifier in the IP header, not the TCP/UDP port. samip; takes no value and requires identical source and destination IPs.

TCP fields and stream selection

flags checks TCP flags: S for SYN, A for ACK, F for FIN, R for RST, U for URG, and P for PSH. Without a modifier, for example, flags:S; checks for exactly SYN. A leading +, *, or ! has the meaning described for fragbits above. flags:+S; therefore permits additional flags. A mask can follow a comma: flags:S,A; excludes ACK from the comparison rather than requiring ACK as well. Such a mask broadens the match. Special cases for reserved TCP bits and “no flags set” are deliberately not approved here as copyable syntax; they require a version-specific confirmed rule and must not be inferred from an unclear label.

For flow, the directions already explained are available along with established, no_stream, and only_stream. flow:established; requires an established TCP connection. flow:no_stream; excludes rebuilt stream packets; flow:only_stream; limits detection to them. These fragments explain individual choices and aren’t an instruction to use both stream options together. bi_direction remains the extension to outbound and return traffic explained above.

seq:1234;, ack:1234;, and window:4096; compare the TCP sequence number, acknowledgment number, and window size with a number. Exact sequence or ACK values change between connections and make sense only when that exact value belongs to the detection case. The TCP window isn’t a payload search window.

ICMP and RPC

itype:8; and icode:0; compare ICMP type and code. Both also support <, >, and ranges such as itype:3<>5;; type and code are different fields. icmp_id:1234; and icmp_seq:1; compare the ICMP identifier and sequence value exactly. Choose values for the intended ICMP case rather than equating them with TCP sequence numbers.

rpc:100000,2,*; checks SUNRPC CALL requests for application number 100000, version 2, and any procedure. The pattern is rpc:APPLICATION,VERSION,PROCEDURE;; * is allowed for version and procedure, not for application number. Replace the numbers with the application, version, and procedure sought. A wildcard doesn’t remove the need to analyze the protocol: it deliberately leaves that part open and therefore broadens detection.

Create the signature and assign it to a policy

Under Intrusion prevention > Custom IPS signatures > Add, specify Name, Protocol, Custom rule, Severity, and Recommended action. A name such as PILOT-TCP-8080-AVANET-TOKEN makes the purpose and test boundary visible. Severity is your own risk assessment; it doesn’t prove that the pattern is malicious.

Keep Recommended action set to Allow packet for the first run. Important: The IPS policy rule action overrides this recommendation, so the pilot rule must also use Allow packet. The other actions have much stronger effects:

• Drop packet discards only the matching packet.
• Drop session terminates the session after the match.
• Reset terminates a TCP session and sends a reset to the originator.
• Bypass session permits the traffic and stops scanning the rest of the session.

For session-based actions, SFOS scans only until the first matching packet. Before changing Allow packet to one of these actions, confirm that this first match belongs to the intended session.

When saving, SFOS reconfigures the IPS engine. This happens without interruption when sufficient free RAM is available. With little free RAM, the engine can restart and cause a brief disruption. Even a seemingly small signature change therefore belongs in an observed maintenance window. After saving, check the IPS status and ips.log for a restart or errors before continuing the pilot.

The often-missed second part follows: Open a deletable policy intended for the pilot under Intrusion prevention > IPS policies, add a rule, choose Custom signature, select the signature, and place the specific rule above broader rules. SFOS evaluates IPS policy rules from top to bottom. Then assign this policy under Rules and policies > Firewall rules > [pilot rule] > Detect and prevent exploits (IPS) to the pilot rule that actually matches.

Perform a positive and negative test

The positive test sends the agreed pattern over the intended port and in the expected direction. In Log viewer, opened from the upper-right corner of WebAdmin, select the IPS module and filter by test time, source, and destination. The signature or SID, action, and timestamp must match the test. ips.log provides additional engine details. Packet capture under Diagnostics > Packet capture additionally shows the Firewall Rule ID and IPS Policy ID. Test a firewall rule systematically explains how to connect the rule, packet capture, and security module.

After the pilot, you can also review the effect under Reports > Network & threats > Intrusion attacks. This report is useful for period comparisons, but it doesn’t replace timely positive and negative tests in Log viewer.

At least one negative test follows: the same service without the token, another port, or a different direction. The signature must not match. For a content pattern, normal application requests are also important because short or generic strings can occur in completely legitimate payloads.

Only after both tests are stable should the intended blocking action be set and tested again. A blocking signature is accepted only when it stops exactly the positive case, the negative case continues, and no other firewall or IPS rule causes the effect.

Check the number of signatures

The count can be found in WebAdmin without using the shell. Under Intrusion prevention > IPS policies, open a deletable policy and add a new policy rule. With Select all, SFOS shows the total above Action. The list is only visible when adding a rule to a deletable policy; close the dialog without saving afterward.

Sophos also documents two read-only queries under 5. Device management > 3. Advanced shell:

psql -U nobody -d signature -p 5434 -c "select count (*) from tblidprules;"
psql -U nobody -d corporate -c "select count (*) from tblidpcustomsignature;"

The first command counts default signatures and the second counts custom signatures. These database queries don’t change data, but still belong in a documented support or diagnostic session. The count isn’t proof of quality or operation and can change with pattern updates.

When the signature doesn’t behave as expected

There is no match

First verify that IPS is active, the traffic matches the expected firewall rule with the correct IPS policy, and the custom signature is actually present in an evaluated policy rule. Then examine protocol, direction, port, encryption, and the real payload. A string shown in a browser doesn’t necessarily appear unchanged in the network packet.

A narrowly filtered packet capture helps confirm visible content and direction. If the pattern isn’t present there, changing the IPS rule can’t create it. If the packets are visible, check offset, depth, distance, within, stream state, and policy-rule order.

Too many connections match

Keep the signature on Allow packet until source, destination, port, direction, or search window has been narrowed. Generic words, short binary sequences, and unbounded regular expressions are typical causes. A broad firewall rule makes the effect even harder to control.

If the firewall shows increased resource usage or an IPS restart after saving, record the time, model, firmware, free resources, and ips.log. Repeatedly saving different variants isn’t a clean test at this point; first clarify the cause and a maintenance window.

Roll back safely

For unexpected matches, first remove the custom rule from the pilot policy or restore the previous IPS policy on the firewall rule. Then verify new sessions with the positive and negative tests. Delete the custom signature only when it has no other use.

Before deletion, document which policy, firewall rule, and application used it. Logs, the tested rule version, and the reason for rollback belong in the change record. Turning IPS off globally or removing the entire production policy isn’t an appropriate rollback for a single defective signature.

FAQ