Skip to content
Avanet
Sign in

Connect cloud flow logs to Sophos XDR and SIEM

Sophos Fusion Admin Sophos SIEM

Cloud flow logs describe network connections, rather than complete packet contents. Sophos XDR can ingest them from AWS VPC Flow Logs, Azure Network Watcher, Google Cloud Platform (GCP) and Oracle Cloud Infrastructure (OCI). Each source has its own path: S3 and Lambda for AWS, Azure Storage and a Function for Network Watcher, Pub/Sub for GCP, and Streaming for OCI.

Netflow telemetry from these integrations requires Sophos Next-Gen SIEM. GCP and OCI also deliver other schemas through the same integrations; this does not make all cloud audit or security data SIEM-only. First confirm the required event categories, tenant, Fusion upgrade and entitlement. SIEM scope and retention planning explains the purchasing considerations.

These direct cloud integrations differ from a syslog collector hosted in the cloud. For that path, use the XDR collector runbook.

Before the first cloud change

You need the appropriate product tile for the target tenant under Security Operations > Integrations > Marketplace, approved cloud administration permissions for the resources concerned, and a team responsible for ongoing costs, keys and data quality. Check availability and displayed fields in your tenant; do not substitute a similarly named integration when a tile is missing.

For each source, record the cloud account or subscription, region, required log categories, target integration and ingestion start time. Limit permissions to the intended data path. Administrative access needed for deployment does not justify permanently using a personal administrator account as the integration identity. Resources such as Lambda, Storage and Pub/Sub can incur separate cloud costs.

A green integration status confirms communication, not complete ingestion. Acceptance requires an approved test event with source evidence and a timestamp, followed by the corresponding normalized record. The integration guide explains permissions, data flow and the distinction from response actions.

AWS VPC Flow Logs through your own S3 bucket

AWS VPC Flow Logs are ingested from your own Amazon S3 bucket. Sophos supplies a CloudFormation template that creates a forwarding Lambda function. You need an AWS account, a bucket containing the flow logs, and permission to create the required Lambda, CloudFormation and IAM resources.

  1. Open AWS VPC Flow Logs in the Sophos Marketplace. Under Configured integrations > Add new, read the S3 information, select Next, enter an identifiable integration name and select Done. Download the CloudFormation template generated after provisioning and store it with restricted access.
  2. In AWS, direct the intended VPC flow log export to that S3 bucket. The logging destination must be Amazon S3. Use only the default AWS format; a custom format can impair normalization and detection.
  3. In the same AWS region as the logs bucket, open CloudFormation > Create stack, upload the downloaded template and choose a stack name without spaces. Set IntegrationType to awsvpcflow and enter the bucket name, not a URL or S3 URI, in NotificationBucket. If intentionally using SNS instead of S3 notifications, enter its ARN in SNSNotificationarn. Check the remaining template parameters against the planned data path.
  4. Review and acknowledge IAM resource creation, create the stack and wait for CREATE_COMPLETE. A separate bucket holding the template does not need to be made public.
  5. In Lambda, open the function created by the stack. Under Function overview > Add Trigger, add the logs bucket as an S3 source. Select Event Type: All Object Create Events. Prefix must contain only the actual log prefix in the bucket; leave it empty if no prefix is used. Leave Suffix empty. Review the cost implications and add the trigger.
  6. Generate an appropriate harmless network flow in the selected VPC scope. Verify the flow log record in the bucket first, then compare the same time range and connection details in the Sophos netflow schema.

The required default format is:

${version} ${account-id} ${interface-id} ${srcaddr} ${dstaddr} ${srcport} ${dstport} ${protocol} ${packets} ${bytes} ${start} ${end} ${action} ${log-status}

If no data arrives, first compare the export destination, newly created objects, region and trigger prefix. The template sets Reserved Concurrency to 5 by default; this is not a sizing guarantee. Assess actual load and CloudWatch metrics. Rate Exceeded can indicate an AWS concurrency quota; repeatedly recreating the Sophos integration does not resolve that limit.

Azure Network Watcher through Storage and an Azure Function

Network Watcher writes NSG Flow Logs or VNet Flow Logs to an Azure Storage Account. New NSG Flow Logs can no longer be created. Plan VNet Flow Logs for new sources. Existing NSG Flow Logs can be connected until they are replaced; Microsoft retires the feature on 30 September 2027. Plan their migration to VNet Flow Logs before that date. The two types require separate Sophos integrations, because they use different blob containers or Data Source Keys.

You need an active Azure subscription, the Storage Account with its blob container and flow logging enabled, and a supported Azure Function service plan. Documented options are Basic with B1, B2 or B3, Dynamic with Y1, and specific Premium codes. Isolated is unsupported. Select a Premium code only from the currently offered list, rather than inferring support from a similar Azure plan name.

  1. In Network Watcher, send VNet Flow Logs for a new source to the chosen Storage Account; use existing NSG Flow Logs only as a transition until migration. Record the Storage Account and blob container names, and check that new flow log objects actually arrive.
  2. In the Sophos Marketplace, open Microsoft Azure Network Watcher - Azure Storage Account. Create an integration under Azure Storage Accounts > Set Up.
  3. Copy Storage Account Name exactly. Function App Name is your own descriptive name, for example NSGFlowLogsForwarder, not a fixed product value. Set Data Source Key to insights-logs-networksecuritygroupflowevent for NSG or insights-logs-flowlogflowevent for VNet.
  4. Choose Azure Function Service Plan Type, the matching Service Plan Code and Num Workers. The default is one worker. Basic B2 or B3 supports a documented maximum of three workers; do not generalize this limit to other plans.
  5. Select Done to generate and download AzureFunction.json. In Azure, load it through Deploy a custom template > Build your own template in the editor. Review the Resource Group and parameters, then select Review + Create and Create.
  6. The Storage Account and Function can be in different subscription IDs or Resource Groups. If so, explicitly set Storage Account Subscription ID and Storage Account Resource Group to identify the actual source.
  7. After deployment, verify an expected flow event in the correct container and then as netflow in Sophos. Accept each integration separately when using both log types.

If the Function is deployed but data is missing, compare the container, Data Source Key, subscription/Resource Group mapping and service plan. A second integration for the same stream can create duplicates; it is not a general repair step.

GCP through Pub/Sub

GCP provides cloudaudit, thirdpartyalert and, with SIEM, netflow. Supported sources include Cloud Audit Logs, Security Command Center Findings, VPC Flow Logs and GKE Dataplane V2 logs. Include only the categories deliberately selected.

  1. In the correct GCP project, create a topic under Pub/Sub > Topics. Under Subscriptions, create a subscription for it and set Acknowledgement deadline to 300 seconds.
  2. Under Logging > Log router > Create Sink, create a sink targeting that Cloud Pub/Sub topic. Limit inclusion filters to the required log sources. Organization and folder logs need an appropriate aggregated sink; a project sink does not automatically cover them.
  3. Create a dedicated service account. Under Add Principal on the Pub/Sub subscription, give its service account ID the Pub/Sub Subscriber role.
  4. Also create a custom role containing monitoring.timeSeries.list and assign it to that service account. An account already used by Sophos XDR can be reused if this required permission is also approved.
  5. Generate a JSON key for the account and store it securely. In the Sophos Marketplace, open GCP > Configured integrations > Add new. Enter the integration name, Project ID and Pub/Sub Subscription ID, then upload the correct JSON file with Upload Key File.
  6. Actually enable the selected sources: Data Access audit logs require explicit service selection under IAM & Admin > Audit Logs. SCC findings use the corresponding Pub/Sub export. Enable VPC Flow Logs at subnet level; sampling limits what they can establish compared with a full packet capture.
  7. GKE Dataplane V2 must already be selected when creating the cluster. Check the required Network Policy Logging configuration and sink filter afterwards. Do not unexpectedly replace an existing cluster merely for a logging pilot.
  8. Generate a suitable test event for every enabled category. Check source evidence, the Pub/Sub path and the normalized schema separately.

For key rotation, create the new JSON key first and update the existing Sophos integration. In API Query Log, secrets cache invalidated. Integration was updated. Restarting. indicates the update. Delete the old GCP key only after ingestion has been verified again. A successful file upload alone is insufficient.

OCI through Streaming

OCI provides cloudaudit, http, nids and, with SIEM, netflow. The path requires an OCI stream, the intended logs and a service connector. The integration identity needs stream-pull to read the stream; broad administrative rights are not required for ongoing retrieval.

  1. Prepare a dedicated logging user and an approved group/compartment structure. Grant only the necessary read access to the intended stream.
  2. Under OCI Streams > Create Stream, select the intended Stream Pool. Record the Stream Pool OCID; it is not interchangeable with an arbitrary stream or compartment ID.
  3. Use Connector Hub to route approved audit, flow, load-balancer, WAF or firewall logs to that stream. Before configuring Sophos, verify that the connector delivers data to the correct Streaming destination. Another connector destination does not automatically populate this stream.
  4. In the Sophos Marketplace, open OCI > Configured integrations > Add new. Enter Name, the dedicated Username, its Auth token, Stream Pool ID, Tenant Name, the stream’s Topic Name, and Region.
  5. For nested compartments, Tenant Name is the tenancy name without the compartment chain. For federated users in the documented OracleIdentityCloudService case, use a value such as acme/OracleIdentityCloudService; acme represents your own tenancy name. Do not mix these cases.
  6. Select Done and verify a suitable source event in OCI, in the Streaming path, then in the expected Sophos schema. A cloud audit event does not test an additional netflow stream.

If ingestion fails, first check Stream Pool, stream name, region, authentication token and stream-pull. If only one category is missing, check source selection and service-connector routing before recreating the whole integration.

Validate data quality and controlled removal

Acceptance records each category’s source, ingestion start, time window, normalized schema and a reproducible example. Compare event time and time zone with the source evidence. An ordinary flow or audit record need not create a detection or MDR case. A green connection or a retrieved record also does not prove years of retention.

Before removal, identify the data path and shared resources. Stop only the approved source export, sink, trigger or connector; do not indiscriminately delete a shared bucket, topic, Storage Account or stream. After confirming that no new records arrive, remove the relevant Sophos configuration and exclusively dedicated keys or cloud resources in accordance with change and retention decisions. Deleting a connection does not establish that historical telemetry has been deleted.

Keep previous required logging paths available during a failed pilot. Rollback means reversing the documented new routing or export change and verifying the result. It does not automatically undo data already sent, cloud costs or missing events.