Add enriched metadata to Amazon VPC flow logs published to CloudWatch Logs and S3

Posted on: May 5, 2020

Now you can include enriched metadata in Amazon Virtual Private Cloud (Amazon VPC) flow logs published to Amazon CloudWatch Logs or Amazon Simple Storage Service (S3). Prior to this launch, custom format VPC flow logs enriched with additional metadata could be published only to S3. With this launch, we are also adding metadata fields that provide insights about the location of the network interface on which flow logs are being captured, such as the AWS Region, AWS Availability Zone, AWS Local Zone, or AWS Outpost where it resides.

Enriched metadata fields in VPC flow logs reduce the cost and operational overhead associated with the additional computations or lookups required to extract meaningful information from log data in a centralized log processing system. You can use VPC flow logs to monitor VPC traffic, understand network dependencies, troubleshoot network connectivity issues, and identify network threats. 

To get started, simply create a new flow log subscription with your chosen set of metadata fields and CloudWatch Logs or S3 as the destination. For either destination, you can choose from a list of available metadata fields, including new fields to identify location such as region, availability zone ID, sublocation ID and sublocation type, and existing fields such as Transmission Control Protocol (TCP) bitmasks to infer flow directionality, packet-level source and destination IPs to identify the source and intended target of flows passing through an intermediate layer such as NAT Gateway or Transit Gateway and resource IDs such as instance ID, VPC ID and subnet ID corresponding to the network interface where flow logs are being captured.

This functionality is available at no additional charge through the AWS Management Console, the AWS Command Line Interface (AWS CLI), and the AWS Software Development Kit (AWS SDK). To learn more about Amazon VPC flow logs, please refer to the documentation.