A CloudTrail Trail Marked 'Logging' That Silently Excludes Data Events

Symptom

A CloudTrail trail in an AWS account reports a healthy state: aws cloudtrail get-trail-status returns IsLogging: true, the S3 delivery bucket receives regular log files, and CloudWatch Logs shows a steady stream of management events. Yet during an incident review, investigators cannot find any record of who read or deleted specific objects from a sensitive S3 bucket, nor any record of a Lambda function being invoked with a particular payload at a particular time. The trail looks operational in every dashboard, but the specific evidence needed does not exist.

False Assumption

The operator assumed that enabling a trail and confirming IsLogging: true means all relevant activity, including object-level S3 access and Lambda invocations, is being captured. In AWS CloudTrail, a trail’s logging status reflects whether management events are being recorded by default; data events for services such as S3 and Lambda are a separate, explicitly opted-in event selector category. A trail can be fully ‘logging’ while data events remain completely disabled, and nothing in the basic status check surfaces that gap.

Root Cause

CloudTrail separates event recording into management events (control-plane API calls such as creating or deleting resources) and data events (data-plane operations such as S3 GetObject or Lambda Invoke). Management event logging is enabled by default when a trail is created. Data event logging for S3, Lambda and other supported resources must be configured explicitly through advanced event selectors or basic data event selectors, and it carries additional cost. When a team configures a trail through the console quick-start path or a minimal Infrastructure-as-Code template, it is common to create the trail, confirm it is logging, and move on without ever adding data event selectors. The trail is genuinely logging, just not the category of event the team believed it covered.

Impact

The organisation loses forensic visibility into data-plane activity on the resources most likely to matter during a security incident: who accessed or exfiltrated objects in a specific S3 bucket, or what parameters were passed to a Lambda function during a suspected compromise. This gap is typically discovered only after an incident, when the absence of expected log entries is mistaken for an attacker having covered their tracks rather than for a configuration gap that existed from day one. Because the trail’s own status check reports success, the missing coverage does not trigger any alert, so the organisation carries this blind spot indefinitely without a signal that anything is wrong.

Diagnosis

Confirm the gap using read-only CloudTrail and IAM API calls rather than assuming from dashboard status alone.

• Run aws cloudtrail get-trail-status --name <trail-name> and confirm IsLogging: true, which only confirms management event delivery, not data event coverage.
• Run aws cloudtrail get-event-selectors --trail-name <trail-name> to inspect configured event selectors. If the output shows only ReadWriteType: All for management events with no DataResources entries for AWS::S3::Object or AWS::Lambda::Function, data events are not being recorded for those resource types.
• Cross-check by attempting to locate a known, recent S3 object read or Lambda invocation in the delivered log files or in CloudWatch Logs Insights. Its absence, combined with the missing DataResources entry, confirms the gap rather than a delivery delay.

Correction

Add explicit advanced event selectors to the existing trail to record the specific data events the workload requires, scoped to avoid unnecessary cost and volume.

aws cloudtrail put-event-selectors 
  --trail-name <trail-name> 
  --advanced-event-selectors '[
    {
      "Name": "S3ObjectDataEvents",
      "FieldSelectors": [
        {"Field": "eventCategory", "Equals": ["Data"]},
        {"Field": "resources.type", "Equals": ["AWS::S3::Object"]},
        {"Field": "resources.ARN", "StartsWith": ["arn:aws:s3:::example-sensitive-bucket/"]}
      ]
    },
    {
      "Name": "LambdaInvokeDataEvents",
      "FieldSelectors": [
        {"Field": "eventCategory", "Equals": ["Data"]},
        {"Field": "resources.type", "Equals": ["AWS::Lambda::Function"]}
      ]
    }
  ]'

Scope the S3 selector to the specific bucket ARN prefix that matters operationally, rather than enabling data events account-wide, to keep log volume and cost proportionate to the actual risk.

Validation

Validation requires generating a known test event and confirming it appears in delivered CloudTrail logs before relying on the corrected configuration.

• Run aws cloudtrail get-event-selectors --trail-name <trail-name> and confirm the output now includes DataResources entries for AWS::S3::Object and AWS::Lambda::Function matching the intended scope.
• In the isolated validation environment, perform a single GetObject against the scoped test bucket and one test invocation of a non-production Lambda function.
• Within the account’s standard CloudTrail delivery latency window, query CloudWatch Logs Insights or the S3 log delivery bucket for the corresponding eventName: GetObject and eventName: Invoke entries with matching timestamps and resource ARNs.
• Pass condition: both test events appear in delivered logs with correct identity, timestamp and resource fields. If either is absent after the expected delivery window, treat the selector configuration as unverified and re-check the ARN scoping before relying on it operationally.

Rollback

If the new event selectors generate unexpected cost or log volume, roll back to the prior selector configuration rather than disabling the trail.

• Record the exact get-event-selectors output before applying any change, so the prior state can be restored precisely.
• To revert, run aws cloudtrail put-event-selectors --trail-name <trail-name> --event-selectors '[{"ReadWriteType": "All", "IncludeManagementEvents": true}]' using the exact prior selector JSON captured beforehand, restoring management-event-only logging.
• Stop condition: if log volume or estimated cost exceeds the agreed threshold within the first monitoring window after applying the change, revert immediately rather than narrowing scope live in production.
• Do not disable the trail itself as a rollback step; disabling logging removes management event coverage that was already working and creates a new, larger visibility gap.

Prevention

Treat data event coverage as a checklist item at trail creation and during periodic access-logging reviews, not an assumption inferred from trail status. Document, per trail, which resource types have explicit data event selectors and which do not, and review that document alongside any change to which S3 buckets or Lambda functions are considered sensitive. Where Infrastructure-as-Code defines the trail, encode the intended event selectors directly in the template so future re-deployments cannot silently drop them, and add a scheduled, read-only check that compares configured selectors against the current inventory of sensitive resources.