IAM privilege escalation and persistence in CloudTrail

TL;DR. After a key leak, the attacker's priority is to not depend on that key. The usual moves, all visible in CloudTrail under iam.amazonaws.com: CreateUser → AttachUserPolicy (AdministratorAccess) → CreateAccessKey and CreateLoginProfile for that user. Quieter variants: a second key on an existing user, an extra principal in a role's trust policy, a new SAML/OIDC provider, a public Lambda or modified EC2 user data. Read requestParameters for the target and responseElements for new key IDs, and remove every one of them before declaring the incident closed.

Revoking the leaked credential is the easy part of an AWS incident. The part that goes wrong is the backdoor you did not find — the one that lets the same actor back in a week later with a key you never saw. This post goes through the IAM persistence and privilege-escalation techniques that show up again and again in the logs, and what each looks like in CloudTrail.

IAM events are global, and logged in us-east-1

IAM is a global service; like most global service events, its CloudTrail events are recorded with awsRegion set to us-east-1. If you only pulled event history for the region where your workloads run, you may have missed every IAM change. Check that region explicitly.

Backdoor users

The loud version, and still the most common:

{"eventSource": "iam.amazonaws.com", "eventName": "CreateUser",
 "requestParameters": {"userName": "backup-admin"}}
{"eventName": "AttachUserPolicy",
 "requestParameters": {"userName": "backup-admin",
   "policyArn": "arn:aws:iam::aws:policy/AdministratorAccess"}}
{"eventName": "CreateAccessKey",
 "requestParameters": {"userName": "backup-admin"},
 "responseElements": {"accessKey": {"accessKeyId": "AKIA..."}}}
{"eventName": "CreateLoginProfile",
 "requestParameters": {"userName": "backup-admin", "passwordResetRequired": false}}

What gives it away:

• The four calls come seconds apart from the same key and IP.
• The name is chosen to blend in: backup, support, svc-…, terraform, something that looks like a service account.
• passwordResetRequired: false on a login profile created by a script.
• The user then signs in to the console without MFA (ConsoleLogin with additionalEventData.MFAUsed = "No").

ATT&CK: T1136.003 Create Account: Cloud Account, T1098.003 Additional Cloud Roles.

Extra credentials on existing identities

Quieter: instead of creating a user, the attacker adds a second access key to an existing one (users can have two), or sets a console password on a service user that never had one.

• CreateAccessKey where requestParameters.userName differs from the caller's own user name — a principal minting a key for someone else is rare in normal operations.
• CreateLoginProfile or UpdateLoginProfile on a user that should be programmatic-only.

The credential report helps here: access_key_2_active = true on a user that has always had one key, or password_enabled = true on a service account, deserve a question.

ATT&CK: T1098.001 Additional Cloud Credentials.

Admin rights: attached, inline or through a group

CreatePolicyVersion deserves attention because it changes what an existing, trusted policy grants without changing its name or its attachments. SetDefaultPolicyVersion can do the same by rolling back to an older, broader version.

Role trust and federation

Roles are harder to spot than users because nothing new appears in the user list.

• UpdateAssumeRolePolicy rewrites who can assume a role. An attacker adds their own AWS account ID or an external principal. Compare the new policyDocument with the previous one (AWS Config history, infrastructure-as-code, or an earlier GetRole).
• CreateSAMLProvider, CreateOpenIDConnectProvider, AddClientIDToOpenIDConnectProvider, UpdateOpenIDConnectProviderThumbprint: whoever controls the identity provider can obtain role credentials in the account via AssumeRoleWithSAML / AssumeRoleWithWebIdentity. ATT&CK: T1484.002 Trust Modification.

When the organisation federates through an external identity provider, the attacker may have come in through it instead of IAM; for Okta-backed access see oktaforensics.com.

Compute-based persistence

Lambda code deployments from CI are normal. The same event from a human's long-term key at an unusual hour is not.

How to be sure you found them all

1. Pivot on every principal the attacker controlled (the leaked key's user, created users, assumed roles) and list their successful writes (readOnly = false, no errorCode).
2. Extract every new key ID from CreateAccessKey responses and pivot on those too.
3. Compare IAM today with IAM before: credential report, aws iam get-account-authorization-details output, or your IaC state.
4. Look for failed attempts: an AccessDenied on CreateUser tells you what the attacker tried, and whether a later success came from another principal.

In the analyzer

AWS Forensics has a rule for each technique above — IAM user created, key created for another user, console password set, role trust changed, identity provider added, user added to an admin group, public Lambda, Lambda code deployed, user data modified, key pair created, admin policy attached, *:* policy written — and the remediation checklist names the exact users, keys and roles to clean up. The events-to-monitor reference lists them with their ATT&CK ids, and the fictional walkthrough shows a backdoor admin being created and found.

Further reading

• IAM security best practices — AWS documentation.
• MITRE ATT&CK T1098 Account Manipulation.
• Remediating potentially compromised AWS credentials.