Cross-account secret sharing in AWS can quickly turn into a security nightmare when AWS Secrets Manager KMS integration goes wrong. Misconfigured KMS key policies and cross-account permissions often leave development teams locked out of critical secrets or, worse, accidentally exposing sensitive data.
Who this guide is for: DevOps engineers, security architects, and AWS administrators managing secrets across multiple AWS accounts who need to avoid common cross-account KMS permissions pitfalls.
We’ll walk through the most frequent cross-account configuration mistakes that break secret access and show you how to implement secure cross-account KMS key policies for Secrets Manager. You’ll also learn proven troubleshooting techniques to quickly resolve access issues when cross-account secret sharing stops working.
Understanding AWS Secrets Manager and KMS Integration Architecture
Core Components of Secrets Manager Encryption
AWS Secrets Manager uses envelope encryption to protect your sensitive data, combining symmetric data keys for fast encryption with KMS keys for secure key management. Every secret gets encrypted with a unique data encryption key (DEK), which is then encrypted using your specified KMS key. The encrypted DEK travels alongside your secret, creating a secure envelope that requires both the encrypted data and valid KMS permissions to decrypt. This dual-layer approach ensures that even if someone gains access to the encrypted secret, they still need proper KMS key permissions to decrypt the underlying data key and access your sensitive information.
How KMS Keys Control Access to Secrets
KMS keys act as the primary gatekeeper for Secrets Manager access, controlling who can decrypt secrets through comprehensive key policies and IAM permissions. When you retrieve a secret, Secrets Manager must first call KMS to decrypt the data encryption key, making KMS permissions absolutely critical for secret access. The key policy determines which AWS accounts, users, and roles can use the KMS key for encryption and decryption operations. Cross-account access requires explicit permissions in both the KMS key policy and the receiving account’s IAM policies, creating multiple layers of authorization that must align perfectly for successful secret retrieval.
Default vs Custom KMS Key Configurations
Secrets Manager offers two encryption options: the AWS managed key aws/secretsmanager or your own customer managed KMS keys. The default AWS managed key provides automatic encryption without additional configuration but limits your control over access policies and cross-account sharing. Customer managed KMS keys give you complete control over key policies, rotation schedules, and cross-account permissions, making them essential for enterprise environments with complex access requirements. While AWS managed keys work well for single-account scenarios, custom keys become necessary when you need to share secrets across AWS accounts or implement granular access controls that the default key cannot provide.
Cross-Account Access Patterns and Dependencies
Cross-account secret sharing creates complex dependency chains between Secrets Manager permissions, KMS key policies, and IAM roles across multiple AWS accounts. The pattern typically involves the secret owner account granting KMS key access to external accounts, while those accounts must have IAM permissions to call Secrets Manager operations. Common patterns include centralized secret management where one account stores secrets for multiple application accounts, and distributed patterns where each account maintains its own secrets but shares specific ones with trusted accounts. These configurations require careful coordination between resource-based policies (KMS key policies) and identity-based policies (IAM), with any misalignment causing access failures that can be challenging to troubleshoot.
Common Cross-Account Configuration Mistakes That Break Secret Access
Misconfigured KMS Key Policies for Cross-Account Access
The most frequent mistake involves setting up KMS key policies that block cross-account access to Secrets Manager resources. When Account A creates a secret encrypted with a KMS key, but Account B needs access, the KMS key policy must explicitly grant permissions to the cross-account role or user. Many developers forget to add the target account’s ARN to the key policy’s Principal statement, leaving the secret encrypted but inaccessible. Another common error occurs when the key policy includes the correct account but restricts actions to only local operations like kms:Decrypt without including essential Secrets Manager actions such as kms:DescribeKey and kms:GenerateDataKey.
Missing Resource-Based Permissions in Secrets Manager
Secret resource policies often get overlooked during cross-account setup, creating invisible barriers that prevent access even when KMS permissions work correctly. Account administrators frequently configure IAM roles with proper Secrets Manager permissions but forget to attach a resource-based policy to the secret itself. This policy must explicitly allow the cross-account principal to perform actions like secretsmanager:GetSecretValue and secretsmanager:DescribeSecret. Without these resource-level permissions, requests fail with access denied errors despite having correct IAM policies. The secret remains locked to its origin account, breaking automated processes and application integrations that depend on cross-account secret sharing.
IAM Role Trust Relationship Gaps
Trust relationships between IAM roles create another layer where cross-account Secrets Manager access breaks down. The role in the target account needs explicit trust policies allowing the source account or specific principals to assume it. Developers often create roles with broad permissions for Secrets Manager and KMS operations but neglect to configure the trust relationship properly. This results in authentication failures before permission checks even begin. Additionally, condition statements in trust policies can inadvertently block legitimate cross-account access when they’re too restrictive about source IP addresses, time ranges, or MFA requirements. These gaps prevent the assume role operation that enables cross-account access to encrypted secrets.
Implementing Secure Cross-Account KMS Key Policies for Secrets Manager
Essential Policy Statements for Cross-Account Secret Access
Creating effective cross-account KMS key policies for AWS Secrets Manager requires specific policy statements that grant the necessary permissions while maintaining security. The key policy must include statements allowing the Secrets Manager service to use the KMS key for encryption and decryption operations. Start with the essential kms:Decrypt, kms:DescribeKey, and kms:GenerateDataKey permissions for the target account’s principals.
{
"Sid": "AllowSecretsManagerCrossAccount",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::TARGET-ACCOUNT:root"
},
"Action": [
"kms:Decrypt",
"kms:DescribeKey",
"kms:GenerateDataKey"
],
"Resource": "*"
}
The policy must also include permissions for the Secrets Manager service itself to perform automatic rotation operations. Add a separate statement that explicitly allows the secretsmanager.amazonaws.com service principal to access the key. This prevents rotation failures that commonly occur in cross-account scenarios.
Principal-Based vs Resource-Based Permission Models
Understanding the difference between principal-based and resource-based permissions is critical for cross-account Secrets Manager KMS integration. Principal-based policies attach to IAM users, roles, or groups and define what resources they can access. Resource-based policies attach directly to KMS keys and specify which principals can perform actions on those keys.
For cross-account secret sharing, resource-based KMS key policies provide more control and flexibility. The key policy acts as the primary authorization mechanism, while IAM policies in the target account provide additional restrictions. This dual-control model ensures that both accounts must explicitly grant access.
| Permission Model | Attachment Point | Cross-Account Benefits | Limitations |
|---|---|---|---|
| Principal-Based | IAM entities | Granular user control | Requires trust relationships |
| Resource-Based | KMS key | Direct cross-account access | Broader permission scope |
Principal-based policies work best when you need fine-grained control over which specific users or roles can access secrets. Resource-based policies excel when you want to grant blanket permissions to an entire account or service.
Condition Keys for Enhanced Security Controls
AWS KMS condition keys provide powerful mechanisms to restrict cross-account secret access based on specific criteria. The kms:ViaService condition key ensures that KMS operations only occur through the Secrets Manager service, preventing direct key usage for other purposes.
{
"Condition": {
"StringEquals": {
"kms:ViaService": [
"secretsmanager.us-east-1.amazonaws.com",
"secretsmanager.us-west-2.amazonaws.com"
]
}
}
}
Time-based conditions using DateGreaterThan and DateLessThan keys allow temporary access grants for cross-account scenarios. IP address restrictions through aws:SourceIp conditions add network-level security controls. The aws:RequestedRegion condition key restricts secret access to specific AWS regions, preventing unauthorized cross-region operations.
Multi-factor authentication requirements using aws:MultiFactorAuthPresent condition keys add an extra security layer for sensitive cross-account secret access. These conditions work particularly well for administrative operations or high-privilege secret access scenarios.
Testing and Validating Cross-Account KMS Permissions
Proper testing of cross-account KMS permissions requires a systematic approach using AWS CLI commands and API calls. Start by testing basic key access using the aws kms describe-key command from the target account. This verifies that the key policy allows cross-account access at the most fundamental level.
aws kms describe-key --key-id arn:aws:kms:region:source-account:key/key-id --region region
Test secret retrieval using the aws secretsmanager get-secret-value command from the target account. Pay attention to error messages that indicate whether the issue stems from KMS permissions, Secrets Manager permissions, or resource-based policy conflicts.
AWS CloudTrail logs provide detailed information about KMS API calls and permission denials. Look for Decrypt and GenerateDataKey events with error codes like AccessDenied or UnauthorizedOperation. These logs help identify exactly which condition keys or policy statements are blocking access.
Use the AWS Policy Simulator to test KMS key policies before implementing them in production. The simulator helps identify potential permission conflicts and validates that your cross-account KMS key policy grants the necessary permissions without over-provisioning access rights.
Best Practices for Managing Secrets Manager KMS Keys Across AWS Accounts
Centralized vs Decentralized Key Management Strategies
Organizations face a critical decision when managing AWS KMS keys for Secrets Manager across multiple accounts. Centralized approaches place all keys in a dedicated security account, enabling unified governance and simplified cross-account KMS key policy management. This strategy reduces administrative overhead and ensures consistent security standards across your AWS environment. However, centralized models can create single points of failure and may introduce latency for applications accessing secrets from distant accounts.
Decentralized strategies distribute keys closer to applications, reducing cross-account dependencies and improving performance. Each account maintains its own KMS keys, providing isolation and reducing blast radius during security incidents. The trade-off comes in increased complexity for Secrets Manager KMS integration and policy coordination across accounts.
| Strategy | Pros | Cons | Best For |
|---|---|---|---|
| Centralized | Unified governance, simplified auditing | Single point of failure, potential latency | Organizations prioritizing compliance |
| Decentralized | Better isolation, reduced dependencies | Complex coordination, inconsistent policies | High-performance applications |
| Hybrid | Balanced approach, flexibility | Increased complexity | Large enterprises with diverse needs |
Automated Key Rotation and Cross-Account Considerations
Automated key rotation becomes complex when dealing with cross-account secret sharing AWS scenarios. The default rotation schedule for AWS KMS keys affects all Secrets Manager resources using those keys, potentially disrupting applications across multiple accounts simultaneously. Cross-account KMS permissions must account for the rotation process, ensuring service roles have access to both old and new key versions during the transition period.
Implement staggered rotation schedules to minimize impact across accounts. Design your cross-account KMS key policy to include time-based conditions that gradually transition permissions from old to new keys. This approach prevents service interruptions while maintaining security standards.
{
"Effect": "Allow",
"Principal": {"AWS": "arn:aws:iam::ACCOUNT:role/SecretsManagerRole"},
"Action": ["kms:Decrypt", "kms:DescribeKey"],
"Resource": "*",
"Condition": {
"StringEquals": {
"kms:ViaService": "secretsmanager.region.amazonaws.com"
}
}
}
Monitoring and Auditing Cross-Account Secret Access
CloudTrail logging becomes your primary tool for tracking Secrets Manager cross-account access patterns. Enable detailed logging for all KMS operations, focusing on decrypt and describe actions that indicate secret retrieval attempts. Cross-account access generates distinct log patterns that help identify unauthorized attempts or configuration issues.
Set up CloudWatch alarms for unusual cross-account activity patterns. Monitor for failed decrypt operations, which often indicate AWS KMS key restrictions or policy misconfigurations. Create custom metrics that track successful cross-account secret retrievals by source account, helping you understand usage patterns and identify potential security risks.
Key monitoring metrics include:
- Failed cross-account decrypt attempts
- Unusual access patterns by time or source account
- KMS key policy violations
- Secrets Manager KMS configuration errors
Cost Optimization Techniques for Multi-Account KMS Usage
Cross-account KMS operations incur charges for each API call, making cost optimization crucial for large-scale deployments. Implement caching strategies in your applications to reduce KMS decrypt operations. AWS SDKs provide built-in caching mechanisms that can significantly reduce your KMS API usage without compromising security.
Consider using KMS key aliases consistently across accounts to simplify management and reduce the likelihood of AWS Secrets Manager security best practices violations. Regional considerations also impact costs – keeping secrets and KMS keys in the same region eliminates cross-region charges while improving performance.
Cost optimization strategies:
- Implement client-side caching for secret values
- Use regional KMS keys to avoid cross-region charges
- Batch secret retrievals when possible
- Regular audit of unused secrets and keys
- Implement lifecycle policies for secret rotation
Disaster Recovery Planning for Cross-Account Secrets
Cross-account secret dependencies create complex disaster recovery scenarios. Your DR plan must account for scenarios where the KMS key account becomes unavailable, potentially breaking secret access across all dependent accounts. Implement cross-region replication for critical secrets, ensuring each replica uses region-specific KMS keys to maintain availability during regional outages.
Establish backup access patterns that don’t rely on cross-account KMS dependencies. Create emergency break-glass procedures that allow temporary secret access through alternative mechanisms. Test your disaster recovery procedures regularly, including scenarios where cross-account KMS permissions fail.
DR planning checklist:
- Cross-region secret replication with local KMS keys
- Break-glass access procedures
- Regular DR testing including cross-account scenarios
- Documentation of KMS key dependencies
- Automated failover mechanisms for critical secrets
Troubleshooting and Resolving Cross-Account Access Issues
Diagnosing Permission Denied Errors
Permission denied errors in cross-account Secrets Manager setups typically stem from misaligned KMS key policies or IAM permissions. Start by checking the KMS key policy in the secret owner’s account to ensure it explicitly allows the consuming account’s roles to decrypt and describe the key. Verify that the consuming account’s IAM policies include the necessary kms:Decrypt, kms:DescribeKey, and secretsmanager:GetSecretValue permissions. Common culprits include hardcoded principal ARNs that don’t account for role assumption chains, missing condition statements that restrict access too broadly, or incorrect resource ARNs in IAM policies.
CloudTrail Analysis for Cross-Account Secret Access
CloudTrail provides detailed audit trails for diagnosing cross-account KMS key restrictions and Secrets Manager access failures. Filter CloudTrail events for secretsmanager:GetSecretValue, kms:Decrypt, and kms:GenerateDataKey API calls from the consuming account. Look for AccessDenied or UnauthorizedOperation error codes that pinpoint exact permission gaps. Pay attention to the sourceIPAddress and userIdentity fields to trace which roles or users are attempting access. Cross-reference failed KMS operations with the corresponding secret retrieval attempts to identify whether the issue lies in KMS key policy configuration or IAM role permissions in the consuming account.
Step-by-Step Resolution Workflows
When facing cross-account Secrets Manager KMS integration errors, follow this systematic approach. First, verify the KMS key policy in the secret owner’s account explicitly grants the consuming account’s root ARN permissions for kms:Decrypt and kms:DescribeKey. Next, check that the consuming account’s IAM role has both KMS permissions and secretsmanager:GetSecretValue for the specific secret ARN. Test access using AWS CLI with explicit credentials from the consuming account to isolate permission issues. If errors persist, temporarily add broader permissions to identify the specific missing action, then narrow down to the minimal required permissions. Document the working configuration and implement infrastructure as code to prevent configuration drift across environments.
Cross-account secrets management doesn’t have to be a headache if you get the fundamentals right. The key is understanding how Secrets Manager and KMS work together, then setting up your cross-account policies with the right permissions from the start. When you nail down the resource-based policies and avoid those common configuration mistakes we covered, you’ll save yourself hours of troubleshooting down the road.
Take the time to implement proper KMS key policies and follow those best practices for cross-account access. Your future self will thank you when secrets flow seamlessly between accounts without any late-night debugging sessions. Start with a clear understanding of your cross-account requirements, test your configurations in a safe environment, and always document your KMS key policies so your team can maintain them easily.
