Break-Glass Emergency Access Runbook

Overview

Break-glass accounts are dedicated super-admin identities that are never used for day-to-day operations. They exist solely for emergency recovery scenarios when primary admin access is unavailable.

Use these accounts only when:

• Primary admin account is locked out of MFA
• Critical misconfiguration needs immediate reversal
• Identity provider outage cascades across your stack
• Primary admin account is compromised

Account Inventory

Platform	Account	MFA Method	Credential Location
Okta	breakglass@tractorbeam.ai	YubiKey 5C NFC	Lockbox
Google Workspace	breakglass@tractorbeam.ai	YubiKey 5C NFC	Lockbox
Apple Business Manager	breakglass@tractorbeam.ai	SMS (prepaid SIM)	Lockbox
AWS (Management Account)	break-glass-emergency	Hardware MFA	Lockbox
GitHub Enterprise	SSO recovery codes	N/A	Lockbox

Physical Credentials Location

Lockbox Location: Wade's house (off-site from office) Lockbox Type: Master Lock 5900D (combination lock) Combination: Known to Wade and Charlie

Lockbox Contents:

• YubiKey 5C NFC (USB-C + NFC tap)
• Maestro U202AA phone with Red Pocket SIM (AT&T network)
• GitHub Enterprise SSO recovery codes
• Printed credentials sheet (passwords, recovery codes)

Emergency Access Procedure

1. Confirm This Is an Emergency

Before using break-glass credentials:

• Verify primary admin access is truly unavailable
• Attempt normal recovery procedures first (password reset, backup MFA)
• Document why break-glass access is required

2. Retrieve Physical Credentials

1. Go to Wade's house to access the lockbox
2. Open lockbox with combination
3. Retrieve the appropriate credentials:
   • YubiKey for Okta/Google Workspace
   • Prepaid phone for Apple Business Manager

3. Platform-Specific Access

Okta

1. Navigate to https://tractorbeam.okta.com
2. Sign in with breakglass@tractorbeam.ai
3. Use password from printed credentials sheet
4. When prompted for MFA, insert YubiKey and touch it
5. Complete the required administrative action
6. Sign out immediately when done

Google Workspace

1. Navigate to https://admin.google.com
2. Sign in with breakglass@tractorbeam.ai
3. Use password from printed credentials sheet
4. When prompted for MFA, insert YubiKey and touch it
5. Complete the required administrative action
6. Sign out immediately when done

Apple Business Manager

1. Navigate to https://business.apple.com
2. Sign in with breakglass@tractorbeam.ai (Managed Apple ID)
3. Use password from printed credentials sheet
4. Power on the prepaid phone
5. Enter the SMS verification code when received
6. Complete the required administrative action
7. Sign out immediately when done
8. Power off the prepaid phone and return to lockbox

AWS (Management Account)

Use only when: Okta SSO and IAM Identity Center are both inaccessible. This is the last resort for AWS Organization administration.

1. Navigate to https://console.aws.amazon.com
2. Sign in with IAM user break-glass-emergency
3. Use password from printed credentials sheet
4. When prompted for MFA, use the hardware MFA device from lockbox
5. Complete the required administrative action
6. Sign out immediately when done

Note: Any login to this account triggers immediate email alerts to the security team via EventBridge → SNS. This is logged in CloudTrail and requires 24-hour documentation per SOC 2 requirements.

GitHub Enterprise

Use only when: Okta SSO is unavailable and you need to access the GitHub organization.

1. Navigate to https://github.com/orgs/tractorbeamai/sso
2. Click "Use a recovery code"
3. Enter one of the recovery codes from the lockbox
4. Cross out the used code on the printed sheet
5. Complete the required administrative action
6. Sign out immediately when done

Note: Recovery codes are single-use. After using one, generate new codes when SSO is restored and update the lockbox.

Alerting

Any login to a break-glass account should trigger immediate alerts to both Wade and Charlie.

Okta

Break-glass account logins generate system log events. Configure alerts in Okta Admin Console:

• Event type: user.session.start
• Filter: actor.alternateId eq "breakglass@tractorbeam.ai"
• Notify: Wade and Charlie via email/Slack

Google Workspace

Configure in Admin Console > Security > Alert Center:

• Create custom alert for super admin sign-ins
• Filter to break-glass account
• Notify: Wade and Charlie

Apple Business Manager

ABM has limited alerting capabilities. Rely on quarterly audit of sign-in history.

AWS (Management Account)

Alerting is configured via Terraform (aws/accounts/management/break-glass.tf):

• EventBridge rules detect console logins and API calls by break-glass-emergency user
• Alerts sent to SNS topic security-alerts → email to security@tractorbeam.ai
• All activity logged in CloudTrail (immutable, stored in Log Archive account)

Account Configuration Details

Platform	Account Type	Role	MFA	Notes
Okta	Local (not federated)	Super Administrator	YubiKey 5C NFC	Excluded from conditional access policies
Google Workspace	Native Google (bypasses Okta SAML)	Super Admin	YubiKey 5C NFC	Recovery phone/email not on tractorbeam.ai domain
Apple Business Manager	Managed Apple ID	Administrator	SMS (prepaid SIM)	Hardware keys not supported by Apple
AWS (Management)	IAM User (bypasses SSO)	AdministratorAccess	Hardware MFA	EventBridge alerts on any login/API call
GitHub Enterprise	SSO recovery codes	Enterprise Owner	N/A	Use when Okta SSO is unavailable

Operational Procedures

Quarterly: Access Verification Drill

• [ ] Open lockbox, verify all items present
• [ ] Power on prepaid phone, confirm signal and can receive SMS
• [ ] Insert YubiKey, confirm laptop recognizes it
• [ ] Log into Okta break-glass account, verify MFA, log out
• [ ] Log into Google Workspace break-glass account, verify MFA, log out
• [ ] Log into ABM break-glass account, verify SMS MFA, log out
• [ ] Log into AWS break-glass account, verify hardware MFA, confirm SNS alert received, log out
• [ ] Verify GitHub recovery codes sheet has unused codes remaining
• [ ] Power off phone, return all items to lockbox
• [ ] Note any issues and remediate

Annual: Credential Rotation

1. Generate new passwords for all break-glass accounts
2. Update passwords in each platform
3. Print new credentials sheet
4. Destroy old credentials sheet (shred or burn)
5. Update lockbox contents

Annual: Prepaid SIM Renewal

Renew the Red Pocket plan before expiration. Scheduled ticket handles reminder.

• Account: https://www.redpocket.com/login
• Plan: $5/month (AT&T/GSMA)
• Grace period: ~60 days after expiration before number is recycled

Adding a New Break-Glass Account

When onboarding a new SaaS platform that needs emergency access:

1. Create a dedicated break-glass account (not your personal admin)
2. Use naming convention: breakglass@tractorbeam.ai or platform equivalent
3. Assign highest admin role available
4. Enroll YubiKey as MFA (or SMS via prepaid phone if hardware keys unsupported)
5. Exclude from conditional access / SSO policies that could block emergency access
6. Add to Account Inventory table (top of this doc)
7. Add to Account Configuration Details table
8. Add login steps to "Platform-Specific Access" section
9. Configure alerting for logins (see Alerting section)
10. Print updated credentials sheet, replace in lockbox

Removing a Break-Glass Account

When decommissioning a platform:

1. Delete or disable the break-glass account in the platform
2. Remove from Account Inventory and Configuration tables
3. Remove platform-specific access steps
4. Print updated credentials sheet, replace in lockbox

After Any Break-Glass Usage

1. Document: date, who, why, what actions taken
2. Investigate root cause of primary access failure
3. Rotate credentials if account may have been observed
4. File incident report if security-related