The Terraform State Trap

Hey there,

You’re in the first technical round for a Senior DevOps role. The engineering manager leans back and asks:

“You’ve just joined a team of four engineers. You’re all deploying infrastructure to the same AWS environment using OpenTofu or Terraform. How do you prevent two engineers from applying conflicting changes at the exact same time?”

If you’ve recently studied for your AWS or Terraform Associate certifications, your reflex is likely the textbook answer: “I’d use a remote backend, like an Amazon S3 bucket, and attach a DynamoDB table for state locking.”

It’s not a wrong answer. But it won’t make you stand out.

The DynamoDB approach is the “classic” way. But it adds unnecessary infrastructure complexity and IAM permission overhead just to maintain a simple lock.

Here is the answer that actually gets you the job.

To truly stand out, you tell the hiring manager that you don’t need DynamoDB anymore.

You explain that you would use S3 Native State Locking. Recent versions of OpenTofu and Terraform can leverage S3 conditional writes to natively lock the state file directly inside the bucket.

If you configure your backend with use_lockfile = true, the tool uses an If-None-Match header to atomically create a lock. If another CI/CD pipeline tries to run simultaneously, the conditional write fails, and the execution is safely halted.

You get the exact same production-grade concurrency protection, but with fewer AWS resources to manage and a much simpler IAM policy.

Here is what that modern, simplified backend block looks like:

terraform {
  backend "s3" {
    bucket         = "cloudqubes-tf-state-production"
    key            = "global/s3/terraform.tfstate"
    region         = "us-east-1"
    
    # The Edge: Native S3 locking using conditional writes (No DynamoDB required!)
    use_lockfile   = true 
  }
}

Certifications teach you the legacy ways to provision infrastructure. Real engineering teaches you how to streamline it.

Bonus tip

There’s a flow in this S3 bucket configuration. Can you find it?

terraform {
  backend "s3" {
    bucket         = "cloudqubes-tf-state-production"
    key            = "global/s3/terraform.tfstate"
    region         = "us-east-1"
    
    # The Edge: Native S3 locking using conditional writes (No DynamoDB required!)
    use_lockfile   = true 
  }
}

Don’t just scroll down. Think.

While this S3 bucket ensure state locking, it does not encrypt the Terraform status file stored in the bucket.

Why should I encrypt the status file?

Terraform and OpenTofu state files are ultimately just giant JSON files. By default, if your infrastructure configuration handles sensitive data, that data is written directly into the state file in plain text.

What kind of secrets end up in the state file?

If your infrastructure touches a secret to deploy a resource, it is likely recorded in the state. Common examples include:

• Database Credentials: Master passwords for RDS instances, ElastiCache clusters, or self-hosted databases.
• API Keys & Tokens: Provider access tokens (like a GitHub Personal Access Token, Cloudflare API key, or Datadog token) used during provisioning.
• Private Keys: SSH private keys generated to access EC2 instances, or TLS/SSL certificate private keys.
• Identity Secrets: Auto-generated console passwords or AWS Access Keys for new IAM users.
• Application Secrets: Any sensitive environment variables passed into Lambda functions, ECS task definitions, or Kubernetes Secrets.

Why do Terraform and OpenTofu do this?

To function correctly, these tools have to map the HCL code you wrote to the actual resources running in the cloud. When a provider generates a password or passes an API key to a server, the tool has to remember that exact value so it can check for “drift” the next time you run a plan or apply. Because the state file is essentially a JSON map of those properties, the secrets are stored exactly as they are evaluated.

The Verdict on Encryption

If you do not encrypt your remote backend, anyone with basic s3:GetObject permissions on your bucket can download the terraform.tfstate file, open it in a text editor, and instantly read your database passwords and API keys.

Including encrypt = true (exactly as you did in your pdf-lambda configuration) is non-negotiable. It ensures that the state file is encrypted at rest using server-side encryption (SSE-S3 or SSE-KMS). This means that even if a malicious actor or an unauthorized junior engineer manages to access the S3 bucket, they cannot read the state file without also having the explicit IAM permissions to decrypt it.

So, here’s how the correct Terraform config should look like:

terraform {
  backend "s3" {
    bucket         = "cloudqubes-tf-state-production"
    key            = "global/s3/terraform.tfstate"
    region         = "us-east-1"
    encrypt        = true  # Enable bucket encryotion                    
    
    # The Edge: Native S3 locking using conditional writes (No DynamoDB required!)
    use_lockfile   = true 
  }
}

Keep building,

Indika Founder, CloudQubes