S3 bucket resource not being found using data

March 9, 2023

I have a main.tf that looks like this:

resource "aws_s3_bucket" "s3_notifications_bucket" {
  bucket        = local.s3_bucket_name
  force_destroy = true
}

module "s3_notification" {
  source              = "..."
  s3_bucket_name      = local.s3_bucket_name
  function_name       = module.lambda.lambda_function_name
  lambda_function_arn = module.lambda.lambda_arn
}

Module s3_notification uses a reference for that bucket. The reason for that is because when I am running locally, I will be creating this bucket. If it’s deployed code, the bucket will already exist and I don’t have to create it.

data "aws_s3_bucket" "s3_notifications_bucket" {
  bucket = var.s3_bucket_name
}

resource "aws_s3_bucket_notification" "aws_lambda_trigger" {
  bucket = data.aws_s3_bucket.s3_notifications_bucket.id

  lambda_function {
    lambda_function_arn = var.lambda_function_arn
    events              = ["s3:ObjectCreated:*"]
  }
}
    
resource "aws_lambda_permission" "lambda_permission" {
  statement_id  = "AllowS3Invoke"
  action        = "lambda:InvokeFunction"
  function_name = var.function_name
  principal     = "s3.amazonaws.com"
  source_arn    = "arn:aws:s3:::${data.aws_s3_bucket.s3_notifications_bucket.id}"
}

However, when I run terraform apply, I get the following error:

Error: Failed getting S3 bucket (my-bucket): NotFound: Not Found
... in data "aws_s3_bucket" "s3_notifications_bucket":

How do I make that bucket to be created before data reference gets processed?

>Solution :

There is actually no need to rely on using a data source in this case, there needs to be one slight change:

module "s3_notification" {
  source              = "..."
  s3_bucket_name      = aws_s3_bucket.s3_notifications_bucket.id
  function_name       = module.lambda.lambda_function_name
  lambda_function_arn = module.lambda.lambda_arn
}

This uses an implicit resource reference, which means that the S3 bucket will be created first, and only then the bucket name attribute will be passed to the s3_bucket_name in the module call.

Additionally, one change needs to happen in the module code as well:

resource "aws_s3_bucket_notification" "aws_lambda_trigger" {
  bucket = var.s3_bucket_name
  lambda_function {
    lambda_function_arn = var.lambda_function_arn
    events              = ["s3:ObjectCreated:*"]
  }
}

and the data source can be dropped for local testing.