[Terraform] AWS Elastic Beanstalk 'Unknown' 상태, terraform으로 aws elastic beanstalk를 관리할 때 유의할 점

AWS IAM

AWS Elastic Beanstalk(이하 EB)에는 서비스를 관리하기 위한 역할(Role)을 최초 애플리케이션 생성시 만들게 된다.

Web Console로 EB를 생성하면 아래와 같은 화면을 볼 수 있다. 이때 '새 서비스 역할 생성 및 사용'을 선택하게 되면 자동으로 EB에 액세스하고 관리하는데 필요한 역할이 부여된 IAM을 생성하게 된다.

 

aws에서 EB 생성시 서비스 액세스를 위한 role을 요구한다.

 

그렇다면 테라폼을 사용해서 EB를 생성할 때 iam을 어떻게 설정해줄 수 있을까.

 

문제 상황

IAM설정을 해주지 않고 terraform apply를 하면, 권한이 없어 EB가 생성되다말고 롤백을 하게 되는 모습을 볼 수 있다. 

아래 사진과 같이 상태가 'Unknown'으로 바뀌고 health check를 못하는 상태가 된다.

 

이벤트 부분을 보면 health check가 안된 EB는 정상적으로 instance가 배포되지 않은 것으로 인식한다.

 

/logs/daemon.log 에서 디테일한 오류를 확인해볼 수 있다.

# Logfile created on 2023-11-10 07:47:23 +0000 by logger.rb/v1.5.0


W, [2023-11-10T07:57:27.665909 #2144]  WARN -- : sending message(s) failed (attempt 10): (Aws::Healthd::Errors::AccessDeniedException) User: arn:aws:sts::158161732663:assumed-role/aws-elasticbeanstalk-ec2-role/i-0feca5ff1006ea9fa is not authorized to perform: elasticbeanstalk:PutInstanceStatistics on resource: arn:aws:elasticbeanstalk:ap-northeast-2:158161732663:application/ddakzip-eb-application. Backing off by 200 seconds in addition to delay interval
W, [2023-11-10T08:00:01.307654 #2144]  WARN -- : log file "/var/log/nginx/healthd/application.log.2023-11-10-08" does not exist
W, [2023-11-10T08:00:57.668536 #2144]  WARN -- : sending message(s) failed (attempt 11): (Aws::Healthd::Errors::AccessDeniedException) User: arn:aws:sts::158161732663:assumed-role/aws-elasticbeanstalk-ec2-role/i-0feca5ff1006ea9fa is not authorized to perform: elasticbeanstalk:PutInstanceStatistics on resource: arn:aws:elasticbeanstalk:ap-northeast-2:158161732663:application/ddakzip-eb-application. Backing off by 200 seconds in addition to delay interval
F, [2023-11-10T08:02:25.229480 #2144] FATAL -- : /opt/elasticbeanstalk/lib/ruby/lib/ruby/gems/3.1.0/gems/puma-6.3.0/lib/puma/launcher.rb:434:in `block in setup_signals': SIGTERM (SignalException)
	from /opt/elasticbeanstalk/lib/ruby/lib/ruby/gems/3.1.0/gems/puma-6.3.0/lib/puma/single.rb:63:in `join'
	from /opt/elasticbeanstalk/lib/ruby/lib/ruby/gems/3.1.0/gems/puma-6.3.0/lib/puma/single.rb:63:in `run'
	from /opt/elasticbeanstalk/lib/ruby/lib/ruby/gems/3.1.0/gems/puma-6.3.0/lib/puma/launcher.rb:194:in `run'
	from /opt/elasticbeanstalk/lib/ruby/lib/ruby/gems/3.1.0/gems/puma-6.3.0/lib/puma/cli.rb:75:in `run'
	from /opt/elasticbeanstalk/lib/ruby/lib/ruby/gems/3.1.0/gems/healthd-1.0.6/bin/healthd:112:in `block in <top (required)>'
	from /opt/elasticbeanstalk/lib/ruby/lib/ruby/gems/3.1.0/gems/healthd-1.0.6/bin/healthd:19:in `chdir'
	from /opt/elasticbeanstalk/lib/ruby/lib/ruby/gems/3.1.0/gems/healthd-1.0.6/bin/healthd:19:in `<top (required)>'
	from /opt/elasticbeanstalk/lib/ruby/bin/healthd:25:in `load'
	from /opt/elasticbeanstalk/lib/ruby/bin/healthd:25:in `<main>'
A, [2023-11-10T08:02:27.869107 #3601]   ANY -- : healthd daemon 1.0.6 initialized
W, [2023-11-10T08:02:38.021989 #3601]  WARN -- : sending message(s) failed (attempt 1): (Aws::Healthd::Errors::AccessDeniedException) User: arn:aws:sts::158161732663:assumed-role/aws-elasticbeanstalk-ec2-role/i-0feca5ff1006ea9fa is not authorized to perform: elasticbeanstalk:PutInstanceStatistics on resource: arn:aws:elasticbeanstalk:ap-northeast-2:158161732663:application/ddakzip-eb-application. Backing off by 2 seconds in addition to delay interval
W, [2023-11-10T08:02:49.911398 #3601]  WARN -- : sending message(s) failed (attempt 2): (Aws::Healthd::Errors::AccessDeniedException) User: arn:aws:sts::158161732663:assumed-role/aws-elasticbeanstalk-ec2-role/i-0feca5ff1006ea9fa is not authorized to perform: elasticbeanstalk:PutInstanceStatistics on resource: arn:aws:elasticbeanstalk:ap-northeast-2:158161732663:application/ddakzip-eb-application. Backing off by 3 seconds in addition to delay interval
W, [2023-11-10T08:03:02.885369 #3601]  WARN -- : sending message(s) failed (attempt 3): (Aws::Healthd::Errors::AccessDeniedException) User: arn:aws:sts::158161732663:assumed-role/aws-elasticbeanstalk-ec2-role/i-0feca5ff1006ea9fa is not authorized to perform: elasticbeanstalk:PutInstanceStatistics on resource: arn:aws:elasticbeanstalk:ap-northeast-2:158161732663:application/ddakzip-eb-application. Backing off by 4 seconds in addition to delay interval

 

해결 방법

해결방법은 간단하다. 테라폼으로 EB를 생성할 때 부여하는 IAM을 직접 정의해주고, policy를 부여하는 방법으로 해결할 수 있다. 

먼저 EB를 직접적으로 생성하는 코드이다. EB에 세부적인 설정은 setting { }을 통해서 추가할 수 있다

# /main.tf
# Create elastic beanstalk application
resource "aws_elastic_beanstalk_application" "eb_app" {
  name = var.eb_application_name
}

# Create elastic beanstalk Environment
resource "aws_elastic_beanstalk_environment" "eb_env" {
  name                = var.eb_environment_name
  application         = aws_elastic_beanstalk_application.eb_app.name
  solution_stack_name = var.solution_stack_name
  tier                = "WebServer" 
  
  # IAM을 추가하는 setting
  setting {
    namespace = "aws:autoscaling:launchconfiguration"
    name      = "IamInstanceProfile"
    value     = aws_iam_instance_profile.eb_instance_profile.name
  }
  
  ...
  
}

 

 

이때 IAM은 다음과 같이 설정할 수 있다.

테라폼에서는 <<EOF EOF를 이용해서 여러줄의 단락을 처리하는 heredoc을 지원한다. 이를 이용하여 json format의 policy를 추가해줄 수 있다.

sts, s3 , ec2 , sqs , ecs , ecr , logs , elasticbeanstalk 권한들을 필요로한다. 실질적으로 오류를 해결하는데 필요한 권한은 

sts:AssumeRole이지만, 기존의 EB 생성시 자동으로 부여되는 IAM을 새로 만드는 것이기에 EB에 필요한 모든 권한을 부여해줘야한다.

 

resource "aws_iam_role" "eb_instance_profile" {
  name = "aws-elasticbeanstalk-ec2-role"
  assume_role_policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "ec2.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
EOF
}

resource "aws_iam_instance_profile" "eb_instance_profile" {
  name = "aws-elasticbeanstalk-ec2-role"
  role = aws_iam_role.eb_instance_profile.name
}

# AWSElasticBeanstalkWebTier 정책 추가
resource "aws_iam_role_policy" "web_tier_policy" {
  name        = "AWSElasticBeanstalkWebTierPolicy"
  role        = aws_iam_role.eb_instance_profile.name
  policy      = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "s3:*",
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": "ec2:*",
      "Resource": "*"
    }
  ]
}
EOF
}

# AWSElasticBeanstalkWorkerTier 정책 추가
resource "aws_iam_role_policy" "worker_tier_policy" {
  name        = "AWSElasticBeanstalkWorkerTierPolicy"
  role        = aws_iam_role.eb_instance_profile.name
  policy      = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "sqs:*",
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": "ec2:*",
      "Resource": "*"
    }
  ]
}
EOF
}

# AWSElasticBeanstalkMulticontainerDocker 정책 추가
resource "aws_iam_role_policy" "multicontainer_docker_policy" {
  name        = "AWSElasticBeanstalkMulticontainerDockerPolicy"
  role        = aws_iam_role.eb_instance_profile.name
  policy      = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "ecs:*",
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": "ecr:*",
      "Resource": "*"
    }
  ]
}
EOF
}

# AWSElasticBeanstalkLogStream을 위한 정책 추가
resource "aws_iam_role_policy" "attach_policy" {
  name   = "ElasticBeanstalkLoggingPolicy"
  role   = aws_iam_role.eb_instance_profile.name
  policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:PutLogEvents",
        "logs:CreateLogStream"
      ],
      "Resource": "*"
    }
  ]
}
EOF
}

# AWSElasticBeanstalkStatisticPolicy 정책 추가
resource "aws_iam_role_policy" "eb_statistic_policy" {
  name        = "AWSElasticBeanstalkStatisticPolicy"
  role        = aws_iam_role.eb_instance_profile.name
  policy      = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "elasticbeanstalk:*",
      "Resource": "*"
    }
  ]
}
EOF
}

 

참조문서

- https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-autoscaling-launchconfiguration.html