Mount EFS on ECS Fargate with Terraform - AWS
On April 8 of 2020 AWS launched the feature where now you are able to mount an Elastic File System volume on your AWS Fargate tasks. So far, the only way to mount an EBS or EFS volumes was with an ECS cluster with EC2 instances. However, running tasks on EC2 stills gives you an operational burden. This is where Fargate comes in, where you can launch tasks in a serverless way, but until April 8, we couldn't mount a volume on Fargate tasks.
A couple of weeks ago, on our app of facial recognition, our engineers were studying a way to improve the speed of the analysis and cache this data for fast access. I suggested them to use the Redis that we already have on our infrastructure, but just Redis wasn't enough, so they needed a way to store this data closer to the microservice, and I suggested then to use EFS, already thinking that I would need to move the microservice from Fargate to EC2, and this was the time that I started googling and found out this announcement from AWS with the news of the new support of EFS with Fargate.
Now that was a win, but I needed to figure out how to do that with Terraform. On a first glance, the only thing that I needed was to specify on the task definition the version of Fargate to 1.4.0, but since I never worked with EFS before, and the Terraform docs forget to mention that you need to implement others resources to work with EFS properly, I suffered a little bit to figure out what I needed to get done. And this is why I am here sharing this, to help you with this task.. =D
First things first. To start you need to build an EFS resource, for this you can use the following terraform code:
# Create the File System
resource "aws_efs_file_system" "this" {
creation_token = "${var.app_name}-efs"
tags = {
Name = "${var.app_name}-efs"
}
}
# Create the access point with the given user permissions
resource "aws_efs_access_point" "this" {
file_system_id = aws_efs_file_system.this.id
posix_user {
gid = 1000
uid = 1000
}
root_directory {
path = "/mnt/efs"
creation_info {
owner_gid = 1000
owner_uid = 1000
permissions = 755
}
}
tags = {
Name = "${var.app_name}-${var.container_name}"
}
}
# Create the mount targets on your private subnets
resource "aws_efs_mount_target" "this" {
count = length(data.aws_subnet_ids.private.ids)
file_system_id = aws_efs_file_system.this.id
subnet_id = tolist(data.aws_subnet_ids.private.ids)[count.index]
security_groups = [aws_security_group.this.id]
}
And then define your Task Definition:
resource "aws_ecs_task_definition" "this" {
family = "${var.app_name}-${var.container_name}"
container_definitions = data.template_file.this.rendered
requires_compatibilities = ["FARGATE"]
network_mode = "awsvpc"
cpu = var.desired_task_cpu
memory = var.desired_task_memory
execution_role_arn = data.aws_iam_role.selected.arn
task_role_arn = data.aws_iam_role.selected.arn
volume {
name = aws_efs_file_system.this.creation_token
efs_volume_configuration {
file_system_id = aws_efs_file_system.this.id
root_directory = "/mnt/efs"
}
}
tags = {
Product = var.app_name
}
}
Don't forget to setup the volume environment and the mount points on your task definition json file:
"mountPoints": [{ "containerPath" : "/mnt/efs",
"sourceVolume" : "volume-name" ,
"readOnly" : false }]
Now, you just need to create the ECS Service, and add the platform version of 1.4.0, this config is needed until the LATEST version of Fargate on ECS update from 1.3.0 to 1.4.0.
platform_version = "1.4.0"
Notes
Always use the private subnet for EFS
The tasks that need to mount the EFS need to have the NFS (2049) port on the inbound of the security group
The EFS needs to have the security group of the tasks/service that needs to access it, note line 32 of the first snippet
You will still need to update your task definition on AWS Console to define the Access Point ID of EFS, because this config is not available on Terraform: Task definitions -> Create New Revision -> Edit Volume:
That's all for today =D