Shedule Tasks in ECS

Malte Hallström / November 20, 2021

4 min read •

It is possible to create scheduled tasks inside of ECS instead of running cron-jobs inside docker containers. This is perfect for scenarios where the container really does not have to be running all the time, but during the short periods of time that they are executing scripts.

However, to streamline the process and make sure that the local infrastructure-as-code was up-to-date and valid. The goal was to make it possible for the CDK scripts to create the scheduled task, never having to touch the console.

I ended up using @aws-cdk/aws-ecs-patterns to achieve the result. Namely, I used the ScheduledFargateTask construct. This required three things:

• An ECS cluster reference
• A schedule expression
• A task definition

The cluster reference

There are (at least) two ways of obtaining a reference to a cluster. You can create one, or you can try to get a reference to an already existing cluster. In my case, I had an already existing cluster in mind, so I went with the latter option.

In order to use the Cluster.fromClusterAttributes static method, I needed to specify a few attributes that could be used to find the designated cluster. These were:

• A reference to a VPC
• A reference to the relevant Security Group(s)
• The cluster name

The VPC & Security Groups got easily be gotten ahold of using the following:

const vpc = ec2.Vpc.fromLookup(this, "Vpc", { vpcId: props.vpcId });

this.securityGroup = new ec2.SecurityGroup(this, "ScheduledTaskSecurityGroup", {
  vpc,
  securityGroupName: "ScheduledTaskSecurityGroup",
});

See CDK Introduction - The security rules for more information and a look at the relevant inputs.

I then continued on and used the above references to get ahold of a cluster reference:

const cluster = ecs.Cluster.fromClusterAttributes(
  this,
  "ScheduledTaskCluster",
  {
    vpc,
    securityGroups: [this.securityGroup],
    clusterName: "cool-cluster",
  }
);

The schedule expression

In order to create a scheduled task on ECS, or anywhere for that matter, a schedule needs to be defined somehow. There are common ways of doing this, and perhaps most well-known is the cron syntax.

In this case, the interface required that the specified schedule was to be defined using the Schedule class from the @aws-cdk/aws-applicationautoscaling module.

The class contained the following methods:

Since I was already familiar with the cron syntax, I went with the cron method. Since I wanted the job to run every 20 minutes, starting at the hour, this was the final result:

schedule: events.Schedule.cron({
  minute: "0",
  hour: "*/2",
}),

The task definition

Although being very similar to [[CDK Introduction#The task definition]], one major difference with this task definition was the jump to AWS Fargate. For a comparison and motivation to use Fargate instead of ECS, see ECS vs. Fargate: What's the difference?

Here's the code for the definition itself, as well as adding the usual container config:

const taskDefinition = new ecs.FargateTaskDefinition(
  this,
  "ScheduledTaskTaskDefinition",
  {
    family: "scheduled-jobs",
  }
);

taskDefinition.addContainer("ScheduledTaskContainer", {
  image: ecs.ContainerImage.fromEcrRepository(props.ecrRepo, props.environment),
  logging: ecs.LogDriver.awsLogs({
    logGroup,
    streamPrefix: "/ecs/ScheduledTask",
  }),
  secrets: {
    PGPASSWORD: getSecret("password"),
    DB_HOST: getSecret("host"),
  },
});

The scheduled task

Now for the grand finale! Like mentioned in the beginning, all of the above steps need to come together in order to make it possible to use the ScheduledFargateTask construct. This is what it looks like:

const scheduledTask = new ecspatterns.ScheduledFargateTask(
  this,
  "ScheduledTask",
  {
    schedule: events.Schedule.cron({
      minute: "0",
      hour: "*/2",
    }),
    cluster,
    platformVersion: ecs.FargatePlatformVersion.LATEST,
    scheduledFargateTaskDefinitionOptions: {
      taskDefinition,
    },
  }
);

It is worth mentioning that this construct defaults to assigning all local subnets to the task. This is convenient of course, but if for some reason you would not like to have that done for you automatically you could use the securityGroups option.