<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
<title>michaelheap.com</title>
<subtitle>Thoughts on leadership, code and how to fix odd edge cases in tools (not necessarily in that order)</subtitle>
<link href="https://michaelheap.com/rss" rel="self"/>
<link href="https://michaelheap.com"/>
<updated>2025-06-02T08:00:13Z</updated>
<id>https://michaelheap.com</id>
<author>
<name>Michael Heap</name>
<email>[email protected]</email>
</author>
<entry>
<title>Run a container on a schedule with ECS</title>
<link href="https://michaelheap.com/ecs-scheduled-container/"/>
<updated>2025-06-02T08:00:13Z</updated>
<id>https://michaelheap.com/ecs-scheduled-container/</id>
<content type="html"><p>I've got a Docker container that I want to run periodically to fetch data and store it in a database. As this is something that needs to run persistently, I'm using Terraform to manage the infrastructure. It took me a while to figure out all the required resources and permissions, so I thought I'd share my solution here.</p>
<p>To securely run a container on a schedule in ECS, I needed to:</p>
<ul>
<li>Initialise Terraform</li>
<li>Configure any secrets that the container needs</li>
<li>Create an ECS Fargate cluster, and a VPC for the container to run in</li>
<li>Create an IAM role for the container to execute as</li>
<li>Create an IAM role for EventBridge to trigger the ECS task</li>
<li>Define an EventBridge schedule trigger</li>
<li>Define a task</li>
</ul>
<h2 id="initialise-terraform" tabindex="-1">Initialise Terraform</h2>
<p>Create <code>providers.tf</code> containing the AWS profile and region you want to use:</p>
<pre class="shiki nord" style="background-color: #2e3440ff; color: #d8dee9ff"><div class="language-id">hcl</div><div class="code-container"><code><div class="line"><span style="color: #81A1C1">provider</span><span style="color: #A3BE8C"> "aws"</span><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">region</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"us-east-2"</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">profile</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"default"</span></div><div class="line"><span style="color: #D8DEE9FF">}</span></div></code></div></pre>
<p>Then run <code>terraform init</code>.</p>
<h2 id="secrets" tabindex="-1">Secrets</h2>
<p>The container that I'm running needs access to some secret values as environment variables. I read the two most sensitive items from <code>tfvars</code>, and hard code the role as the Terraform repo is private anyway. The <code>test/area/role</code> secret is really just configuration rather than being a secret:</p>
<pre class="shiki nord" style="background-color: #2e3440ff; color: #d8dee9ff"><div class="language-id">hcl</div><div class="code-container"><code><div class="line"><span style="color: #81A1C1">variable</span><span style="color: #A3BE8C"> "AREA_USERNAME"</span><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">description</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"Username for AREA access"</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">type</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> string</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">sensitive</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #B48EAD">true</span></div><div class="line"><span style="color: #D8DEE9FF">}</span></div><div class="line"></div><div class="line"><span style="color: #81A1C1">variable</span><span style="color: #A3BE8C"> "AREA_PASSWORD"</span><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">description</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"Password for AREA access"</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">type</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> string</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">sensitive</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #B48EAD">true</span></div><div class="line"><span style="color: #D8DEE9FF">}</span></div><div class="line"></div><div class="line"><span style="color: #81A1C1">locals</span><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">my_secrets</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> "test/area/username" = var.AREA_USERNAME</span></div><div class="line"><span style="color: #D8DEE9FF"> "test/area/password" = var.AREA_PASSWORD</span></div><div class="line"><span style="color: #D8DEE9FF"> "test/area/role" = "some_role"</span></div><div class="line"><span style="color: #D8DEE9FF"> }</span></div><div class="line"><span style="color: #D8DEE9FF">}</span></div><div class="line"></div><div class="line"><span style="color: #81A1C1">resource</span><span style="color: #A3BE8C"> "aws_secretsmanager_secret" "my_secrets"</span><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">for_each</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> local</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">my_secrets</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">name</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> each</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">key</span></div><div class="line"><span style="color: #D8DEE9FF">}</span></div><div class="line"></div><div class="line"><span style="color: #81A1C1">resource</span><span style="color: #A3BE8C"> "aws_secretsmanager_secret_version" "my_secret_values"</span><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">for_each</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> local</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">my_secrets</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">secret_id</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> aws_secretsmanager_secret</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">my_secrets[each</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">key].id</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">secret_string</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> each</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">value</span></div><div class="line"><span style="color: #D8DEE9FF">}</span></div></code></div></pre>
<h2 id="ecs-cluster-and-vpc" tabindex="-1">ECS Cluster and VPC</h2>
<p>ECS allows you to scale to zero, but you still need a VPC to spin up containers in. My containers need internet access, so here's how I create an ECS cluster and VPC with internet access:</p>
<pre class="shiki nord" style="background-color: #2e3440ff; color: #d8dee9ff"><div class="language-id">hcl</div><div class="code-container"><code><div class="line"><span style="color: #616E88"># Create an ECS cluster</span></div><div class="line"><span style="color: #81A1C1">resource</span><span style="color: #A3BE8C"> "aws_ecs_cluster" "my_cluster"</span><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">name</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"demo-fargate-cluster"</span></div><div class="line"><span style="color: #D8DEE9FF">}</span></div><div class="line"></div><div class="line"><span style="color: #616E88"># Create a VPC and network, allowing all egress traffic</span></div><div class="line"><span style="color: #81A1C1">resource</span><span style="color: #A3BE8C"> "aws_vpc" "main"</span><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">cidr_block</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"10.0.0.0/16"</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">enable_dns_hostnames</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #B48EAD">true</span></div><div class="line"><span style="color: #D8DEE9FF">}</span></div><div class="line"></div><div class="line"><span style="color: #81A1C1">resource</span><span style="color: #A3BE8C"> "aws_subnet" "public"</span><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">vpc_id</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> aws_vpc</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">main</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">id</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">cidr_block</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"10.0.1.0/24"</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">availability_zone</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"us-east-2a"</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">map_public_ip_on_launch</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #B48EAD">true</span></div><div class="line"><span style="color: #D8DEE9FF">}</span></div><div class="line"></div><div class="line"><span style="color: #616E88"># You could also use a NAT gateway instead. We use an internet gateway</span></div><div class="line"><span style="color: #616E88"># due to speed / cost reasons in this example</span></div><div class="line"><span style="color: #81A1C1">resource</span><span style="color: #A3BE8C"> "aws_internet_gateway" "gw"</span><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">vpc_id</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> aws_vpc</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">main</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">id</span></div><div class="line"><span style="color: #D8DEE9FF">}</span></div><div class="line"></div><div class="line"><span style="color: #81A1C1">resource</span><span style="color: #A3BE8C"> "aws_route_table" "public"</span><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">vpc_id</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> aws_vpc</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">main</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">id</span></div><div class="line"></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">route</span><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">cidr_block</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"0.0.0.0/0"</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">gateway_id</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> aws_internet_gateway</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">gw</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">id</span></div><div class="line"><span style="color: #D8DEE9FF"> }</span></div><div class="line"><span style="color: #D8DEE9FF">}</span></div><div class="line"></div><div class="line"><span style="color: #81A1C1">resource</span><span style="color: #A3BE8C"> "aws_route_table_association" "public_assoc"</span><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">subnet_id</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> aws_subnet</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">public</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">id</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">route_table_id</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> aws_route_table</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">public</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">id</span></div><div class="line"><span style="color: #D8DEE9FF">}</span></div><div class="line"></div><div class="line"><span style="color: #81A1C1">resource</span><span style="color: #A3BE8C"> "aws_security_group" "ecs_tasks"</span><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">name</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"ecs-scheduled-tasks-sg"</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">description</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"Allow outbound traffic"</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">vpc_id</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> aws_vpc</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">main</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">id</span></div><div class="line"></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">egress</span><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">from_port</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #B48EAD">0</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">to_port</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #B48EAD">0</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">protocol</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"-1"</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">cidr_blocks</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> [</span><span style="color: #A3BE8C">"0.0.0.0/0"</span><span style="color: #D8DEE9FF">]</span></div><div class="line"><span style="color: #D8DEE9FF"> }</span></div><div class="line"><span style="color: #D8DEE9FF">}</span></div></code></div></pre>
<h2 id="execution-iam-role" tabindex="-1">Execution IAM role</h2>
<p>The container that runs does not have access to AWS Secrets Manager by default. To allow access, I create a new IAM role that the container will assume using <code>sts:AssumeRole</code> before running. This new role has a policy attached that allows access to specific secrets in AWS Secrets Manager.</p>
<pre class="shiki nord" style="background-color: #2e3440ff; color: #d8dee9ff"><div class="language-id">hcl</div><div class="code-container"><code><div class="line"><span style="color: #616E88"># Allow ECS to assume this role</span></div><div class="line"><span style="color: #81A1C1">resource</span><span style="color: #A3BE8C"> "aws_iam_role" "ecs_task_execution"</span><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">name</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"ecs-task-execution-role"</span></div><div class="line"></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">assume_role_policy</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> jsonencode({</span></div><div class="line"><span style="color: #D8DEE9FF"> Version = "2012-10-17",</span></div><div class="line"><span style="color: #D8DEE9FF"> Statement = [{</span></div><div class="line"><span style="color: #D8DEE9FF"> Effect = "Allow",</span></div><div class="line"><span style="color: #D8DEE9FF"> Principal = {</span></div><div class="line"><span style="color: #D8DEE9FF"> Service = "ecs-tasks.amazonaws.com"</span></div><div class="line"><span style="color: #D8DEE9FF"> },</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">Action</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"sts:AssumeRole"</span></div><div class="line"><span style="color: #D8DEE9FF"> }]</span></div><div class="line"><span style="color: #D8DEE9FF"> })</span></div><div class="line"><span style="color: #D8DEE9FF">}</span></div><div class="line"></div><div class="line"><span style="color: #616E88"># Create a policy that allows access to the secrets we defined earlier</span></div><div class="line"><span style="color: #81A1C1">resource</span><span style="color: #A3BE8C"> "aws_iam_policy" "ecs_execution_secrets_access"</span><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">name</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"ecs-exec-secrets-access"</span></div><div class="line"></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">policy</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> jsonencode({</span></div><div class="line"><span style="color: #D8DEE9FF"> Version = "2012-10-17",</span></div><div class="line"><span style="color: #D8DEE9FF"> Statement = [{</span></div><div class="line"><span style="color: #D8DEE9FF"> Effect = "Allow",</span></div><div class="line"><span style="color: #D8DEE9FF"> Action = [</span></div><div class="line"><span style="color: #D8DEE9FF"> "secretsmanager:GetSecretValue",</span></div><div class="line"><span style="color: #D8DEE9FF"> "secretsmanager:DescribeSecret"</span></div><div class="line"><span style="color: #D8DEE9FF"> ],</span></div><div class="line"><span style="color: #D8DEE9FF"> Resource = [</span></div><div class="line"><span style="color: #D8DEE9FF"> aws_secretsmanager_secret.my_secrets["test/area/username"].arn,</span></div><div class="line"><span style="color: #D8DEE9FF"> aws_secretsmanager_secret.my_secrets["test/area/password"].arn,</span></div><div class="line"><span style="color: #D8DEE9FF"> aws_secretsmanager_secret.my_secrets["test/area/role"].arn,</span></div><div class="line"><span style="color: #D8DEE9FF"> ]</span></div><div class="line"><span style="color: #D8DEE9FF"> }]</span></div><div class="line"><span style="color: #D8DEE9FF"> })</span></div><div class="line"><span style="color: #D8DEE9FF">}</span></div><div class="line"></div><div class="line"><span style="color: #616E88"># Attach the above policy to the IAM role</span></div><div class="line"><span style="color: #81A1C1">resource</span><span style="color: #A3BE8C"> "aws_iam_role_policy_attachment" "ecs_exec_secrets_access_attach"</span><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">role</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> aws_iam_role</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">ecs_task_execution</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">name</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">policy_arn</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> aws_iam_policy</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">ecs_execution_secrets_access</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">arn</span></div><div class="line"><span style="color: #D8DEE9FF">}</span></div><div class="line"></div><div class="line"><span style="color: #616E88"># Attach the ECS Task execution policy</span></div><div class="line"><span style="color: #81A1C1">resource</span><span style="color: #A3BE8C"> "aws_iam_role_policy_attachment" "ecs_task_execution_attach"</span><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">role</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> aws_iam_role</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">ecs_task_execution</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">name</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">policy_arn</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy"</span></div><div class="line"><span style="color: #D8DEE9FF">}</span></div><div class="line"></div></code></div></pre>
<h2 id="trigger-iam-role" tabindex="-1">Trigger IAM role</h2>
<p>Amazon EventBridge also needs an IAM role in order to trigger the ECS task. Here's an IAM role that has permission to run the defined tasks explicitly.</p>
<p>It took me a <em>long</em> time to realise that not only did I need to provide a <code>runTask</code> permission to the tasks, I also needed to allow <code>runTask</code> to the ECS cluster that was being used.</p>
<pre class="shiki nord" style="background-color: #2e3440ff; color: #d8dee9ff"><div class="language-id">hcl</div><div class="code-container"><code><div class="line"><span style="color: #616E88"># Allow EventBridge to assume this role</span></div><div class="line"><span style="color: #81A1C1">resource</span><span style="color: #A3BE8C"> "aws_iam_role" "eventbridge_invoke_ecs"</span><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">name</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"eventbridge-ecs-invoke-role"</span></div><div class="line"></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">assume_role_policy</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> jsonencode({</span></div><div class="line"><span style="color: #D8DEE9FF"> Version = "2012-10-17",</span></div><div class="line"><span style="color: #D8DEE9FF"> Statement = [{</span></div><div class="line"><span style="color: #D8DEE9FF"> Effect = "Allow",</span></div><div class="line"><span style="color: #D8DEE9FF"> Principal = {</span></div><div class="line"><span style="color: #D8DEE9FF"> Service = "events.amazonaws.com"</span></div><div class="line"><span style="color: #D8DEE9FF"> },</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">Action</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"sts:AssumeRole"</span></div><div class="line"><span style="color: #D8DEE9FF"> }</span></div><div class="line"><span style="color: #D8DEE9FF"> ]</span></div><div class="line"><span style="color: #D8DEE9FF"> })</span></div><div class="line"><span style="color: #D8DEE9FF">}</span></div><div class="line"></div><div class="line"><span style="color: #616E88"># Create a new IAM policy that allows us to run all of the defined tasks</span></div><div class="line"><span style="color: #616E88"># We need to explicitly allow ecs:RunTask for the ecs:cluster too</span></div><div class="line"><span style="color: #81A1C1">resource</span><span style="color: #A3BE8C"> "aws_iam_role_policy" "eventbridge_ecs_policy"</span><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">name</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"invoke-ecs"</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">role</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> aws_iam_role</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">eventbridge_invoke_ecs</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">id</span></div><div class="line"></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">policy</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> jsonencode({</span></div><div class="line"><span style="color: #D8DEE9FF"> Version = "2012-10-17",</span></div><div class="line"><span style="color: #D8DEE9FF"> Statement = [</span></div><div class="line"><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> Effect = "Allow",</span></div><div class="line"><span style="color: #D8DEE9FF"> Action = "ecs:RunTask",</span></div><div class="line"><span style="color: #D8DEE9FF"> Resource = aws_ecs_task_definition.scheduled_my_command.arn</span></div><div class="line"><span style="color: #D8DEE9FF"> },</span></div><div class="line"><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">Effect</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"Allow"</span><span style="color: #D8DEE9FF">,</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">Action</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"iam:PassRole"</span><span style="color: #D8DEE9FF">,</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">Resource</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> aws_iam_role</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">ecs_task_execution</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">arn</span></div><div class="line"><span style="color: #D8DEE9FF"> },</span></div><div class="line"><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">Effect</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"Allow"</span><span style="color: #D8DEE9FF">,</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">Action</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"ecs:RunTask"</span><span style="color: #D8DEE9FF">,</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">Resource</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"*"</span><span style="color: #D8DEE9FF">,</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">Condition</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"ArnEquals"</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">:</span><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> "ecs:cluster" : aws_ecs_cluster.my_cluster.arn</span></div><div class="line"><span style="color: #D8DEE9FF"> }</span></div><div class="line"><span style="color: #D8DEE9FF"> }</span></div><div class="line"><span style="color: #D8DEE9FF"> }</span></div><div class="line"><span style="color: #D8DEE9FF"> ]</span></div><div class="line"><span style="color: #D8DEE9FF"> })</span></div><div class="line"><span style="color: #D8DEE9FF">}</span></div><div class="line"></div></code></div></pre>
<h2 id="eventbridge-schedule-trigger" tabindex="-1">EventBridge schedule trigger</h2>
<p>We need to define an event rule that runs the container on a schedule. I also create a Cloudwatch log group to capture task logs.</p>
<pre class="shiki nord" style="background-color: #2e3440ff; color: #d8dee9ff"><div class="language-id">hcl</div><div class="code-container"><code><div class="line"><span style="color: #616E88"># Run the task at 23:59 every night</span></div><div class="line"><span style="color: #81A1C1">resource</span><span style="color: #A3BE8C"> "aws_cloudwatch_event_rule" "run_at_23_59"</span><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">name</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"run-ecs-task-schedule"</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">schedule_expression</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"cron(59 23 * * ? *)"</span></div><div class="line"><span style="color: #D8DEE9FF">}</span></div><div class="line"></div><div class="line"><span style="color: #616E88"># And create a Cloudwatch log group to send logs to</span></div><div class="line"><span style="color: #81A1C1">resource</span><span style="color: #A3BE8C"> "aws_cloudwatch_log_group" "my_scheduled_task"</span><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">name</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"/ecs/my-scheduled-task-logs"</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">retention_in_days</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #B48EAD">1</span></div><div class="line"><span style="color: #D8DEE9FF">}</span></div></code></div></pre>
<h2 id="task-definition" tabindex="-1">Task Definition</h2>
<p>Finally, we need to define the task to run. You'll need to upload the docker image to ECR before defining a task. You'll also need to specify all of the secrets that the container needs access to.</p>
<pre class="shiki nord" style="background-color: #2e3440ff; color: #d8dee9ff"><div class="language-id">hcl</div><div class="code-container"><code><div class="line"><span style="color: #616E88"># Define the container and command to run, plus CPU/Memory usage</span></div><div class="line"><span style="color: #81A1C1">resource</span><span style="color: #A3BE8C"> "aws_ecs_task_definition" "scheduled_my_command"</span><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">family</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"scheduled-my-command"</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">requires_compatibilities</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> [</span><span style="color: #A3BE8C">"FARGATE"</span><span style="color: #D8DEE9FF">]</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">network_mode</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"awsvpc"</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">cpu</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"256"</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">memory</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"512"</span></div><div class="line"></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">execution_role_arn</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> aws_iam_role</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">ecs_task_execution</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">arn</span></div><div class="line"></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">container_definitions</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> jsonencode([</span></div><div class="line"><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> name = </span><span style="color: #A3BE8C">"demo"</span><span style="color: #D8DEE9FF">,</span></div><div class="line"><span style="color: #D8DEE9FF"> image = </span><span style="color: #A3BE8C">"hello-world"</span><span style="color: #D8DEE9FF">,</span></div><div class="line"><span style="color: #D8DEE9FF"> essential = </span><span style="color: #B48EAD">true</span><span style="color: #D8DEE9FF">,</span></div><div class="line"><span style="color: #D8DEE9FF"> secrets = [</span></div><div class="line"><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> name = </span><span style="color: #A3BE8C">"AREA_USER"</span><span style="color: #D8DEE9FF">,</span></div><div class="line"><span style="color: #D8DEE9FF"> valueFrom = aws_secretsmanager_secret</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">my_secrets[</span><span style="color: #A3BE8C">"test/area/username"</span><span style="color: #D8DEE9FF">].arn</span></div><div class="line"><span style="color: #D8DEE9FF"> },</span></div><div class="line"><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">name</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"AREA_PASSWORD"</span><span style="color: #D8DEE9FF">,</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">valueFrom</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> aws_secretsmanager_secret</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">my_secrets[</span><span style="color: #A3BE8C">"test/area/password"</span><span style="color: #D8DEE9FF">].arn</span></div><div class="line"><span style="color: #D8DEE9FF"> },</span></div><div class="line"><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">name</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"AREA_ROLE"</span><span style="color: #D8DEE9FF">,</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">valueFrom</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> aws_secretsmanager_secret</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">my_secrets[</span><span style="color: #A3BE8C">"test/area/role"</span><span style="color: #D8DEE9FF">].arn</span></div><div class="line"><span style="color: #D8DEE9FF"> }</span></div><div class="line"><span style="color: #D8DEE9FF"> ],</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">logConfiguration</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> logDriver = "awslogs",</span></div><div class="line"><span style="color: #D8DEE9FF"> options = {</span></div><div class="line"><span style="color: #D8DEE9FF"> awslogs-group = "/ecs/my-scheduled-task-logs"</span></div><div class="line"><span style="color: #D8DEE9FF"> awslogs-region = "us-east-2"</span></div><div class="line"><span style="color: #D8DEE9FF"> awslogs-stream-prefix = "demo"</span></div><div class="line"><span style="color: #D8DEE9FF"> }</span></div><div class="line"><span style="color: #D8DEE9FF"> }</span></div><div class="line"><span style="color: #D8DEE9FF"> }</span></div><div class="line"><span style="color: #D8DEE9FF"> ])</span></div><div class="line"><span style="color: #D8DEE9FF">}</span></div><div class="line"></div><div class="line"><span style="color: #616E88"># Trigger this task using the scheduled event</span></div><div class="line"><span style="color: #81A1C1">resource</span><span style="color: #A3BE8C"> "aws_cloudwatch_event_target" "ecs_my_command_target"</span><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">rule</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> aws_cloudwatch_event_rule</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">run_at_23_59</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">name</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">role_arn</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> aws_iam_role</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">eventbridge_invoke_ecs</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">arn</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">target_id</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"ecs-task-my-command"</span></div><div class="line"></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">arn</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> aws_ecs_cluster</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">my_cluster</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">arn</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">ecs_target</span><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">task_definition_arn</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> aws_ecs_task_definition</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">scheduled_my_command</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">arn</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">launch_type</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"FARGATE"</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">network_configuration</span><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">subnets</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> [aws_subnet</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">public</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">id]</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">security_groups</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> [aws_security_group</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">ecs_tasks</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">id]</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">assign_public_ip</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #B48EAD">true</span></div><div class="line"><span style="color: #D8DEE9FF"> }</span></div><div class="line"><span style="color: #D8DEE9FF"> }</span></div><div class="line"></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #616E88"># This is only needed for debugging. See the final section</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #616E88"># dead_letter_config {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #616E88"># arn = aws_sqs_queue.failed_invocations.arn</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #616E88"># }</span></div><div class="line"><span style="color: #D8DEE9FF">}</span></div></code></div></pre>
<h2 id="help%2C-it's-not-working!" tabindex="-1">Help, it's not working!</h2>
<p>Last, but not least, debugging! If your task isn't triggering as expected, you can configure a DeadLetterQueue (DLQ) to receieve the error messages from AWS. If you choose to do this, uncomment the <code>dead_letter_config</code> section of the task definition above.</p>
<pre class="shiki nord" style="background-color: #2e3440ff; color: #d8dee9ff"><div class="language-id">hcl</div><div class="code-container"><code><div class="line"><span style="color: #81A1C1">resource</span><span style="color: #A3BE8C"> "aws_sqs_queue" "failed_invocations"</span><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">name</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"eventbridge-ecs-dlq"</span></div><div class="line"><span style="color: #D8DEE9FF">}</span></div><div class="line"></div><div class="line"><span style="color: #81A1C1">resource</span><span style="color: #A3BE8C"> "aws_sqs_queue_policy" "allow_eventbridge"</span><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">queue_url</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> aws_sqs_queue</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">failed_invocations</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">id</span></div><div class="line"></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">policy</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> jsonencode({</span></div><div class="line"><span style="color: #D8DEE9FF"> Version = "2012-10-17",</span></div><div class="line"><span style="color: #D8DEE9FF"> Statement = [</span></div><div class="line"><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> Sid = "AllowEventBridgeToSendMessages",</span></div><div class="line"><span style="color: #D8DEE9FF"> Effect = "Allow",</span></div><div class="line"><span style="color: #D8DEE9FF"> Principal = {</span></div><div class="line"><span style="color: #D8DEE9FF"> Service = "events.amazonaws.com"</span></div><div class="line"><span style="color: #D8DEE9FF"> },</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">Action</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"sqs:SendMessage"</span><span style="color: #D8DEE9FF">,</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">Resource</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> aws_sqs_queue</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">failed_invocations</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">arn,</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">Condition</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> ArnEquals = {</span></div><div class="line"><span style="color: #D8DEE9FF"> "aws:SourceArn" = aws_cloudwatch_event_rule.run_at_23_59.arn</span></div><div class="line"><span style="color: #D8DEE9FF"> }</span></div><div class="line"><span style="color: #D8DEE9FF"> }</span></div><div class="line"><span style="color: #D8DEE9FF"> }</span></div><div class="line"><span style="color: #D8DEE9FF"> ]</span></div><div class="line"><span style="color: #D8DEE9FF"> })</span></div><div class="line"><span style="color: #D8DEE9FF">}</span></div></code></div></pre>
<p>To read messages from the dead letter queue, use the <code>aws</code> CLI tool:</p>
<pre class="shiki nord" style="background-color: #2e3440ff; color: #d8dee9ff"><div class="language-id">bash</div><div class="code-container"><code><div class="line"><span style="color: #616E88"># Fetch the queue URL</span></div><div class="line"><span style="color: #D8DEE9FF">aws sqs get-queue-url --queue-name eventbridge-ecs-dlq</span></div><div class="line"></div><div class="line"><span style="color: #616E88"># Read messages - change the URL for your ID</span></div><div class="line"><span style="color: #D8DEE9FF">aws sqs receive-message \</span></div><div class="line"><span style="color: #D8DEE9FF"> --queue-url https://sqs.us-east-2.amazonaws.com/111111111111/eventbridge-ecs-dlq \</span></div><div class="line"><span style="color: #D8DEE9FF"> --max-number-of-messages 10 \</span></div><div class="line"><span style="color: #D8DEE9FF"> --wait-time-seconds 5 \</span></div><div class="line"><span style="color: #D8DEE9FF"> --message-attribute-names All \</span></div><div class="line"><span style="color: #D8DEE9FF"> --attribute-names All</span></div></code></div></pre>
<h2 id="conclusion" tabindex="-1">Conclusion</h2>
<p>The parts of this that gave me the most trouble were figuring out how to debug using <code>dead_letter_queue</code> and that the IAM policy for <code>runTask</code> needed access to the cluster too.</p>
<p>Hopefully this has helped you (or will help me again in the future) to deploy scheduled tasks on ECS.</p>
</content>
</entry>
<entry>
<title>Pin your GitHub Actions</title>
<link href="https://michaelheap.com/pin-your-github-actions/"/>
<updated>2025-03-15T20:19:27Z</updated>
<id>https://michaelheap.com/pin-your-github-actions/</id>
<content type="html"><p>Way back in 2019, Julien Renaux published <a href="https://julienrenaux.fr/2019/12/20/github-actions-security-risk/">Use GitHub Actions at your own risk</a>. While the title is a little sensational, it correctly pointed out that any maintainer can update a branch or tag to point at new code without you knowing. This means that if any action is compromised, you'll start leaking secrets without knowing it.</p>
<p>Today, <code>tj-actions/changed-files</code>, a widely used GitHub Action <a href="https://www.stepsecurity.io/blog/harden-runner-detection-tj-actions-changed-files-action-is-compromised">was compromised</a> and started leaking secrets.</p>
<p>Hopefully this was the wakeup call the industry needed to start paying attention to supply chain security.</p>
<h2 id="solving-the-problem" tabindex="-1">Solving the problem</h2>
<p>Security is always a trade-off. You can solve the supply chain problem by specifying a full length commit SHA, but fetching that SHA for every action is a painful process. Then, whenever you want to upgrade your action you have to do it all again.</p>
<p>Thankfully, there are tools and automations available to solve this problem. You can be secure <em>and</em> feel minimal pain thanks to these projects:</p>
<ol>
<li><a href="https://github.com/mheap/pin-github-action">pin-github-action</a> - This is one of my projects. It takes a directory of workflows and uses the GitHub API to convert tag and branch references to a full length commit SHA.</li>
<li><a href="https://github.com/zgosalvez/github-actions-ensure-sha-pinned-actions">github-actions-ensure-sha-pinned-actions</a> - A community action that will fail the build if it detects any unpinned actions being used in the current repository.</li>
<li><a href="https://docs.github.com/en/code-security/dependabot/working-with-dependabot/keeping-your-actions-up-to-date-with-dependabot">Dependabot</a> / <a href="https://docs.renovatebot.com/modules/manager/github-actions/">Renovate</a> - Dependency management systems that submits PRs to upgrade GitHub Actions. They both natively understand the <code># &lt;ref&gt;</code> comments in workflow files.</li>
</ol>
<h3 id="pin-to-a-sha" tabindex="-1">Pin to a SHA</h3>
<p>The first thing to do is update all of your existing workflows to use the long commit SHA using <code>pin-github-action</code>.</p>
<p>You can run it with <code>npx</code> or <code>docker</code>. If you're not sure which to use, use <code>docker</code> through the following alias:</p>
<pre class="shiki nord" style="background-color: #2e3440ff; color: #d8dee9ff"><div class="language-id">bash</div><div class="code-container"><code><div class="line"><span style="color: #88C0D0">alias</span><span style="color: #D8DEE9FF"> pin-github-action=</span><span style="color: #ECEFF4">"</span><span style="color: #A3BE8C">docker run --rm -v </span><span style="color: #ECEFF4">$(</span><span style="color: #A3BE8C">pwd</span><span style="color: #ECEFF4">)</span><span style="color: #A3BE8C">:/workflows -e GITHUB_TOKEN mheap/pin-github-action</span><span style="color: #ECEFF4">"</span></div></code></div></pre>
<p>If you're working with a large number of workflows, or any private actions you'll need to set the <code>GITHUB_TOKEN</code> environment variable to prevent rate limiting or provide valid access credentials to a repository:</p>
<pre class="shiki nord" style="background-color: #2e3440ff; color: #d8dee9ff"><div class="language-id">bash</div><div class="code-container"><code><div class="line"><span style="color: #81A1C1">export</span><span style="color: #D8DEE9FF"> GITHUB_TOKEN=</span><span style="color: #ECEFF4">"</span><span style="color: #A3BE8C">ghp_YOUR_TOKEN_HERE</span><span style="color: #ECEFF4">"</span></div></code></div></pre>
<p>Finally, change directory to your repository and run <code>pin-github-action</code>:</p>
<pre class="shiki nord" style="background-color: #2e3440ff; color: #d8dee9ff"><div class="language-id">bash</div><div class="code-container"><code><div class="line"><span style="color: #88C0D0">cd</span><span style="color: #D8DEE9FF"> my-repo</span></div><div class="line"><span style="color: #D8DEE9FF">pin-github-action .github/workflows</span></div></code></div></pre>
<p>If you run <code>git diff .github/workflows</code> you'll see that all of your actions have been updated to the long commit SHA:</p>
<pre class="shiki nord" style="background-color: #2e3440ff; color: #d8dee9ff"><div class="language-id">diff</div><div class="code-container"><code><div class="line"><span style="color: #D8DEE9FF"> steps:</span></div><div class="line"><span style="color: #ECEFF4">-</span><span style="color: #BF616A"> - uses: actions/checkout@v4</span></div><div class="line"><span style="color: #ECEFF4">+</span><span style="color: #A3BE8C"> - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4</span></div><div class="line"><span style="color: #ECEFF4">-</span><span style="color: #BF616A"> - uses: MyOrg/some-action@v1</span></div><div class="line"><span style="color: #ECEFF4">+</span><span style="color: #A3BE8C"> - uses: MyOrg/some-action@25ed13d0628a1601b4b44048e63cc4328ed03633 # v1</span></div></code></div></pre>
<p>I recommend pinning all actions to a SHA, but this may not be feasible for some companies that use internal actions. If you want to trust internal actions, you can pass the <code>--allow</code> flag to <code>pin-github-action</code> to add a specific prefix to an allowlist:</p>
<pre class="shiki nord" style="background-color: #2e3440ff; color: #d8dee9ff"><div class="language-id">bash</div><div class="code-container"><code><div class="line"><span style="color: #D8DEE9FF">pin-github-action --allow </span><span style="color: #ECEFF4">"</span><span style="color: #A3BE8C">MyOrg/*</span><span style="color: #ECEFF4">"</span><span style="color: #D8DEE9FF"> .github/workflows</span></div></code></div></pre>
<p>This will ignore any actions with the prefix <code>MyOrg</code>:</p>
<pre class="shiki nord" style="background-color: #2e3440ff; color: #d8dee9ff"><div class="language-id">diff</div><div class="code-container"><code><div class="line"><span style="color: #D8DEE9FF"> steps:</span></div><div class="line"><span style="color: #ECEFF4">-</span><span style="color: #BF616A"> - uses: actions/checkout@v4</span></div><div class="line"><span style="color: #ECEFF4">+</span><span style="color: #A3BE8C"> - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4</span></div><div class="line"><span style="color: #D8DEE9FF"> - uses: MyOrg/some-action@v1</span></div></code></div></pre>
<h3 id="prevent-regressions" tabindex="-1">Prevent regressions</h3>
<p>Pinning all actions to a specific SHA solves the problem today, but it doesn't guarantee that a new action won't be added in the future without using a SHA. To prevent that happening, <a href="https://github.com/zgosalvez/github-actions-ensure-sha-pinned-actions">github-actions-ensure-sha-pinned-actions</a> can be used to fail the build when any unpinned actions are detected.</p>
<p>The README for the action contains examples, but if you're using <code>pin-github-action</code> you can automatically add a new workflow using the <code>--enforce</code> flag. The <code>--enforce</code> flag writes a workflow containing <code>github-actions-ensure-sha-pinned-actions</code> to the path provided, including adding any actions passed in <code>--allow</code> to the <code>allowlist</code> input for the action.</p>
<p>The following command will create a workflow at <code>.github/workflows/security.yaml</code> that ensures all actions are using the long commit SHA, <em>unless</em> they are actions from <code>MyOrg</code>:</p>
<pre class="shiki nord" style="background-color: #2e3440ff; color: #d8dee9ff"><div class="language-id">bash</div><div class="code-container"><code><div class="line"><span style="color: #D8DEE9FF">pin-github-action --allow </span><span style="color: #ECEFF4">"</span><span style="color: #A3BE8C">MyOrg/*</span><span style="color: #ECEFF4">"</span><span style="color: #D8DEE9FF"> --enforce .github/workflows/security.yaml .github/workflows </span></div></code></div></pre>
<p>The created workflow looks like this:</p>
<pre class="shiki nord" style="background-color: #2e3440ff; color: #d8dee9ff"><div class="language-id">yaml</div><div class="code-container"><code><div class="line"><span style="color: #81A1C1">on</span><span style="color: #ECEFF4">:</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">push</span></div><div class="line"></div><div class="line"><span style="color: #8FBCBB">name</span><span style="color: #ECEFF4">:</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">Security</span></div><div class="line"></div><div class="line"><span style="color: #8FBCBB">jobs</span><span style="color: #ECEFF4">:</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #8FBCBB">ensure-pinned-actions</span><span style="color: #ECEFF4">:</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #8FBCBB">runs-on</span><span style="color: #ECEFF4">:</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">ubuntu-latest</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #8FBCBB">steps</span><span style="color: #ECEFF4">:</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #ECEFF4">-</span><span style="color: #D8DEE9FF"> </span><span style="color: #8FBCBB">name</span><span style="color: #ECEFF4">:</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">Checkout code</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #8FBCBB">uses</span><span style="color: #ECEFF4">:</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683</span><span style="color: #D8DEE9FF"> </span><span style="color: #616E88"># v4</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #ECEFF4">-</span><span style="color: #D8DEE9FF"> </span><span style="color: #8FBCBB">name</span><span style="color: #ECEFF4">:</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">Ensure SHA pinned actions</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #8FBCBB">uses</span><span style="color: #ECEFF4">:</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">zgosalvez/github-actions-ensure-sha-pinned-actions@25ed13d0628a1601b4b44048e63cc4328ed03633</span><span style="color: #D8DEE9FF"> </span><span style="color: #616E88"># v3</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #8FBCBB">with</span><span style="color: #ECEFF4">:</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #8FBCBB">allowlist</span><span style="color: #ECEFF4">:</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">|</span></div><div class="line"><span style="color: #A3BE8C"> MyOrg/</span></div></code></div></pre>
<h3 id="automating-updates" tabindex="-1">Automating updates</h3>
<p>Finally, we need to keep our dependencies up to date. You have two options here:</p>
<ol>
<li>Run <code>pin-github-action</code> again</li>
<li>Use Dependabot or Renovate</li>
</ol>
<p><strong>I recommend using Dependabot or Renovate.</strong></p>
<p>Running <code>pin-github-action</code> on a repository with pinned SHAs and will extract the target version from the comment and update the SHA like so:</p>
<pre class="shiki nord" style="background-color: #2e3440ff; color: #d8dee9ff"><div class="language-id">diff</div><div class="code-container"><code><div class="line"><span style="color: #D8DEE9FF"> steps:</span></div><div class="line"><span style="color: #ECEFF4">-</span><span style="color: #BF616A"> - uses: actions/checkout@eef61447b9ff4aafe5dcd4e0bbf5d482be7e7871 # v4</span></div><div class="line"><span style="color: #ECEFF4">+</span><span style="color: #A3BE8C"> - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4</span></div></code></div></pre>
<p>This is great for updating actions in bulk, but it doesn't improve your security posture that much. You're still blindly upgrading to the latest version.</p>
<p>I recommend using Dependabot or Renovate to update your actions. Both tools create Pull Requests with updated SHAs and provide a diff for review. Each pull request contains a link to the <code>compare</code> view on GitHub that you can use to audit the changes since your last update and ensure that the action has not been compromised.</p>
<h3 id="stay-safe%2C-pin-your-dependencies" tabindex="-1">Stay safe, pin your dependencies</h3>
<p>The compromise of <code>tj-actions/changed-files</code> serves as a crucial reminder of the security risks in GitHub Actions. While pinning actions to commit SHAs adds an extra step, it significantly reduces the risk of supply chain attacks.</p>
<p>By leveraging tools like <code>pin-github-action</code>, enforcing SHA pinning with <code>github-actions-ensure-sha-pinned-actions</code>, and automating updates with Dependabot or Renovate, teams can secure their workflows without unnecessary overhead.</p>
<p>Supply chain security is an ongoing effort, but with these practices in place, you can proactively protect your repositories and secrets from potential threats.</p>
</content>
</entry>
<entry>
<title>Output a JSON file with Jekyll</title>
<link href="https://michaelheap.com/jekyll-output-json/"/>
<updated>2025-03-14T21:14:14Z</updated>
<id>https://michaelheap.com/jekyll-output-json/</id>
<content type="html"><p>I needed to expose some of the data we had in <code>app/_data</code> in our Jekyll deployment for consumption in another project. Instead of copy/pasting the data, I chose to expose it as an API.</p>
<p>To do the same, create a file named <code>app/_plugins/your_name_here_hook.rb</code> with the following contents:</p>
<pre class="shiki nord" style="background-color: #2e3440ff; color: #d8dee9ff"><div class="language-id">ruby</div><div class="code-container"><code><div class="line"><span style="color: #81A1C1">module</span><span style="color: #D8DEE9FF"> MyApi</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">def</span><span style="color: #D8DEE9FF"> </span><span style="color: #88C0D0">self.process</span><span style="color: #ECEFF4">(</span><span style="color: #D8DEE9">site</span><span style="color: #ECEFF4">)</span><span style="color: #D8DEE9FF"> </span></div><div class="line"><span style="color: #D8DEE9FF"> api_prefix </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #ECEFF4">'</span><span style="color: #A3BE8C">api</span><span style="color: #ECEFF4">'</span></div><div class="line"><span style="color: #D8DEE9FF"> the_data </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> site</span><span style="color: #ECEFF4">.</span><span style="color: #D8DEE9FF">data</span><span style="color: #ECEFF4">[</span><span style="color: #ECEFF4">'</span><span style="color: #A3BE8C">path_to</span><span style="color: #ECEFF4">'</span><span style="color: #ECEFF4">][</span><span style="color: #ECEFF4">'</span><span style="color: #A3BE8C">your</span><span style="color: #ECEFF4">'</span><span style="color: #ECEFF4">][</span><span style="color: #ECEFF4">'</span><span style="color: #A3BE8C">data</span><span style="color: #ECEFF4">'</span><span style="color: #ECEFF4">]</span></div><div class="line"></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #8FBCBB">FileUtils</span><span style="color: #ECEFF4">.</span><span style="color: #D8DEE9FF">mkdir_p</span><span style="color: #ECEFF4">(</span><span style="color: #ECEFF4">"</span><span style="color: #81A1C1">#{</span><span style="color: #A3BE8C">site</span><span style="color: #ECEFF4">.</span><span style="color: #A3BE8C">dest</span><span style="color: #81A1C1">}</span><span style="color: #A3BE8C">/</span><span style="color: #81A1C1">#{</span><span style="color: #A3BE8C">api_prefix</span><span style="color: #81A1C1">}</span><span style="color: #ECEFF4">"</span><span style="color: #ECEFF4">)</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #8FBCBB">File</span><span style="color: #ECEFF4">.</span><span style="color: #D8DEE9FF">write</span><span style="color: #ECEFF4">(</span><span style="color: #ECEFF4">"</span><span style="color: #81A1C1">#{</span><span style="color: #A3BE8C">site</span><span style="color: #ECEFF4">.</span><span style="color: #A3BE8C">dest</span><span style="color: #81A1C1">}</span><span style="color: #A3BE8C">/</span><span style="color: #81A1C1">#{</span><span style="color: #A3BE8C">api_prefix</span><span style="color: #81A1C1">}</span><span style="color: #A3BE8C">/your-data.json</span><span style="color: #ECEFF4">"</span><span style="color: #ECEFF4">,</span><span style="color: #D8DEE9FF"> the_data</span><span style="color: #ECEFF4">.</span><span style="color: #D8DEE9FF">to_json</span><span style="color: #ECEFF4">)</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">end</span></div><div class="line"><span style="color: #81A1C1">end</span></div><div class="line"></div><div class="line"><span style="color: #8FBCBB">Jekyll</span><span style="color: #ECEFF4">::</span><span style="color: #8FBCBB">Hooks</span><span style="color: #ECEFF4">.</span><span style="color: #D8DEE9FF">register </span><span style="color: #ECEFF4">:</span><span style="color: #D8DEE9FF">site</span><span style="color: #ECEFF4">,</span><span style="color: #D8DEE9FF"> </span><span style="color: #ECEFF4">:</span><span style="color: #D8DEE9FF">post_write </span><span style="color: #81A1C1">do</span><span style="color: #D8DEE9FF"> </span><span style="color: #ECEFF4">|</span><span style="color: #D8DEE9">site</span><span style="color: #ECEFF4">,</span><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">_</span><span style="color: #ECEFF4">|</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #8FBCBB">MyApi</span><span style="color: #ECEFF4">.</span><span style="color: #D8DEE9FF">process</span><span style="color: #ECEFF4">(</span><span style="color: #D8DEE9FF">site</span><span style="color: #ECEFF4">)</span></div><div class="line"><span style="color: #81A1C1">end</span></div></code></div></pre>
<p>Then you can access <code>/api/your-data.json</code> on your Jekyll deployment to see the data.</p>
</content>
</entry>
<entry>
<title>Running Scrumdog with Docker</title>
<link href="https://michaelheap.com/scrumdog-docker/"/>
<updated>2025-02-25T19:25:33Z</updated>
<id>https://michaelheap.com/scrumdog-docker/</id>
<content type="html"><p>I was intrigued by the idea of <a href="https://github.com/whoek/scrumdog">Scrumdog</a>, which allows you to export Jira tickets to a sqlite database.</p>
<p>Unfortunately the website is now offline, and the provided binaries don't work on non-intel macs. Thankfully, Scrumdog is open source so I can just build it myself... except that it's written in OCaml and I don't have any of the toolchain installed and don't really want to install it just for one project.</p>
<p>So, I reached for Docker.</p>
<p>It seems as though the <code>opam</code> dependency file isn't configured correctly for scrumdog, so here's a <code>Dockerfile</code> that explicitly installs the correct libraries:</p>
<pre class="shiki nord" style="background-color: #2e3440ff; color: #d8dee9ff"><div class="language-id">bash</div><div class="code-container"><code><div class="line"><span style="color: #616E88"># Build a base image with updated dependencies</span></div><div class="line"><span style="color: #D8DEE9FF">FROM ocaml/opam:alpine AS init-opam</span></div><div class="line"></div><div class="line"><span style="color: #D8DEE9FF">RUN </span><span style="color: #88C0D0">set</span><span style="color: #D8DEE9FF"> -x </span><span style="color: #81A1C1">&&</span><span style="color: #D8DEE9FF"> \</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #88C0D0">:</span><span style="color: #D8DEE9FF"> </span><span style="color: #ECEFF4">"</span><span style="color: #A3BE8C">Update and upgrade default packages</span><span style="color: #ECEFF4">"</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">&&</span><span style="color: #D8DEE9FF"> \</span></div><div class="line"><span style="color: #D8DEE9FF"> sudo apk update </span><span style="color: #81A1C1">&&</span><span style="color: #D8DEE9FF"> sudo apk upgrade </span><span style="color: #81A1C1">&&</span><span style="color: #D8DEE9FF"> \</span></div><div class="line"><span style="color: #D8DEE9FF"> sudo apk add gmp-dev sqlite-dev</span></div><div class="line"></div><div class="line"></div><div class="line"><span style="color: #616E88"># Install opam dependencies explicitly then build scrumdog</span></div><div class="line"><span style="color: #D8DEE9FF">FROM init-opam AS ocaml-app-base</span></div><div class="line"><span style="color: #D8DEE9FF">COPY </span><span style="color: #88C0D0">.</span><span style="color: #D8DEE9FF"> </span><span style="color: #88C0D0">.</span></div><div class="line"><span style="color: #D8DEE9FF">RUN </span><span style="color: #88C0D0">set</span><span style="color: #D8DEE9FF"> -x </span><span style="color: #81A1C1">&&</span><span style="color: #D8DEE9FF"> \</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #88C0D0">:</span><span style="color: #D8DEE9FF"> </span><span style="color: #ECEFF4">"</span><span style="color: #A3BE8C">Install related packages</span><span style="color: #ECEFF4">"</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">&&</span><span style="color: #D8DEE9FF"> \</span></div><div class="line"><span style="color: #D8DEE9FF"> opam install dune yojson sqlite3 cohttp-lwt-unix tls-lwt </span><span style="color: #81A1C1">&&</span><span style="color: #D8DEE9FF"> \</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #88C0D0">eval</span><span style="color: #D8DEE9FF"> </span><span style="color: #ECEFF4">$(</span><span style="color: #A3BE8C">opam env</span><span style="color: #ECEFF4">)</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">&&</span><span style="color: #D8DEE9FF"> \</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #88C0D0">:</span><span style="color: #D8DEE9FF"> </span><span style="color: #ECEFF4">"</span><span style="color: #A3BE8C">Build applications</span><span style="color: #ECEFF4">"</span></div><div class="line"><span style="color: #D8DEE9FF">RUN </span><span style="color: #88C0D0">eval</span><span style="color: #D8DEE9FF"> </span><span style="color: #ECEFF4">$(</span><span style="color: #A3BE8C">opam env</span><span style="color: #ECEFF4">)</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">&&</span><span style="color: #D8DEE9FF"> dune build</span></div><div class="line"></div><div class="line"></div><div class="line"><span style="color: #616E88"># Install dependencies + copy the built executable</span></div><div class="line"><span style="color: #D8DEE9FF">FROM alpine AS ocaml-app</span></div><div class="line"></div><div class="line"><span style="color: #D8DEE9FF">COPY --from=ocaml-app-base /home/opam/_build/default/bin/scrumdog.exe /home/bin/main.exe</span></div><div class="line"><span style="color: #D8DEE9FF">RUN </span><span style="color: #88C0D0">set</span><span style="color: #D8DEE9FF"> -x </span><span style="color: #81A1C1">&&</span><span style="color: #D8DEE9FF"> \</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #88C0D0">:</span><span style="color: #D8DEE9FF"> </span><span style="color: #ECEFF4">"</span><span style="color: #A3BE8C">Update and upgrade default packages</span><span style="color: #ECEFF4">"</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">&&</span><span style="color: #D8DEE9FF"> \</span></div><div class="line"><span style="color: #D8DEE9FF"> apk update </span><span style="color: #81A1C1">&&</span><span style="color: #D8DEE9FF"> apk upgrade </span><span style="color: #81A1C1">&&</span><span style="color: #D8DEE9FF"> \</span></div><div class="line"><span style="color: #D8DEE9FF"> apk add gmp-dev sqlite-dev </span><span style="color: #81A1C1">&&</span><span style="color: #D8DEE9FF"> \</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #88C0D0">:</span><span style="color: #D8DEE9FF"> </span><span style="color: #ECEFF4">"</span><span style="color: #A3BE8C">Create a user to execute application</span><span style="color: #ECEFF4">"</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">&&</span><span style="color: #D8DEE9FF"> \</span></div><div class="line"><span style="color: #D8DEE9FF"> adduser -D app </span><span style="color: #81A1C1">&&</span><span style="color: #D8DEE9FF"> \</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #88C0D0">:</span><span style="color: #D8DEE9FF"> </span><span style="color: #ECEFF4">"</span><span style="color: #A3BE8C">Change owner to app</span><span style="color: #ECEFF4">"</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">&&</span><span style="color: #D8DEE9FF"> \</span></div><div class="line"><span style="color: #D8DEE9FF"> chown app:app /home/bin/main.exe</span></div><div class="line"></div><div class="line"><span style="color: #D8DEE9FF">WORKDIR /home/app</span></div><div class="line"><span style="color: #D8DEE9FF">USER app</span></div><div class="line"><span style="color: #D8DEE9FF">ENTRYPOINT </span><span style="color: #ECEFF4">[</span><span style="color: #ECEFF4">"</span><span style="color: #A3BE8C">/home/bin/main.exe</span><span style="color: #ECEFF4">"</span><span style="color: #ECEFF4">]</span></div></code></div></pre>
<p>Finally, build the image and run it.</p>
<pre class="shiki nord" style="background-color: #2e3440ff; color: #d8dee9ff"><div class="language-id">bash</div><div class="code-container"><code><div class="line"><span style="color: #D8DEE9FF">docker build -t scrumdog-local </span><span style="color: #88C0D0">.</span></div><div class="line"><span style="color: #D8DEE9FF">docker run --rm -v </span><span style="color: #81A1C1">$</span><span style="color: #D8DEE9">PWD</span><span style="color: #D8DEE9FF">:/home/app scrumdog-local -j</span></div></code></div></pre>
<p>This generates a <code>sample.jql</code> file in the current directory that you can edit and use as a configuration file:</p>
<pre class="shiki nord" style="background-color: #2e3440ff; color: #d8dee9ff"><div class="language-id">bash</div><div class="code-container"><code><div class="line"><span style="color: #D8DEE9FF">docker run --rm -v </span><span style="color: #81A1C1">$</span><span style="color: #D8DEE9">PWD</span><span style="color: #D8DEE9FF">:/home/app scrumdog-local sample.jql</span></div></code></div></pre>
</content>
</entry>
<entry>
<title>Quality > Speed > Scope</title>
<link href="https://michaelheap.com/quality-speed-scope/"/>
<updated>2025-02-23T13:56:47Z</updated>
<id>https://michaelheap.com/quality-speed-scope/</id>
<content type="html"><p>In the world of product development, prioritization is everything. Should we optimize for quality, ship as fast as possible, or build as many features as we can?</p>
<ol>
<li>
<p><strong>Never compromise on quality.</strong><br />
A product that doesn’t work well, isn’t reliable, or doesn’t meet user expectations will ultimately fail. It doesn't matter how quickly you ship it or how many features it has. Users won’t stick around for a bad experience. A great product that solves a real problem with elegance and consistency will always win in the long run.</p>
</li>
<li>
<p><strong>Never compromise on speed.</strong><br />
Perfect is the enemy of good. If you’re not shipping, all the work you've done so far is delivering <em>zero</em> value. It’s easy to get caught up in refining every detail, but a product that never sees the light of day doesn’t help anyone. Shipping fast ensures that you’re constantly learning, iterating, and improving based on real-world feedback.</p>
</li>
<li>
<p><strong>Compromise on scope.</strong><br />
Instead of trying to build everything at once, focus on solving a narrow use case extremely well. Identify the core problem your users face, build the simplest and most effective solution for it, and get it in their hands as soon as possible. A great MVP isn’t about having the most features; it’s about having the right ones.</p>
</li>
</ol>
<p>Then, <strong>iterate relentlessly</strong>. Ship, learn, refine, and ship again. Small, incremental progress compounds over time.</p>
<p>If you follow this approach consistently, you’ll be amazed at what kind of product you can build in just six months. The best products don’t emerge from grand, ambitious plans. They evolve through <em>disciplined execution</em> and <em>continuous improvement</em>.</p>
</content>
</entry>
<entry>
<title>Using AWS credential_process and 1Password</title>
<link href="https://michaelheap.com/aws-credential-helper-1password/"/>
<updated>2025-02-18T17:15:37Z</updated>
<id>https://michaelheap.com/aws-credential-helper-1password/</id>
<content type="html"><p>A while back I read an excellent post from Paul Galow on <a href="https://paulgalow.com/securing-aws-credentials-macos-lastpass">securing AWS credentials with LastPass</a>. I wanted exactly this, but with 1Password instead. Here's how to do it.</p>
<h2 id="configure-1password" tabindex="-1">Configure 1Password</h2>
<p>If you don’t already have <code>op</code> installed, you’ll need to install the <a href="https://developer.1password.com/docs/cli/">op CLI</a> from the 1Password website.</p>
<p>Once that’s done, create a new vault for storing secrets that are accessible from the CLI:</p>
<pre class="shiki nord" style="background-color: #2e3440ff; color: #d8dee9ff"><div class="language-id">bash</div><div class="code-container"><code><div class="line"><span style="color: #D8DEE9FF">op vault create CLI</span></div></code></div></pre>
<p>Then create a new item in that vault, making sure to replace <code>XXX</code> with your actual access key and secret:</p>
<pre class="shiki nord" style="background-color: #2e3440ff; color: #d8dee9ff"><div class="language-id">bash</div><div class="code-container"><code><div class="line"><span style="color: #D8DEE9FF"> op item create --category Password --vault CLI --title AWSCredentials ACCESS_KEY=XXX SECRET_KEY=XXX</span></div></code></div></pre>
<blockquote>
<p>The above command is prefixed with a space so that it is not written to your shell history if <code>HIST_IGNORE_SPACE</code> is enabled</p>
</blockquote>
<h2 id="create-the-credential_process" tabindex="-1">Create the credential_process</h2>
<p>The AWS CLI has the ability to read credentials from a process rather than a static configuration file. It expects that the credential process will return a JSON document containing <code>AccessKeyid</code> and <code>SecretAccessKey</code>.</p>
<p>Copy and paste the following in to a terminal to create the credentials script:</p>
<pre class="shiki nord" style="background-color: #2e3440ff; color: #d8dee9ff"><div class="language-id">bash</div><div class="code-container"><code><div class="line"><span style="color: #81A1C1">export</span><span style="color: #D8DEE9FF"> AWS_HELPER=</span><span style="color: #ECEFF4">"</span><span style="color: #81A1C1">$</span><span style="color: #D8DEE9">HOME</span><span style="color: #A3BE8C">/bin/aws-1password</span><span style="color: #ECEFF4">"</span></div><div class="line"><span style="color: #D8DEE9FF">mkdir -p </span><span style="color: #81A1C1">$</span><span style="color: #D8DEE9">HOME</span><span style="color: #D8DEE9FF">/bin</span></div><div class="line"></div><div class="line"><span style="color: #88C0D0">echo</span><span style="color: #D8DEE9FF"> </span><span style="color: #ECEFF4">'</span><span style="color: #A3BE8C">#!/usr/bin/env bash</span></div><div class="line"><span style="color: #A3BE8C">readonly opVault="CLI"</span></div><div class="line"><span style="color: #A3BE8C">readonly opEntry="AWSCredentials"</span></div><div class="line"><span style="color: #A3BE8C">readonly accessKeyId=$(op read "op://$opVault/$opEntry/ACCESS_KEY")</span></div><div class="line"><span style="color: #A3BE8C">readonly secretAccessKey=$(op read "op://$opVault/$opEntry/SECRET_KEY")</span></div><div class="line"></div><div class="line"><span style="color: #A3BE8C"># Create JSON object that AWS CLI expects</span></div><div class="line"><span style="color: #A3BE8C">jq -n \</span></div><div class="line"><span style="color: #A3BE8C"> --arg accessKeyId "$accessKeyId" \</span></div><div class="line"><span style="color: #A3BE8C"> --arg secretAccessKey "$secretAccessKey" \</span></div><div class="line"><span style="color: #A3BE8C"> ".Version = 1</span></div><div class="line"><span style="color: #A3BE8C"> | .AccessKeyId = \$accessKeyId</span></div><div class="line"><span style="color: #A3BE8C"> | .SecretAccessKey = \$secretAccessKey"</span><span style="color: #ECEFF4">'</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">&gt;</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">$</span><span style="color: #D8DEE9">AWS_HELPER</span><span style="color: #81A1C1">;</span></div><div class="line"></div><div class="line"><span style="color: #D8DEE9FF">chmod +x </span><span style="color: #81A1C1">$</span><span style="color: #D8DEE9">AWS_HELPER</span></div><div class="line"></div></code></div></pre>
<h2 id="configure-aws" tabindex="-1">Configure AWS</h2>
<p>The final thing to do is to create an AWS config file that uses that script to fetch authentication credentials:</p>
<pre class="shiki nord" style="background-color: #2e3440ff; color: #d8dee9ff"><div class="language-id">bash</div><div class="code-container"><code><div class="line"><span style="color: #D8DEE9FF">mkdir -p </span><span style="color: #81A1C1">$</span><span style="color: #D8DEE9">HOME</span><span style="color: #D8DEE9FF">/.aws</span></div><div class="line"><span style="color: #88C0D0">echo</span><span style="color: #D8DEE9FF"> </span><span style="color: #ECEFF4">"</span><span style="color: #A3BE8C">[default]</span></div><div class="line"><span style="color: #A3BE8C">credential_process = </span><span style="color: #81A1C1">$</span><span style="color: #D8DEE9">AWS_HELPER</span><span style="color: #ECEFF4">"</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">&gt;</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">$</span><span style="color: #D8DEE9">HOME</span><span style="color: #D8DEE9FF">/.aws/config</span></div></code></div></pre>
<p>To check if it worked, run <code>aws configure list</code>. You should be prompted for your 1Password credentials to unlock the vault.</p>
<pre class="shiki nord" style="background-color: #2e3440ff; color: #d8dee9ff"><div class="language-id">bash</div><div class="code-container"><code><div class="line"><span style="color: #D8DEE9FF">$ aws configure list</span></div><div class="line"><span style="color: #D8DEE9FF"> Name Value Type Location</span></div><div class="line"><span style="color: #D8DEE9FF"> ---- ----- ---- --------</span></div><div class="line"><span style="color: #D8DEE9FF"> profile </span><span style="color: #81A1C1">&lt;</span><span style="color: #D8DEE9FF">not set</span><span style="color: #81A1C1">&gt;</span><span style="color: #D8DEE9FF"> None None</span></div><div class="line"><span style="color: #D8DEE9FF">access_key </span><span style="color: #81A1C1">****************</span><span style="color: #D8DEE9FF">XYZ custom-process</span></div><div class="line"><span style="color: #D8DEE9FF">secret_key </span><span style="color: #81A1C1">****************</span><span style="color: #D8DEE9FF">XYZ custom-process</span></div><div class="line"><span style="color: #D8DEE9FF"> region </span><span style="color: #81A1C1">&lt;</span><span style="color: #D8DEE9FF">not set</span><span style="color: #81A1C1">&gt;</span><span style="color: #D8DEE9FF"> None None</span></div></code></div></pre>
</content>
</entry>
<entry>
<title>Slack channels are free</title>
<link href="https://michaelheap.com/slack-channels-are-free/"/>
<updated>2025-02-12T20:51:45Z</updated>
<id>https://michaelheap.com/slack-channels-are-free/</id>
<content type="html"><p>“Should we create a new Slack channel for this?”</p>
<p>I’ve not seen a topic as divisive as this since the days of tabs vs spaces (tl;dr: <a href="https://adamtuttle.codes/blog/2021/tabs-vs-spaces-its-an-accessibility-issue/">use tabs</a>). Whenever there’s a suggestion to create a new channel, there are fervent discussions about why it’s a good/bad idea, with strong feelings on both sides.</p>
<p>I’m firmly in camp “as many channels as needed, sometimes more channels than employees”. Once you hit a certain size, having a few firehose channels for everything means that 80%+ of the messages in a channel are irrelevant to the majority of the audience.</p>
<p>Instead, spin up a new channel for each area of focus. This allows you to keep your working groups small (12 rather than 1200 people) and lets people focus on the channels that are important day to day.</p>
<p><strong>Extra credit:</strong> If you really want to help people focus, ensure that you use threads to prevent unread notifications. If you see a topic that you want to follow, click the “get notified about new replies” option in the context menu for the top level message.</p>
<h2 id="the-power-of-focused-channels" tabindex="-1">The Power of Focused Channels</h2>
<p>Using specific channels for each topic has immediate benefits. Scoping conversations down to a topic rather than it being a free for all provides consumers with:</p>
<ol>
<li>Notification control</li>
<li>Easier searching</li>
<li>Single purpose channels (e.g. discussion, alerts etc)</li>
<li>The correct audience</li>
</ol>
<p>Even Slack recommends using more, topic specific channels:</p>
<blockquote>
<p>When you set up multiple topic- and project-specific channels, groups can focus their discussions among smaller numbers of people, helping them to align and move faster. And having lots of specific channels means that each person can participate in fewer channels, because only a handful of them will be necessary for their daily work.<br />
<br />
via <a href="https://slack.com/intl/en-gb/resources/using-slack/how-to-organize-your-slack-channels">https://slack.com/intl/en-gb/resources/using-slack/how-to-organize-your-slack-channels</a></p>
</blockquote>
<h3 id="fewer-unnecessary-notifications" tabindex="-1">Fewer Unnecessary Notifications</h3>
<p>Being able to manage notifications is my favourite reason to use more Slack channels. Not all messages are created equally, but when they land in the same channel I have to manually read and filter the messages in my head.</p>
<p>For concrete example, imagine a company that has a single <code>#general</code> channel. It’s a place where company updates are shared, but also a place where people chat about the weekend. As a consumer I have to follow the channel in case I miss anything but the signal to noise ratio is low.</p>
<p>Now imagine that this channel is split in to <code>#announcements</code> and <code>#watercooler</code>. I can safely mute or leave the water cooler channel while paying close attention to <code>#announcements</code>.</p>
<h3 id="find-what-you-need%E2%80%94fast" tabindex="-1">Find What You Need—Fast</h3>
<p>Once you have separate channels, searching becomes much easier. All you need is a keyword or two and the channel that you recall seeing the conversation in.</p>
<p>When working on some new documentation, I searched in the <code>#support</code> channel for “flubjam”, the name of the product I was working on (no, thats not the real product name). This surfaced all the issues customers were having with the product that I can now weave in to the updated documentation.</p>
<h3 id="dedicated-channels%2C-better-workflows" tabindex="-1">Dedicated Channels, Better Workflows</h3>
<p>If we take the notification control benefit and supercharge it, we get to “single use channels”. Slack can do so much more than be a chat room for your team.</p>
<p>A <em>large</em> portion of my Slack usage is monitoring events:</p>
<ul>
<li>A new post on Stack Overflow with a specific tag goes to <code>#stack-overflow</code></li>
<li>New GitHub issues and releases for the flubjam project are piped in to <code>#notify-flubjam</code></li>
<li>All GitHub activity for the flubjam project is piped in to <code>#firehose-flubjam</code></li>
<li>Posts elsewhere on Slack that get a floppy disk icon are cross posted to <code>#zmeta-saved</code></li>
</ul>
<p>Creating single use channels gives people the information that they need, when they need it.</p>
<h3 id="reach-the-right-people%2C-every-time" tabindex="-1">Reach the Right People, Every Time</h3>
<p>Finally, using single purpose channels means that your audience is probably the one you’re looking for. Instead of blasting your question to 1200 people in a general channel, you can ask a targeted group your specific question.</p>
<p>I find this much easier than trying to figure out who’s involved in a project to set up a five way DM (which will inevitably miss out a key person who then feels slighted that I forgot them).</p>
<h2 id="finding-the-right-channels" tabindex="-1">Finding the Right Channels</h2>
<p>So if focused channels are so great, why don’t we all do it? There’s an easy answer:</p>
<p><strong>Finding new channels is hard</strong>.</p>
<p>There are a couple of ways to solve this problem. The most effective ones I’ve seen are:</p>
<ul>
<li>Have a channel that announces new channels (very meta!)</li>
<li>Set a reminder to check for new channels weekly using the Channels-&gt;Browse Channels option. Usually there haven’t been that many created in the last 7 days.</li>
<li>You can rely on people to invite you. Usually they won’t invite you directly. Instead, they’ll mention you by your Slack username, realise that you’re not there and then click “invite”. This means you only join channels when there is something for you to actively participate in</li>
<li>Finally, update your onboarding documentation with links to relevant channels (you do have onboarding documentation, right? 😁)</li>
</ul>
<h2 id="where-should-you-post%3F" tabindex="-1">Where Should You Post?</h2>
<p>Ok, so you have lots of tightly scoped channels. You’ve joined the channels relevant to your day to day work, and now you have a question about the “flubjam” project.</p>
<p>Do you post in <code>#team-product</code>, <code>#flubjam-is-cool</code> or <code>#general</code>?</p>
<p>Channel naming schemes can help with this. One of the patterns I’ve seen at a few different places now is to have an “<code>#ask-team-name</code>” channel that is public. Have a question for finance? <code>#ask-finance</code>! Curious about the product roadmap? <code>#ask-product</code>!</p>
<p>If you’re small enough that you have 1-3 products, keeping a single <code>#ask-product</code> channel makes sense.</p>
<p>As you expand your product offering, you may want to expand to “<code>#ask-product-name</code>” style channels, such as “<code>#ask-dev-portal</code>”. This allows the <a href="https://medium.com/design-bootcamp/the-power-of-the-product-triad-0e76801a384d">product triad</a> to answer the questions as a team rather than the questions going to a specific department all the time.</p>
<p>Finally, each product is made up of various initiatives and deliverables. You likely have a “<code>#private-team-a</code>” channel that you use to discuss ongoing projects, but I encourage you to bring that conversation in to “<code>#project-name</code>” channels. Not only does it give you a single place to read for any given project, it invites collaboration with other teams who may have suggestions based on their experience. These channels are short lived and should be archived as soon as the project is completed.</p>
<p>To recap:</p>
<ul>
<li><code>#ask-team-name</code> to get started</li>
<li><code>#ask-product-name</code> as the team grows</li>
<li><code>#project-name</code> to stop conversation sprawl and build a culture of collaboration</li>
</ul>
<p>With concrete examples:</p>
<ul>
<li><code>#ask-product</code> for general product questions</li>
<li><code>#ask-flubjam</code> for all flubjam related questions</li>
<li><code>#project-flubjam-capacitors</code> is a short term channel to talk about implementing capacitors in flubjam</li>
</ul>
<h2 id="create-that-slack-channel" tabindex="-1">Create That Slack Channel</h2>
<p>Focused channels improve notification control, make searching easier, ensure the right audience sees your messages, and unlock powerful workflows beyond just chat. While discoverability can be a challenge, a few simple strategies—like clear naming conventions—can make it easy for people to find the right spaces.</p>
<p>At the end of the day, good Slack hygiene isn’t about having fewer channels—it’s about having the <strong>right</strong> channels.</p>
<h2 id="update%3A-super-bonus-content" tabindex="-1">Update: Super Bonus Content</h2>
<p>I shared a preview of this post with some colleagues and <a href="https://www.linkedin.com/in/jasonhnaustin">Jason</a> had some great feedback on how to cope with channel sprawl:</p>
<ul>
<li><strong>Housecleaning built-in:</strong> If you have a topic that is not meant to be long-lived, a convention that flags the channel (usually <code>#temp-</code>) signifies that when the topic is resolved, the channel dies. This helps address the channel sprawl problem, and intentionally redirects folks back to long-lived comms channels, vs tactical problem solving.</li>
<li><strong>Join many, star few:</strong> the likelihood that you keep up with the conversation in every channel you join, in a &quot;Slack channels are free&quot; environment, is pretty much zero. Missing critical notifications on people you are working with regularly is a big risk to productivity, and job security in some cases. Star key channels, and mark your closest collaborators and reporting chain as &quot;VIP&quot;, so they are on a short-list to focus your attention.</li>
<li><strong>Ruthless prioritization on &quot;Leave Channel&quot;:</strong> if you're not contributing or engaging in a channel, leave it immediately. If you're mentioned, you'll be notified, and you can rejoin.</li>
</ul>
<p>As someone that left <em>175</em> Slack channels in his January cleanup, that last point on ruthless priorization is key. I'm already back in some of them, but I know it's because I'm actively engaged rather than through inertia.</p>
</content>
</entry>
<entry>
<title>Excellent Pull Request Reviews</title>
<link href="https://michaelheap.com/pull-request-reviews/"/>
<updated>2025-02-05T03:55:49Z</updated>
<id>https://michaelheap.com/pull-request-reviews/</id>
<content type="html"><p>Pull request reviews are a critical part of building high-quality products, but too often, they become a rubber-stamping exercise—skim, “LGTM,” approve. This kind of review can lead to broken code, unclear documentation, and missed opportunities for improvement.</p>
<p>A great PR review isn’t just about glancing at the code; it’s about <strong>running the changes</strong>, <strong>thinking critically about what’s missing</strong>, and <strong>providing actionable feedback</strong>.</p>
<h2 id="run-the-thing" tabindex="-1">Run the thing</h2>
<p>If I could only have one rule for reviewing pull requests, it would be “did you run it?”. Far too often I see people skimming over the content, giving it a “LGTM” and pressing approve.</p>
<p>This happens <em>everywhere</em>. My most recent example is a project that is migrating documentation from one platform to another. All of the content already exists, so it’s just about reformatting and publishing it elsewhere, right?</p>
<p>No. No, no, <em>no</em>. You see, a lot of the documentation doesn’t work. Maybe it didn’t work when it was originally published (“LGTM!”). Maybe it did, and it’s rotted over time and no longer works.</p>
<p>So now we have a new definition of done for the migration work. Before you submit a pull request for review, you need to work through the content from top to bottom and ensure it works. Then your reviewer has to do the same.</p>
<p>Having two people test the same thing ensures that the documentation is clear, isn’t missing implicit knowledge and most of all, that it works.</p>
<p>If you’re building software rather than writing docs, good news, the same process works! Having two people <em>use</em> the software makes sure that it meets the requirements, uncovers edge cases and generally helps build better products.</p>
<h2 id="what%E2%80%99s-missing%3F" tabindex="-1">What’s missing?</h2>
<p>Looking at a pull request and saying “LGTM” is only half of the review. You also need to think about what you’re <strong>not</strong> looking at. What should be there, but isn’t?</p>
<p>Are there any missing tests? Missing docs? How about example configuration or usage examples for the feature that you just added?</p>
<p>Spotting what isn’t there is one of the hardest things to learn when reviewing PRs, but it’s also one of the most impactful. Being able to say “this looks great, but I think we should add a test for when X” helps build better products.</p>
<h2 id="is-this-blocking%3F" tabindex="-1">Is this blocking?</h2>
<p>As you add more comments to a pull request (and you <em>will</em> add more comments if you’re actually running what’s there and thinking about what’s missing), it can be hard to understand which pieces of feedback are opinions, and which require changes.</p>
<p>To make it obvious for the original author, I recommend prefixing feedback if it doesn’t need to be actioned immediately.</p>
<ul>
<li><em>No prefix</em>: This is important feedback for the current PR</li>
<li><em>future</em>: Could we refactor this into something that’s reusable?</li>
<li><em>nit</em>: I find it easier to parse when using early returns rather than else blocks</li>
</ul>
<p>By following this pattern of feedback you can ensure that you provide all the context you have in the PR without worrying about preventing the original author from shipping.</p>
<p>I’ve been on the receiving end of this type of feedback, and it’s been great to learn what <em>could</em> be done given more time. I don’t always go back and fix the <em>nit</em> comments, but it always makes me think about what I can do differently next time.</p>
<h2 id="do-the-work" tabindex="-1">Do the work</h2>
<p>Last, but by no means least, do the work yourself. I don’t mean “pull down the branch and make changes” (though in at least one of my teams that <em>is</em> the default operating model and it works well).</p>
<p>I mean use GitHub’s suggestions macro when reviewing a pull request. If you have a suggestion how to improve something, instead of trying to explain it in text you can edit the lines yourself using the following format:</p>
<pre class="shiki nord" style="background-color: #2e3440ff; color: #d8dee9ff"><div class="language-id">bash</div><div class="code-container"><code><div class="line"><span style="color: #ECEFF4">```</span><span style="color: #A3BE8C">suggestion</span></div><div class="line"><span style="color: #A3BE8C">This line will be suggested instead of the one that you selected</span></div><div class="line"><span style="color: #ECEFF4">```</span></div></code></div></pre>
<p>Not only is making the actual change easier to parse for the original author, they can quickly apply any changes they agree with by clicking “add to batch” in the web UI.</p>
<h2 id="go-forth-and-review" tabindex="-1">Go forth and review</h2>
<p>By making sure the code works, thinking critically about what’s missing, and clearly distinguishing between blocking and non-blocking comments, reviewers can significantly improve the quality of the product. And when possible, making direct suggestions instead of just pointing out issues can streamline the process for everyone.</p>
<p>A great review isn’t just about catching issues. It's about fostering collaboration, improving maintainability, and ultimately, building better products. So next time you review a PR, take the time to do it right. Your team (and your future self) will thank you.</p>
</content>
</entry>
<entry>
<title>Run go test -tags in VSCode</title>
<link href="https://michaelheap.com/go-test-tags/"/>
<updated>2025-01-10T13:01:15Z</updated>
<id>https://michaelheap.com/go-test-tags/</id>
<content type="html"><p>I've been writing more Go recently, and the <a href="https://code.visualstudio.com/docs/languages/go#_test">Go extension</a> for VSCode provides an awesome <em>Run Test</em> button inline.</p>
<p>In many of our projects, we have a combination of unit and integration tests. To keep <code>go test</code> quick, the integration tests have an <code>integration</code> tag enabled. This means that the <em>Run Test</em> button doesn't work for integration tests.</p>
<p>To make them work, add the following to <code>.vscode/settings.json</code> in your project:</p>
<pre class="shiki nord" style="background-color: #2e3440ff; color: #d8dee9ff"><div class="language-id">json</div><div class="code-container"><code><div class="line"><span style="color: #ECEFF4">{</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #ECEFF4">"</span><span style="color: #8FBCBB">go.testFlags</span><span style="color: #ECEFF4">"</span><span style="color: #ECEFF4">:</span><span style="color: #D8DEE9FF"> </span><span style="color: #ECEFF4">[</span><span style="color: #ECEFF4">"</span><span style="color: #A3BE8C">-v</span><span style="color: #ECEFF4">"</span><span style="color: #ECEFF4">],</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #ECEFF4">"</span><span style="color: #8FBCBB">go.testTags</span><span style="color: #ECEFF4">"</span><span style="color: #ECEFF4">:</span><span style="color: #D8DEE9FF"> </span><span style="color: #ECEFF4">"</span><span style="color: #A3BE8C">integration</span><span style="color: #ECEFF4">"</span></div><div class="line"><span style="color: #ECEFF4">}</span></div></code></div></pre>
</content>
</entry>
<entry>
<title>Flatten nested foreach loops with Terraform</title>
<link href="https://michaelheap.com/terraform-flatten-nested-loops/"/>
<updated>2024-09-12T09:26:22Z</updated>
<id>https://michaelheap.com/terraform-flatten-nested-loops/</id>
<content type="html"><p>Given a list of identifiers and an associated list of values, here's how to flatten them all down to a single list of unique values.</p>
<pre class="shiki nord" style="background-color: #2e3440ff; color: #d8dee9ff"><div class="language-id">hcl</div><div class="code-container"><code><div class="line"><span style="color: #616E88"># Define your input data. We'll convert this in to a list of ${name}-${version} values</span></div><div class="line"><span style="color: #81A1C1">variable</span><span style="color: #A3BE8C"> "ai_providers"</span><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">type</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> list(object({</span></div><div class="line"><span style="color: #D8DEE9FF"> name = string</span></div><div class="line"><span style="color: #D8DEE9FF"> versions = list(string)</span></div><div class="line"><span style="color: #D8DEE9FF"> }))</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">default</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> [</span></div><div class="line"><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> name = </span><span style="color: #A3BE8C">"OpenAI"</span></div><div class="line"><span style="color: #D8DEE9FF"> versions = [</span><span style="color: #A3BE8C">"gpt-3.5-turbo"</span><span style="color: #D8DEE9FF">, </span><span style="color: #A3BE8C">"gpt-4o"</span><span style="color: #D8DEE9FF">]</span></div><div class="line"><span style="color: #D8DEE9FF"> },</span></div><div class="line"><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">name</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"Anthropic"</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">versions</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> [</span><span style="color: #A3BE8C">"Opus"</span><span style="color: #D8DEE9FF">, </span><span style="color: #A3BE8C">"Sonnet"</span><span style="color: #D8DEE9FF">, </span><span style="color: #A3BE8C">"Haiku"</span><span style="color: #D8DEE9FF">]</span></div><div class="line"><span style="color: #D8DEE9FF"> },</span></div><div class="line"><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">name</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"Mistral"</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">versions</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> [</span><span style="color: #A3BE8C">"7B"</span><span style="color: #D8DEE9FF">, </span><span style="color: #A3BE8C">"8x7B"</span><span style="color: #D8DEE9FF">, </span><span style="color: #A3BE8C">"8x22B"</span><span style="color: #D8DEE9FF">]</span></div><div class="line"><span style="color: #D8DEE9FF"> },</span></div><div class="line"><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">name</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> </span><span style="color: #A3BE8C">"llama3"</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">versions</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> [</span><span style="color: #A3BE8C">"8B"</span><span style="color: #D8DEE9FF">, </span><span style="color: #A3BE8C">"70B"</span><span style="color: #D8DEE9FF">]</span></div><div class="line"><span style="color: #D8DEE9FF"> },</span></div><div class="line"><span style="color: #D8DEE9FF"> ]</span></div><div class="line"><span style="color: #D8DEE9FF">}</span></div><div class="line"></div><div class="line"><span style="color: #616E88">## Flatmap using nested loops and flatten()</span></div><div class="line"><span style="color: #81A1C1">locals</span><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">ai_providers_flattened</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> flatten([</span></div><div class="line"><span style="color: #D8DEE9FF"> for provider in var</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">ai_providers : [</span></div><div class="line"><span style="color: #D8DEE9FF"> for version in provider</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">versions : {</span></div><div class="line"><span style="color: #D8DEE9FF"> version = version</span></div><div class="line"><span style="color: #D8DEE9FF"> provider = provider</span></div><div class="line"><span style="color: #D8DEE9FF"> key = </span><span style="color: #A3BE8C">"</span><span style="color: #81A1C1">${provider</span><span style="color: #D8DEE9FF">.name</span><span style="color: #81A1C1">}</span><span style="color: #A3BE8C">-</span><span style="color: #81A1C1">${version}</span><span style="color: #A3BE8C">"</span></div><div class="line"><span style="color: #D8DEE9FF"> }</span></div><div class="line"><span style="color: #D8DEE9FF"> ]</span></div><div class="line"><span style="color: #D8DEE9FF"> ])</span></div><div class="line"><span style="color: #D8DEE9FF">}</span></div><div class="line"></div><div class="line"><span style="color: #616E88"># Iterate over the flattened list to create resources</span></div><div class="line"><span style="color: #81A1C1">resource</span><span style="color: #A3BE8C"> "demo_foo" "my_resource"</span><span style="color: #D8DEE9FF"> {</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">for_each</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> { for api in local.ai_providers_flattened : api.key =&gt; api }</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">name</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> each</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">value</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">provider</span></div><div class="line"><span style="color: #D8DEE9FF"> </span><span style="color: #D8DEE9">version</span><span style="color: #D8DEE9FF"> </span><span style="color: #81A1C1">=</span><span style="color: #D8DEE9FF"> each</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">value</span><span style="color: #B48EAD">.</span><span style="color: #D8DEE9FF">version</span></div><div class="line"><span style="color: #D8DEE9FF">}</span></div></code></div></pre>
<p>The <code>for_each = { for api in local.ai_providers_flattened : api.key =&gt; api }</code> line confused me for a while. This line is used to identify the resource later as <code>my_resource</code> is now a map of objects rather than a single object. Let's walk through it step by step</p>
<ul>
<li><code>demo_foo.my_resource</code> is a map of objects</li>
<li><code>demo_foo.my_resource[api.key]</code> points to a single resource</li>
<li><code>api.key</code> is defined as <code>&quot;${provider.name}-${version}&quot;</code></li>
<li>Details for the <code>demo_foo</code> resource for OpenAI gpt-4o is available at <code>demo_foo.my_resource[&quot;OpenAI-gpt-4o&quot;]</code></li>
</ul>
</content>
</entry>
</feed>