Skip to content

Add home-unite-us production service and adopt its ECR repository - #187

Merged
ale210 merged 2 commits into
mainfrom
166-home-unite-us-prod-service
Aug 30, 2026
Merged

Add home-unite-us production service and adopt its ECR repository#187
ale210 merged 2 commits into
mainfrom
166-home-unite-us-prod-service

Conversation

@ale210

@ale210 ale210 commented Aug 30, 2026

Copy link
Copy Markdown
Member

Second of three PRs for #166, following #186 which imported the production Cognito pool.

What this does

  • Adds environment-prod.tf, standing up the production stack through modules/container with the correct naming (home-unite-us-fullstack-prod).
  • Adopts the existing homeuniteus ECR repository as a plain aws_ecr_repository resource in ecr.tf, with an import block. It is not routed through modules/ecr because that module derives the repository name from the project name, and this repository's name is referenced by the running task definition and by the project's own CI.

Nothing here takes production traffic

The new listener rule is at priority 17; the live unmanaged rule is at 16, and the load balancer evaluates the lower number first. So this stack comes up alongside production, serving nothing, and can be verified before anything moves. Removing rule 16 is the cutover and belongs to the next PR, where the rollback is simply putting it back.

Choices worth reviewing

launch_type = "ec2", not Fargate. The unmanaged service runs Fargate; the module's own guidance is to use EC2 unless there is a reason not to. The cluster has room — two container instances with roughly 6 GB free each, against a 512 MB task.

The image is the tag production already serves, 2dc864ae50.20250220-172003, pulled from the adopted repository. It is date-and-SHA stamped rather than a moving tag, so it cannot drift the way the mutable :dev/:qa tags elsewhere in this account have. Building a fresh production image is being handled separately with the HomeUniteUs team.

Every environment variable is declared explicitly. The image carries a baked-in .env with the same keys, and environment variables take precedence over it — declaring the full set is what ends production's dependence on a file that exists only inside a container. These point at the production Cognito pool from #186, not the QA pool.

Two of those values look wrong and are not:

  • HUU_ENVIRONMENT = "qa". The image is built with HUU_TARGET_ENV=qa, which bakes /var/www/qa.homeunite.us and /var/log/qa.homeunite.us into it. It has to stay qa while this tag is in use.
  • hostname = "qa.homeunite.us" with www.homeunite.us as an additional host. The frontend bundle is compiled with VITE_HUU_API_BASE_URL=https://qa.homeunite.us/api, and the backend's ROOT_URL has to agree or the OAuth redirect breaks. This mirrors the live rule at priority 16, which already serves both names. Moving the application onto www properly needs a rebuilt image.

COGNITO_ACCESS_ID and COGNITO_ACCESS_KEY are set to empty strings so boto3 falls back to the task role, rather than the static access key of the homeuniteus-app IAM user that the baked .env supplies. Retiring that user and its key is part of the cleanup PR.

DATABASE_URL is assembled into an SSM parameter, matching how QA passes it. The production database already exists inside the shared instance and is not created here; its password is read from the homeuniteus-rds-password secret left behind by the Terragrunt stack.

What to check in the plan

One import (the ECR repository, which should be tag-only changes), and creates for the new service, task definition, target group, listener rule, log group, security group, task role and policy, and the SSM parameter. No existing resource should be modified or destroyed beyond the ECR tag.

Verification

  • terraform validate passes on 1.16.0. terraform fmt is clean on the added blocks; the pre-existing ecr_fullstack block is left unformatted rather than mixing unrelated churn into this PR.
  • After this merges, confirm the new service reaches a steady 1/1 and its target group shows a healthy target, and that www.homeunite.us and qa.homeunite.us are still served by the old stack — nothing should change for users in this PR.

Not in this PR

The cutover, and deleting the legacy service, task definition family, target group, security group, log group, homeuniteus-app IAM user and key, the two IAM policies, and the four redundant secrets.

@github-actions

github-actions Bot commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

Terraform plan in terraform
With backend config files: terraform/prod.backend.tfvars

Plan: 1 to import, 14 to add, 1 to change, 0 to destroy.
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+   create
!~  update in-place

Terraform will perform the following actions:

  # module.home-unite-us.aws_ecr_repository.homeuniteus_prod will be updated in-place
  # (imported from "homeuniteus")
!~  resource "aws_ecr_repository" "homeuniteus_prod" {
        arn                  = "arn:aws:ecr:us-west-2:035866691871:repository/homeuniteus"
        id                   = "homeuniteus"
        image_tag_mutability = "MUTABLE"
        name                 = "homeuniteus"
        region               = "us-west-2"
        registry_id          = "************"
        repository_url       = "035866691871.dkr.ecr.us-west-2.amazonaws.com/homeuniteus"
!~      tags                 = {
-           "Organization" = "Hack for LA" -> null
        }
!~      tags_all             = {
-           "Organization" = "Hack for LA" -> null
+           "managed-by"   = "terraform-incubator"
        }

        encryption_configuration {
            encryption_type = "AES256"
            kms_key         = null
        }

        image_scanning_configuration {
            scan_on_push = true
        }
    }

  # module.home-unite-us.aws_iam_role_policy_attachment.prod_cognito will be created
+   resource "aws_iam_role_policy_attachment" "prod_cognito" {
+       id         = (known after apply)
+       policy_arn = "arn:aws:iam::aws:policy/AmazonCognitoPowerUser"
+       role       = "ecs-container-home-unite-us-fullstack-prod"
    }

  # module.home-unite-us.module.db_url_prod.aws_ssm_parameter.this will be created
+   resource "aws_ssm_parameter" "this" {
+       arn            = (known after apply)
+       data_type      = (known after apply)
+       has_value_wo   = (known after apply)
+       id             = (known after apply)
+       insecure_value = (known after apply)
+       key_id         = (known after apply)
+       name           = "/home-unite-us/fullstack-prod-database_url"
+       region         = "us-west-2"
+       tags           = {
+           "project" = "home-unite-us"
        }
+       tags_all       = {
+           "managed-by" = "terraform-incubator"
+           "project"    = "home-unite-us"
        }
+       tier           = (known after apply)
+       type           = "SecureString"
+       value          = (sensitive value)
+       value_wo       = (write-only attribute)
+       version        = (known after apply)
    }

  # module.home-unite-us.module.db_url_prod.random_password.password will be created
+   resource "random_password" "password" {
+       bcrypt_hash      = (sensitive value)
+       id               = (known after apply)
+       length           = 48
+       lower            = true
+       min_lower        = 0
+       min_numeric      = 0
+       min_special      = 0
+       min_upper        = 0
+       number           = true
+       numeric          = true
+       override_special = "!#$-"
+       result           = (sensitive value)
+       special          = true
+       upper            = true
    }

  # module.home-unite-us.module.prod_service.aws_cloudwatch_log_group.this will be created
+   resource "aws_cloudwatch_log_group" "this" {
+       arn                         = (known after apply)
+       deletion_protection_enabled = (known after apply)
+       id                          = (known after apply)
+       log_group_class             = (known after apply)
+       name                        = "/ecs/home-unite-us-fullstack-prod"
+       name_prefix                 = (known after apply)
+       region                      = "us-west-2"
+       retention_in_days           = 0
+       skip_destroy                = false
+       tags_all                    = {
+           "managed-by" = "terraform-incubator"
        }
    }

  # module.home-unite-us.module.prod_service.aws_ecs_service.fargate[0] will be created
+   resource "aws_ecs_service" "fargate" {
+       arn                                = (known after apply)
+       availability_zone_rebalancing      = (known after apply)
+       cluster                            = "incubator-prod"
+       deployment_maximum_percent         = 200
+       deployment_minimum_healthy_percent = 100
+       desired_count                      = 1
+       enable_ecs_managed_tags            = false
+       enable_execute_command             = true
+       iam_role                           = (known after apply)
+       id                                 = (known after apply)
+       launch_type                        = "EC2"
+       name                               = "home-unite-us-fullstack-prod"
+       platform_version                   = (known after apply)
+       region                             = "us-west-2"
+       scheduling_strategy                = "REPLICA"
+       tags_all                           = {
+           "managed-by" = "terraform-incubator"
        }
+       task_definition                    = (known after apply)
+       triggers                           = (known after apply)
+       wait_for_steady_state              = false

+       deployment_configuration (known after apply)

+       load_balancer {
+           container_name   = "home-unite-us-fullstack-prod"
+           container_port   = 80
+           target_group_arn = (known after apply)
#            (1 unchanged attribute hidden)
        }

+       network_configuration {
+           assign_public_ip = false
+           security_groups  = (known after apply)
+           subnets          = [
+               "subnet-03ed55f60a6c28e72",
+               "subnet-089e80a53e1522e28",
            ]
        }

+       ordered_placement_strategy {
+           field = "attribute:ecs.availability-zone"
+           type  = "spread"
        }
+       ordered_placement_strategy {
+           field = "instanceId"
+           type  = "spread"
        }
    }

  # module.home-unite-us.module.prod_service.aws_ecs_task_definition.task will be created
+   resource "aws_ecs_task_definition" "task" {
+       arn                      = (known after apply)
+       arn_without_revision     = (known after apply)
+       container_definitions    = (sensitive value)
+       enable_fault_injection   = (known after apply)
+       execution_role_arn       = "arn:aws:iam::035866691871:role/incubator-prod-ecs-task-role"
+       family                   = "home-unite-us-fullstack-prod"
+       id                       = (known after apply)
+       network_mode             = "awsvpc"
+       region                   = "us-west-2"
+       requires_compatibilities = [
+           "EC2",
        ]
+       revision                 = (known after apply)
+       skip_destroy             = false
+       tags_all                 = {
+           "managed-by" = "terraform-incubator"
        }
+       task_role_arn            = (known after apply)
+       track_latest             = false
    }

  # module.home-unite-us.module.prod_service.aws_iam_policy.container_policy will be created
+   resource "aws_iam_policy" "container_policy" {
+       arn              = (known after apply)
+       attachment_count = (known after apply)
+       id               = (known after apply)
+       name             = "home-unite-us-fullstack-prod-task-policy"
+       name_prefix      = (known after apply)
+       path             = "/"
+       policy           = jsonencode(
            {
+               Statement = [
+                   {
+                       Action   = [
+                           "ssmmessages:CreateControlChannel",
+                           "ssmmessages:CreateDataChannel",
+                           "ssmmessages:OpenControlChannel",
+                           "ssmmessages:OpenDataChannel",
                        ]
+                       Effect   = "Allow"
+                       Resource = "*"
                    },
                ]
+               Version   = "2012-10-17"
            }
        )
+       policy_id        = (known after apply)
+       tags_all         = {
+           "managed-by" = "terraform-incubator"
        }
#        (1 unchanged attribute hidden)
    }

  # module.home-unite-us.module.prod_service.aws_iam_role.instance will be created
+   resource "aws_iam_role" "instance" {
+       arn                   = (known after apply)
+       assume_role_policy    = jsonencode(
            {
+               Statement = [
+                   {
+                       Action    = "sts:AssumeRole"
+                       Effect    = "Allow"
+                       Principal = {
+                           Service = "ecs-tasks.amazonaws.com"
                        }
+                       Sid       = ""
                    },
                ]
+               Version   = "2012-10-17"
            }
        )
+       create_date           = (known after apply)
+       force_detach_policies = false
+       id                    = (known after apply)
+       managed_policy_arns   = (known after apply)
+       max_session_duration  = 3600
+       name                  = "ecs-container-home-unite-us-fullstack-prod"
+       name_prefix           = (known after apply)
+       path                  = "/"
+       tags                  = {
+           "tag-key" = "*********"
        }
+       tags_all              = {
+           "managed-by" = "terraform-incubator"
+           "tag-key"    = "*********"
        }
+       unique_id             = (known after apply)

+       inline_policy (known after apply)
    }

  # module.home-unite-us.module.prod_service.aws_iam_role_policy_attachment.task_policy will be created
+   resource "aws_iam_role_policy_attachment" "task_policy" {
+       id         = (known after apply)
+       policy_arn = (known after apply)
+       role       = "ecs-container-home-unite-us-fullstack-prod"
    }

  # module.home-unite-us.module.prod_service.aws_lb_listener_rule.static will be created
+   resource "aws_lb_listener_rule" "static" {
+       arn          = (known after apply)
+       id           = (known after apply)
+       listener_arn = "arn:aws:elasticloadbalancing:us-west-2:035866691871:listener/app/incubator-prod-lb/7451adf77133ef36/390a225766a4daf3"
+       priority     = 17
+       region       = "us-west-2"
+       tags_all     = {
+           "managed-by" = "terraform-incubator"
        }

+       action {
+           order            = (known after apply)
+           target_group_arn = (known after apply)
+           type             = "forward"
        }

+       condition {
+           host_header {
+               regex_values = []
+               values       = [
+                   "qa.homeunite.us",
+                   "www.homeunite.us",
                ]
            }
        }
+       condition {
+           path_pattern {
+               regex_values = []
+               values       = [
+                   "/*",
                ]
            }
        }
    }

  # module.home-unite-us.module.prod_service.aws_lb_target_group.this will be created
+   resource "aws_lb_target_group" "this" {
+       arn                                = (known after apply)
+       arn_suffix                         = (known after apply)
+       connection_termination             = (known after apply)
+       deregistration_delay               = "10"
+       id                                 = (known after apply)
+       ip_address_type                    = (known after apply)
+       lambda_multi_value_headers_enabled = false
+       load_balancer_arns                 = (known after apply)
+       load_balancing_algorithm_type      = (known after apply)
+       load_balancing_anomaly_mitigation  = (known after apply)
+       load_balancing_cross_zone_enabled  = (known after apply)
+       name                               = "home-unite-us-fullstack-prod-tg"
+       name_prefix                        = (known after apply)
+       port                               = 80
+       preserve_client_ip                 = (known after apply)
+       protocol                           = "HTTP"
+       protocol_version                   = (known after apply)
+       proxy_protocol_v2                  = false
+       region                             = "us-west-2"
+       slow_start                         = 0
+       tags_all                           = {
+           "managed-by" = "terraform-incubator"
        }
+       target_type                        = "ip"
+       vpc_id                             = "*********************"

+       health_check {
+           enabled             = true
+           healthy_threshold   = 3
+           interval            = 30
+           matcher             = "200,400,404"
+           path                = "/"
+           port                = "traffic-port"
+           protocol            = "HTTP"
+           timeout             = (known after apply)
+           unhealthy_threshold = 3
        }

+       stickiness (known after apply)

+       target_failover (known after apply)

+       target_group_health (known after apply)

+       target_health_state {
+           enable_unhealthy_connection_termination = false
+           unhealthy_draining_interval             = 0
        }
    }

  # module.home-unite-us.module.prod_service.aws_security_group.container will be created
+   resource "aws_security_group" "container" {
+       arn                    = (known after apply)
+       description            = "Container home-unite-us-fullstack-prod"
+       egress                 = (known after apply)
+       id                     = (known after apply)
+       ingress                = (known after apply)
+       name                   = "ecs-container-home-unite-us-fullstack-prod"
+       name_prefix            = (known after apply)
+       owner_id               = (known after apply)
+       region                 = "us-west-2"
+       revoke_rules_on_delete = false
+       tags                   = {
+           "Name" = "ecs-container-home-unite-us-fullstack-prod"
        }
+       tags_all               = {
+           "Name"       = "ecs-container-home-unite-us-fullstack-prod"
+           "managed-by" = "terraform-incubator"
        }
+       vpc_id                 = "*********************"
    }

  # module.home-unite-us.module.prod_service.aws_vpc_security_group_egress_rule.allow_all_traffic will be created
+   resource "aws_vpc_security_group_egress_rule" "allow_all_traffic" {
+       arn                    = (known after apply)
+       cidr_ipv4              = "0.0.0.0/0"
+       id                     = (known after apply)
+       ip_protocol            = "-1"
+       region                 = "us-west-2"
+       security_group_id      = (known after apply)
+       security_group_rule_id = (known after apply)
+       tags_all               = {
+           "managed-by" = "terraform-incubator"
        }
    }

  # module.home-unite-us.module.prod_service.aws_vpc_security_group_ingress_rule.container_ingress_port will be created
+   resource "aws_vpc_security_group_ingress_rule" "container_ingress_port" {
+       arn                    = (known after apply)
+       cidr_ipv4              = "10.10.0.0/16"
+       from_port              = 80
+       id                     = (known after apply)
+       ip_protocol            = "tcp"
+       region                 = "us-west-2"
+       security_group_id      = (known after apply)
+       security_group_rule_id = (known after apply)
+       tags_all               = {
+           "managed-by" = "terraform-incubator"
        }
+       to_port                = 80
    }

Plan: 1 to import, 14 to add, 1 to change, 0 to destroy.

✅ Plan applied in Terraform apply (OIDC) #69

@ale210
ale210 merged commit ba93a1e into main Aug 30, 2026
ale210 added a commit that referenced this pull request Aug 30, 2026
…hecks (#188)

Third of four PRs for #166, after #186 (Cognito
import) and #187 (production service). It corrects how the new service
is configured. It still takes no traffic — the cutover is the next PR.

## Production does not use RDS, and never has

#187 set `DATABASE_URL` from the `homeuniteus-rds-password` secret,
following the abandoned Terragrunt state, which describes a Postgres
database on the shared instance. The service came up and its API died
immediately:

```
FATAL: password authentication failed for user "homeuniteus"
ERROR: Application startup failed. Exiting.
```

The credential is not stale — the premise was wrong. The live production
container's `.env` points `DATABASE_URL` at **`/opt/huu/homeuniteus.db`,
a SQLite file on the container's own filesystem**. Nothing has connected
to that Postgres database since January 2025, which is why the password
no longer matches anything.

This PR sets `DATABASE_URL` to the same SQLite path so the new service
matches production, and removes the Postgres wiring: the
`homeuniteus-rds-password` data source, the `aws_db_instance` lookup,
and the `db_url_prod` SSM parameter, which was created holding a value
that never worked.

## What that means, recorded here deliberately

The production task definition declares no volume and no mount point. So
that SQLite file is created empty when a task starts and is destroyed
when the task is replaced — by a deploy, a host drain, or an AZ
rebalance. **The log group holds 743 task generations.** Application
data has never survived any of them.

It goes unnoticed because Cognito holds the user accounts and Cognito is
durable — 96 of them. The site loads, sign-in works, and only the
application's own tables reset. It fails in the one way that looks fine
from outside.

The current file was inspected before writing this: 53,248 bytes,
containing nothing beyond what the application recreates for itself on
startup via `create_all` and the `after_create` role seed. So there is
nothing to migrate, and nothing was lost in arriving at this PR.

**This PR reproduces that behaviour rather than fixing it**, so that the
cutover changes nothing about how the application runs. Making
production durable is a real change to a live service and belongs in its
own piece of work, not inside a migration. The comment in
`environment-prod.tf` says so at the point where the next reader will
meet it.

## Health checks now test the API

The target group checked `/`, which nginx serves whether or not the API
is alive — the container runs nginx in the foreground and the API as a
background process, so the API dying does not even stop the task. When
#187's service came up with a dead API, **the target group reported
healthy the entire time**.

It now checks `/api/health/`, which is served by the API process and
returns `"UP"`. Had this been in place, #187 would have failed visibly
instead of looking green.

## `LOG_LEVEL` restored to `DEBUG`

#187 set `INFO`; production runs `DEBUG`. Reverted to match, keeping
this migration behaviour-preserving. Whether `DEBUG` is appropriate for
production is a separate question and not settled here.

## What to check in the plan

A new task definition revision and a service update to use it, an
in-place update to the target group's health check, and the destruction
of the unused `db_url_prod` SSM parameter and its `random_password`.
Nothing else should change, and no other resource should be replaced.

## Verification

- `terraform validate` and `terraform fmt` pass on 1.16.0.
- **After this merges**, confirm from the task's CloudWatch logs that
the API reaches startup without the SQLAlchemy error — not from the
target group, which is what misled us last time. With the new health
check path a failed start should also surface as an unhealthy target.
- Production is unaffected either way: rule 16 still routes all traffic
to the old service.

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant