2021 08 21 (Saturday) Deployment

General Notes

This deployment includes several bugfixes and batch 6.

Due to the high amount of code changes, we will be performing this deployment after hours and on the weekend. It has been tested in mig and test environments.

Timeline

• 22:00 - ran account specific

Terraform will perform the following actions:

  # aws_iam_role_policy.migration_segments_policy will be updated in-place
  ~ resource "aws_iam_role_policy" "migration_segments_policy" {
        id     = "migration_segments_role_prod:migration_segments_policy_prod"
        name   = "migration_segments_policy_prod"
      ~ policy = jsonencode(
          ~ {
              ~ Statement = [
                    # (2 unchanged elements hidden)
                    {
                        Action   = [
                            "sqs:DeleteMessage",
                            "sqs:SendMessage",
                            "sqs:SendMessageBatch",
                            "sqs:ReceiveMessage",
                            "sqs:GetQueueAttributes",
                        ]
                        Effect   = "Allow"
                        Resource = "arn:aws:sqs:us-east-1:980423577122:*"
                    },
                  + {
                      + Action   = [
                          + "s3:GetObject",
                          + "s3:PutObject",
                        ]
                      + Effect   = "Allow"
                      + Resource = [
                          + "arn:aws:s3:::dawson.ustaxcourt.gov-documents-*",
                        ]
                    },
                ]
                # (1 unchanged element hidden)
            }
        )
        # (1 unchanged attribute hidden)
    }

• 22:05 - made the Pull Request
• 22:10 - Ensured that destination table is deleted (efcms-prod-alpha)
• 22:12 - merged PR ✅; CircleCI build
• 22:14 - while tests were running, to ensure we had a clean slate and because I had not the previous time, I destroyed the migration infrastructure: npm run destroy:migration -- ${ENV}
• 22:15 - observed a new requirement for var.documents_bucket_name, and used dawson.ustaxcourt.gov-documents-prod-us-east-1
• 22:21 - completed the destruction of migration infrastructure Destroy complete! Resources: 7 destroyed.
• 22:24 - tests pass ✅
• 22:46 - observed setup of migration worked! ✅ 😎
  • migrate: true
  • destination-table-version: alpha
  • source-table-version: beta
• 22:57 - Migration kicked off ✅
• 23:16 - Created a PR for removing migrations that have been run on every Court environment.
• 23:56 - migration finished; ✅ indexing begins 🕐
• 01:00 - migration rolling along

┌─────────┬───────────────────────┬────────────┬───────────┬─────────┐
│ (index) │       indexName       │ countAlpha │ countBeta │  diff   │
├─────────┼───────────────────────┼────────────┼───────────┼─────────┤
│    0    │     'efcms-case'      │  1450636   │  1992576  │ 541940  │
│    1    │ 'efcms-case-deadline' │    3102    │   16468   │  13366  │
│    2    │ 'efcms-docket-entry'  │  13907091  │ 18270116  │ 4363025 │
│    3    │    'efcms-message'    │   56535    │  300626   │ 244091  │
│    4    │     'efcms-user'      │   927765   │  2004226  │ 1076461 │
│    5    │   'efcms-user-case'   │  1040375   │  1981774  │ 941399  │
│    6    │   'efcms-work-item'   │   395824   │  767286   │ 371462  │
└─────────┴───────────────────────┴────────────┴───────────┴─────────┘
Total Difference: -7551744 (17781328/25333072) 70.19% 

• 01:45 - clusters quieted, testing looks good, switched colors

    Comparison between ES Clusters (Alpha was destination; Beta was source)

┌─────────┬───────────────────────┬────────────┬───────────┬──────┐
│ (index) │       indexName       │ countAlpha │ countBeta │ diff │
├─────────┼───────────────────────┼────────────┼───────────┼──────┤
│    0    │     'efcms-case'      │  1992576   │  1992576  │  0   │
│    1    │ 'efcms-case-deadline' │   16468    │   16468   │  0   │
│    2    │ 'efcms-docket-entry'  │  18270116  │ 18270116  │  0   │
│    3    │    'efcms-message'    │   300626   │  300626   │  0   │
│    4    │     'efcms-user'      │  2004226   │  2004226  │  0   │
│    5    │   'efcms-user-case'   │  1981774   │  1981774  │  0   │
│    6    │   'efcms-work-item'   │   767286   │  767286   │  0   │
└─────────┴───────────────────────┴────────────┴───────────┴──────┘

Batch 6 in the books